Interface interaction method and device and computer equipment

Abstract

The invention relates to an interface interaction method and device and computer equipment. The method comprises the steps that under the condition that a display associated with a main operation device displays a first interface, a first mode switching request is received, the first interface displays a three-dimensional image collected by an end instrument associated with the main operation device, and the display is controlled to display the first interface and a configuration interface in response to the first mode switching request, the configuration interface displays configuration parameters of at least one of the main operation device, the tail end instrument and the auxiliary device associated with the main operation device, and the configuration parameters in the configuration interface are adjusted according to operation data collected by the main operation device. By adopting the method, the control flexibility and reliability of the surgical robot can be effectively ensured.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical devices, in particular to an interface interaction method and device and computer equipment. BACKGROUND

[0002] With the continuous development of the field of medical devices, surgical robots have emerged. Surgical robots adopt master-slave control, and an operator can control the end instrument installed on the slave mechanical arm to follow the movement to perform corresponding surgical operations by controlling the master hand of the surgical robot.

[0003] However, the current surgical robot is generally equipped with two displays, which respectively display three-dimensional images collected by the end instrument and device configuration interfaces related to the operation. When the operator needs to change the parameter configuration in the device configuration interface, the line of sight will be transferred from the stereoscopic picture to the device configuration interface, and the change of the stereoscopic picture cannot be monitored in real time, thereby reducing the flexibility and reliability of the control of the surgical robot. SUMMARY

[0004] Therefore, it is necessary to provide an interface interaction method, device and computer equipment to effectively ensure the flexibility and reliability of the control of the surgical robot by displaying three-dimensional images and configuration interfaces in the same display.

[0005] In a first aspect, the present application provides an interface interaction method, comprising:

[0006] In the case that the display associated with the master operating device displays a first interface, a first mode switching request is received; the first interface displays a three-dimensional image collected by an end instrument associated with the master operating device;

[0007] In response to the first mode switching request, the display is controlled to display the first interface and a configuration interface; wherein the configuration interface displays configuration parameters of at least one of the master operating device, the end instrument and an auxiliary device associated with the master operating device;

[0008] According to the operation data collected by the master operating device, the configuration parameters in the configuration interface are adjusted.

[0009] In one of the embodiments, according to the operation data collected by the master operating device, the configuration parameters in the configuration interface are adjusted, comprising:

[0010] The operation data collected by the master operating device is mapped to the configuration interface to obtain interface interaction data of the configuration interface; the interface interaction data is used to adjust the configuration parameters in the configuration interface.

[0011] In one of the embodiments, the interface interaction data comprises position parameters and adjustment parameters.

[0012] Adjust the configuration parameter in the configuration interface by using the interface interaction data, including:

[0013] Determine the target adjustment control in the configuration interface according to the position parameter;

[0014] Adjust the configuration parameter associated with the target adjustment control according to the adjustment parameter.

[0015] In one of the embodiments, the method further includes:

[0016] Controlling the display to display the configuration interface on the first interface; or, controlling the display to display the first interface in a first area of the display screen and display the configuration interface in a second area of the display screen; wherein the overlap between the first area and the second area is less than a preset threshold.

[0017] In one of the embodiments, the area of the second area is smaller than the area of the first area.

[0018] In one of the embodiments, the method further includes:

[0019] In response to the first mode switching request, obtaining the first pose information of the main operating device; and stopping controlling the end effector according to the pose change of the main operating device;

[0020] After the display displays the first interface and the configuration interface, the method further includes:

[0021] In response to the second mode switching request, controlling the display to display the first interface and obtaining the second pose information of the main operating device; in the case that the first pose information is inconsistent with the second pose information, adjusting the pose of the main operating device according to the first pose information; and resuming controlling the end effector according to the pose change of the main operating device.

[0022] In one of the embodiments, the first mode switching request is generated based on the trigger operation on the physical control; wherein the physical control is a clutch switch or a foot pedal associated with the main operating device.

[0023] In one of the embodiments, the second mode switching request is generated based on the trigger operation on the physical control, or based on the trigger operation on the exit control in the configuration interface; wherein the physical control is a clutch switch or a foot pedal associated with the main operating device.

[0024] In a second aspect, the application further provides an interface interaction device, including:

[0025] The request receiving module is configured to receive a first mode switching request in a case where the display associated with the master operating device displays a first interface; the first interface displays a three-dimensional image collected by an end instrument associated with the master operating device;

[0026] The interface display module is configured to control the display to display the first interface and a configuration interface in response to the first mode switching request; the configuration interface displays configuration parameters of at least one of the master operating device, the end instrument, and an auxiliary device associated with the master operating device;

[0027] The configuration adjustment module is configured to adjust the configuration parameters in the configuration interface according to operation data collected by the master operating device.

[0028] In a third aspect, the present application further provides a computer device, comprising a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0029] The request receiving module is configured to receive a first mode switching request in a case where the display associated with the master operating device displays a first interface; the first interface displays a three-dimensional image collected by an end instrument associated with the master operating device;

[0030] The interface display module is configured to control the display to display the first interface and a configuration interface in response to the first mode switching request; the configuration interface displays configuration parameters of at least one of the master operating device, the end instrument, and an auxiliary device associated with the master operating device;

[0031] The configuration adjustment module is configured to adjust the configuration parameters in the configuration interface according to operation data collected by the master operating device.

[0032] In a fourth aspect, the present application further provides a computer readable storage medium, which stores a computer program; the computer program is executed by a processor to implement the following steps:

[0033] The request receiving module is configured to receive a first mode switching request in a case where the display associated with the master operating device displays a first interface; the first interface displays a three-dimensional image collected by an end instrument associated with the master operating device;

[0034] The interface display module is configured to control the display to display the first interface and a configuration interface in response to the first mode switching request; the configuration interface displays configuration parameters of at least one of the master operating device, the end instrument, and an auxiliary device associated with the master operating device;

[0035] The configuration adjustment module is configured to adjust the configuration parameters in the configuration interface according to operation data collected by the master operating device.

[0036] In a fifth aspect, the present application further provides a computer program product, comprising a computer program; the computer program is executed by a processor to implement the following steps:

[0037] In a case where the display associated with the master operating device displays a first interface, a first mode switching request is received; the first interface displays a three-dimensional image collected by an end instrument associated with the master operating device;

[0038] In response to the first mode switching request, the display is controlled to display the first interface and a configuration interface; the configuration interface displays configuration parameters of at least one of the master operating device, the end instrument, and an auxiliary device associated with the master operating device;

[0039] The configuration parameters in the configuration interface are adjusted according to operation data collected by the master operating device.

[0040] The interface interaction method, the device, and the computer device described above, in a case where the display associated with the master operating device displays a first interface, in response to a first mode switching request, the display is controlled to display the first interface and a configuration interface; the first interface displays a three-dimensional image collected by an end instrument associated with the master operating device; the configuration interface displays configuration parameters of at least one of the master operating device, the end instrument, and an auxiliary device associated with the master operating device; and then the configuration parameters in the configuration interface are adjusted according to operation data collected by the master operating device. Compared with the related art, which requires two displays to display a three-dimensional image and a configuration interface respectively, the method described above, in a case where configuration adjustment is required, displays a three-dimensional image and a configuration interface in the same display, and adjusts the configuration parameters in the configuration interface according to operation data collected by the master operating device, so that the operator's line of sight is still real-time monitoring of the three-dimensional image when adjusting the configuration parameters, and the hand does not leave the master operating device, thereby effectively ensuring the flexibility and reliability of the surgical robot control. BRIEF DESCRIPTION OF DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related art, the drawings needed to be used in the description of the embodiments of the present application or the related art will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other related drawings can also be obtained without creative labor.

[0042] Figure 1 A flowchart of the interface interaction method in one embodiment;

[0043] Figure 2 A flowchart of adjusting the configuration parameters in one embodiment;

[0044] Figure 3 A schematic diagram of displaying a page in one embodiment;

[0045] Figure 4 Fig. 2 is a schematic diagram of a display page in another embodiment;

[0046] Figure 5 Fig. 3 is a schematic diagram of a device control flow in an embodiment;

[0047] Figure 6 Fig. 4 is a schematic diagram of an interface interaction method in another embodiment;

[0048] Figure 7 Fig. 5 is a structural block diagram of an interface interaction apparatus in an embodiment;

[0049] Figure 8 Fig. 6 is an internal structure diagram of a computer device in an embodiment. DETAILED DESCRIPTION

[0050] In order to make the purposes, technical solutions and advantages of the present application clearer, further detailed description will be given below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application.

[0051] With the continuous development of the medical instrument field, surgical robots have emerged. Surgical robots adopt master-slave control, and an operator can control the end instrument installed on the slave mechanical arm to follow the movement to perform corresponding surgical operations by controlling the master hand of the surgical robot.

[0052] However, the current surgical robot is generally configured with two displays, which respectively display three-dimensional images collected by the end instrument and device configuration interfaces related to the surgery. When the operator needs to change the parameter configuration in the device configuration interface, the line of sight will be transferred from the stereoscopic picture to the device configuration interface, and the change of the stereoscopic picture cannot be monitored in real time, thereby reducing the flexibility and reliability of the control of the surgical robot.

[0053] Based on this, in an exemplary embodiment, an interface interaction method is provided, which is described by taking the control unit in the master operating device as an example. The so-called master operating device is the master control device in the surgical robot, which can be the master hand. As shown in Figure 1 The specific steps include the following steps:

[0054] S101, receiving a first mode switching request in a case where a display associated with the master operating device displays a first interface.

[0055] The display can be a display screen associated with the master operating device. The first interface displays a three-dimensional image collected by the terminal instrument associated with the master operating device. The first mode switching request is used to indicate that the master operating device enters the configuration adjustment mode from the control mode. The control mode is to control the terminal instrument by using the master operating device. The configuration adjustment mode is to adjust the configuration parameters by using the master operating device.

[0056] Further, the terminal instrument is an actuator installed on the slave manipulator in the surgical robot, which is controlled by the master control device, and can be an endoscope, a surgical instrument (such as a forceps, an ultrasonic knife head), etc. In actual application, the motion of the slave manipulator can be controlled by controlling the motion of the master operating device, so as to drive the terminal instrument on the slave manipulator to perform corresponding actions.

[0057] Optionally, when the master operating device is in the control mode, the slave manipulator can be controlled to move according to the pose change of the master operating device, so as to drive the terminal instrument to move. At this time, the terminal instrument can collect a three-dimensional image of the position in real time and feed back to the display, and the display can display a first interface containing the three-dimensional image. For example, the three-dimensional image of the position can be collected in real time by using an endoscope.

[0058] Further, when the operator has a configuration adjustment requirement, the first mode switching request can be generated by triggering the physical control associated with the master operating device. And the first mode switching request is sent to the control unit. The configuration adjustment requirement can include, but is not limited to, adjusting the seat, adjusting the armrest, adjusting the display screen configuration, controlling other auxiliary devices associated with the master operating device, etc.

[0059] It is worth noting that, in order to avoid the abnormal movement of the master operating device caused by the hand of the operator leaving the master operating device, the physical control for initiating the first mode switching request can be arranged at an operating position which can be directly touched by the hand of the operator, or at a position which does not need to be touched by the hand.

[0060] For example, the first mode switching request is generated based on the triggering operation of the physical control. The physical control is a clutch switch or a foot pedal associated with the master operating device. Specifically, the clutch switch on the clamping part of the master operating device can be used as the physical control for initiating the first mode switching request. Alternatively, the foot pedal associated with the master operating device can be used as the physical control for initiating the first mode switching request.

[0061] For example, when the master operating device is in the control mode, if the operator has a configuration adjustment requirement, the first mode switching request can be initiated by stepping on the foot pedal associated with the master operating device, and the first mode switching request is sent to the control unit.

[0062] S102, in response to the first mode switching request, controlling the display to display the first interface and a configuration interface.

[0063] The configuration interface is a display page for adjusting configuration information. Further, the configuration interface displays configuration parameters of at least one of the master operating device, the end effector, and the auxiliary device associated with the master operating device. The auxiliary device can be a medical device such as a pneumoperitoneum machine that can operate in cooperation with the master operating device, or a regular device such as a seat or a handrail.

[0064] In response to the first mode switching request, in an optional manner, a target configuration interface can be selected from the candidate configuration interfaces according to the historical parameter adjustment records. Then, the display can be controlled to display the first interface and the target configuration interface. At this time, if the target configuration interface does not have parameters to be adjusted by the operator, the operator can adjust and display other configuration interfaces by controlling the master operating device. The candidate configuration interfaces can include a configuration interface corresponding to the master operating device, a configuration interface corresponding to the end effector, and a configuration interface corresponding to the auxiliary device.

[0065] For example, the candidate configuration interface corresponding to the latest parameter adjustment record can be selected as the target configuration interface. Alternatively, the candidate configuration interface with the highest usage frequency can be selected as the target configuration interface according to the usage frequencies of the candidate configuration interfaces in the historical parameter adjustment records.

[0066] In another optional manner, a master configuration interface associated with the candidate configuration interfaces can be displayed. The master configuration interface includes selection controls of the candidate configuration interfaces. The user can trigger the selection control of any candidate configuration interface by controlling the master operating device, so as to control the display to replace the master configuration interface with the corresponding candidate configuration interface.

[0067] S103, adjusting the configuration parameters in the configuration interface according to operation data collected by the master operating device.

[0068] The configuration parameters are specific parameter values under each configuration type. For example, for the remote operation ratio configuration type, the configuration parameter can be a specific ratio value.

[0069] Optionally, after the display displays the first interface and the configuration interface, the operator can operate the master operating device to adjust the configuration parameters in the configuration interface. At this time, in order to avoid abnormal movement of the slave robot driven by the end controller, the control of the slave robot needs to be stopped.

[0070] Specifically, after the operation data of the operator acting on the master operating device is collected by the sensor in the master operating device, the operation data can be converted into operation data for the configuration interface, so as to adjust the configuration parameters in the configuration interface. It is worth noting that at this time, the line of sight of the operator will not deviate from the three-dimensional image in the first interface, thereby ensuring real-time monitoring of the three-dimensional image.

[0071] In the above interface interaction method, in the case that the first interface is displayed on the display associated with the master operating device, the display is controlled to display the first interface and the configuration interface in response to the first mode switching request; wherein the first interface displays a three-dimensional image collected by an end instrument associated with the master operating device; the configuration interface displays configuration parameters of at least one of the master operating device, the end instrument and the auxiliary device associated with the master operating device; and then the configuration parameters in the configuration interface are adjusted according to the operation data collected by the master operating device. Compared with the related art, which requires two displays to display the three-dimensional image and the configuration interface respectively, the above method displays the three-dimensional image and the configuration interface in the same display in the case of having configuration adjustment requirements, and adjusts the configuration parameters in the configuration interface according to the operation data collected by the master operating device, so that the operator's line of sight still monitors the three-dimensional image in real time when adjusting the configuration parameters, and the hand does not leave the master operating device, thereby effectively ensuring the flexibility and reliability of the surgical robot control.

[0072] In order to ensure the reliability of page interaction, on the basis of the above embodiment, in the embodiment of the present application, an optional way of adjusting the configuration parameters is provided, as shown in Figure 2 The specific steps include the following steps:

[0073] S201, mapping the operation data collected by the master operating device to the configuration interface to obtain interface interaction data of the configuration interface.

[0074] The so-called interface interaction data is data for interacting with the configuration interface.

[0075] Optionally, after the operation data of the operator acting on the master operating device is collected by the sensor, the movement and triggering of the master operating device represented by the operation data can be mapped to the configuration interface by using a preset data mapping logic to obtain interface interaction data for the configuration interface. At this time, the interface interaction data can be displayed through the interaction arrow in the configuration interface. For example, when the operator moves the master operating device to the left, the interaction arrow in the configuration interface will also move to the left.

[0076] Exemplarily, a two-dimensional plane with the same scale as the interface size of the configuration interface can be preset in the workspace of the main operation device. Then, the operation data can be mapped into the two-dimensional plane, so as to obtain the two-dimensional coordinates of the interactive arrow in the configuration interface and the specific operation parameters under each two-dimensional coordinate.

[0077] In S202, the configuration parameters in the configuration interface are adjusted according to the interface interaction data.

[0078] Optionally, the configuration parameters in the configuration interface can be adjusted according to each interactive operation of the configuration interface in the interface interaction data. Exemplarily, the parameter adjustment logic contained in the interface interaction data can be obtained by analyzing the interface interaction data according to the standard signal features sent by the user when triggering each control in the main operation device and the adjustment logic corresponding to each control. Then, the configuration parameters in the configuration interface can be adjusted according to the parameter adjustment logic.

[0079] In the embodiments of the present application, the operation data is converted into adjustment operations for the configuration parameters by mapping the operation data into the configuration interface and adjusting the configuration parameters in the configuration interface according to the interface interaction data, so as to ensure the reliability of the configuration parameter adjustment.

[0080] In order to ensure the reliability of the parameter adjustment, in the embodiments of the present application, the interface interaction data includes position parameters and adjustment parameters. Further, another optional way of adjusting the configuration parameters is provided, that is, the target adjustment control in the configuration interface is determined according to the position parameters, and the configuration parameters associated with the target adjustment control are adjusted according to the adjustment parameters.

[0081] The position parameters are parameters in the position movement dimension. For example, they can be two-dimensional coordinate parameters. The adjustment parameters are parameters in the adjustment operation dimension. For example, they can be parameters representing click operations. The target adjustment control is the control to be adjusted by the operator in the configuration interface.

[0082] Optionally, in the case where the interface interaction data contains a control selection signal, the target adjustment control can be located in the configuration interface according to the position parameters associated with the control selection signal. Then, the configuration parameters associated with the target adjustment control can be adjusted according to the parameter adjustment mode represented by the adjustment parameters, while the target adjustment control corresponding to the position parameters remains unchanged.

[0083] For example, when the operator controls the interactive arrow movement through the main operating device control configuration interface, the operator can click the target adjustment control selected by the interactive arrow in the configuration interface through pinching operation of the clamping part of the main operating device; then, the operator can adjust the configuration parameter associated with the target adjustment control through adjustment of the pinching times.

[0084] In the embodiment of the present application, by adjusting the configuration parameter associated with the target adjustment control according to the adjustment parameter after positioning the target adjustment control according to the position parameter, the operator can be more intuitively shown the target adjustment control to be adjusted during the parameter adjustment process, thereby ensuring the reliability of the parameter adjustment operation.

[0085] To ensure the reliability of the parameter adjustment, on the basis of the above embodiment, in the embodiment of the present application, an interface display mode is provided, specifically, the display is controlled to superimposedly display the configuration interface on the first interface.

[0086] Optionally, to not affect the display of the three-dimensional image, the display can be controlled to superimposedly display the configuration interface in the form of a floating layer or a pop-up window on a non-key region in the first interface. The configuration interface is relatively transparent. For example, the display page shown in FIG. 1 can be referred to. Figure 3 The display can be controlled to display the configuration interface 1 at the corner of the three-dimensional image.

[0087] Further, in the embodiment of the present application, another interface display mode is also provided, specifically, the display is controlled to display the first interface in a first region of the display screen and display the configuration interface in a second region of the display screen.

[0088] The first region is the region in which the first interface is displayed, and the second region is the region in which the configuration interface is displayed. Further, the first region and the second region can overlap or not overlap, and in the case of overlapping, the overlap degree between the first region and the second region is less than a preset threshold. The preset threshold is used to measure the overlap degree between the first region and the second region.

[0089] Optionally, the operator can customize the first region and the second region, and after the first region and the second region are configured, the overlap degree between the first region and the second region can be determined according to the position information of the first region and the position information of the second region. In the case where the overlap degree is less than the preset threshold, it is determined that the region configuration operation is valid. In the case where the overlap degree is greater than or equal to the preset threshold, it is determined that the region configuration operation is invalid, and at this time, the operator can be fed back a large overlap degree prompt.

[0090] Further, after it is determined that the region configuration operation is valid, the display can be controlled to display the first interface in the first region and display the configuration interface in the second region.

[0091] It is worth noting that, in order to ensure that the operator monitors the three-dimensional image in real time, a larger display space needs to be allocated to the first area, that is, the area of the second area is smaller than the area of the first area. For example, referring to the display page shown in FIG. 2, the three-dimensional image can be displayed in the first area 3, and the configuration interface can be displayed in the second area 2, without overlapping between the first area and the second area. Figure 4

[0092] In the embodiments of the present application, various ways of simultaneously displaying the first interface and the configuration interface are provided, which improves the flexibility of interface display on the basis of ensuring the rationality of the interface.

[0093] In order to ensure the reliability of the control of the surgical robot, on the basis of the above-mentioned embodiments, in the embodiments of the present application, an optional way of device control is provided, as shown in FIG. 3, which specifically includes the following steps: Figure 5

[0094] S501, in response to a first mode switching request, acquiring first pose information of a master operating device.

[0095] The so-called first pose information is the last pose information of the master operating device when the master operating device controls the end effector.

[0096] It can be understood that when the configuration parameters in the configuration interface are adjusted according to the operation data, the pose of the master operating device changes, and when the master operating device is subsequently restored to control the end effector, there may be a situation that the pose of the end effector does not match the pose of the master operating device. Therefore, after detecting the first mode switching request, the first pose information of the master operating device at the current time can be acquired through the sensor on the master operating device, so as to subsequently restore the pose of the master operating device.

[0097] S502, stopping controlling the end effector according to the pose change of the master operating device.

[0098] It can be understood that, in order to avoid the pose change of the master operating device when adjusting the configuration parameters affecting the action of the end effector, when the first mode switching request is detected, the end effector needs to be stopped from being controlled according to the pose change of the master operating device, that is, at this time, the pose change of the master operating device is only used to adjust the configuration parameters in the configuration interface.

[0099] S503, in response to a second mode switching request, controlling the display to display the first interface, and acquiring second pose information of the master operating device.

[0100] ​​The second mode switching request is used to indicate that the master operating device enters the control mode from the configuration adjustment mode. The second pose information is the last pose information of the master operating device when the master operating device interacts with the configuration interface.

[0101] Optionally, after the display displays the first interface and the configuration interface, the configuration parameters in the configuration interface can be adjusted according to the operation data. After the operation personnel parameter adjustment is completed, a second mode switching request can be generated by triggering a physical control associated with the master operating device or triggering a virtual control in the configuration interface. The second mode switching request is sent to the control unit.

[0102] For example, the second mode switching request is generated based on the triggering operation of the physical control or based on the triggering operation of the exit control in the configuration interface. The exit control is a control in the configuration interface for exiting the configuration adjustment mode.

[0103] Specifically, a clutch switch on the clamping part of the master operating device can be used as the physical control for initiating the second mode switching request. Alternatively, a foot pedal associated with the master operating device can be used as the physical control for initiating the second mode switching request. Alternatively, the master operating device can be controlled to trigger the exit control in the configuration interface to initiate the second mode switching request.

[0104] For example, when the master operating device is in the configuration adjustment mode, if the operation personnel completes the configuration parameter adjustment, the master operating device can be controlled to click the exit control in the configuration interface to initiate the second mode switching request and send the second mode switching request to the control unit.

[0105] Optionally, in response to the second mode switching request, the display can be controlled to display only the first interface, and the second pose information of the master operating device can be obtained through the sensor on the master operating device.

[0106] S504, in the case where the first pose information and the second pose information are inconsistent, the pose of the master operating device is adjusted according to the first pose information.

[0107] Optionally, the consistency of the first pose information and the second pose information can be verified. In the case where the first pose information and the second pose information are consistent, the control of the end effector according to the pose change of the master operating device can be directly restored.

[0108] In the case where the first pose information and the second pose information are inconsistent, the pose of the master operating device can be adjusted according to the first pose information, so that the pose of the master operating device is restored to the pose before entering the configuration adjustment mode, thereby ensuring the matching between the pose of the end controller and the pose of the master operating device.

[0109] S505, resuming the control of the end effector according to the pose change of the master operating device.

[0110] Optionally, after adjusting the pose of the master operating device, the control of the end effector according to the pose change of the master operating device can be resumed, that is, at this time, the end effector can be controlled to operate again by controlling the master operating device.

[0111] It is worth noting that, in addition to ensuring the matching between the end controller and the master operating device by recording the pose information of the master operating device, after receiving the first mode switching request, the encoder readings of the joints in the master operating device at the current time can be obtained as the first encoder readings. After the display displays the first interface and the configuration interface, in response to the second mode switching request, the encoder readings of the joints in the master operating device at this time are obtained as the second encoder readings.

[0112] In the case where the first encoder readings are consistent with the second encoder readings, the control of the end effector according to the pose change of the master operating device can be directly resumed. In the case where the first encoder readings are inconsistent with the second encoder readings, the positions of the joints in the master operating device can be adjusted according to the first encoder readings, so that the pose of the master operating device is restored to the pose before entering the configuration adjustment mode, thereby ensuring the matching between the pose of the end controller and the pose of the master operating device.

[0113] In the embodiments of the present application, by recording the first pose information of the master operating device after receiving the first mode switching request, it can be ensured that the position information of the master operating device matches the pose of the end effector when the master operating device controls the end effector again, thereby ensuring the reliability of the device control.

[0114] Figure 6 For the flowchart of the interface interaction method in another embodiment, on the basis of the above-mentioned embodiments, the present embodiment provides an optional example of an interface interaction method. In combination with Figure 6 , the specific implementation process is as follows:

[0115] S601, in the case where the display associated with the master operating device displays a first interface, receiving a first mode switching request.

[0116] The first interface displays a three-dimensional image collected by the end effector associated with the master operating device.

[0117] S602, in response to the first mode switching request, controlling the display to superimpose and display a configuration interface on the first interface, or controlling the display to display the first interface in a first area of the display screen and display the configuration interface in a second area of the display screen.

[0118] The overlap between the first region and the second region is less than a preset threshold. The configuration interface displays configuration parameters of at least one of the master operating device, the terminal instrument, and the auxiliary device associated with the master operating device. The area of the second region is less than the area of the first region.

[0119] The first mode switching request is generated based on a trigger operation on a physical control. The physical control is a clutch switch or a foot pedal associated with the master operating device.

[0120] S603, obtaining first pose information of the master operating device, and stopping control of the terminal instrument according to a change in the pose of the master operating device.

[0121] S604, mapping operation data collected through the master operating device to the configuration interface to obtain interface interaction data of the configuration interface, and adjusting configuration parameters in the configuration interface using the interface interaction data.

[0122] The interface interaction data includes a position parameter and an adjustment parameter.

[0123] Optionally, a target adjustment control in the configuration interface can be determined according to the position parameter, and a configuration parameter associated with the target adjustment control can be adjusted according to the adjustment parameter.

[0124] S605, in response to the second mode switching request, controlling the display to display the first interface, and obtaining second pose information of the master operating device.

[0125] The second mode switching request is generated based on a trigger operation on a physical control, or based on a trigger operation on an exit control in the configuration interface.

[0126] S606, in a case where the first pose information and the second pose information are inconsistent, adjusting the pose of the master operating device according to the first pose information, and resuming control of the terminal instrument according to a change in the pose of the master operating device.

[0127] The specific process of S601-S606 described above can be referred to the description of the method embodiments, which has similar implementation principles and technical effects, and will not be repeated here.

[0128] It should be understood that although each step in the flowchart involved in the embodiments described above is shown in sequence according to the arrow, these steps are not necessarily executed in the order indicated by the arrow. Unless explicitly stated herein, the execution of these steps is not strictly limited in sequence, and these steps can be executed in other orders. Moreover, at least part of the steps in the flowchart involved in the embodiments described above can include multiple steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily sequential, but can be alternately executed with at least part of other steps or steps or stages in other steps.

[0129] Based on the same inventive concept, the embodiments of the present application also provide an interface interaction device for implementing the interface interaction method described above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme described in the above method, so the specific limitations in one or more interface interaction device embodiments provided below can refer to the limitations of the interface interaction method described above, which will not be repeated here.

[0130] In an exemplary embodiment, as shown in Figure 7 An interface interaction device 7 is provided, comprising: a request receiving module 710, an interface display module 720, and a configuration adjustment module 730, wherein:

[0131] The request receiving module 710 is configured to receive a first mode switching request when a display associated with a master operating device displays a first interface; the first interface displays a three-dimensional image collected by an end instrument associated with the master operating device;

[0132] The interface display module 720 is configured to control the display to display the first interface and a configuration interface in response to the first mode switching request; wherein the configuration interface displays configuration parameters of at least one of the master operating device, the end instrument, and an auxiliary device associated with the master operating device;

[0133] The configuration adjustment module 730 is configured to adjust the configuration parameters in the configuration interface according to operation data collected by the master operating device.

[0134] In an exemplary embodiment, the configuration adjustment module 730 is specifically configured to:

[0135] map the operation data collected by the master operating device to the configuration interface to obtain interface interaction data of the configuration interface; and adjust the configuration parameters in the configuration interface using the interface interaction data.

[0136] In an example embodiment, the interface interaction data comprises a position parameter and an adjustment parameter; the configuration adjustment module 730 is further configured to:

[0137] According to the position parameter, a target adjustment control in the configuration interface is determined; and according to the adjustment parameter, a configuration parameter associated with the target adjustment control is adjusted.

[0138] In an example embodiment, the interface display module 720 is specifically configured to:

[0139] The display is controlled to display the configuration interface on the first interface; or, the display is controlled to display the first interface on a first region of the display screen and display the configuration interface on a second region of the display screen; wherein an overlap degree between the first region and the second region is less than a preset threshold.

[0140] In an example embodiment, an area of the second region is less than an area of the first region.

[0141] In an example embodiment, the interface interaction apparatus 7 further comprises a pose adjustment module, wherein the pose adjustment module is specifically configured to:

[0142] In response to a first mode switching request, first pose information of the master operating device is acquired; and the control on the terminal instrument according to the pose change of the master operating device is stopped;

[0143] After the display displays the first interface and the configuration interface, the pose adjustment module is further configured to:

[0144] In response to a second mode switching request, the display is controlled to display the first interface, and second pose information of the master operating device is acquired; in a case where the first pose information is inconsistent with the second pose information, the pose of the master operating device is adjusted according to the first pose information; and the control on the terminal instrument according to the pose change of the master operating device is resumed.

[0145] In an example embodiment, the first mode switching request is generated based on a trigger operation on a physical control; wherein the physical control is a clutch switch or a foot pedal associated with the master operating device.

[0146] In an example embodiment, the second mode switching request is generated based on a trigger operation on a physical control, or based on a trigger operation on a quit control in the configuration interface; wherein the physical control is a clutch switch or a foot pedal associated with the master operating device.

[0147] The modules in the interface interaction device can be implemented by software, hardware, or a combination thereof. The modules can be embedded in or independent of a processor in a computer device in hardware form, or stored in a memory in the computer device in software form, so that the processor can call and execute the operations corresponding to the modules.

[0148] In an exemplary embodiment, a computer device, which can be a terminal, has an internal structure as shown in Figure 8 The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit, and an input device. The processor, the memory, and the input / output interface are connected through a system bus, and the communication interface, the display unit, and the input device are connected to the system bus through the input / output interface. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for running the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is configured to exchange information between the processor and external devices. For example, the input interface can be configured to receive operation data collected by a master operating device; and the output interface can be configured to transmit pose information sent by a control unit to an end effector. The communication interface of the computer device is configured to perform wired or wireless communication with an external terminal. The wireless communication can be achieved through WIFI, mobile cellular network, near field communication (NFC), or other technologies. The computer program is executed by the processor to implement an interface interaction method. The display unit of the computer device is configured to form a visually visible picture, which can include a display associated with the master operating device, a projection device, or a virtual reality imaging device. The display can include a liquid crystal display or an electronic ink display capable of displaying a configuration interface. The input device of the computer device can include a master hand, a foot pedal, a touch layer overlaid on the display, or a keyboard, a trackball, or a touchpad arranged on the housing of the computer device. The input device can also include an external keyboard, a touchpad, or a mouse.

[0149] Those skilled in the art can understand that Figure 8 The structure shown in the above

[0150] In an embodiment, a computer device is also provided, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps in the above method embodiments when executing the computer program.

[0151] In an embodiment, a computer readable storage medium is provided, which stores a computer program, and the computer program is executed by a processor to implement the steps in the above method embodiments.

[0152] In an embodiment, a computer program product is provided, comprising a computer program, and the computer program is executed by a processor to implement the steps in the above method embodiments.

[0153] It should be noted that the data (including but not limited to operation data) involved in the present application is all data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data need to comply with relevant regulations.

[0154] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when executed, can include the processes of the above-mentioned embodiment methods. Any reference to memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. The non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. The volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration but not limitation, the RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The database involved in the embodiments provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments provided in the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., without being limited thereto.

[0155] The technical features of the above embodiments can be combined arbitrarily. In order to make the description simple, all possible combinations of the technical features in the above embodiments are not described, however, as long as the combinations of the technical features do not exist contradictory, they should be considered as the scope of the present application.

[0156] The above-described embodiments are merely illustrative of several embodiments of the present application, and the description is relatively specific and detailed, but should not be understood as a limitation on the scope of the patent. It should be noted that for those skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are all within the scope of the present application. Therefore, the scope of protection of the present application should be subject to the appended claims.

Claims

1. An interface interaction method, characterized by, The method comprises: In the case that the display associated with the master operating device displays a first interface, a first mode switching request is received; the first interface displays a three-dimensional image collected by an end instrument associated with the master operating device; In response to the first mode switching request, the display is controlled to display the first interface and a configuration interface; wherein the configuration interface displays configuration parameters of at least one of the master operating device, the end instrument and an auxiliary device associated with the master operating device; According to operation data collected by the master operating device, the configuration parameters in the configuration interface are adjusted.

2. The method of claim 1, wherein, The adjustment of the configuration parameters in the configuration interface according to the operation data collected by the master operating device comprises: The operation data collected by the master operating device is mapped to the configuration interface to obtain interface interaction data of the configuration interface; The configuration parameters in the configuration interface are adjusted by using the interface interaction data.

3. The method of claim 2, wherein, The interface interaction data comprises position parameters and adjustment parameters. The adjustment of the configuration parameters in the configuration interface by using the interface interaction data comprises: According to the position parameters, a target adjustment control in the configuration interface is determined; According to the adjustment parameters, the configuration parameters associated with the target adjustment control are adjusted.

4. The method according to any one of claims 1 to 3, characterized in that, The control of the display to display the first interface and the configuration interface comprises: The display is controlled to display the configuration interface on the first interface; or The display is controlled to display the first interface in a first area of the display screen and the configuration interface in a second area of the display screen; wherein the overlap degree between the first area and the second area is less than a preset threshold.

5. The method of claim 4, wherein, The area of the second area is smaller than the area of the first area.

6. The method according to any one of claims 1 to 3, characterized in that, The method further comprises: In response to the first mode switching request, first pose information of the master operating device is obtained; The control of the end instrument according to the pose change of the master operating device is stopped; After the display displays the first interface and the configuration interface, the method further comprises: In response to a second mode switching request, the display is controlled to display the first interface, and second pose information of the master operating device is obtained; In the case that the first pose information is inconsistent with the second pose information, the pose of the master operating device is adjusted according to the first pose information; The control of the end instrument according to the pose change of the master operating device is resumed.

7. The method according to any one of claims 1 to 3, characterized in that, The first mode switching request is generated based on a trigger operation on a physical control; wherein the physical control is a clutch switch or a foot pedal associated with the master operating device.

8. The method of claim 6, wherein, The second mode switching request is generated based on a trigger operation on a physical control, or based on a trigger operation on a quit control in the configuration interface; wherein the physical control is a clutch switch or a foot pedal associated with the master operating device.

9. A page interaction apparatus characterized by comprising: The device comprises: The request receiving module is configured to receive a first mode switching request in a case where a display associated with the master operating device displays a first interface; the first interface displays a three-dimensional image collected by an end instrument associated with the master operating device; The interface display module is configured to control the display to display the first interface and a configuration interface in response to the first mode switching request; wherein the configuration interface displays configuration parameters of at least one of the master operating device, the end instrument, and an auxiliary device associated with the master operating device; The configuration adjustment module is configured to adjust the configuration parameters in the configuration interface according to operation data collected by the master operating device. 10.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is configured to perform the method according to any one of claims 1-9. The processor implements the steps of the method in any one of claims 1 to 8 when executing the computer program.

Images