Master-slave device control system and method for controlling same

Abstract

Provided are a master-slave device control system and a method for controlling same. The master-slave device control system comprises a master device (1) and a slave device (2). The master device (1) is connected to a target model (3), and the target model (3) is configured for describing a target object (4). A position conversion relationship between the target model (3) and a reference model (6) is designated as a first positional conversion relationship, and the reference model (6) is configured for describing an additional object (5) other than the target object (4). The slave device (2) is connected to the target object (4), and the positional conversion relationship between the target object (4) and the additional object (5) is a second position conversion relationship. The first positional conversion relationship corresponds to the second positional conversion relationship. The control method comprises: acquiring a second positional conversion relationship via a master-slave device control system, and determining a first positional conversion relationship according to the second position conversion relationship (201); updating a model position of the target model according to the first positional conversion relationship to give an updated calibration target model (202); determining a pseudo-position corresponding to the calibration target model, and determining a third positional conversion relationship on the basis of the pseudo-position (203); and determining a positional mapping relationship between the master device and the slave device according to the third positional conversion relationship, so as to control the master-slave device control system to execute a control operation on the basis of the positional mapping relationship (204).

Description

A master-slave device control system and control method thereof

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on March 29, 2024, with application number 202410382226.6 and invention name “A master-slave device control system and control method thereof”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application belongs to the field of medical device technology, and in particular relates to a master-slave device control system and a control method thereof. Background Art

[0003] With the advancement of medical technology, master-slave operating systems, featuring high precision and reliability, are widely used in the medical field. Specifically, a master-slave operating system consists of an active and a passive terminal. During actual operation, doctors access patient information from the passive terminal in real time and use this information to operate the active terminal. Through master-slave control, doctors manipulate the medical devices on the passive terminal to perform diagnosis or surgery on the patient.

[0004] Currently, cameras are usually used to obtain image information of patients at the slave end. However, the image information often only carries two-dimensional plane information, and the patient is only described on a two-dimensional level. Information other than the two-dimensional plane information is lacking, resulting in insufficient description of the patient, thereby reducing the reliability of surgery based on image information.

[0005] Therefore, the idea has arisen to perform 3D reconstruction on patients, essentially scanning and digitizing them, to help doctors directly obtain 3D information about the patient. However, 3D scanning is expensive to implement and is susceptible to light and shadow effects, which can reduce the effectiveness of 3D scanning. This also reduces the reliability of surgeries based on the information obtained from 3D reconstruction. Technical issues

[0006] The embodiments of the present application provide a master-slave device control system and a control method thereof, aiming to solve the problem that the existing three-dimensional scanning is easily affected by light and shadow when obtaining three-dimensional information of a patient, thereby reducing the scanning effect and reliability of the three-dimensional scanning. Technical Solutions

[0007] In a first aspect, an embodiment of the present application provides a master-slave device control system, the master-slave device control system comprising an active device and a slave device, the active device being configured to respond to a movement instruction and perform a movement operation, and the slave device being configured to respond to the movement operation and perform a control operation corresponding to the movement operation;

[0008] The active device is connected to a target model, the target model is used to describe the target object, the position conversion relationship between the target model and a reference model is a first position conversion relationship, and the reference model is used to describe other objects except the target object;

[0009] The slave device is connected to the target object, and the position conversion relationship between the target object and the other object is a second position conversion relationship;

[0010] The first position conversion relationship corresponds to the second position conversion relationship.

[0011] In a possible implementation of the first aspect above, the master-slave device control system further includes a calibration module;

[0012] The calibration module is used to calibrate the position conversion relationship between the active device, the target model and the reference model;

[0013] Wherein, the calibration module is arranged on the active device.

[0014] In a possible implementation of the first aspect, the master-slave device control system further includes a reference module, the reference module being configured to determine a position conversion relationship between the slave device and the reference model;

[0015] The reference module includes a first reference module and a second reference module. The first reference module is set at the slave device, and the second reference module is set at the target object.

[0016] In a second aspect, an embodiment of the present application provides a control method for a master-slave device control system. The method is applied to a master-slave device control system, wherein the master-slave device control system includes an active device and a slave device, wherein the active device is connected to a target model, and the method includes:

[0017] The master-slave device control system acquires a second position conversion relationship, and determines a first position conversion relationship based on the second position conversion relationship, where the first position conversion relationship is a position conversion relationship between a first desired position of the target model and a reference model;

[0018] updating the model position of the target model according to the first position conversion relationship to obtain an updated calibrated target model;

[0019] Determining a pseudo position corresponding to the calibration target model, and determining a third position conversion relationship based on the pseudo position, wherein the third position conversion relationship is a position conversion relationship between the pseudo position and the calibration target model;

[0020] According to the third position conversion relationship, a position mapping relationship between the active device and the slave device is determined, so as to control the master-slave device control system to perform a control operation based on the position mapping relationship.

[0021] In a possible implementation of the second aspect, updating the model position of the target model according to the first position conversion relationship to obtain an updated calibrated target model includes:

[0022] determining a second expected position corresponding to the active device according to the first position conversion relationship;

[0023] The model position of the target model is updated based on the second expected position to obtain an updated calibrated target model.

[0024] In a possible implementation of the second aspect, the master-slave device control system further includes a calibration module, and determining the second expected position corresponding to the active device based on the first position conversion relationship includes:

[0025] determining a fourth position conversion relationship between the calibration module and the active device, and determining a fifth position conversion relationship between the target model and the calibration module;

[0026] A second expected position corresponding to the active device is determined according to the first position conversion relationship, the fourth position conversion relationship, and the fifth position conversion relationship.

[0027] In a possible implementation of the second aspect, determining the third position conversion relationship according to the pseudo position includes:

[0028] determining a seventh position conversion relationship between the active device and the target model according to the fourth position conversion relationship and the fifth position conversion relationship;

[0029] A third position conversion relationship is determined according to the pseudo position and the seventh position conversion relationship.

[0030] In a possible implementation of the second aspect, the target model is used to describe a target object, and determining the position mapping relationship between the active device and the slave device based on the third position conversion relationship includes:

[0031] determining an eighth position conversion relationship between the target object and the driven device;

[0032] A position mapping relationship between the active device and the passive device is determined according to the third position conversion relationship and the eighth position conversion relationship.

[0033] In a possible implementation of the second aspect, the master-slave device control system further includes a reference module, the reference module including a first reference module and a second reference module, and determining the eighth position conversion relationship between the target object and the slave device includes:

[0034] determining a ninth position conversion relationship between the target object and the first reference module, determining a tenth position conversion relationship between the first reference module and the second reference module, and determining an eleventh position conversion relationship between the second reference module and the slave device;

[0035] An eighth position conversion relationship between the target object and the driven device is determined according to the ninth position conversion relationship, the tenth position conversion relationship, and the eleventh position conversion relationship.

[0036] In a possible implementation of the second aspect, the method further includes:

[0037] Acquire an original position of the active device, and acquire an end position of the active device after performing a move operation in response to a move instruction;

[0038] generating a control instruction for the driven device based on the original posture, the terminal posture, and the position mapping relationship;

[0039] The slave device is controlled to move to a target posture corresponding to the control instruction according to the control instruction.

[0040] In a third aspect, an embodiment of the present application provides a control device for a master-slave device control system, the device being applied to a master-slave device control system, the master-slave device control system including an active device and a slave device, the active device being connected to a target model, the device including:

[0041] The acquisition module is used to acquire a second position conversion relationship and determine a first position conversion relationship based on the second position conversion relationship, where the first position conversion relationship is a position conversion relationship between a first desired position of the target model and a reference model.

[0042] An updating module is used to update the model position of the target model according to the first position conversion relationship to obtain an updated calibrated target model.

[0043] The determining module is configured to determine a pseudo position corresponding to the calibration target model, and determine a third position conversion relationship based on the pseudo position, where the third position conversion relationship is a position conversion relationship between the pseudo position and the calibration target model.

[0044] A mapping module is configured to determine a position mapping relationship between the active device and the slave device according to the third position conversion relationship, so as to control the master-slave device control system to perform a control operation based on the position mapping relationship.

[0045] In a fourth aspect, an embodiment of the present application provides a terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the control method of the master-slave device control system provided in the second aspect above is implemented.

[0046] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the control method of the master-slave device control system provided in the second aspect above.

[0047] It can be understood that the beneficial effects of the first aspect and the third aspect to the fifth aspect can be found in the relevant description of the second aspect, and will not be repeated here. Beneficial effects

[0048] Compared with the prior art, the embodiments of the present application have the following beneficial effects:

[0049] In an embodiment of the present application, a second position conversion relationship is obtained through a master-slave device control system, and a first position conversion relationship is determined based on the second position conversion relationship. The model position of the target model is updated based on the first position conversion relationship to obtain an updated calibration target model. Then, information of the target object on a three-dimensional level can be obtained by obtaining information of the calibration target model, thereby reducing the situation where the obtained information does not adequately describe the patient. In addition, a pseudo position corresponding to the calibration target model is determined, and a third position conversion relationship is determined based on the pseudo position. Based on the third position conversion relationship, a position mapping relationship between the active device and the slave device is determined, so as to control the master-slave device control system to perform control operations based on the position mapping relationship, thereby controlling the movement of the target object connected to the slave device to perform surgery on the target object, thereby improving the reliability of surgery based on the information described by the target model, and the implementation cost is low. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] FIG1 is a schematic structural diagram of a master-slave device control system provided by an embodiment of the present application;

[0051] FIG2 is a flowchart of a control method for a master-slave device control system provided by an embodiment of the present application;

[0052] FIG3 is a schematic diagram of an example of target model initialization provided by an embodiment of the present application;

[0053] FIG4 is a flowchart of another method for controlling a master-slave device control system according to an embodiment of the present application;

[0054] FIG5 is a schematic structural diagram of a control device of a master-slave device control system provided by an embodiment of the present application;

[0055] FIG6 is a structural block diagram of a terminal device provided in an embodiment of the present application. Modes for Carrying Out the Invention

[0056] In order to make the technical problems, technical solutions and beneficial effects to be solved by this application more clearly understood, this application is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0057] Referring to FIG. 1 , FIG. 1 shows a schematic structural diagram of a master-slave device control system provided by an embodiment of the present application. As shown in FIG. 1 , the master-slave device control system includes an active device 1 and a slave device 2 .

[0058] The active device 1 can be a device that can control other devices to perform corresponding operations, such as a computer, a robotic arm, a robot, etc. Other devices can be devices that respond to the operation of the active device 1 and perform corresponding operations, that is, the slave device 2, such as a scanner, a robotic arm, etc.

[0059] The active device 1 can be communicatively connected with the slave device 2. The active device 1 can be used to respond to movement instructions and perform movement operations. The movement instructions can be generated by the user based on the movement requirements of the active device 1 and / or the slave device 2. The movement operation can be an operation of moving the active device 1 to a target posture corresponding to the movement requirement. The slave device 2 can be used to respond to the movement operation and perform control operations corresponding to the movement operation.

[0060] In actual applications, based on the movement requirements of the active device 1 and / or the slave device 2, the user can generate a movement instruction for the active device 1. The active device 1 can then respond to the movement instruction and perform a movement operation. The active device 1 can also send a control instruction corresponding to the movement operation to the slave device 2 via a communication connection with the slave device 2. The slave device 2 can respond to the control instruction corresponding to the movement operation and perform the control operation corresponding to the movement operation, such as controlling the slave device to move to a target position corresponding to the movement operation, or adjusting the posture of the slave device to a target posture corresponding to the movement operation. Specifically, the control instruction can be generated based on the movement operation of the active device 1.

[0061] The active device 1 can be connected to the target model 3. The target model 3 can be a model that describes things through entities, such as a model generated by 3D printing technology, or a model manufactured by casting, forging, welding, etc. The target model 3 can also be a model that describes things through virtual data, such as a three-dimensional model imaged by holographic projection.

[0062] The target model 3 may be used to describe a target object 4 . The target object 4 may be an object for which a user needs to obtain information. The slave device 2 may be connected to the target object 4 .

[0063] The target object may be composed of at least one component.

[0064] In specific implementations, the master-slave device control system can be applied to scenarios that require master-slave control, such as smart home scenarios, surgical scenarios, mechanical operation scenarios, etc., wherein surgical scenarios may include orthopedic surgical scenarios, such as pelvic surgical scenarios, and mechanical operation scenarios may include mechanical manufacturing scenarios, mechanical maintenance scenarios, etc. In different application scenarios, the active device, the slave device, the target model, and the target object may also be different. For example, when the application scenario is a pelvic surgical scenario, the active device and the slave device may be a robotic arm used to perform pelvic surgery, the target model may be a pelvic model generated by 3D printing technology to describe the pelvis, and the target object may be the pelvis that requires pelvic surgery. When the application scenario is a mechanical maintenance scenario, the active device and the slave device may be a robot used to perform mechanical maintenance, the target model may be a three-dimensional model of a product imaged by holographic projection, and the target object may be a product that needs to be maintained.

[0065] As an example, the target model can be connected to the active device by bolts, or fixed to any end of the active device by a fixing component. For example, a fixing bracket can be installed on the active device, and the target model can be fixed to the active device through the fixing bracket.

[0066] The position conversion relationship between the target model 3 and the reference model 6 may be a first position conversion relationship, and the position conversion relationship between the target object 4 and the other objects 5 may be a second position conversion relationship. The first position conversion relationship may correspond to the second position conversion relationship.

[0067] The reference model 6 can be used to describe other objects 5 besides the target object. Similarly, the reference model 6 can be a model that describes things through entities, such as a model generated by 3D printing technology, or a model manufactured by casting, forging, welding, etc. The reference model 6 can also be a model that describes things through virtual data, such as a three-dimensional model imaged by holographic projection.

[0068] It should be understood that in different application scenarios, in addition to the target object, there are other objects that cannot be ignored, namely other objects. For example, in a pelvic surgery scenario, since the pelvis is usually connected by two hip bones on the left and right sides, when performing surgery on a pelvis with a damage on one side, that is, when performing surgery on the target object, it will inevitably be affected by the pelvis on the other side that is not damaged, such as blocking light, blocking the operation of the robotic arm, etc. Therefore, it is necessary to consider the positional relationship between the pelvis on the other side that is not damaged and the driven equipment and the target object, and then it is necessary to generate a reference model for describing other objects. For another example, in a mechanical maintenance scenario, since the product being maintained can usually be composed of multiple parts, and the target object can be a damaged part among all the parts of the product, when maintaining the damaged part, it will also be affected by other parts among all the parts except the damaged part, such as blocking light, blocking the operation of the robotic arm, etc. Therefore, it is necessary to consider the positional relationship between other parts and the driven equipment and the target object, and then it is necessary to generate a reference model for describing other objects.

[0069] In one embodiment of the present application, the master-slave device control system may further include a calibration module 7. The calibration module 7 may be provided on the active device 1 and may be used to calibrate the position conversion relationship between the active device 1, the target model 3, and the reference model 6. The calibration module 7 may be used to assist in determining the position conversion relationship, thereby improving the efficiency of determining the position conversion relationship. Specifically, the calibration module 7 may be a calibration board.

[0070] In one embodiment of the present application, the master-slave device control system may further include a reference module 8. Reference module 8 may be used to determine the position conversion relationship between slave device 2 and reference model 6. Reference module 8 may be used to assist in determining the position conversion relationship, thereby improving the efficiency of determining the position conversion relationship. Specifically, reference module 8 may be a reference frame.

[0071] The reference module 8 may include a first reference module 81 and a second reference module 82 . The first reference module 81 may be disposed on the slave device 2 , and the second reference module 82 may be disposed on the target object 4 .

[0072] In one embodiment of the present application, the master-slave device control system may further include a position relationship determination module, which may be used to obtain the position information and posture information of an object. The position relationship determination module may include a structured light camera, an optical positioning system and other devices.

[0073] As an example, when the active device and / or the slave device is a robot or a robotic arm, the active device and the slave device can be provided with a camera, and then the hand-eye calibration method can be used to determine the position conversion relationship between the camera and the device. Based on the position conversion relationship between the camera and the device, the position information and posture information of objects within the shooting range of the camera can be determined.

[0074] In an embodiment of the present application, a master-slave device control system includes an active device and a slave device. The active device is connected to a target model, and the target model is used to describe a target object. The slave device is connected to the target object, and then the target model can be operated to obtain information about the target object on a three-dimensional level, thereby reducing the situation where the acquired information does not adequately describe the patient. In addition, the active device can be used to respond to movement instructions and perform movement operations, and the slave device is used to respond to movement operations and perform control operations corresponding to the movement operations, wherein the position conversion relationship between the active device and the target model is a first position conversion relationship, and the position conversion relationship between the slave device and the target object is a second position conversion relationship, and the first position conversion relationship corresponds to the second position conversion relationship. Then, the user can control the active device through the information described by the target model, and control the movement of the slave device through the correspondence between the first position conversion relationship and the second position conversion relationship, thereby controlling the movement of the target object connected to the slave device to perform surgery on the target object, thereby improving the reliability of surgery based on the information described by the target model, and the implementation cost is low.

[0075] 2 , which shows a flowchart of a control method for a master-slave device control system according to an embodiment of the present application, which may include the following steps:

[0076] Step 201: The master-slave device control system obtains a second position conversion relationship, and determines a first position conversion relationship according to the second position conversion relationship.

[0077] When a master-slave device control system is required to perform an operation, the target object whose information needs to be obtained and other objects related to the target object can be determined, and then the master-slave device control system can obtain target object information of the target object and other object information of other objects.

[0078] The target object information may include the target object's structural information, dimensional information, and positional information. Structural information may include the connection relationships between each component in the target object, which may include detachable and fixed connections. Detachable connections include bolt connections, interference fit connections, and fixed connections include bonding and welding. Dimensional information may include the dimensions of each component in the target object, such as length, width, height, diameter, and radian. Positional information may include the relative positional relationships between each component in the target object, including vertical and horizontal positional relationships. Similarly, other object information may include the structural information, dimensional information, and positional information of other objects.

[0079] In practical applications, target object information of a target object can be predetermined and stored. For example, a three-dimensional scan of the target object can be performed, and the target object information can be extracted from the scan results and stored in the master-slave device control system, thereby allowing the master-slave device control system to directly obtain the target object information of the target object. Similarly, the determination of other object information can refer to the above-mentioned target object information determination steps.

[0080] After obtaining the target object information and other object information, a second position conversion relationship between the target object and other objects may be determined based on the target object information and other object information.

[0081] In practical applications, the first target component that has interactions with other objects can be determined from all components of the target object, and the second target component that has interactions with the first target component can be determined from all components of the other objects. Then, the first structural information and first position information between all components of the target object and the first target component can be determined, as well as the second structural information and second position information between all components of the other objects and the second target component can be determined. Then, the second position conversion relationship between the target object and other objects can be determined based on the first structural information, first position information, second structural information, second position information and the relative position relationship between the first target component and the second target component.

[0082] As an example, the second position conversion relationship between the target object and other objects can be obtained by performing three-dimensional scanning on the target object and other objects at the same time.

[0083] For example, when the target object is a damaged pelvis and the other object is a healthy pelvis without damage, the entire pelvis including the damaged pelvis and the healthy pelvis can be CT scanned at the same time, and the second position conversion relationship between the damaged pelvis and the healthy pelvis can be obtained.

[0084] After the target object and other objects are determined, a target model for describing the target object and a reference model for describing other objects may be generated.

[0085] In practical applications, 3D printing technology can be used to print a target model describing the target object at a certain scale according to the target object's target object information. Component models of all components in the target object can also be printed at a certain scale according to the target object's target object information. All component models can then be spliced ​​together to obtain the target model. Holographic projection imaging can also be used according to the target object's target object information to obtain a virtual target model. Similarly, the same method can be used to generate a reference model.

[0086] After the target model and the reference model are obtained and the second position conversion relationship is determined, the first position conversion relationship between the target model and the reference model can be determined according to the second position conversion relationship.

[0087] The first position conversion relationship is a position conversion relationship between a first expected position of the target model and the reference model, and the first expected position may be a position where the target model should be located.

[0088] In practical applications, a structured light camera and a hand-eye calibration method can be used to determine the first position transformation relationship between the target model and the reference model.

[0089] It is important to understand that users need to obtain information about the target object through the target model. However, in different application scenarios, there may be objects that affect the user's ability to obtain information about the target object or affect the user's ability to perform operations in the application scenario. These are other objects that cannot be ignored. Therefore, when obtaining information about the target object, it is also necessary to obtain information about other objects, that is, it is also necessary to obtain information about other objects through the reference model. In addition, in order to enable users to obtain more accurate information, the relative positional relationship between the target model and the reference model can be adjusted to accurately restore the relative positional relationship between the target object and other objects. In other words, the relative positional relationship between the target object and other objects corresponds to the relative positional relationship between the target model and the reference model.

[0090] Based on this, the first position conversion relationship between the target model and the reference model can be determined through the second position conversion relationship.

[0091] Specifically, the second position conversion relationship and the first position conversion relationship can be expressed by the following formula:

[0092] Among them, target model 3 * can be expressed as the first desired position of the target model, It can be expressed as the first position transformation relationship between the target model 3 and the reference model 6, It can be expressed as a second position conversion relationship between the target object 4 and other objects 5.

[0093] Step 202 : updating the model position of the target model according to the first position conversion relationship to obtain an updated calibrated target model.

[0094] The model position can be the position of the target model in the global coordinate system. The global coordinate system can be any coordinate system of the active device, the driven device, the target model, the reference model, or a coordinate system set by the user. The calibrated target model can be the target model that has completed the model position initialization.

[0095] After obtaining the first position conversion relationship, the target model can be moved to the first desired position by the active device to obtain the moved target model, which is the calibrated target model after completing the initialization calibration.

[0096] In practical applications, the reference model can be fixed, and the target model can be moved to the first desired position so that the relative position relationship between the target model and the reference model corresponds to the relative position relationship between the target object and other objects. This can complete the position calibration of the target model, that is, initialize the model position of the target model to obtain the calibrated target model.

[0097] 3 , which shows a schematic diagram of an example of initializing a target model provided by an embodiment of the present application. As shown in FIG3 , the dotted box A may be the first expected position of the target model 3 .

[0098] In a specific implementation, the first position conversion relationship between the target model 3 and the reference model 6 should correspond to the second position conversion relationship between the target object 4 and other objects 5. Then, the target model 3 can be moved to the dotted box A based on the second position conversion relationship, so that the position conversion relationship between the moved target model 3 and the reference model 6 corresponds to the position conversion relationship between the target object 4 and other objects 5, thereby completing the position calibration of the target model 3.

[0099] Step 203: Determine a pseudo position corresponding to the calibration target model, and determine a third position conversion relationship according to the pseudo position.

[0100] The pseudo position may be a position reached by the target model when the simulated user operates the active device, and the third position conversion relationship may be a position conversion relationship between the pseudo position and the calibrated target model.

[0101] After obtaining the calibrated target model, the pseudo position to which the target model is moved when the user operates the active device can be simulated, and then a third position conversion relationship between the pseudo position and the position of the calibrated target model can be determined.

[0102] In practical applications, the model position of the calibration target model in the global coordinate system can be determined, as well as the position of the pseudo position in the global coordinate system. Then, the conversion relationship between the pseudo position and the model position of the target model can be determined to obtain a third position conversion relationship.

[0103] Specifically, the third position conversion relationship can be expressed by the following formula:

[0104] The pseudo position of the target model may be represented as 31 , and the model position of the calibrated target model may be represented as 32 .

[0105] Step 204 : determining a position mapping relationship between the active device and the slave device according to the third position conversion relationship, so as to control the master-slave device control system to perform a control operation based on the position mapping relationship.

[0106] After the third position conversion relationship is obtained, a position mapping relationship between the device position of the active device and the device position of the slave device may be determined according to the third position conversion relationship.

[0107] It needs to be understood that in order to obtain information about the target object, the user can control the active device to move the target model so that the target model reaches a position that is convenient for observation or information acquisition. In order to be able to perform operations on the target object based on the information of the target model, the target object needs to be moved so that it can reach a position corresponding to the target model. This can be done by establishing a position mapping relationship between the active device and the slave device, and then, based on the position mapping relationship, by controlling the movement of the active device to link the movement of the slave device, so that the target object connected to the slave device moves to a position corresponding to the target model, thereby being able to accurately perform operations on the target object based on the information of the target model.

[0108] For example, in a pelvic surgery scenario, the user can be the doctor performing the surgery, and the target object can be the pelvis that needs surgery. For example, if the patient has a damaged pelvis, the target model can be a pelvic model used to describe the damaged pelvis. The doctor needs to obtain information about the damaged pelvis in order to perform surgery on the damaged pelvis. The information of the damaged pelvis can be obtained through the target model and used to perform surgery. The target object on the slave device can be moved through the position mapping relationship between the active device and the slave device, that is, the damaged pelvis can be moved to a position that is convenient for observation or information acquisition. In addition, the damaged pelvis can also be moved to a position that is convenient for performing surgery.

[0109] In practical applications, in order to establish the third position conversion relationship, a pseudo position can be obtained by simulating the user moving the active device.

[0110] In an embodiment of the present application, step 204 may include steps 2041 and 2042:

[0111] Step 2041: Determine an eighth position conversion relationship between the target object and the slave device.

[0112] After the third position conversion relationship is obtained, an eighth position conversion relationship between the target object and the slave device may be determined.

[0113] Specifically, the eighth position conversion relationship can be expressed by the following formula:

[0114] Among them, the target object can be represented as 4, and the slave device can be represented as 2.

[0115] In one embodiment of the present application, the master-slave device control system may further include a reference module, which may include a first reference module and a second reference module. The first reference module may be used to assist in determining the tenth position conversion relationship, and the second reference module may be used to assist in determining the eleventh position conversion relationship. Step 2041 may be implemented in the following manner:

[0116] After obtaining the third position conversion relationship, the ninth position conversion relationship between the target object and the first reference module, the tenth position conversion relationship between the first reference module and the second reference module, and the eleventh position conversion relationship between the second reference module and the slave device can be determined.

[0117] In practical applications, the eleventh position conversion relationship can be determined by hand-eye calibration, the tenth position conversion relationship can be determined by an optical positioning system, and the ninth position conversion relationship can be obtained by performing a three-dimensional scan of the target object and the first reference module.

[0118] It should be understood that the above determination of the position conversion relationship is for illustration only. In specific implementation, the determination of the position conversion relationship may depend on the actual situation, that is, whether the operating conditions of the methods for determining the position conversion relationship such as hand-eye calibration, optical positioning system, three-dimensional scanning, and structured light camera are met according to the conditions of the current environment, and the one with better effect is selected from all methods that meet the operating conditions. This embodiment does not limit this.

[0119] Specifically, the ninth position conversion relationship can be expressed by the following formula:

[0120] The first reference module may be represented as 81 .

[0121] Specifically, the tenth position conversion relationship can be expressed by the following formula:

[0122] The second reference module may be represented as 82 .

[0123] Specifically, the eleventh position conversion relationship can be expressed by the following formula:

[0124] After obtaining the ninth, tenth and eleventh position conversion relationships, the eighth position conversion relationship between the target object and the driven device may be determined based on the ninth, tenth and eleventh position conversion relationships.

[0125] Specifically, the eighth position conversion relationship can be determined by the following formula:

[0126] Step 2042: Determine a position mapping relationship between the active device and the passive device according to the third position conversion relationship and the eighth position conversion relationship.

[0127] After the eighth position conversion relationship is obtained, the position mapping relationship between the active device and the slave device may be determined according to the third position conversion relationship and the eighth position conversion relationship.

[0128] Specifically, the position mapping relationship can be determined by the following formula:

[0129] Among them, the virtual position corresponding to the slave device can be expressed as 21, Can be T B , Can be T B -1 , the virtual position corresponding to the target object can be expressed as 41.

[0130] It should be understood that in order to determine the position mapping relationship, it is necessary to simulate the user moving the target model to the pseudo position when operating the active device. Since the target model and the target object have a linkage relationship, that is, the target object on the slave device can be linked by moving the target model, when the user moves the target model to the pseudo position when operating the active device, the target object should also be linked to move to the virtual position corresponding to the pseudo position, that is, 21.

[0131] In one embodiment of the present application, the method further includes the following steps:

[0132] The original posture of the active device is obtained, and the terminal posture of the active device after performing the moving operation in response to the moving instruction is obtained. Based on the original posture, the terminal posture and the position mapping relationship, a control instruction for the slave device is generated, and the slave device is controlled to move to the target posture corresponding to the control instruction according to the control instruction.

[0133] Among them, the original posture can be the posture of the active device when the master-slave device control system does not perform any operation, the end posture can be the posture of the target position reached by the master-slave device control system after moving the active device in response to the user's movement instruction, the movement instruction can be generated by the user based on the movement requirements, and the control instruction can be used to control the movement of the slave device.

[0134] After obtaining the position mapping relationship, the master-slave device control system can obtain the original position of the active device in real time. At the same time, the master-slave device control system can respond to the user's instructions to perform operations based on the position mapping relationship, so that the user can generate movement instructions for moving the target object to the target position based on the information acquisition requirements and / or movement requirements of the target object.

[0135] In actual applications, users can input information acquisition requirements and / or movement requirements based on the target object into a control device that is communicatively connected to the active device, and then the control device can generate movement instructions for moving the target object to the target position. Specifically, the control device can be set in a master-slave device control system, or it can be a cloud platform that is communicatively connected to the master-slave device control system.

[0136] After receiving the movement instruction, the master-slave device control system can feed back the movement instruction to the active device. The active device can respond to the movement instruction and move to the end position corresponding to the movement instruction, and then determine the end position when it is at the end position, and feed back the end position to the master-slave device control system.

[0137] As an example, the user may also directly operate the active device to move it to a location the user wants to reach.

[0138] In practical applications, a program coordinate system can be built into the master-slave device control system, and the position conversion relationship between the program coordinate system and the mechanical coordinate system can be determined by methods such as hand-eye calibration. The mechanical coordinate system can be a global coordinate system, so that the original position and end position of the active device can be determined based on the position of the active device in the global coordinate system and the position conversion relationship between the program coordinate system and the mechanical coordinate system.

[0139] Specifically, the original position and endpoint position of the active device can be directly obtained through methods such as an optical positioning system or a structured light camera.

[0140] After receiving the final posture feedback from the active device, the master-slave device control system can generate control instructions for the slave device based on the original posture, final posture and position mapping relationship.

[0141] In a specific implementation, the master-slave device control system can determine the displacement of the active device from the original posture to the end posture based on the original posture and the end posture, and determine the expected displacement of the slave device based on the displacement and the position mapping relationship, and then generate a control instruction for the slave device based on the expected displacement of the slave device.

[0142] After receiving the control instruction for the slave device, the master-slave device control system can send the control instruction to the slave device, and then the slave device can respond to the control instruction and move to the target position corresponding to the control instruction, and the posture of the slave device when it is at the target position corresponds to the target posture of the control instruction.

[0143] In an embodiment of the present application, a second position conversion relationship is obtained through a master-slave device control system, and a first position conversion relationship is determined based on the second position conversion relationship. The model position of the target model is updated based on the first position conversion relationship to obtain an updated calibration target model. Then, information of the target object on a three-dimensional level can be obtained by obtaining information of the calibration target model, thereby reducing the situation where the obtained information does not adequately describe the patient. In addition, a pseudo position corresponding to the calibration target model is determined, and a third position conversion relationship is determined based on the pseudo position. Based on the third position conversion relationship, a position mapping relationship between the active device and the slave device is determined, so as to control the master-slave device control system to perform control operations based on the position mapping relationship, thereby controlling the movement of the target object connected to the slave device to perform surgery on the target object, thereby improving the reliability of surgery based on the information described by the target model, and the implementation cost is low.

[0144] 4 , which shows a flowchart of another method for controlling a master-slave device control system according to an embodiment of the present application, which may include the following steps:

[0145] Step 401: The master-slave device control system obtains a second position conversion relationship, and determines a first position conversion relationship according to the second position conversion relationship.

[0146] For the relevant description of step 401, please refer to step 201 and will not be repeated.

[0147] Step 402: Determine a second expected position corresponding to the active device according to the first position conversion relationship.

[0148] The second expected position may be a position where the active device should be located.

[0149] After obtaining the first position conversion relationship, since the target model is connected to the active device, the second expected position where the active device should be located can be determined based on the first expected position.

[0150] In practical applications, the coordinate system of the reference model can be used as the global coordinate system, and the first expected position can be determined in the global coordinate system. The second expected position can be determined based on the positional relationship between the target model and the active device and the first expected position.

[0151] Specifically, the second expected position can be determined by the following formula:

[0152] Among them, the second expected position of the active device can be expressed as 1 * , the reference model can be expressed as 6, and It can be expressed as a first position conversion relationship.

[0153] In one embodiment of the present application, the master-slave device control system may further include a calibration module, and step 402 may include steps 4021 to 4022:

[0154] Step 4021 : Determine a fourth position conversion relationship between the calibration module and the active device, and determine a fifth position conversion relationship between the target model and the calibration module.

[0155] After obtaining the first position conversion relationship, the fourth position conversion relationship between the calibration module and the active device can be determined. Specifically, the fourth position conversion relationship can be expressed by the following formula:

[0156] Among them, the calibration module can be represented as 7.

[0157] At the same time, the fifth position conversion relationship between the target model and the calibration module can also be determined. Specifically, the fifth position conversion relationship can be expressed by the following formula:

[0158] Step 4022: Determine a second expected position corresponding to the active device according to the first position conversion relationship, the fourth position conversion relationship, and the fifth position conversion relationship.

[0159] After obtaining the fourth position conversion relationship and the fifth position conversion relationship, a seventh position conversion relationship between the second desired position and the reference model may be derived based on the first position conversion relationship, the fourth position conversion relationship, and the fifth position conversion relationship.

[0160] Specifically, the second expected position can also be determined by the following formula:

[0161] Step 403: Update the model position of the target model based on the second expected position to obtain an updated calibrated target model.

[0162] After the second expected position is obtained, the model position of the target model may be updated based on the second expected position to obtain an updated calibrated target model.

[0163] In practical applications, the position conversion relationship between the second desired position and the device position of the active device can be determined, and based on the position conversion relationship, a conversion method for moving the active device to the second desired position can be determined, such as how many units of displacement in the horizontal direction of the global coordinate system toward the positive direction of the x-axis, etc. Then, the active device can be moved based on the determined conversion method to obtain the moved active device, and the model position of the target model in the moved active device can be updated.

[0164] Specifically, the second desired position DR * The position conversion relationship between the device position of the active device DR and the device position can be determined by the following formula:

[0165] in, The position conversion relationship between the reference model and the device position of the active device can be expressed as a position conversion relationship between the reference model and the device position of the active device, and the position conversion relationship between the reference model and the device position of the active device can be determined by using a structured light camera or the like.

[0166] Step 404: Determine a pseudo position corresponding to the calibration target model.

[0167] For the relevant description of step 404, please refer to step 203 and will not be repeated here.

[0168] Step 405 : Determine a seventh position conversion relationship between the active device and the target model based on the fourth position conversion relationship and the fifth position conversion relationship.

[0169] After obtaining the pseudo position, the seventh position conversion relationship between the active device and the target model can be determined based on the fourth position conversion relationship and the fifth position conversion relationship. Specifically, the seventh position conversion relationship can be determined by the following formula:

[0170] Step 406: Determine a third position conversion relationship based on the pseudo position and the seventh position conversion relationship.

[0171] After obtaining the seventh position conversion relationship, the third position conversion relationship can be determined according to the pseudo position and the seventh position conversion relationship. Specifically, the third position conversion relationship can also be determined by the following formula:

[0172] Among them, the virtual position corresponding to the active device can be expressed as 11, TA It can be expressed as the seventh position conversion relationship, that is, T A Expressed as The rest of the description can refer to the description of the above embodiment.

[0173] Step 407 : determining a position mapping relationship between the active device and the slave device according to the third position conversion relationship, so as to control the master-slave device control system to perform a control operation based on the position mapping relationship.

[0174] For the relevant description of step 407, please refer to step 204 and will not be repeated.

[0175] In an embodiment of the present application, a second position conversion relationship is obtained by a master-slave device control system, and a first position conversion relationship is determined based on the second position conversion relationship. A second desired position corresponding to the active device is determined based on the first position conversion relationship, and a fourth position conversion relationship between the second desired position and the device position of the active device is determined. The model position of the target model is updated based on the fourth position conversion relationship to obtain an updated calibrated target model. Thus, information about the target object on a three-dimensional level can be obtained by obtaining information of the calibrated target model, thereby reducing the situation where the obtained information does not adequately describe the patient. Furthermore, a pseudo position corresponding to the calibrated target model is determined, and an eighth position conversion relationship between the active device and the target model is determined based on the fifth position conversion relationship and the sixth position conversion relationship. A third position conversion relationship is determined based on the pseudo position and the eighth position conversion relationship. Based on the third position conversion relationship, a position mapping relationship between the active device and the slave device is determined. The master-slave device control system is controlled to perform a control operation based on the position mapping relationship, thereby controlling the movement of the target object connected to the slave device to perform surgery on the target object. This improves the reliability of surgery based on the information described by the target model and reduces implementation costs.

[0176] 5 , which shows a schematic diagram of the structure of a control device for a master-slave device control system according to an embodiment of the present application, wherein the control device for the master-slave device control system is applied to a master-slave device control system, which includes an active device and a slave device. The active device is connected to a target model and may specifically include the following modules:

[0177] The acquisition module 501 is configured to acquire a second position conversion relationship and determine a first position conversion relationship based on the second position conversion relationship, where the first position conversion relationship is a position conversion relationship between a first desired position of the target model and the reference model.

[0178] The updating module 502 is configured to update the model position of the target model according to the first position conversion relationship to obtain an updated calibrated target model.

[0179] The determination module 503 is configured to determine a pseudo position corresponding to the calibration target model, and determine a third position conversion relationship based on the pseudo position, where the third position conversion relationship is a position conversion relationship between the pseudo position and the calibration target model.

[0180] The mapping module 504 is configured to determine a position mapping relationship between the active device and the passive device according to the third position conversion relationship, so as to control the active and passive device control systems to perform control operations based on the position mapping relationship.

[0181] In one implementation, the updating module 502 may be used to:

[0182] Determining a second expected position corresponding to the active device according to the first position conversion relationship;

[0183] determining a fourth position conversion relationship between the second desired position and the device position of the active device;

[0184] The model position of the target model is updated based on the fourth position conversion relationship to obtain an updated calibrated target model.

[0185] In one implementation, the master-slave device control system further includes a calibration module, and the update module 502 can be used to:

[0186] Determining a fifth position conversion relationship between the calibration module and the active device, and determining a sixth position conversion relationship between the target model and the calibration module;

[0187] Determining a seventh position conversion relationship between the second desired position and the reference model based on the first position conversion relationship, the fifth position conversion relationship, and the sixth position conversion relationship;

[0188] A fourth position conversion relationship between the second desired position and the active device is determined according to the seventh position conversion relationship.

[0189] In one implementation, the determining module 503 may be configured to:

[0190] determining an eighth position conversion relationship between the active device and the target model based on the fifth position conversion relationship and the sixth position conversion relationship;

[0191] The third position conversion relationship is determined according to the pseudo position and the eighth position conversion relationship.

[0192] In one implementation, the target model is used to describe the target object, and the mapping module 504 can be used to:

[0193] determining a ninth position conversion relationship between the target object and the driven device;

[0194] A position mapping relationship between the active device and the passive device is determined according to the third position conversion relationship and the ninth position conversion relationship.

[0195] In one implementation, the master-slave device control system further includes a reference module, which includes a first reference module and a second reference module. The mapping module 504 can be used to:

[0196] determining a tenth position conversion relationship between the target object and the first reference module, determining an eleventh position conversion relationship between the first reference module and the second reference module, and determining a twelfth position conversion relationship between the second reference module and the slave device;

[0197] A ninth position conversion relationship between the target object and the driven device is determined according to the tenth position conversion relationship, the eleventh position conversion relationship, and the twelfth position conversion relationship.

[0198] The acquisition module 501 is further configured to acquire the original posture of the active device and the final posture of the active device after executing the movement operation in response to the movement instruction.

[0199] In one implementation, the method further includes:

[0200] The generation module is used to generate control instructions for the driven device based on the original posture, the end posture and the position mapping relationship.

[0201] The control module is used to control the slave device to move to the target posture corresponding to the control instruction according to the control instruction.

[0202] In an embodiment of the present application, a second position conversion relationship is obtained through a master-slave device control system, and a first position conversion relationship is determined based on the second position conversion relationship. The model position of the target model is updated based on the first position conversion relationship to obtain an updated calibration target model. Then, information of the target object on a three-dimensional level can be obtained by obtaining information of the calibration target model, thereby reducing the situation where the obtained information does not adequately describe the patient. In addition, a pseudo position corresponding to the calibration target model is determined, and a third position conversion relationship is determined based on the pseudo position. Based on the third position conversion relationship, a position mapping relationship between the active device and the slave device is determined, so as to control the master-slave device control system to perform control operations based on the position mapping relationship, thereby controlling the movement of the target object connected to the slave device to perform surgery on the target object, thereby improving the reliability of surgery based on the information described by the target model, and the implementation cost is low.

[0203] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices are based on the same concept as the method embodiment of this application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0204] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0205] Referring to Figure 6, Figure 6 shows a structural block diagram of a terminal device provided by an embodiment of the present application. The terminal device 61 includes: at least one processor 611, a memory 612, and a computer program 6121 stored in the memory 612 and executable on the at least one processor 611. When the processor 611 executes the computer program 6121, the steps in any of the above-mentioned method embodiments are implemented.

[0206] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of any one of the above method embodiments can be implemented.

[0207] An embodiment of the present application provides a computer program product. When the computer program product is run on a mobile terminal, the mobile terminal can implement the steps of the above-mentioned various method embodiments when executing the computer program product.

[0208] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, it can implement the steps of the above-mentioned various method embodiments. Among them, the computer program includes computer program code, which can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device capable of carrying the computer program code to the camera / terminal device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal and a software distribution medium.

[0209] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the scope of protection of the present application.

Claims

1. A master-slave device control system, characterized in that: The master-slave device control system includes an active device and a slave device, wherein the active device is used to respond to a movement instruction and perform a movement operation, and the slave device is used to respond to the movement operation and perform a control operation corresponding to the movement operation; The active device is connected to a target model, the target model is used to describe the target object, the position conversion relationship between the target model and a reference model is a first position conversion relationship, and the reference model is used to describe other objects except the target object; The slave device is connected to the target object, and the position conversion relationship between the target object and the other object is a second position conversion relationship; The first position conversion relationship corresponds to the second position conversion relationship.

2. The master-slave device control system according to claim 1, wherein: The master-slave device control system further includes a calibration module; The calibration module is used to calibrate the position conversion relationship between the active device, the target model and the reference model; Wherein, the calibration module is arranged on the active device.

3. The master-slave device control system according to claim 1, wherein: The master-slave device control system further includes a reference module, wherein the reference module is used to determine a position conversion relationship between the slave device and the reference model; The reference module includes a first reference module and a second reference module. The first reference module is set at the slave device, and the second reference module is set at the target object.

4. A control method for a master-slave device control system, characterized in that: The method is applied to a master-slave device control system, the master-slave device control system includes an active device and a slave device, the active device is connected to a target model, and the method includes: The master-slave device control system acquires a second position conversion relationship, and determines a first position conversion relationship based on the second position conversion relationship, where the first position conversion relationship is a position conversion relationship between a first desired position of the target model and a reference model; updating the model position of the target model according to the first position conversion relationship to obtain an updated calibrated target model; Determining a pseudo position corresponding to the calibration target model, and determining a third position conversion relationship based on the pseudo position, wherein the third position conversion relationship is a position conversion relationship between the pseudo position and the calibration target model; According to the third position conversion relationship, a position mapping relationship between the active device and the slave device is determined, so as to control the master-slave device control system to perform a control operation based on the position mapping relationship.

5. The control method of the master-slave device control system according to claim 4, characterized in that: Updating the model position of the target model according to the first position conversion relationship to obtain an updated calibrated target model includes: determining a second expected position corresponding to the active device according to the first position conversion relationship; The model position of the target model is updated based on the second expected position to obtain an updated calibrated target model.

6. The control method of the master-slave device control system according to claim 5, characterized in that: The master-slave device control system further includes a calibration module, and determining the second expected position corresponding to the active device according to the first position conversion relationship includes: determining a fourth position conversion relationship between the calibration module and the active device, and determining a fifth position conversion relationship between the target model and the calibration module; A second expected position corresponding to the active device is determined according to the first position conversion relationship, the fourth position conversion relationship, and the fifth position conversion relationship.

7. The control method of the master-slave device control system according to claim 6, characterized in that: The determining of a third position conversion relationship according to the pseudo position includes: determining a seventh position conversion relationship between the active device and the target model according to the fourth position conversion relationship and the fifth position conversion relationship; A third position conversion relationship is determined according to the pseudo position and the seventh position conversion relationship.

8. The control method of the master-slave device control system according to any one of claims 4 to 7, characterized in that: The target model is used to describe a target object, and determining the position mapping relationship between the active device and the slave device according to the third position conversion relationship includes: determining an eighth position conversion relationship between the target object and the driven device; A position mapping relationship between the active device and the passive device is determined according to the third position conversion relationship and the eighth position conversion relationship.

9. The control method of the master-slave device control system according to claim 8, characterized in that: The master-slave device control system further includes a reference module, which includes a first reference module and a second reference module. Determining the eighth position conversion relationship between the target object and the slave device includes: determining a ninth position conversion relationship between the target object and the first reference module, determining a tenth position conversion relationship between the first reference module and the second reference module, and determining an eleventh position conversion relationship between the second reference module and the slave device; An eighth position conversion relationship between the target object and the driven device is determined according to the ninth position conversion relationship, the tenth position conversion relationship, and the eleventh position conversion relationship.

10. The control method of the master-slave device control system according to claim 4, wherein: The method further comprises: Acquire an original position of the active device, and acquire an end position of the active device after performing a move operation in response to a move instruction; generating a control instruction for the driven device based on the original posture, the terminal posture, and the position mapping relationship; The slave device is controlled to move to a target posture corresponding to the control instruction according to the control instruction.

Images