Control method, system and equipment of surgical robot and medium

Abstract

The invention discloses a control method, system and equipment of a surgical robot and a medium. The control method comprises the following steps: acquiring an image comprising a sleeve and depth information of the sleeve; identifying the sleeve in the image, and obtaining feature points of the sleeve according to an identification result; obtaining target depth information corresponding to the feature points; based on the feature points and the target depth information, obtaining positioning information of the sleeve; and controlling the operation end of the surgical robot to move based on the positioning information. The feature point and the depth information of the cannula are obtained by recognizing the cannula in the scene where the surgical robot is located, the cannula is obtained based on the feature point and the depth information, the position information of the cannula is accurately obtained, automatic movement of a mechanical arm of the robot based on the cannula position information is achieved, manual dragging operation is replaced, and the movement time of the surgical robot is shortened.

Description

Technical Field

[0001] The present invention relates to the field of robot control technology, and in particular to a control method, system, equipment and medium for a surgical robot. Background Art

[0002] In the current medical field, surgical robots have been widely used due to their precision and efficiency. During surgery, doctors need to insert several cannulas into the patient's body. Common abdominal surgery robots do not have the function of automatic positioning, and multiple robotic arms of the robot need to be manually dragged to connect with multiple cannulas to carry out surgery. Since the robot cannot accurately know the spatial position of the cannulas, it needs to be manually dragged to the vicinity of the cannulas to complete the docking. Manual operation is difficult, time-consuming and laborious. Summary of the invention

[0003] The technical problem to be solved by the present invention is to overcome the defect in the prior art that the robot cannot determine the spatial position of the cannula, resulting in the inability of the surgical robot to move automatically, and to provide a control method, system, device and medium for a surgical robot.

[0004] The present invention solves the above technical problems through the following technical solutions:

[0005] A first aspect provides a control method for a surgical robot, the control method comprising:

[0006] Acquire an image including the casing and depth information of the casing;

[0007] Identify the sleeve in the image, and obtain feature points of the sleeve according to the identification result;

[0008] Obtaining target depth information corresponding to the feature point;

[0009] Obtaining positioning information of the casing based on the feature points and the target depth information;

[0010] The operating end of the surgical robot is controlled to move based on the positioning information.

[0011] Preferably, the step of identifying the sleeve in the image and obtaining the characteristic points of the sleeve according to the identification result comprises:

[0012] Inputting the image into a pre-trained cannula recognition model and outputting a mask of the cannula;

[0013] The characteristic points of the sleeve are obtained based on the mask recognition.

[0014] Preferably, the step of obtaining target depth information corresponding to the feature point comprises:

[0015] The region of the sleeve in the image is used as the foreground, and the other regions are used as the background;

[0016] Acquire first depth information of foreground pixels and second depth information of background pixels within a preset range centered at the feature point;

[0017] The target depth information corresponding to the feature point is calculated according to the first depth information and the second depth information.

[0018] Preferably, the step of obtaining the positioning information of the casing based on the feature points and the target depth information comprises:

[0019] Obtaining the geometric center of gravity of the casing;

[0020] In response to the feature point meeting a preset condition, taking the feature point as a target feature point;

[0021] The preset conditions include that the distance between the feature point and the geometric center of gravity meets the first distance threshold requirement, and the target depth information meets the depth threshold requirement;

[0022] The positioning information of the casing is obtained based on the target feature points and the corresponding target depth information.

[0023] Preferably, the control method further includes:

[0024] In response to the feature point not meeting the preset condition, the positioning information of the casing is obtained based on the geometric center of gravity and the depth information corresponding to the geometric center of gravity.

[0025] Preferably, after identifying the cannula in the image, the method further includes:

[0026] identifying a target object in the image;

[0027] Wherein, the sleeve is arranged on the target object;

[0028] Obtaining a contact area between the target object and the sleeve based on the feature points;

[0029] The positioning information of the sleeve is obtained based on the feature points and the contact area.

[0030] Preferably, the step of obtaining the positioning information of the casing based on the feature points and the contact area comprises:

[0031] In response to the distance between the feature point and the contact area meeting a second distance threshold requirement, taking the position information of any feature point meeting the requirement as the positioning information of the casing;

[0032] Alternatively, the feature point with the smallest distance from the contact area is selected, and the position information of the feature point is used as the positioning information of the sleeve.

[0033] Preferably, the step of controlling the operating end to move based on the positioning information includes:

[0034] Converting the positioning information of the sleeve from a pixel coordinate system to a robot coordinate system;

[0035] The operating end is controlled to move based on the robot coordinate system.

[0036] Preferably, the number of the operating ends is greater than or equal to the number of the sleeves, and the control method further comprises:

[0037] In response to the number of the sleeves identified in the image being no less than two, acquiring the positioning information corresponding to each sleeve;

[0038] The operating ends having the same number as the sleeves are controlled to match the positioning information of the corresponding sleeves, and each of the operating ends moves based on the positioning information of the corresponding sleeve.

[0039] Preferably, the step of controlling the operating ends having the same number as the sleeves to match the positioning information of the corresponding sleeves comprises:

[0040] Acquire the moving path of each of the sleeves and each of the operating ends;

[0041] Selecting the moving path that does not intersect with other moving paths as the target path;

[0042] A corresponding number of the target paths are selected based on the number of the sleeves, and the operating ends corresponding to the target paths are controlled to move.

[0043] A second aspect provides a control system for a surgical robot, the control system comprising an image acquisition module, a recognition module, a positioning module and a control module;

[0044] The image acquisition module is used to acquire an image including the casing and depth information of the casing;

[0045] The recognition module is used to recognize the casing in the image and obtain the characteristic points of the casing according to the recognition result;

[0046] The positioning module is used to obtain the target depth information corresponding to the feature point; and is also used to obtain the positioning information of the casing based on the feature point and the target depth information;

[0047] The control module is used to control the operating end of the surgical robot to move based on the positioning information.

[0048] Preferably, the recognition module includes a mask output unit and a feature point recognition unit;

[0049] The mask output unit is used to input the image into a pre-trained casing recognition model and output a mask of the casing;

[0050] The feature point recognition unit is used to obtain the feature points of the sleeve based on the mask recognition.

[0051] Preferably, the positioning module is also used to take the area of ​​the sleeve in the image as the foreground and other areas as the background; obtain the first depth information of the foreground pixels and the second depth information of the background pixels within a preset range centered on the feature point; and calculate the target depth information corresponding to the feature point based on the first depth information and the second depth information.

[0052] As an implementable manner, the positioning module is further used to obtain the geometric center of gravity of the casing; in response to the feature point meeting a preset condition, the feature point is used as a target feature point;

[0053] The preset conditions include that the distance between the feature point and the geometric center of gravity meets the first distance threshold requirement, and the target depth information meets the depth threshold requirement;

[0054] The positioning module is also used to obtain the positioning information of the casing based on the target feature points and the corresponding target depth information.

[0055] Preferably, the positioning module is further used for obtaining the positioning information of the casing based on the geometric center of gravity and the depth information corresponding to the geometric center of gravity in response to the feature point not meeting the preset condition.

[0056] Preferably, the image acquisition module is also used to identify a target object in the image; wherein the sleeve is arranged on the target object;

[0057] The image acquisition module is also used to obtain the contact area between the target object and the sleeve based on the feature points;

[0058] The positioning module is also used to obtain positioning information of the sleeve based on the feature points and the contact area.

[0059] Preferably, the positioning module is further used for taking the position information of any of the feature points meeting the requirement as the positioning information of the casing in response to the distance between the feature point and the contact area meeting the second distance threshold requirement;

[0060] Alternatively, the feature point with the smallest distance from the contact area is selected, and the position information of the feature point is used as the positioning information of the sleeve.

[0061] Preferably, the control module includes a coordinate system conversion unit and a displacement control unit;

[0062] The coordinate system conversion unit is used to convert the positioning information of the sleeve from the pixel coordinate system to the robot coordinate system;

[0063] The displacement control unit is used to control the operation end to move based on the robot coordinate system.

[0064] Preferably, the number of the operating ends is greater than or equal to the number of the sleeves, and the displacement control unit is further used to obtain the positioning information corresponding to each sleeve in response to the number of the sleeves identified in the image being not less than two; control the operating ends having the same number as the sleeves to match the positioning information of the corresponding sleeves, and each operating end moves based on the positioning information of the corresponding sleeve.

[0065] Preferably, the control module further includes a path planning unit;

[0066] The path planning unit is used to obtain the moving path of each sleeve and each operating end; and select the moving path that does not intersect with other moving paths as the target path;

[0067] The displacement control unit is further used to select a corresponding number of the target paths based on the number of the sleeves, and control the operation ends corresponding to the target paths to move.

[0068] A third aspect provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and used to run on the processor, wherein the processor implements the control method of the surgical robot as described above when executing the computer program.

[0069] A fourth aspect provides a computer storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the control method of the surgical robot as described above.

[0070] The positive progressive effect of the present invention is that by identifying the cannula in the scene where the surgical robot is located, the characteristic points and depth information of the cannula are obtained, and the position information of the cannula is accurately obtained based on the characteristic points and depth information, so as to realize the automatic movement of the robot's mechanical arm based on the cannula position information, replace the manual dragging operation, and shorten the movement time of the surgical robot. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] Figure 1 This is a flow chart of a control method for a surgical robot according to Embodiment 1 of the present invention;

[0072] Figure 2 This is a schematic diagram of mask recognition according to Embodiment 1 of the present invention;

[0073] Figure 3 Schematic diagram of characteristic points of the casing in Example 1 of the present invention;

[0074] Figure 4 This is a flow chart of obtaining depth information of feature points in the control method of a surgical robot according to Embodiment 1 of the present invention;

[0075] Figure 5 This is a flow chart of obtaining positioning information of a cannula in the control method of a surgical robot according to Embodiment 1 of the present invention;

[0076] Figure 6 This is a schematic diagram of a scene of a sleeve and a target object according to Embodiment 1 of the present invention;

[0077] Figure 7 This is a module schematic diagram of a control system of a surgical robot according to Embodiment 2 of the present invention;

[0078] Figure 8 This is a schematic diagram of the hardware structure of an electronic device according to Embodiment 3 of the present invention. DETAILED DESCRIPTION

[0079] The present invention is further described below by way of examples, but the present invention is not limited to the scope of the examples.

[0080] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the referred device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present invention.

[0081] In addition, the terms "first", "second", and "third" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, a feature defined as "first", "second", and "third" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0082] Example 1

[0083] This embodiment provides a control method for a surgical robot. Figure 1 As shown, the control method includes:

[0084] S101, acquiring an image including a casing and depth information of the casing;

[0085] S102, identifying the casing in the image, and obtaining feature points of the casing according to the identification result;

[0086] S103, obtaining target depth information corresponding to the feature point;

[0087] S104, obtaining positioning information of the casing based on the feature points and the target depth information;

[0088] S105: Control the operating end of the surgical robot to move based on the positioning information.

[0089] In this solution, a camera and a depth sensor are set up to obtain the cannula image and the depth information of the cannula. Based on the recognition of the cannula and the corresponding depth information, the cannula positioning information can be directly obtained accurately and quickly. There is no need for three-dimensional data alignment of the cannula. It has better universality for different types of cannulas. The surgical robot is controlled to move automatically based on the cannula positioning information, and the operating end of the surgical robot can be deployed before the operation, which reduces the user's operating burden and improves the user experience.

[0090] As an implementable manner, step S102 includes:

[0091] Inputting the image into a pre-trained cannula recognition model and outputting a mask of the cannula;

[0092] The characteristic points of the sleeve are obtained based on the mask recognition.

[0093] In this scheme, if Figure 2 As shown, according to the user's usage scenario and usage requirements, the acquired picture can be an RGB (a color standard) image or a grayscale image. The band space of the image is not limited to visible light, and a false color image generated by infrared, ultraviolet and other bands can be used; preferably, the camera is a multi-eye camera or a structured light camera, and the depth sensor is a lidar; the pre-trained casing recognition model uses a mask extraction model including but not limited to the Mask R-CNN (Mask Region Convolutional Neural Network) model, and the mask corresponding to the casing obtained by the mask region convolutional neural network model output is realized. The casing in the image is recognized, and the feature points corresponding to the casing are obtained according to the results of mask recognition; in one embodiment, as Figure 3As shown, the edge of the casing is extracted by the feature point through the Canny (an edge detection operator) operator and other algorithms, and the edge is sampled at equal intervals to obtain the feature point 1 and its point coordinates. It should be noted that the feature point can be the geometric center of the image or other points that characterize the characteristics of the casing, which can quickly realize the preliminary positioning of the casing and has a high recognition accuracy.

[0094] As a feasible approach, Figure 4 As shown, step S103 includes:

[0095] S1031, taking the region of the sleeve in the image as the foreground and other regions as the background;

[0096] S1032, obtaining first depth information of foreground pixels and second depth information of background pixels within a preset range centered at the feature point;

[0097] S1033: Calculate the target depth information corresponding to the feature point according to the first depth information and the second depth information.

[0098] In this solution, the image is divided into a foreground area and a background area based on the mask recognition result, and the depth information corresponding to the feature point is calculated according to the depth information of the foreground pixels and background pixels around the feature point. The calculation method includes but is not limited to calculating the mean, variance value or median of the first depth information and the second depth information, or setting weights for the first depth information and the second depth information respectively, and calculating the weighted average of the first depth information and the second depth information to improve the accuracy of the target depth information corresponding to the feature point.

[0099] As a feasible approach, Figure 5 As shown, step S104 includes:

[0100] S1041, obtaining the geometric center of gravity of the casing;

[0101] S1042, in response to the feature point meeting a preset condition, taking the feature point as a target feature point;

[0102] The preset conditions include that the distance between the feature point and the geometric center of gravity meets the first distance threshold requirement, and the target depth information meets the depth threshold requirement;

[0103] Among them, meeting the first distance threshold requirement and meeting the depth threshold requirement can be set to be greater than, less than or equal to according to actual usage.

[0104] S1043: Obtain the positioning information of the casing based on the target feature point and the corresponding target depth information.

[0105] In this scheme, in order to improve the positioning accuracy of the cannula, the fixed feature points of the cannula, such as the geometric center of gravity, are used to screen the feature points based on preset conditions, and the distance between each feature point on the cannula and the center point is calculated as the morphological feature. The Euclidean distance, Manhattan distance or Chebyshev distance can be optionally used to obtain high-precision feature points based on distance and depth screening as the positioning information of the moving target of the surgical robot.

[0106] As an achievable manner, the control method further includes:

[0107] In response to the feature point not meeting the preset condition, the positioning information of the casing is obtained based on the geometric center of gravity and the depth information corresponding to the geometric center of gravity.

[0108] In this solution, when the cannula mask recognition result is obtained, if the feature point cannot meet the requirements of the preset conditions, the fixed feature point of the cannula, such as the geometric center of gravity, is selected as the positioning reference of the cannula, and the positioning information of the cannula is obtained by combining the depth information of the geometric center of gravity. Preferably, the depth information of the geometric center of gravity is calculated based on the depth information of the pixel points within a preset range around the geometric center of gravity. Considering the morphological characteristics of the cannula, the geometric center of gravity of the cannula is above the slender rod of the cannula, and the position information of the geometric center of gravity is adjusted in combination with the depth value of the cannula inserted into the wound.

[0109] As an achievable method, in response to the feature point not meeting the preset condition, the user can actively select the identified feature point, obtain the depth information of the selected feature point, and obtain the positioning information of the casing. The active selection method includes but is not limited to dragging, clicking, double-clicking or circling through the touch screen, mouse or key.

[0110] As an achievable manner, after the sleeve in the image is identified, the following further comprises:

[0111] identifying a target object in the image;

[0112] Wherein, the sleeve is arranged on the target object;

[0113] Obtaining a contact area between the target object and the sleeve based on the feature points;

[0114] The positioning information of the sleeve is obtained based on the feature points and the contact area.

[0115] In this scheme, if Figure 6 As shown, the camera obtains images of the cannula and the patient on whom the cannula is set, and has higher stability and accuracy by identifying the position information of feature points in the contact area between the cannula and the patient. Feature points in or near the area are selected and combined with the depth information of the corresponding feature points to obtain high-precision cannula positioning information.

[0116] As an achievable manner, the step of obtaining the positioning information of the casing based on the feature points and the contact area includes:

[0117] In response to the distance between the feature point and the contact area meeting a second distance threshold requirement, taking the position information of any feature point meeting the requirement as the positioning information of the casing;

[0118] Alternatively, the feature point with the smallest distance from the contact area is selected, and the position information of the feature point is used as the positioning information of the sleeve.

[0119] In this solution, feature points near or closest to the contact area are selected based on a preset distance as the positioning points of the sleeve. In one embodiment, feature points on the contact edge between the sleeve and the target object can be selected as the positioning information of the sleeve to avoid the image of the positioning information caused by the position deviation of the feature points caused by the swing of the sleeve, thereby improving the accuracy of the sleeve positioning information.

[0120] As an implementable manner, step S105 includes:

[0121] Converting the positioning information of the sleeve from a pixel coordinate system to a robot coordinate system;

[0122] The operating end is controlled to move based on the robot coordinate system.

[0123] In this solution, the internal parameters of the camera and external parameters and matrices such as the depth sensor are combined to realize the conversion of the coordinate system of the cannula positioning information through Zhang Zhengyou's calibration method, so that the surgical robot can move to the position of the positioning information; the user can confirm or correct the conversion result of the coordinate system to further improve the accuracy of the subsequent movement of the surgical robot.

[0124] As an achievable manner, the number of the operating ends is greater than or equal to the number of the sleeves, and the control method further includes:

[0125] In response to the number of the sleeves identified in the image being no less than two, acquiring the positioning information corresponding to each sleeve;

[0126] The operating ends having the same number as the sleeves are controlled to match the positioning information of the corresponding sleeves, and each of the operating ends moves based on the positioning information of the corresponding sleeve.

[0127] In this solution, multiple cannulas are usually arranged on the patient's body during abdominal surgery. The positioning information of each cannula is identified and obtained respectively, and the corresponding number of operating ends of the robot are controlled to move toward the target positioning information, thereby realizing the automatic docking between the operating end of the surgical robot and the cannula.

[0128] As a feasible method, a recognizable mark is set on the cannula, and the mark is used to characterize the surgical instrument or endoscope matching the cannula. According to the recognition result of the cannula, the operating end of the surgical robot with the corresponding surgical instrument or endoscope is controlled to move toward the corresponding cannula based on the mark on the cannula; wherein the mark includes but is not limited to recognizable graphics such as QR codes, bar codes, etc., or the surface or edge of the cannula has a recognizable shape, structure, etc. Accurate pairing and automated movement of multiple cannulas and multiple operating ends are achieved.

[0129] As an achievable manner, the step of controlling the same number of the operating ends as the sleeves to match the positioning information of the corresponding sleeves includes:

[0130] Acquire the moving path of each of the sleeves and each of the operating ends;

[0131] Selecting the moving path that does not intersect with other moving paths as the target path;

[0132] A corresponding number of the target paths are selected based on the number of the sleeves, and the operating ends corresponding to the target paths are controlled to move.

[0133] In this solution, in order to avoid collisions between multiple operating ends of the surgical robot, non-intersecting moving paths are selected to achieve the accuracy of the automated movement of the operating end of the surgical robot and ensure the safety of the operation of the surgical robot.

[0134] The control method of the surgical robot provided in this embodiment obtains the feature points and depth information of the cannula by identifying the cannula in the scene where the surgical robot is located, and accurately obtains the position information of the cannula based on the feature points and the depth information, thereby realizing the automatic movement of the robot's mechanical arm based on the cannula position information, replacing the manual dragging operation, and shortening the movement time of the surgical robot.

[0135] Example 2

[0136] This embodiment provides a control system 200 of a surgical robot, such as Figure 7 As shown, the control system includes an image acquisition module 201, a recognition module 202, a positioning module 203 and a control module 204;

[0137] The image acquisition module 201 is used to acquire an image including the casing and depth information of the casing;

[0138] The recognition module 202 is used to recognize the casing in the image and obtain the characteristic points of the casing according to the recognition result;

[0139] The positioning module 203 is used to obtain the target depth information corresponding to the feature point; and is also used to obtain the positioning information of the casing based on the feature point and the target depth information;

[0140] The control module 204 is used to control the operating end of the surgical robot to move based on the positioning information.

[0141] As an implementable manner, the recognition module 202 includes a mask output unit and a feature point recognition unit;

[0142] The mask output unit is used to input the image into a pre-trained casing recognition model and output a mask of the casing;

[0143] The feature point recognition unit is used to obtain the feature points of the sleeve based on the mask recognition.

[0144] As an implementable method, the positioning module 203 is also used to take the area of ​​the sleeve in the image as the foreground and other areas as the background; obtain the first depth information of the foreground pixels and the second depth information of the background pixels within a preset range centered on the feature point; and calculate the target depth information corresponding to the feature point based on the first depth information and the second depth information.

[0145] As an implementable manner, the positioning module 203 is further used to obtain the geometric center of gravity of the casing; in response to the feature point meeting the preset condition, the feature point is used as the target feature point;

[0146] The preset conditions include that the distance between the feature point and the geometric center of gravity meets the first distance threshold requirement, and the target depth information meets the depth threshold requirement;

[0147] The positioning module 203 is further configured to obtain the positioning information of the casing based on the target feature points and the corresponding target depth information.

[0148] As an implementable manner, the positioning module 203 is further configured to obtain the positioning information of the casing based on the geometric center of gravity and the depth information corresponding to the geometric center of gravity in response to the feature point not meeting the preset condition.

[0149] As an implementable manner, the image acquisition module 201 is further used to identify a target object in the image; wherein the sleeve is disposed on the target object;

[0150] The image acquisition module 201 is also used to obtain the contact area between the target object and the cannula based on the feature points;

[0151] The positioning module 203 is further configured to obtain positioning information of the casing based on the feature points and the contact area.

[0152] As an implementable manner, the positioning module 203 is further configured to, in response to the distance between the feature point and the contact area meeting a second distance threshold requirement, use the position information of any feature point meeting the requirement as the positioning information of the casing;

[0153] Alternatively, the feature point with the smallest distance from the contact area is selected, and the position information of the feature point is used as the positioning information of the sleeve.

[0154] As an implementable manner, the control module 204 includes a coordinate system conversion unit and a displacement control unit;

[0155] The coordinate system conversion unit is used to convert the positioning information of the sleeve from the pixel coordinate system to the robot coordinate system;

[0156] The displacement control unit is used to control the operation end to move based on the robot coordinate system.

[0157] As an achievable manner, the number of the operating ends is greater than or equal to the number of the sleeves, and the displacement control unit is further used to obtain the positioning information corresponding to each sleeve in response to the number of the sleeves identified in the image being no less than two; control the operating ends having the same number as the sleeves to match the positioning information of the corresponding sleeves, and each operating end moves based on the positioning information of the corresponding sleeve.

[0158] As an implementable manner, the control module 204 further includes a path planning unit;

[0159] The path planning unit is used to obtain the moving path of each sleeve and each operating end; and select the moving path that does not intersect with other moving paths as the target path;

[0160] The displacement control unit is further used to select a corresponding number of the target paths based on the number of the sleeves, and control the operation ends corresponding to the target paths to move.

[0161] It should be noted that the working principle of the control system 200 of the surgical robot in this embodiment is the same as the working principle of the control method of the surgical robot in Example 1, so it will not be repeated here.

[0162] The control system of the surgical robot provided in this embodiment obtains the feature points and depth information of the cannula by identifying the cannula in the scene where the surgical robot is located, and accurately obtains the position information of the cannula based on the feature points and the depth information, thereby realizing the automatic movement of the robot's mechanical arm based on the cannula position information, replacing the manual dragging operation, and shortening the movement time of the surgical robot.

[0163] Example 3

[0164] like Figure 8 FIG. 3 is a schematic diagram of the structure of an electronic device provided in Embodiment 3 of the present invention, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the control method of the surgical robot in Embodiment 1 is implemented. Figure 8 The electronic device 30 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0165] The electronic device 30 may be in the form of a general-purpose computing device, for example, it may be a server device. The components of the electronic device 30 may include, but are not limited to: at least one processor 31, at least one memory 32, and a bus 33 connecting different system components (including the memory 32 and the processor 31).

[0166] The bus 33 includes a data bus, an address bus, and a control bus.

[0167] The memory 32 may include a volatile memory, such as a random access memory (RAM) 321 and / or a cache memory 322 , and may further include a read-only memory (ROM) 323 .

[0168] The memory 32 may also include a program / utility 325 having a set (at least one) of program modules 324, such program modules 324 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0169] The processor 31 executes various functional applications and data processing by running the computer program stored in the memory 32, such as the control method of the surgical robot in Example 1 of the present invention.

[0170] The electronic device 30 may also communicate with one or more external devices 34 (e.g., keyboards, pointing devices, etc.). Such communication may be performed via an input / output (I / O) interface 35. Furthermore, the model-generated device 30 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 36. As shown, the network adapter 36 communicates with other modules of the model-generated device 30 via a bus 33. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the model-generated device 30, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID (RAID) systems, tape drives, and data backup storage systems.

[0171] It should be noted that although several units / modules or sub-units / modules of the electronic device are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, according to an embodiment of the present invention, the features and functions of two or more units / modules described above can be embodied in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided into multiple units / modules to be embodied.

[0172] Example 4

[0173] The present invention also provides a computer-readable storage medium having a computer program stored thereon, and when the program is executed by a processor, the control method of the surgical robot of Example 1 is implemented.

[0174] The readable storage medium may include but is not limited to: a portable disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory, an optical storage device, a magnetic storage device or any suitable combination of the above.

[0175] In a possible implementation manner, the present invention may also be implemented in the form of a program product, which includes a program code. When the program product is run on a terminal device, the program code is used to enable the terminal device to execute the control method of the surgical robot of Example 1.

[0176] Among them, the program code for executing the present invention can be written in any combination of one or more programming languages, and the program code can be executed completely on the user device, partially on the user device, as an independent software package, partially on the user device and partially on a remote device, or completely on the remote device.

[0177] Although the specific embodiments of the present invention are described above, it should be understood by those skilled in the art that this is only for illustration and the protection scope of the present invention is defined by the appended claims. Those skilled in the art may make various changes or modifications to these embodiments without departing from the principles and essence of the present invention, but these changes and modifications all fall within the protection scope of the present invention.

Claims

1. A control method for a surgical robot, characterized in that: The control method comprises: Acquire an image including the casing and depth information of the casing; Identify the sleeve in the image, and obtain feature points of the sleeve according to the identification result; Obtaining target depth information corresponding to the feature point; Obtaining positioning information of the casing based on the feature points and the target depth information; The operating end of the surgical robot is controlled to move based on the positioning information.

2. The control method of the surgical robot according to claim 1, characterized in that: The step of identifying the sleeve in the image and obtaining the characteristic points of the sleeve according to the identification result comprises: Inputting the image into a pre-trained cannula recognition model and outputting a mask of the cannula; The characteristic points of the sleeve are obtained based on the mask recognition.

3. The control method of the surgical robot according to claim 2, characterized in that: The step of obtaining target depth information corresponding to the feature point comprises: The region of the sleeve in the image is used as the foreground, and the other regions are used as the background; Acquire first depth information of foreground pixels and second depth information of background pixels within a preset range centered at the feature point; The target depth information corresponding to the feature point is calculated according to the first depth information and the second depth information.

4. The control method of the surgical robot according to claim 3, characterized in that: The step of obtaining the positioning information of the casing based on the feature points and the target depth information comprises: Obtaining the geometric center of gravity of the casing; In response to the feature point meeting a preset condition, taking the feature point as a target feature point; The preset conditions include but are not limited to that the distance between the feature point and the geometric center of gravity meets a distance threshold requirement, and the target depth information meets a depth threshold requirement; The positioning information of the casing is obtained based on the target feature points and the corresponding target depth information.

5. The control method of the surgical robot according to claim 4, characterized in that: The control method further comprises: In response to the feature point not meeting the preset condition, the positioning information of the casing is obtained based on the geometric center of gravity and the depth information corresponding to the geometric center of gravity.

6. The control method of the surgical robot according to claim 1, characterized in that: After the sleeve in the image is identified, the method further includes: identifying a target object in the image; Wherein, the sleeve is arranged on the target object; Obtaining a contact area between the target object and the sleeve based on the feature points; The positioning information of the sleeve is obtained based on the feature points and the contact area.

7. The control method of the surgical robot according to claim 6, characterized in that: The step of obtaining the positioning information of the casing based on the feature points and the contact area comprises: In response to the distance between the feature point and the contact area satisfying a second distance threshold condition, taking the position information of any feature point that meets the requirement as the positioning information of the casing; Alternatively, the feature point with the smallest distance from the contact area is selected, and the position information of the feature point is used as the positioning information of the sleeve.

8. The control method of the surgical robot according to claim 1, characterized in that: The step of controlling the operating end to move based on the positioning information includes: Converting the positioning information of the sleeve from a pixel coordinate system to a robot coordinate system; The operating end is controlled to move based on the robot coordinate system.

9. The control method of the surgical robot according to claim 1, characterized in that: The number of the operating ends is greater than or equal to the number of the sleeves, and the control method further comprises: In response to the number of the sleeves identified in the image being no less than two, acquiring the positioning information corresponding to each sleeve; The operating ends having the same number as the sleeves are controlled to match the positioning information of the corresponding sleeves, and each of the operating ends moves based on the positioning information of the corresponding sleeve.

10. The control method of the surgical robot according to claim 9, characterized in that: The step of controlling the same number of the operating ends as the sleeves to match the positioning information of the corresponding sleeves comprises: Acquire the moving path of each of the sleeves and each of the operating ends; Selecting the moving path that does not intersect with other moving paths as the target path; A corresponding number of the target paths are selected based on the number of the sleeves, and the operating ends corresponding to the target paths are controlled to move.

11. A control system for a surgical robot, characterized in that: The control system includes an image acquisition module, a recognition module, a positioning module and a control module; The image acquisition module is used to acquire an image including the casing and depth information of the casing; The recognition module is used to recognize the casing in the image and obtain the characteristic points of the casing according to the recognition result; The positioning module is used to obtain the target depth information corresponding to the feature point; and is also used to obtain the positioning information of the casing based on the feature point and the target depth information; The control module is used to control the operating end of the surgical robot to move based on the positioning information.

12. An electronic device comprising a memory, a processor, and a computer program stored in the memory and used to run on the processor, characterized in that: When the processor executes the computer program, the control method of the surgical robot according to any one of claims 1 to 10 is implemented.

13. A computer storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the control method of the surgical robot according to any one of claims 1 to 10 is implemented.

Images