Surgical robot planning navigation method, device and equipment and storage medium

Abstract

The embodiment of the invention discloses a surgical robot planning navigation method, device and equipment and a storage medium, and the method comprises the steps: determining a three-dimensional model of a to-be-operated region of a target object, and determining a reference moving path based on a part interaction instruction for a target part in the three-dimensional model; acquiring a robot operation instruction, and controlling a target surgical robot to move based on the surgical robot operation instruction; and under the condition that the target surgical robot abnormally moves, correcting the moving path of the target surgical robot based on the reference moving path. According to the technical scheme, the safety problem that in the prior art, when the surgical robot is operated, other tissues are likely to be touched by mistake is solved, the reference moving path can be determined through preoperative planning on the basis of the interaction instruction, the abnormal action of the surgical robot is subsequently corrected on the basis of the reference moving path, and the operation efficiency is improved. Damage to surrounding tissues in an operation is reduced, and operation safety is improved.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of surgical robot control technology, and in particular, to a surgical robot planning and navigation method, device, equipment and storage medium. Background Art

[0002] Laparoscopic surgical robots are medical devices used to assist minimally invasive surgery in the abdominal cavity. They are usually remotely controlled in a master-slave mode and consist of a user console, a patient surgical platform, and a three-dimensional high-definition imaging system. Similar to traditional minimally invasive surgery, laparoscopic surgical robots insert endoscopes and surgical instruments into the affected area of ​​the human body through natural holes in the human body or through multiple minimally invasive incisions in the abdomen, pelvis, or chest. Users perform related procedures remotely in vitro through real-time images fed back by the endoscope. Compared with traditional minimally invasive surgery, laparoscopic surgical robots are safer and more precise. At the same time, the three-dimensional high-definition imaging system can provide a wider field of view, which greatly expands the surgical capabilities of surgical users.

[0003] Although laparoscopic surgical robots have provided a wider field of view and precise operational assistance, the intracavitary environment is still very complex for laparoscopic surgery. The three-dimensional spatial relationship between surgical instruments and organs cannot be directly observed at the user console. When the user performs the procedure, there are still safety issues such as instruments out of field of view and accidental contact with other tissues. Summary of the invention

[0004] The embodiments of the present invention provide a surgical robot planning and navigation method, device, equipment and storage medium, which can reduce damage to surrounding tissues during surgery and improve surgical safety.

[0005] In a first aspect, an embodiment of the present invention provides a surgical robot planning and navigation method, the method comprising:

[0006] Determine a three-dimensional model of the target object's area to be operated on, and determine a reference movement path based on part interaction instructions for the target part in the three-dimensional model; obtain robot operation instructions, and control the movement of the target surgical robot based on the surgical robot operation instructions; in the event of abnormal movement of the target surgical robot, correct the movement path of the target surgical robot based on the reference movement path.

[0007] In a second aspect, an embodiment of the present invention provides a surgical robot planning and navigation device, the device comprising:

[0008] A reference path determination module is used to determine a three-dimensional model of the target object's area to be operated on, and to determine a reference movement path based on part interaction instructions for the target part in the three-dimensional model; a movement control module is used to obtain robot operation instructions, and to control the movement of the target surgical robot based on the surgical robot operation instructions; and a movement path correction module is used to correct the movement path of the target surgical robot based on the reference movement path in the event of abnormal movement of the target surgical robot.

[0009] In a third aspect, an embodiment of the present invention provides a computer device, the computer device comprising:

[0010] one or more processors;

[0011] A memory for storing one or more programs;

[0012] When the one or more programs are executed by the one or more processors, the one or more processors implement the surgical robot planning and navigation method described in any embodiment.

[0013] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the surgical robot planning and navigation method described in any embodiment.

[0014] The technical solution provided by the embodiment of the present invention determines a three-dimensional model of the target object's surgical area, determines a reference movement path based on the part interaction instructions for the target part in the three-dimensional model; obtains robot operation instructions, and controls the target surgical robot to move based on the surgical robot operation instructions; and in the event of abnormal movement of the target surgical robot, corrects the movement path of the target surgical robot based on the reference movement path. The technical solution of the embodiment of the present invention solves the safety problem of the prior art that it is easy to accidentally touch other tissues when operating a surgical robot. It can determine the reference movement path based on the interactive instructions for preoperative planning, and subsequently correct the abnormal movements of the surgical robot based on the reference movement path, thereby reducing damage to surrounding tissues during surgery and improving surgical safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 This is a flow chart of a surgical robot planning and navigation method provided by an embodiment of the present invention;

[0016] Figure 2 This is a flow chart of another surgical robot planning and navigation method provided by an embodiment of the present invention;

[0017] Figure 3 This is an example diagram of a part within a surgical area provided by an embodiment of the present invention;

[0018] Figure 4 is an example diagram of a panoramic model provided by an embodiment of the present invention;

[0019] Figure 5 It is a structural schematic diagram of a surgical robot planning and navigation device provided by an embodiment of the present invention;

[0020] Figure 6 It is a structural schematic diagram of a computer device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0022] Figure 1 It is a flow chart of a surgical robot planning and navigation method provided by an embodiment of the present invention. The embodiment of the present invention can be applied to the scenario of controlling the movement of a laparoscopic surgical robot. The method can be executed by a surgical robot planning and navigation device, which can be implemented by software and / or hardware.

[0023] like Figure 1 As shown, the surgical robot planning and navigation method includes the following steps:

[0024] S110 , determining a three-dimensional model of the target object's area to be operated on, and determining a reference moving path based on a part interaction instruction for the target part in the three-dimensional model.

[0025] Among them, the target object may be an object that needs to be operated on. The area to be operated on may be an area where the target object needs to be operated on. Specifically, the area to be operated on may be manually specified. Exemplarily, the user may manually specify the lesion area that needs to be operated on as the area to be operated on. Exemplarily, the scanning data of the area to be operated on may be obtained, and then a three-dimensional model of the area to be operated on may be constructed based on the scanning data. Specifically, the three-dimensional model may include parts such as organs, blood vessels, and tissues. The target part may be a part that needs to be operated on. The part interaction instruction may be an interaction instruction for a part in the initial three-dimensional model. The reference movement path may be a reference path of the surgical robot during the surgical movement process. Exemplarily, the target part may be determined from the three-dimensional model based on the part interaction instruction, and then all the target parts may be connected to obtain the reference movement path.

[0026] S120, obtaining robot operation instructions, and controlling the target surgical robot to move based on the surgical robot operation instructions.

[0027] The robot operation instruction may be an instruction to control the movement of the surgical robot. For example, the user may issue the robot operation instruction through manual operation at the control end. The target surgical robot may be a surgical robot that needs to perform a surgical operation in the area to be operated. Further, the target surgical robot may be controlled to move in the area to be operated according to the robot operation instruction.

[0028] S130. When the target surgical robot moves abnormally, correct the moving path of the target surgical robot based on the reference moving path.

[0029] In the case where the moving path of the target surgical robot deviates greatly from the reference moving path, it can be determined that the target surgical robot has abnormal movement. Exemplarily, based on the deviation distance between the real-time moving path of the target surgical robot and the reference moving path, if the deviation distance exceeds a preset deviation threshold, it can be determined that the target surgical robot has abnormal movement.

[0030] Furthermore, when the target surgical robot moves abnormally, a corresponding correction method can be formulated based on the deviation between the reference moving path and the moving path of the target surgical robot, so as to reduce the risk of accidental damage to organs and tissues. Exemplary correction methods include but are not limited to applying resistance to the target surgical robot, sending a return command, and sending a stop movement command.

[0031] The technical solution provided by the embodiment of the present invention determines the three-dimensional model of the target object's surgical area, determines the reference movement path based on the part interaction instructions for the target part in the three-dimensional model; obtains the robot operation instructions, and controls the target surgical robot to move based on the surgical robot operation instructions; in the case of abnormal movement of the target surgical robot, the movement path of the target surgical robot is corrected based on the reference movement path. The technical solution of the embodiment of the present invention solves the safety problems in the prior art that when operating the surgical robot, it is easy to accidentally touch other tissues. The reference movement path can be determined based on the interactive instructions for preoperative planning, and the abnormal movements of the surgical robot can be corrected based on the reference movement path, thereby reducing damage to surrounding tissues during surgery and improving surgical safety.

[0032] Figure 2It is another flow chart of a surgical robot planning and navigation method provided by an embodiment of the present invention. The embodiment of the present invention can be applied to the scenario of controlling the movement of a laparoscopic surgical robot. Based on the above embodiment, this embodiment further explains how to determine a three-dimensional model of the target object's surgical area to be operated on, and determine a reference movement path based on part interaction instructions for the target part in the three-dimensional model; and how to correct the movement path of the target surgical robot based on the reference movement path when the target surgical robot has abnormal movement. The device can be implemented by software and / or hardware and integrated in a computer device with application development function.

[0033] like Figure 2 As shown, the surgical robot planning and navigation method includes the following steps:

[0034] S210, acquiring image scanning data of the area to be operated on of the target object, and determining an initial three-dimensional model according to the image scanning data.

[0035] Among them, the target object may be an object that needs to undergo surgery. The area to be operated on may be an area on the target object that needs to undergo surgery. Specifically, the area to be operated on may be manually specified. Exemplarily, the user may manually specify a lesion area that needs surgery as the area to be operated on. The image scanning data may be image data obtained by scanning the area to be operated on the target object. Exemplarily, the image scanning data may include CT image data and MRI image data. The initial three-dimensional image may be a three-dimensional model of the area to be operated on itself. Specifically, the CT image data and the MRI image data may be input into a pre-trained three-dimensional model generation network, so that the network outputs an initial three-dimensional model corresponding to the area to be operated on.

[0036] S220: Input the initial three-dimensional model into a target part segmentation model to determine a plurality of candidate parts in the initial three-dimensional model.

[0037] Among them, the target part segmentation model can be a pre-trained model for filling in the gaps. The target part segmentation model can divide the organs, blood vessels, tissues and other parts in the input initial three-dimensional model. Furthermore, the candidate parts may be parts that may need to be surgically operated on later. Specifically, the initial three-dimensional model can be input into the target part segmentation model so that the model divides the organs, blood vessels, tissues and other parts in the initial three-dimensional model, and each divided part is used as a candidate part. Exemplarily, 3D-Unet or other three-dimensional image semantic segmentation technologies can be used to extract models of organs, blood vessels, tissues and other parts related to the patient's surgical procedure, and present a list of segmented organs and tissues to the user. Exemplarily, Figure 3This is an example diagram of a part in a surgical area provided by an embodiment of the present invention. Figure 3 As shown, a round sphere represents a part.

[0038] S230. Determine at least one target part from the candidate parts based on the part interaction instruction, and determine the reference moving path according to the target part.

[0039] Among them, the part interaction instruction can be an interaction instruction about the candidate part in the initial three-dimensional model. Exemplarily, the user can issue the part interaction instruction by electric shock, dragging or manual input. The target part can be the part that needs surgery. Specifically, based on the part interaction instruction, the corresponding part can be selected from the candidate parts as the target part. Furthermore, the reference moving path can be a reference path of the surgical robot during the surgical movement. Exemplarily, each target part can be taken as a mobile node, and finally all the mobile nodes are connected to obtain the reference moving path. Optionally, the reference moving path can be three-dimensional. Among them, in the process of connecting the mobile nodes, it is necessary to ensure that the connected path will not touch other parts except the target part.

[0040] S240, obtaining robot operation instructions, and controlling the target surgical robot to move based on the surgical robot operation instructions.

[0041] The robot operation instruction may be an instruction to control the movement of the surgical robot. For example, the user may issue the robot operation instruction through manual operation at the control end. The target surgical robot may be a surgical robot that needs to perform a surgical operation in the area to be operated. Further, the target surgical robot may be controlled to move in the area to be operated according to the robot operation instruction.

[0042] S250, determining a reference moving space based on the reference moving path; sending a resistance feedback instruction to the target surgical robot when the operator of the target surgical robot approaches the edge of the reference moving space; sending a movement termination instruction to the target surgical robot when the operator of the target surgical robot exceeds the edge of the reference moving space.

[0043] Among them, the reference moving space can be a reference space in which the surgical robot can move. Exemplarily, a circular area with a preset radius value can be arranged around the reference moving path, and the obtained circular area is used as the reference moving space. Optionally, the reference moving space can also be determined by interactive instructions. For example, the user can select the corresponding organ tissue through input methods such as a touch screen or a mouse and keyboard, and plan the procedure on the patient's three-dimensional model, including but not limited to manually correcting the identification boundary of the organ model, setting special landmarks, and setting the operation space of the procedure and other preoperative planning activities.

[0044] The resistance feedback instruction may be a feedback message sent to the control end of the surgical robot. Specifically, the resistance feedback instruction may cause the control console of the control end to provide operation resistance, thereby reminding the operator to stop moving in the wrong direction. Optionally, the real-time position information of the operator of the target surgical robot may be obtained, and then the distance between the operator and the edge of the reference moving space may be determined based on the real-time position information. When the distance is less than a preset first distance threshold, resistance feedback information may be sent to the target surgical robot.

[0045] Furthermore, the movement termination instruction may be an instruction to control the target surgical robot to stop moving. Optionally, the distance between the target surgical robot and the edge of the reference movement space may be determined, and when the distance is less than a preset second threshold, a movement termination instruction may be sent to the target surgical robot to stop the robot from moving. The first distance threshold and the second distance threshold may be manually set, and the first distance threshold may be greater than the second distance threshold.

[0046] Optionally, when a navigation mode selection instruction is received, the target navigation mode can be determined according to the navigation mode selection instruction; data feedback information of the target surgical robot can be obtained, and the target three-dimensional model can be determined according to the data feedback information and the initial three-dimensional model; based on the target three-dimensional model and the target navigation mode, the movement feedback information of the target surgical robot can be determined, and the movement feedback information can be displayed on a preset display screen.

[0047] The navigation mode selection instruction may be an instruction for selecting a navigation mode. The target navigation mode may be a navigation mode finally determined according to the navigation mode selection instruction. Exemplarily, a plurality of navigation modes may be displayed on the preset interface, and the user may select the corresponding navigation mode as needed, and then issue a navigation mode selection instruction, and the device then uses the navigation mode corresponding to the navigation mode selection instruction as the target navigation mode. Optionally, the selectable navigation modes include a follow-up navigation mode and a panoramic navigation mode. Furthermore, the data feedback information may include image data of the endoscope and ultrasonic data of the ultrasonic probe. The target three-dimensional model may be a three-dimensional model of the area to be operated on behind the target surgical robot. Specifically, according to the data feedback information, a three-dimensional model of the target surgical robot may be reconstructed in the initial three-dimensional model to obtain the target three-dimensional model.

[0048] Furthermore, the movement feedback information may be feedback information about the movement state of the target surgical robot. Specifically, the display angle of the target three-dimensional model may be determined according to the target navigation mode, and the movement feedback information corresponding to the target navigation mode may be obtained. By displaying the movement feedback information on a preset display screen, the spatial structural changes of the organ tissue may be dynamically displayed, allowing the operator to control the surgical instrument more accurately.

[0049] Optionally, the target three-dimensional model is determined based on the data feedback information and the initial three-dimensional model, including: determining the real-time depth information of the endoscope of the target surgical robot relative to the surgical area based on the data feedback information, and constructing a three-dimensional surface based on the real-time depth information; obtaining the joint encoder information of the surgical arm in the target surgical robot, and determining the real-time posture information of the endoscope based on the joint encoder information; and determining the target three-dimensional model based on the three-dimensional surface, the real-time posture information and the initial three-dimensional model.

[0050] Among them, the real-time depth information can be the real-time distance information between the operator of the target surgical robot and the edge of the area to be operated. The three-dimensional surface can be the surface shape of the edge of the area to be operated in three-dimensional space. Specifically, the real-time distance information between the operator of the target surgical robot and each node in the edge of the area to be operated can be determined based on the real-time depth information, and the real-time depth information can be constructed based on the real-time distance information of all nodes in the edge of the area to be operated. The joint encoder information can be the encoder numerical information of each joint in the operator. The real-time posture information can be the real-time position and posture information of the endoscope of the target surgical machine. Specifically, the joint encoder information can be solved by forward kinematics to obtain the real-time posture information. Further, a coordinate system can be established with the endoscope of the target surgical machine as the origin, and then the registration is performed based on the three-dimensional surface, the real-time posture information and the initial three-dimensional model to obtain the target three-dimensional model. Exemplarily, the iterative closest point algorithm (Iterative Closest Point, ICP) can be used for registration, and then the homogeneous transformation matrix obtained by the ICP algorithm is used to rotate and translate the obtained three-dimensional model.

[0051] Optionally, based on the target three-dimensional model and the target navigation mode, the movement feedback information of the target surgical robot is determined, and the movement feedback information is displayed on a preset display screen, including: when the target navigation mode is the follow-up navigation mode, the target three-dimensional model is made semi-transparent, and the semi-transparent target three-dimensional model is superimposed with the endoscopic image and displayed on the preset display screen; when a display adjustment instruction is received, the transparency and color of the target three-dimensional model are adjusted based on the display adjustment instruction.

[0052] Among them, the follow navigation mode can be understood as a navigation mode with the endoscope angle as the first perspective. When the target navigation mode is the follow navigation mode, the target three-dimensional model can be made semi-transparent, and the transparency and color of the target three-dimensional model can be adjusted based on the display adjustment instructions, and the semi-transparent target three-dimensional model and the endoscope image are superimposed and displayed on the preset display screen. Among them, the display adjustment instructions can be adjustment instructions for the target three-dimensional model in the target navigation mode. The main advantage of the follow navigation mode is that the user can directly observe the spatial relationship and safety boundaries of the organs in real time, allowing the operator to control the surgical instruments more accurately.

[0053] Optionally, based on the target three-dimensional model and the target navigation mode, the movement feedback information of the target surgical robot is determined, and the movement feedback information is displayed on a preset display screen. It also includes: when the target navigation mode is a panoramic navigation mode, a panoramic model of the area to be operated on is displayed on the preset display screen; when a viewing angle adjustment instruction is received, the display parameters of the panoramic model are adjusted based on the viewing angle adjustment instruction.

[0054] Among them, the panoramic navigation mode can be understood as a mode of navigation from a third perspective. The panoramic model can be understood as a model obtained by observing the target three-dimensional model from a third perspective. The perspective adjustment instruction can be an adjustment instruction for the display parameters of the panoramic model in the panoramic navigation mode. Exemplarily, the user can issue a perspective adjustment instruction by rotating, translating, and scaling the three-dimensional model through an interactive touch screen. The display parameters can be parameters for displaying the panoramic model. Exemplarily, the display parameters may include: display perspective and display ratio and other parameters. In the case of receiving the perspective adjustment instruction, the display parameters of the panoramic model can be adjusted accordingly based on the perspective adjustment instruction. The panoramic navigation mode can be used to assist the user in mastering the panoramic spatial relationship of the operation during the operation. Its main functions include: assisting the user to control the lens to find the target area more easily; assisting the user to determine whether the operation interferes with other tissues according to the spatial position of the instrument and the organ; when the instrument moves out of the endoscope field of view, the instrument can be moved back to the field of view more safely and accurately by observing the panoramic view of the endoscope field of view.

[0055] For example, Figure 4 is an example diagram of a panoramic model provided by an embodiment of the present invention. Figure 4 As shown, the figure may include interactive controls such as translation, rotation, and scaling, and the user can click the corresponding interactive controls to adjust the display parameters of the panoramic model.

[0056] The technical solution provided by the embodiment of the present invention can introduce the system into the preoperative planning and intraoperative navigation stages of the medical process, support the reconstruction of three-dimensional models and three-dimensional image semantic segmentation to assist users in preoperative planning, provide three-dimensional model overlay following and panoramic view presentation methods, so that users can more intuitively grasp the spatial relationship of the surgical procedure to effectively improve the efficiency of the operation, and provide a safety fence function to assist user operation to further reduce damage to surrounding tissues during surgery and improve surgical safety.

[0057] The technical solution provided by the embodiment of the present invention obtains image scanning data of the surgical area to be operated on the target object, determines an initial three-dimensional model according to the image scanning data; inputs the initial three-dimensional model into the target part division model, determines multiple candidate parts in the initial three-dimensional model; determines at least one target part from the candidate parts based on the part interaction instruction, and determines the reference movement path according to the target part; obtains robot operation instructions, controls the target surgical robot to move based on the surgical robot operation instructions; determines the reference movement space based on the reference movement path; sends a resistance feedback instruction to the target surgical robot when the operator of the target surgical robot approaches the edge of the reference movement space; sends a movement termination instruction to the target surgical robot when the operator of the target surgical robot exceeds the edge of the reference movement space. The technical solution of the embodiment of the present invention solves the safety problems in the prior art that it is easy to accidentally touch other tissues when operating the surgical robot. The reference movement path can be determined based on the interaction instruction for preoperative planning, and the abnormal movement of the surgical robot can be corrected based on the reference movement path to reduce the damage to the surrounding tissues during the operation and improve the safety of the operation.

[0058] Figure 5 It is a structural schematic diagram of a surgical robot planning and navigation device provided in an embodiment of the present invention. The embodiment of the present invention can be applied to the scenario of controlling the movement of a laparoscopic surgical robot. The device can be implemented by software and / or hardware and integrated into a computer device with application development capabilities.

[0059] like Figure 5 As shown, the surgical robot planning and navigation device includes: a reference path determination module 310 , a movement control module 320 and a movement path correction module 330 .

[0060] Among them, the reference path determination module 310 is used to determine the three-dimensional model of the target object's surgical area, and determine the reference movement path based on the part interaction instructions for the target part in the three-dimensional model; the movement control module 320 is used to obtain the robot operation instructions, and control the movement of the target surgical robot based on the surgical robot operation instructions; the movement path correction module 330 is used to correct the movement path of the target surgical robot based on the reference movement path when the target surgical robot has abnormal movement.

[0061] The technical solution provided by the embodiment of the present invention determines a three-dimensional model of the target object's surgical area, determines a reference movement path based on the part interaction instructions for the target part in the three-dimensional model; obtains robot operation instructions, and controls the target surgical robot to move based on the surgical robot operation instructions; and in the event of abnormal movement of the target surgical robot, corrects the movement path of the target surgical robot based on the reference movement path. The technical solution of the embodiment of the present invention solves the safety problem of the prior art that it is easy to accidentally touch other tissues when operating a surgical robot. It can determine the reference movement path based on the interactive instructions for preoperative planning, and subsequently correct the abnormal movements of the surgical robot based on the reference movement path, thereby reducing damage to surrounding tissues during surgery and improving surgical safety.

[0062] In an optional embodiment, the reference path determination module 310 is specifically used to: obtain image scanning data of the target object's area to be operated on, and determine an initial three-dimensional model based on the image scanning data; input the initial three-dimensional model into a target part division model to determine multiple candidate parts in the initial three-dimensional model; based on the part interaction instructions, determine at least one target part from the candidate parts, and determine the reference movement path based on the target part.

[0063] In an optional embodiment, the movement path correction module 330 is specifically used to: determine a reference movement space based on the reference movement path; send a resistance feedback instruction to the target surgical robot when the operator of the target surgical robot approaches the edge of the reference movement space; and send a movement termination instruction to the target surgical robot when the operator of the target surgical robot exceeds the edge of the reference movement space.

[0064] In an optional embodiment, the surgical robot planning and navigation device also includes: a path navigation module, which is used to: upon receiving a navigation mode selection instruction, determine a target navigation mode according to the navigation mode selection instruction; obtain data feedback information of the target surgical robot, and determine a target three-dimensional model according to the data feedback information and the initial three-dimensional model; based on the target three-dimensional model and the target navigation mode, determine movement feedback information of the target surgical robot, and display the movement feedback information on a preset display screen.

[0065] In an optional embodiment, the path navigation module includes: a target three-dimensional model determination module, which is used to: determine the real-time depth information of the endoscope of the target surgical robot relative to the surgical area according to the data feedback information, and construct a three-dimensional surface based on the real-time depth information; obtain the joint encoder information of the surgical arm in the target surgical robot, and determine the real-time posture information of the endoscope according to the joint encoder information; determine the target three-dimensional model according to the three-dimensional surface, the real-time posture information and the initial three-dimensional model.

[0066] In an optional embodiment, the path navigation module includes: a first movement feedback module, used to: when the target navigation mode is a follow-up navigation mode, perform semi-transparent processing on the target three-dimensional model, and superimpose the semi-transparent target three-dimensional model with the endoscopic image and display it on the preset display screen; when a display adjustment instruction is received, adjust the transparency and color of the target three-dimensional model based on the display adjustment instruction.

[0067] In an optional embodiment, the path navigation module includes: a second mobile feedback module, used to: when the target navigation mode is a panoramic navigation mode, display a panoramic model of the area to be operated on the preset display screen; when a perspective adjustment instruction is received, adjust the display parameters of the panoramic model based on the perspective adjustment instruction.

[0068] The surgical robot planning and navigation device provided in the embodiment of the present invention can execute the surgical robot planning and navigation method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0069] Figure 6 A schematic diagram of the structure of a computer device provided in an embodiment of the present invention. Figure 6 A block diagram of an exemplary computer device 12 suitable for use in implementing embodiments of the present invention is shown. Figure 6 The computer device 12 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention. The computer device 12 can be any terminal device with computing capability and can be configured in the surgical robot planning and navigation device.

[0070] like Figure 6 As shown, the computer device 12 is in the form of a general-purpose computing device. The components of the computer device 12 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 that connects various system components (including the system memory 28 and the processing unit 16).

[0071] The bus 18 may be one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus architectures. For example, these architectures include, but are not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MAC) bus, an Enhanced ISA bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.

[0072] The computer device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the computer device 12, including volatile and non-volatile media, removable and non-removable media.

[0073] The system memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache 32. The computer device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 34 may be used to read and write non-removable, non-volatile magnetic media ( Figure 6 not shown, usually called a "hard drive"). Although Figure 6 Not shown in the figure, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to the bus 18 via one or more data medium interfaces. The system memory 28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present invention.

[0074] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in system memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. Program modules 42 generally perform the functions and / or methods of the embodiments described herein.

[0075] The computer device 12 may also communicate with one or more external devices 14 (e.g., keyboards, pointing devices, displays 24, etc.), one or more devices that enable a user to interact with the computer device 12, and / or any device that enables the computer device 12 to communicate with one or more other computing devices (e.g., network cards, modems, etc.). Such communication may be performed through an input / output (I / O) interface 22. Furthermore, the computer device 12 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) through a network adapter 20. Figure 6 As shown, the network adapter 20 communicates with other modules of the computer device 12 via the bus 18. It should be understood that although Figure 6 Not shown, other hardware and / or software modules may be used in conjunction with computer device 12, including but not limited to microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0076] The processing unit 16 executes various functional applications and data processing by running the program stored in the system memory 28, for example, implementing the surgical robot planning and navigation method provided in the embodiment of the present invention, which includes:

[0077] Determine a three-dimensional model of the target object's area to be operated on, and determine a reference movement path based on part interaction instructions for the target part in the three-dimensional model; obtain robot operation instructions, and control the movement of the target surgical robot based on the surgical robot operation instructions; in the event of abnormal movement of the target surgical robot, correct the movement path of the target surgical robot based on the reference movement path.

[0078] This embodiment provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, the surgical robot planning and navigation method provided in any embodiment of the present invention is implemented, including:

[0079] Determine a three-dimensional model of the target object's area to be operated on, and determine a reference movement path based on part interaction instructions for the target part in the three-dimensional model; obtain robot operation instructions, and control the movement of the target surgical robot based on the surgical robot operation instructions; in the event of abnormal movement of the target surgical robot, correct the movement path of the target surgical robot based on the reference movement path.

[0080] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to: an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0081] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0082] The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0083] Computer program code for performing the operations of the present invention may be written in one or more programming languages ​​or combinations thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0084] It should be understood by those skilled in the art that the modules or steps of the present invention described above can be implemented by a general-purpose computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, optionally, they can be implemented by a program code executable by a computer device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0085] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A surgical robot planning and navigation method, characterized in that: include: Determine a three-dimensional model of the target object's area to be operated on, and determine a reference movement path based on a site interaction instruction for the target site in the three-dimensional model; Obtaining robot operation instructions, and controlling the target surgical robot to move based on the surgical robot operation instructions; In the case where the target surgical robot moves abnormally, the moving path of the target surgical robot is corrected based on the reference moving path.

2. The method according to claim 1, characterized in that: The step of determining a three-dimensional model of the target object's area to be operated on, and determining a reference moving path based on a part interaction instruction for the target part in the three-dimensional model, comprises: Acquire image scanning data of the area to be operated on of the target object, and determine an initial three-dimensional model according to the image scanning data; Inputting the initial three-dimensional model into a target part segmentation model to determine a plurality of candidate parts in the initial three-dimensional model; Based on the part interaction instruction, at least one target part is determined from the candidate parts, and the reference moving path is determined according to the target part.

3. The method according to claim 2, characterized in that When the target surgical robot moves abnormally, the moving path of the target surgical robot is corrected based on the reference moving path, including: determining a reference movement space based on the reference movement path; When the operator of the target surgical robot approaches an edge of the reference movement space, sending a resistance feedback instruction to the target surgical robot; When the operator of the target surgical robot exceeds the edge of the reference movement space, a movement suspension instruction is sent to the target surgical robot.

4. The method according to claim 2 further comprises: In case of receiving a navigation mode selection instruction, determining a target navigation mode according to the navigation mode selection instruction; Acquiring data feedback information of the target surgical robot, and determining a target three-dimensional model according to the data feedback information and the initial three-dimensional model; Based on the target three-dimensional model and the target navigation mode, movement feedback information of the target surgical robot is determined, and the movement feedback information is displayed on a preset display screen.

5. The method according to claim 4, characterized in that The step of determining the target three-dimensional model according to the data feedback information and the initial three-dimensional model includes: Determine the real-time depth information of the endoscope of the target surgical robot relative to the surgical area according to the data feedback information, and construct a three-dimensional surface based on the real-time depth information; Acquire joint encoder information of a surgical arm in the target surgical robot, and determine real-time position and posture information of the endoscope according to the joint encoder information; A target three-dimensional model is determined according to the three-dimensional surface, the real-time posture information and the initial three-dimensional model.

6. The method according to claim 4, characterized in that The step of determining the movement feedback information of the target surgical robot based on the target three-dimensional model and the target navigation mode, and displaying the movement feedback information on a preset display screen, comprises: When the target navigation mode is the follow-up navigation mode, the target three-dimensional model is semi-transparently processed, and the semi-transparently processed target three-dimensional model is superimposed with the endoscope image and displayed on the preset display screen; When a display adjustment instruction is received, the transparency and color of the target three-dimensional model are adjusted based on the display adjustment instruction.

7. The method according to claim 4, characterized in that The step of determining the movement feedback information of the target surgical robot based on the target three-dimensional model and the target navigation mode, and displaying the movement feedback information on a preset display screen, comprises: When the target navigation mode is a panoramic navigation mode, a panoramic model of the area to be operated on is displayed on the preset display screen; When a viewing angle adjustment instruction is received, display parameters of the panoramic model are adjusted based on the viewing angle adjustment instruction.

8. A surgical robot planning and navigation device, characterized in that: The device comprises: A reference path determination module, used to determine a three-dimensional model of the target object's area to be operated on, and determine a reference movement path based on a site interaction instruction for the target site in the three-dimensional model; A movement control module, used to obtain robot operation instructions, and control the target surgical robot to move based on the surgical robot operation instructions; The moving path correction module is used to correct the moving path of the target surgical robot based on the reference moving path when the target surgical robot moves abnormally.

9. A computer device, characterized in that: The computer device comprises: one or more processors; A memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the surgical robot planning and navigation method as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the surgical robot planning and navigation method as described in any one of claims 1-7 is implemented.

Images