A surgical robot

Abstract

This invention discloses a surgical robot. The surgical robot includes: at least one binocular camera acquiring depth images of the surgical environment; a cone-beam computed tomography (CBCT) device acquiring head scan data; a robot body including a robotic arm and a data processing module, the end of the robotic arm being equipped with a surgical instrument adapter for mounting surgical instruments; a surgical environment 3D model generation unit generating a 3D model of the surgical environment based on the depth images; a 3D head model generation unit generating a 3D head model based on the head scan data; and a surgical path planning unit planning the movement path of the surgical instruments based on the 3D surgical environment model and the 3D head model. This technical solution achieves real-time detection of the surgical environment and the head region, providing accurate and reliable 3D surgical environment models and 3D head models, as well as automatic path planning, thereby improving the accuracy and safety of the surgery.

Description

Technical Field

[0001] This invention relates to the field of medical robot technology, and more particularly to a surgical robot. Background Technology

[0002] With the continuous development of robotics technology, robots are being widely used in the medical field.

[0003] Currently, the surgical robots used in ear surgery have limited access to relevant surgical information, resulting in issues with low surgical accuracy and safety. Summary of the Invention

[0004] This invention provides a surgical robot that enables real-time detection of the surgical environment and head region, providing accurate and reliable 3D models of the surgical environment and head, as well as automatic path planning, thereby improving the precision and safety of the surgery.

[0005] According to one aspect of the present invention, a surgical robot is provided, comprising:

[0006] At least one binocular camera is used to acquire depth images of the surgical environment;

[0007] Cone-beam computed tomography (CBCT) imaging equipment used to acquire head scan data;

[0008] The robot body includes a robotic arm and a data processing module, wherein the data processing module is electrically connected to the binocular camera, the cone-beam computed tomography imaging device, and the robotic arm.

[0009] The end of the robotic arm is equipped with a surgical instrument adapter, which is used to mount surgical instruments.

[0010] The data processing module includes a surgical environment 3D model generation unit, a 3D head model generation unit, and a surgical path planning unit;

[0011] The surgical environment 3D model generation unit is used to generate a surgical environment 3D model based on the depth image of the surgical environment.

[0012] The three-dimensional head model generation unit is used to generate a three-dimensional head model based on the head scan data;

[0013] The surgical path planning unit is used to plan the movement path of the surgical instruments based on the three-dimensional model of the surgical environment and the three-dimensional head model.

[0014] Optionally, the surgical environment 3D model generation unit includes:

[0015] A point cloud data conversion subunit is used to convert the depth image of the surgical environment into point cloud data of the surgical environment;

[0016] The point cloud data filtering subunit is used to filter the surgical environment point cloud data to obtain filtered surgical environment point cloud data.

[0017] The surface reconstruction subunit is used to perform surface reconstruction based on the filtered surgical environment point cloud data to obtain a three-dimensional model of the surgical environment.

[0018] Optionally, the three-dimensional head model generation unit includes:

[0019] The head scan data filtering subunit is used to filter the head scan data to obtain filtered head scan data.

[0020] The head region segmentation subunit is used to perform region segmentation based on the filtered head scan data to obtain multiple head tissue block features.

[0021] The head 3D reconstruction subunit is used to perform 3D reconstruction of the features of the multiple head tissue blocks to obtain a 3D head model.

[0022] Optionally, the data processing module further includes:

[0023] The surgical safety warning unit is used to obtain the position of the surgical instrument and to issue a warning when the position of the surgical instrument exceeds the range of a first preset safety area. The warning method includes sound warning, vibration warning and / or light warning.

[0024] Optionally, the data processing module further includes:

[0025] The surgical safety braking unit is used to obtain the position of the surgical instrument and lock the joints of the robotic arm when the position of the surgical instrument exceeds the range of a second preset safety zone.

[0026] Optionally, the data processing module further includes:

[0027] The instrument position compensation unit is used to obtain the position of the surgical instrument, compare the position of the surgical instrument with the target position of the surgical instrument, and if the position of the surgical instrument has not reached the target position, control the surgical instrument to move to the target position.

[0028] Optionally, the data processing module further includes:

[0029] The head tissue segmentation unit is used to input the head scan data into a pre-trained head segmentation model to obtain a head segmentation result, wherein the head segmentation result includes one or more head tissues.

[0030] Optionally, the robot body is provided with a robot body target for identifying the position of the robot body, the end of the robotic arm is provided with an instrument target for identifying the position of surgical instruments, and the headrest is provided with a headrest target for identifying the position of the headrest.

[0031] Optionally, the surgical robot further includes:

[0032] An imaging trolley, which is communicatively connected to the data processing module, is used to display the three-dimensional model of the surgical environment and / or the three-dimensional head model.

[0033] Optionally, the surgical robot further includes:

[0034] The operating table has degrees of freedom in raising and lowering, translation, and moving forward and backward; the operating table includes a headrest, which is a device for supporting the head.

[0035] The surgical robot of this invention includes: at least one binocular camera for acquiring depth images of the surgical environment; a cone-beam computed tomography (CBCT) imaging device for acquiring head scan data; a robot body including a robotic arm and a data processing module, the data processing module being electrically connected to the binocular camera, the CBCT imaging device, and the robotic arm; a surgical instrument adapter at the end of the robotic arm for mounting surgical instruments; and a data processing module including a surgical environment 3D model generation unit, a 3D head model generation unit, and a surgical path planning unit. The surgical environment 3D model generation unit generates a 3D model of the surgical environment based on the depth images of the surgical environment; the 3D head model generation unit generates a 3D head model based on head scan data; and the surgical path planning unit plans the movement path of the surgical instruments based on the surgical environment 3D model and the 3D head model. This technical solution enables real-time detection of the surgical environment and head region, providing accurate and reliable 3D models of the surgical environment and the 3D head model, as well as automatic path planning, thereby improving the accuracy and safety of the surgery.

[0036] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0037] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0038] Figure 1 This is a schematic diagram of the structure of a surgical robot according to an embodiment of the present invention;

[0039] Figure 2 This is a schematic diagram of a counterweight structure provided according to an embodiment of the present invention;

[0040] Figure 3 This is a schematic diagram of the structure of a robotic arm according to an embodiment of the present invention;

[0041] Figure 4 This is a schematic diagram of the structure of a surgical robot according to an embodiment of the present invention;

[0042] Figure 5 This is a schematic diagram of the structure of a surgical robot according to an embodiment of the present invention;

[0043] Figure 6 This is a schematic diagram of the structure of a robot body target and a device target according to an embodiment of the present invention;

[0044] Figure 7 This is a schematic diagram of the structure of a head-supported target provided according to an embodiment of the present invention. Detailed Implementation

[0045] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0046] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be used interchangeably where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices. The acquisition, storage, use, and processing of data in the technical solutions of this application all comply with the relevant provisions of national laws and regulations.

[0047] Figure 1 This is a schematic diagram of the structure of a surgical robot provided in an embodiment of the present invention. This embodiment is applicable to surgeries such as ear surgery. Figure 1 As shown, the device includes: at least one binocular camera 110 for acquiring depth images of the surgical environment; a cone-beam computed tomography (CBCT) imaging system 120 for acquiring head scan data; a robot body 130, including a robotic arm 131 and a data processing module (not shown in the figure), the data processing module being electrically connected to the binocular camera 110, the CBCT imaging system 120, and the robotic arm 131 respectively; a surgical instrument adapter is provided at the end of the robotic arm 131 for mounting surgical instruments 132; the data processing module includes a surgical environment 3D model generation unit, a 3D head model generation unit, and a surgical path planning unit; the surgical environment 3D model generation unit is used to generate a 3D model of the surgical environment based on the depth images of the surgical environment; the 3D head model generation unit is used to generate a 3D head model based on head scan data; the surgical path planning unit is used to plan the motion path of the surgical instruments based on the surgical environment 3D model and the 3D head model.

[0048] In this embodiment, the binocular camera 110 can be used to acquire real-time depth images of the surgical environment. These depth images may include, but are not limited to, depth information of robot components such as the computed tomography (CT) device 120 and the robot body 130, as well as other objects within the camera's field of view. The cone beam computed tomography (CBCT) device 120 can be used to acquire head scan data, which refers to the brain structure data obtained from scanning the user's head. The robot body 130 may include, but is not limited to, a robot base, a robotic arm 131, surgical instruments 132, and a data processing module. The data processing module is electrically connected to both the binocular camera 110 and the CBCT device 120, meaning it can receive depth images of the surgical environment transmitted by the binocular camera 110 and head scan data transmitted by the CBCT device 120.

[0049] After the data processing module receives the depth image of the surgical environment and the head scan data, the surgical environment 3D model generation unit can generate a 3D model of the surgical environment based on the depth image of the surgical environment; the 3D head model generation unit can generate a 3D head model based on the head scan data; and the surgical path planning unit can plan the movement path of the surgical instruments based on the surgical environment 3D model and the 3D head model.

[0050] In this embodiment, the 3D model of the surgical environment enables the surgical robot to precisely perceive the surgical area and warn of potential hazards, thereby improving surgical safety. For example, the data processing module can determine whether the surgical instrument 132 exceeds the safe surgical area based on the 3D model of the surgical environment. If the surgical instrument 132 exceeds the safe surgical area, it will issue a warning through sound, light, or vibration.

[0051] In this embodiment, the three-dimensional model of the surgical environment and the three-dimensional head model can be used together for surgical path planning to ensure that the surgical instrument 132 avoids the target tissue structure inside the head and reduces surgical risks.

[0052] The surgical instrument 132 may include, but is not limited to, grinding or drilling tools, etc., without specific limitations. The robotic arm 131 may be a passive device, that is, the user needs to drag the robotic arm 131 or the surgical instrument 132 to achieve movement.

[0053] In some embodiments, each joint of the robotic arm 131 is provided with an absolute encoder, which can be used to measure the absolute angle of rotation of each joint of the robotic arm 131. In some embodiments, the connection part of the robotic arm 131 can be a quick-release tenon and mortise structure, and the electrical connection part can be a pluggable plug and socket structure.

[0054] Optionally, the robotic arm 131 includes six degrees of freedom, and each joint of the robotic arm 131 is provided with a counterweight structure 133, which is used to balance the weight of the joint itself and the weight of the surgical instruments.

[0055] For example, Figure 2 This is a schematic diagram of a counterweight structure provided in an embodiment of the present invention. It should be noted that the counterweight structure 133 can be used to balance the weight of the joint itself and the weight of the surgical instruments, thereby reducing the force required by the user to operate the robotic arm.

[0056] For example, Figure 3 This is a schematic diagram of a robotic arm provided in an embodiment of the present invention. The three rotation axes of the robotic arm 131 can be set to a vertical Z-axis, and the fourth degree of freedom can be set to linear vertical motion. The advantage of this configuration is that the entire weight of the robotic arm 131 and the surgical instrument 132 can be borne by the joints, eliminating the need for the user to manually apply lifting force to the robotic arm 131 and the surgical instrument 132.

[0057] Optionally, the cone-beam computed tomography imaging device 120 is positioned in front of the robot body and includes rotational and vertical degrees of freedom.

[0058] Specifically, the head position is precisely scanned by rotating and / or raising the cone beam imaging computer reconstruction tomography device 120.

[0059] Optionally, the surgical environment 3D model generation unit includes: a point cloud data conversion subunit, used to convert the depth image of the surgical environment into surgical environment point cloud data; a point cloud data filtering subunit, used to filter the surgical environment point cloud data to obtain filtered surgical environment point cloud data; and a surface reconstruction subunit, used to perform surface reconstruction based on the filtered surgical environment point cloud data to obtain a surgical environment 3D model.

[0060] For example, the depth image of the surgical environment can be converted into surgical environment point cloud data, where the surgical environment point cloud data can be represented by P = (x, y, D), where x and y represent pixel coordinates and D represents the depth value; further, the surgical environment point cloud data can be filtered by voxel mesh filtering or statistical filtering to obtain filtered surgical environment point cloud data, so as to remove noise points and redundant points; further, the surface can be reconstructed from the filtered surgical environment point cloud data by the Marching Cubes algorithm to obtain a three-dimensional model of the surgical environment.

[0061] Optionally, the three-dimensional head model generation unit includes: a head scan data filtering subunit, used to filter the head scan data to obtain filtered head scan data; a head region segmentation subunit, used to perform region segmentation based on the filtered head scan data to obtain multiple head tissue block features; and a head three-dimensional reconstruction subunit, used to perform three-dimensional reconstruction of the multiple head tissue block features to obtain a three-dimensional head model.

[0062] For example, the head scan data can be filtered by a head scan data filtering subunit using Gaussian filtering, median filtering, or bilateral filtering to obtain filtered head scan data. Then, a head region segmentation subunit can perform region segmentation based on the filtered head scan data to obtain multiple head tissue block features. The region segmentation can include, but is not limited to, image segmentation methods such as threshold segmentation, Otus method, region growing, level set method, or graph cut. Finally, a head 3D reconstruction subunit can perform 3D reconstruction processing on the multiple head tissue block features using volume rendering, surface reconstruction, or Delaunay triangulation to obtain a 3D head model.

[0063] Optionally, the data processing module further includes: a surgical safety warning unit, used to obtain the position of surgical instruments and to issue a warning when the position of surgical instruments exceeds the range of a first preset safety area, wherein the warning method includes sound warning, vibration warning and / or light warning.

[0064] For example, the data processing module can determine whether the surgical instrument has exceeded the safe surgical area based on its position. If the surgical instrument exceeds the safe surgical area, a warning will be issued by means of sound, light and / or vibration.

[0065] Optionally, the data processing module further includes a surgical safety braking unit, used to acquire the position of the surgical instrument and lock the joints of the robotic arm 131 when the position of the surgical instrument exceeds the range of the second preset safety area.

[0066] For example, the data processing module can determine whether the surgical instrument has exceeded the safe surgical area based on its position. If the surgical instrument exceeds the safe surgical area, the braking is activated to lock the joints of the robotic arm 131 to prevent injury to the patient.

[0067] Optionally, the data processing module further includes: an instrument position compensation unit, used to acquire the position of the surgical instrument, compare the position of the surgical instrument with the target position of the surgical instrument, and if the position of the surgical instrument has not reached the target position of the surgical instrument, control the surgical instrument to move to the target position of the surgical instrument.

[0068] For example, the position of the surgical instrument can be obtained through a binocular camera. If the position of the surgical instrument has not reached the target position, the surgical instrument can be controlled to move to the target position through PID compensation.

[0069] Optionally, the data processing module further includes a head tissue segmentation unit, used to input the head scan data into a pre-trained head segmentation model to obtain a head segmentation result, wherein the head segmentation result includes one or more head tissues.

[0070] Among them, the head segmentation model refers to a pre-trained neural network model.

[0071] Specifically, a large amount of head scan sample data and the corresponding tissue category labels can be obtained. The head scan sample data is then input into an initial neural network, which outputs a predicted tissue category. The model loss is determined based on the predicted tissue category and the tissue category label. The parameters of the initial neural network are adjusted based on the model loss until the model training stopping condition is met, thus obtaining a head segmentation model.

[0072] Optionally, the surgical robot also includes an operating table 140, which has lifting, translation, and forward / backward degrees of freedom; the operating table 140 includes a headrest, which is a device for supporting the head.

[0073] For example, Figure 4 This is a schematic diagram of a surgical robot provided in an embodiment of the present invention. The head can be moved to the CBCT scanning area via an adaptive operating table 140 that can be raised, lowered, translated, and / or moved forward and backward.

[0074] Optionally, the surgical robot also includes an imaging carriage 150, which is communicatively connected to a data processing module for displaying a three-dimensional model of the surgical environment and / or a three-dimensional head model.

[0075] For example, Figure 5 This is a schematic diagram of a surgical robot provided in an embodiment of the present invention. The auxiliary imaging cart 150 assists the user in performing surgery by displaying a three-dimensional model of the surgical environment and / or a three-dimensional head model.

[0076] Optionally, the robot body 130 is provided with a robot body target for identifying the position of the robot body, the end of the robotic arm is provided with an instrument target for identifying the position of surgical instruments, and the headrest is provided with a headrest target for identifying the position of the headrest.

[0077] For example, Figure 6 This is a schematic diagram of the structure of a robot main target and a device target provided in an embodiment of the present invention. Figure 7This is a schematic diagram of a headrest target provided in an embodiment of the present invention. Specifically, within the field of view of the binocular camera, the position of the robot body can be obtained in real time based on the robot target, the position of the surgical instrument can be obtained in real time based on the surgical instrument target, and the position of the headrest can be obtained in real time based on the headrest target.

[0078] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A surgical robot, characterized by, The application relates to a surgical robot system, comprising: at least one binocular camera for collecting a depth image of a surgical environment; a cone beam projection computerized tomography device for collecting head scanning data; a robot body comprising a mechanical arm and a data processing module, the data processing module being electrically connected with the binocular camera, the cone beam projection computerized tomography device and the mechanical arm respectively; a surgical instrument adapter is arranged at the end of the mechanical arm, and the surgical instrument adapter is used for mounting a surgical instrument; the data processing module comprises a surgical environment three-dimensional model generation unit, a three-dimensional head model generation unit and a surgical path planning unit; the surgical environment three-dimensional model generation unit is used for generating a surgical environment three-dimensional model based on the depth image of the surgical environment; the three-dimensional head model generation unit is used for generating a three-dimensional head model based on the head scanning data; the surgical path planning unit is used for planning a movement path of the surgical instrument based on the surgical environment three-dimensional model and the three-dimensional head model.

2. The surgical robot of claim 1, wherein, the surgical environment three-dimensional model generation unit comprises: a point cloud data conversion subunit for converting the depth image of the surgical environment into surgical environment point cloud data; a point cloud data filtering subunit for filtering the surgical environment point cloud data to obtain filtered surgical environment point cloud data; a surface reconstruction subunit for performing surface reconstruction based on the filtered surgical environment point cloud data to obtain a surgical environment three-dimensional model.

3. The surgical robot of claim 1, wherein, the three-dimensional head model generation unit comprises: a head scanning data filtering subunit for filtering the head scanning data to obtain filtered head scanning data; a head region segmentation subunit for performing region segmentation based on the filtered head scanning data to obtain a plurality of head tissue block features; a three-dimensional head reconstruction subunit for performing three-dimensional reconstruction on the plurality of head tissue block features to obtain a three-dimensional head model.

4. The surgical robot of claim 1, wherein, the data processing module further comprises: a surgical safety warning unit for acquiring a surgical instrument position, and performing a warning prompt in the case that the surgical instrument position exceeds a first preset safety area range, wherein the warning prompt mode comprises a sound prompt, a vibration prompt and / or a light prompt.

5. The surgical robot of claim 1, wherein, the data processing module further comprises: a surgical safety braking unit for acquiring a surgical instrument position, and locking a joint of the mechanical arm in the case that the surgical instrument position exceeds a second preset safety area range.

6. The surgical robot of claim 1, wherein, the data processing module further comprises: an instrument position compensation unit for acquiring a surgical instrument position, comparing the surgical instrument position with a surgical instrument target position, and controlling the surgical instrument to move to the surgical instrument target position if the surgical instrument position has not reached the surgical instrument target position.

7. The surgical robot of claim 1, wherein, the data processing module further comprises: a head tissue segmentation unit for inputting the head scanning data into a pre-trained head segmentation model to obtain a head segmentation result, wherein the head segmentation result comprises one or more head tissues.

8. The surgical robot of claim 1, wherein, The robot body is provided with a robot body target for identifying the position of the robot body, the end of the mechanical arm is provided with an instrument target for identifying the position of the surgical instrument, and the head holder is provided with a head holder target for identifying the position of the head holder.

9. The surgical robot of claim 1, wherein, The surgical robot further comprises: An image trolley in communication connection with the data processing module, for displaying the three-dimensional model of the surgical environment and / or the three-dimensional head model.

10. The surgical robot of claim 1, wherein, The surgical robot further comprises: A surgical bed having lifting, translational and advancing and retreating degrees of freedom; the surgical bed comprises a head holder which is a device for supporting the head.

Images