Orthopedic surgical robot visual tactile sensing guide device, surgical system and method

Abstract

The present invention relates to the field of medical equipment technology, and more particularly to a visual-tactile sensor guide device, a surgical system and a method for an orthopedic surgical robot, wherein the visual-tactile sensor guide device comprises a mechanical arm connector and a visual-tactile sensor guide assembly, wherein the mechanical arm connector is connected to the mechanical arm; the visual-tactile sensor guide assembly comprises a main body module, a light-transmitting elastic module is arranged at the end of the main body module, a bracket interface is arranged on the side of the main body module, the bracket interface is connected to the mechanical arm connector, and a surgical instrument interface is arranged on the main body module. The present invention can directly obtain the shape and texture information of the target bone surface through a small number of range data acquisition, complete the calculation of the target bone posture, solve the problem of "many steps, long time, and easy occlusion" in the registration process of the optical navigation and positioning system, maintain high resolution and bone surface recognition ability through visual-tactile sensing, effectively avoid the interference of liquid (blood, tissue fluid) during pure visual acquisition, and provide a larger space for the integration of computer vision and artificial intelligence.

Description

Technical Field

[0001] The present invention relates to the technical field of medical equipment, and in particular to a visual-tactile sensing guide device, a surgical system and a method for an orthopedic surgical robot. Background Art

[0002] Orthopedic surgical robots are increasingly being used in surgeries involving multiple joints, such as the shoulder, hip, knee, and spine, because they can provide surgeons with intuitive and detailed preoperative planning, as well as precise and stable intraoperative navigation and manipulation assistance, effectively optimizing surgical outcomes. At the start of surgery, to ensure that the robot's end-use tool and the patient's actual joint position on the operating table are shared in a world coordinate system, the preoperative image and intraoperative data of the bone on the side of the patient's joint to be operated on are first aligned. This establishes the bone's spatial position, enabling robot trajectory planning and intraoperative navigation. Currently, the most common navigation method involves placing tracers with optical markers on the bones and monitoring them with a global-view infrared optical camera (such as the NDI optical positioning system) to obtain pose information. Typically, tracers are implanted in all target patient bones, and probes equipped with the tracers are used for calibration and registration to locate the patient's bones. During surgery, the robot's end-use tool, equipped with a tracer, is used to synchronously obtain the relative positions of the patient and tool to monitor and control the trajectory of the intraoperative procedure.

[0003] The navigation system driven by the optical camera and the tracer has the following problems that need to be optimized: (1) Intraoperative registration requires the doctor to first drill one or two holes in the unrelated bones around the patient's target bone surface to fix the tracer. For joints with smaller structures, such as the scapula in the shoulder joint, the location available for drilling and installing the tracer is very small, making it challenging to install it firmly, and the possibility of loosening and occlusion during surgery also increases. (2) After the tracer is installed, the doctor needs to hold the probe with the tracer and click dozens or hundreds of times on the bone feature points according to the system prompts, supplemented by preoperative CT images, to reconstruct the actual exposed bone posture of the patient during surgery. This is time-consuming and the accuracy is affected by the doctor's click position. (3) During the intraoperative operation, the relative position between the patient's bones and the robot needs to be captured and updated in real time. If the tracer of either side is blocked in the middle, the operation needs to be paused and recalibrated appropriately, which causes unnecessary interruptions.

[0004] In addition to the use of optical cameras and tracers, other methods such as the use of depth cameras to directly observe the target bone surface and three-dimensional reconstruction navigation strategies, the inevitable interference of blood and tissue fluid flow in the surgical field and the patient's joint-specific wear will significantly interfere with image processing and judgment, reduce the accuracy of preoperative data training, and lead to possible faults, misjudgments and misleading.

[0005] Therefore, how to effectively shorten the time doctors spend on initial registration at the start of surgery, increase the stability of intraoperative navigation, and maintain sensitivity and accuracy has become an urgent problem to be solved. Summary of the Invention

[0006] In response to the problems existing in the prior art when performing intraoperative navigation with orthopedic surgical robots, the present invention provides a visual-tactile sensing guidance device, surgical system and method for orthopedic surgical robots, which changes the current methods of intraoperative alignment and navigation, and has the advantages of saving intraoperative operation time, strong robustness, high resolution, and intuitive interaction.

[0007] The present invention provides a visual-tactile sensing and guiding device for an orthopedic surgical robot, comprising a robotic arm connector and a visual-tactile sensing and guiding assembly, wherein the first end of the robotic arm connector is connected to the robotic arm of the orthopedic surgical robot; the visual-tactile sensing and guiding assembly is used to directly obtain the shape and texture information of the target bone surface to complete the calculation of the target bone posture; the visual-tactile sensing and guiding assembly comprises a main body module, the end of the main body module is provided with a light-transmitting elastic module, the side of the main body module is provided with a bracket interface, the bracket interface is connected to the second end of the robotic arm connector, and the main body module is also provided with a surgical instrument interface for connecting surgical instruments.

[0008] According to a visual-tactile sensing and guiding device for an orthopedic surgical robot provided by the present invention, the robotic arm connecting part includes an end flange, a connecting bracket and a sensor guide interface, the first end face of the end flange is rigidly connected to the robotic arm, the second end face of the end flange is connected to the first end of the connecting bracket, the connecting bracket is provided with a bending portion to change the connection angle of the robotic arm, the second end of the connecting bracket constructs the sensor guide interface, and the sensor guide interface is rigidly matched with the bracket interface.

[0009] According to a visual-tactile sensing and guiding device for an orthopedic surgical robot provided by the present invention, the main body module includes a shell, a light-guiding lens, a light source and a camera, wherein the shell is a wrapping structure with an open end, the elastic module is installed at the open end of the shell, and the bracket interface is provided on the outer wall of the shell; the light-guiding lens is fixed inside the shell and is located on the side of the elastic module facing the shell; the light source is located inside the shell to emit monochromatic and stable light in the direction of the elastic module; the camera is located inside the shell to capture deformation image information of the elastic module.

[0010] According to a visual-tactile sensing and guiding device for an orthopedic surgical robot provided by the present invention, the elastic module is a light-transmitting solid colloid, one end of the light-transmitting solid colloid is rigidly installed in the shell, and the other end is exposed outside the shell, and the end of the light-transmitting solid colloid exposed outside the shell is provided with a plurality of opaque characteristic protrusions distributed in an array, and the characteristic protrusions are constructed from the surface of the light-transmitting solid colloid toward the interior of the light-transmitting solid colloid.

[0011] According to the visual-tactile sensing and guiding device of an orthopedic surgical robot provided by the present invention, the surgical instrument interface is constructed as a guide hole that sequentially passes through the main body module and the elastic module.

[0012] According to the visual-tactile sensing and guiding device of an orthopedic surgical robot provided by the present invention, the main body module and the elastic module are provided with withdrawal grooves in the same side direction, and the surgical instrument interface is constructed at the inner end of the withdrawal groove.

[0013] According to the visual-tactile sensing guide device of an orthopedic surgical robot provided by the present invention, the surgical instrument interface is a hollow guide cylinder, and the hollow guide cylinder is arranged parallel to the outer periphery of the main body module along the length direction of the main body module.

[0014] According to the visual-tactile sensing and guiding device of an orthopedic surgical robot provided by the present invention, the surgical instrument interface is a guide block, and the guide block is connected to the outer peripheral surface of the main body module through a connecting piece.

[0015] The present invention also provides a surgical system for an orthopedic surgical robot, which is suitable for any of the above-mentioned visual-tactile sensing and guiding devices of the orthopedic surgical robot, and includes an input information unit, a system information processing unit, and an output information unit, wherein the input information unit is used to obtain external input data that has not been processed by the intraoperative computer system; the system information processing unit is connected to the input information unit, and is used to collect and process the data of the input information unit; the output information unit is connected to the system information processing unit, and is used to obtain the data processing results of the system information processing unit and output a control signal to control the orthopedic surgical robot to assist in completing the operation.

[0016] According to a surgical system of an orthopedic surgical robot provided by the present invention, the external input data obtained by the input information unit includes: image data obtained from the patient's preoperative CT scan, orthopedic surgical robot definition data, the patient's actual intraoperative bone surface structure data, preoperative planning result data, and customized design results of the patient's anatomical structure obtained by a visual-tactile sensor guide.

[0017] The system information processing unit collects and processes the data of the input information unit, including: reconstructing a target bone 3D model from a preoperative CT image and obtaining the local posture of the target bone surface, collecting target bone surface data with a visual-tactile sensor guide and recording the global posture of the collection point in combination with the posture of the robotic arm, determining the key collection area based on preoperative planning and determining its local posture on the target bone, data smoothing, image stitching, matching processing of the collected data with the preoperative CT image, overlay comparison processing between the reconstructed models, data visualization processing, direct model modification processing based on human-computer interaction, and finding the local posture of the collection area in the target bone surface.

[0018] The control signal output by the output information unit includes: the global posture of the target bone including the target bone surface in the world coordinate system, the global posture of the bone feature position in the world coordinate system during intraoperative operation and the trajectory planning result, the global posture and trajectory planning result of the robotic arm when performing intraoperative operation, the intraoperative reconstruction model of the target bone and bone surface model, the visual overlay comparison result of the target bone surface and the key acquisition area in the CT reconstruction model and the intraoperative reconstruction model, and the fusion reconstruction model of the target bone and bone surface.

[0019] The present invention also provides a surgical method for an orthopedic surgical robot, which is applicable to any of the above-mentioned visual-tactile sensing and guiding devices for the orthopedic surgical robot and any of the above-mentioned surgical systems for the orthopedic surgical robot, and includes steps S1 to S4.

[0020] S1. Mount the visual-tactile sensor guide device on the orthopedic surgical robot, move the orthopedic surgical robot to the operating position and start it, so that the surgical system of the orthopedic surgical robot intervenes in the surgical process.

[0021] S2. Use the visual-tactile sensor guidance device to reconstruct two-dimensional and three-dimensional data of the patient's target bones and target bone surfaces, and perform preoperative / intraoperative alignment of the visual-tactile sensor guidance device.

[0022] S3. Select a surgical instrument suitable for the surgical operation and install it on the surgical instrument interface of the visual-tactile sensing guide device, and perform orthopedic surgical robot-assisted intraoperative operation based on the data model reconstructed in step S2.

[0023] S4. After all operations requiring the assistance of the orthopedic surgical robot are completed, the orthopedic surgical robot is put on standby and withdraws from the surgical process, and the orthopedic surgical robot-assisted surgery part is completed.

[0024] According to a surgical method for an orthopedic surgical robot provided by the present invention, the process of performing pre-operative / intra-operative registration of the visual-tactile sensing guide device in step S2 includes S21 to S29.

[0025] S21. Determine the area around the feature points on the target bone surface and the area involved in the intraoperative operation as the acquisition area.

[0026] S22. Pull the robotic arm to bring the visual-tactile sensing guide device closer to the target bone surface, and adjust the posture of the visual-tactile sensing guide device so that the elastic module is aligned with the collection area defined in step S21.

[0027] S23 , contacting the elastic module with the collection area defined in step S21 , gradually applying stress until the elastic module fully contacts the target bone surface, and collecting surface data information of the target bone surface.

[0028] S24: Determine whether the target bone surface data information collected in step S23 meets the threshold for intraoperative model reconstruction. If the judgment is no, execute step S25; if the judgment is yes, execute step S26.

[0029] S25. Determine the target bone surface area with insufficient data, add it as a supplementary acquisition area, and repeat steps S22 to S24.

[0030] S26. Reconstruct the target bone and target bone surface of the patient in two dimensions and three dimensions according to the target bone surface data information acquired in step S23.

[0031] S27. The preoperative CT scan and the data model reconstructed in step S26 are simultaneously displayed on the main control trolley display screen, and key view angles and areas are superimposed for comparison to determine whether the reconstructed model result is reliable. If not, execute step S28; if yes, execute step S29.

[0032] S28: Adjust the weights of the reconstructed model and the preoperative CT model, correct the current reconstructed model, form a new fusion reconstructed model, and then re-execute step S27.

[0033] S29. Confirm to the surgical system of the orthopedic surgical robot that the current model is reliable and effective, and complete the preoperative / intraoperative alignment of the visual-tactile sensing guide device.

[0034] According to the surgical method of an orthopedic surgical robot provided by the present invention, the intraoperative operation process in step S3 includes S31 to S39.

[0035] S31. Select a surgical instrument and assemble a visual-tactile sensing guide device according to the intraoperative operation to be performed.

[0036] S32. Based on the data model reconstructed in step S2, the surgical system of the orthopedic surgical robot records the position of the target bone surface, and the robotic arm automatically plans the trajectory and moves to the position of the target bone surface suitable for intraoperative operation, so that the elastic module of the visual-tactile sensing guide device fully contacts the target bone surface.

[0037] S33. The visual-tactile sensing guidance device collects surface information of the target bone surface, obtains the local pose of the collection area on the target bone surface, and calculates the current global pose of the target bone surface.

[0038] S34. Compare the current global posture of the target bone surface calculated in step S33 with the posture of the target bone surface recorded by the surgical system of the orthopedic surgical robot in step S32, and determine whether the two postures are consistent. If not, execute step S35; if yes, execute step S36.

[0039] S35: The robotic arm slightly tracks to a suitable operating posture under the current target bone surface posture, and then proceeds to step S36.

[0040] S36. The doctor begins the intraoperative operation.

[0041] S37. Determine whether the doctor has finished the intraoperative operation in this link. If the judgment is no, execute step S38; if the judgment is yes, execute step S39.

[0042] S38: The doctor has not completed the intraoperative operation in this link, and steps S33 to S37 are executed again.

[0043] S39. The doctor has completed the intraoperative operation of this link, confirmed that the operation is completed, and executed the end operation instruction. The robotic arm drives the visual and tactile sensing guide device to leave the target bone surface.

[0044] The orthopedic surgical robot visual-tactile sensing and guidance device, surgical system, and method provided by the present invention are different from the currently commonly used "optical navigation and positioning system" sensing and guidance system. It can directly obtain the shape and texture information of the target bone surface through a small number of range data acquisitions (rather than a probe collecting one point at a time), complete the calculation of the target bone posture, and solve the problems of "many steps, long time, and easy occlusion" in the registration process of the optical navigation and positioning system. And through visual-tactile sensing, it maintains high resolution and bone surface recognition capabilities. The present invention eliminates the need for tracers on the patient's bones and surgical instruments, as well as the installation of optical cameras on the trolley. Therefore, there is no need to consider the tracer installation position, depth, and stability. It is more adaptable to bones of different sizes and wear conditions and has strong robustness. The sensing and guidance functions are coupled, with high integration and compact structure. The visual-tactile acquisition method effectively avoids the interference of liquids (blood, tissue fluid) during pure visual acquisition, providing greater space for the integration of computer vision and artificial intelligence. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0046] Figure 1 It is a structural schematic diagram of the visual-tactile sensing and guiding device of the orthopedic surgical robot provided by the present invention.

[0047] Figure 2 It is a structural schematic diagram of the visual-tactile sensor guide assembly provided by the present invention.

[0048] Figure 3 It is a structural schematic diagram of the robotic arm connecting piece provided by the present invention.

[0049] Figure 4 It is a schematic diagram of the exploded structure of the visual-tactile sensor guide assembly provided by the present invention.

[0050] Figure 5 It is a schematic cross-sectional structure diagram of the visual-tactile sensor guide assembly provided by the present invention.

[0051] Figure 6 This is one of the schematic diagrams of the upward structure of the visual-tactile sensor guide assembly provided by the present invention.

[0052] Figure 7 This is the second schematic diagram of the upward structure of the visual-tactile sensor guide assembly provided by the present invention.

[0053] Figure 8 This is the third schematic diagram of the upward-looking structure of the visual-tactile sensor guide assembly provided by the present invention.

[0054] Figure 9 This is the fourth schematic diagram of the upward-looking structure of the visual-tactile sensor guide assembly provided by the present invention.

[0055] Figure 10 This is one of the schematic diagrams of the assembly structure of the visual-tactile sensor guide assembly and the surgical instrument provided by the present invention.

[0056] Figure 11 This is the second schematic diagram of the assembly structure of the visual-tactile sensor guide assembly and the surgical instrument provided by the present invention.

[0057] Figure 12 This is the third schematic diagram of the assembly structure of the visual-tactile sensor guide assembly and the surgical instrument provided by the present invention.

[0058] Figure 13 It is a schematic diagram of the surgical system of the orthopedic surgical robot provided by the present invention.

[0059] Figure 14 This is one of the schematic diagrams of the surgical system workflow of the orthopedic surgical robot provided by the present invention.

[0060] Figure 15 This is the second schematic diagram of the surgical system workflow of the orthopedic surgical robot provided by the present invention.

[0061] Figure 16 It is a flow chart of the orthopedic surgical robot method provided by the present invention.

[0062] Figure 17 1 is a schematic diagram of the implementation flow of step S2 of the orthopedic surgical robot method provided by the present invention.

[0063] Figure 18 2 is a schematic diagram of the implementation process of step S21 of the orthopedic surgical robot method provided by the present invention.

[0064] Figure 19 3 is a schematic diagram of the implementation flow of step S3 of the orthopedic surgical robot method provided by the present invention.

[0065] Figure numerals: 1. Robotic arm connector; 11. End flange; 12. Connecting bracket; 13. Sensor guide interface; 2. Robotic arm; 3. Visual-tactile sensor guide assembly; 31. Main body module; 311. Housing; 312. Light guide lens; 313. Light source; 314. Camera; 32. Elastic module; 321. Transparent solid colloid; 322. Feature protrusion; 33. Bracket interface; 34. Surgical instrument interface; 341. Withdrawal slot; 4. Surgical instrument; 5. Input information unit; 6. System information processing unit; 7. Output information unit. DETAILED DESCRIPTION

[0066] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0067] The following combination Figures 1 to 19 The present invention describes an orthopedic surgical robot visual-tactile sensing guide device, surgical system and method.

[0068] An embodiment of the present invention provides a visual-tactile sensor guide device for an orthopedic surgical robot, Figure 1 and Figure 2As shown, it includes a robotic arm connector 1 and a visual-tactile sensor guide assembly 3, wherein the first end of the robotic arm connector 1 is connected to the robotic arm 2 of the orthopedic surgical robot; the visual-tactile sensor guide assembly 3 is used to directly obtain the shape and texture information of the target bone surface to complete the calculation of the target bone posture; the visual-tactile sensor guide assembly 3 includes a main body module 31, and a translucent elastic module 32 is provided at the end of the main body module 31, and a bracket interface 33 is provided on the side of the main body module 31, and the bracket interface 33 is connected to the second end of the robotic arm connector 1, and a surgical instrument interface 34 for connecting a surgical instrument 4 is also provided on the main body module 31.

[0069] It is understandable that the visual-tactile sensing guide device of the orthopedic surgical robot in this embodiment can be optionally installed at an applicable position of the robotic arm according to the configuration of the robotic arm and the requirements of the surgical procedure. For example, in this embodiment, see Figure 1 As shown, the visual-tactile sensor guide device for an orthopedic surgical robot is connected to the end of a robotic arm 2 via a robotic arm connector 1. The hardware components of the visual-tactile sensor guide device for an orthopedic surgical robot primarily include the robotic arm connector 1 and a visual-tactile sensor guide assembly 3. One end of the robotic arm connector 1 is rigidly connected to the robotic arm 2, while the other end is rigidly mounted with the visual-tactile sensor guide assembly 3. The visual-tactile sensor guide assembly 3 includes an interface for connecting to a surgical instrument 4, allowing the surgical instrument 4 to be aligned with the visual-tactile sensor guide assembly 3 in a defined manner and position. The instrument can then be rigidly fixed or subjected to controlled relative motion, depending on the instrument's characteristics and intraoperative operating requirements, for interactive use by the surgeon.

[0070] Specifically, see Figure 2 As shown, the visual-tactile sensor guide assembly 3 consists of a main body module 31, an elastic module 32, a bracket interface 33 and a surgical instrument interface 34. The bracket interface 33 is used to rigidly connect the robotic arm connector 1 and is integrally manufactured on the side of the main body module 31; the main body module 31 is an integrated optical module, and a light-transmitting elastic module 32 is installed at the end. The material of the elastic module 32 is a transparent superelastic colloid, such as resin or silicone; in this embodiment, the surgical instrument interface 34 is a rigid hollow guide cylinder for guiding the Kirschner wire, which has a diameter larger than the Kirschner wire and a possible transition sleeve, and allows the Kirschner wire to rotate around the long axis in the surgical instrument interface 34 and move in both directions along the long axis at the same time, so as to complete the Kirschner wire drilling into the target bone along the guide cylinder. The surgical instrument interface 34 passes through the main body module 31 and the elastic module 32 and terminates at the height of the upper edge of the curved surface part of the elastic module 32, so as to reserve a distance for the curved surface part of the elastic module 32 to deform and sink.

[0071] It is understood that the orthopedic surgical robot visual tactile sensing and guidance device in the above embodiment is a mechanical arm integrated device with sensing and guidance coupling functions. Unlike the currently used "optical navigation and positioning system" sensing and guidance system, it can directly obtain the shape and texture information of the target bone surface through a small number of range data acquisition (rather than a probe collecting one point at a time), complete the calculation of the target bone posture, and solve the problem of "many steps, long time, and easy occlusion" in the registration process of the optical navigation and positioning system, and maintain high resolution through visual tactile sensing. The present invention eliminates the need for tracers on the patient's bones and surgical instruments, as well as the installation of optical cameras on the trolley. Therefore, there is no need to consider the tracer installation position, depth, and stability. It is more adaptable to bones of different sizes and wear conditions and has strong robustness; the sensing and guidance functions are coupled, the integration is high, and the structure is compact; through the visual tactile acquisition method, the interference of liquids (blood, tissue fluid) during pure visual acquisition is effectively avoided, providing greater space for the integration of computer vision and artificial intelligence.

[0072] In some embodiments of the visual-tactile sensing and guiding device for an orthopedic surgical robot provided by the present invention, see Figure 3 As shown, the robot arm connector 1 includes an end flange 11, a connecting bracket 12 and a sensor guide interface 13. The first end face of the end flange 11 is rigidly connected to the robot arm 2, and the second end face of the end flange 11 is connected to the first end of the connecting bracket 12. The connecting bracket 12 is provided with a bending portion to change the connection angle of the robot arm 2. The second end of the connecting bracket 12 constructs the sensor guide interface 13, and the sensor guide interface 13 is rigidly connected to the bracket interface 33.

[0073] In some specific examples, the schematic diagram of the robotic arm connector 1 is as follows: Figure 2 As shown, it mainly consists of an end flange 11, a connecting bracket 12 and a sensor guide interface 13. The end flange 11 is a flange with positioning features and holes. The manipulator arm side of the flange is rigidly connected to the manipulator 2 through fasteners, and the sensor guide side of the flange is integrated with the connecting bracket 12. The connecting bracket 12 extends forward from the end flange 11 and changes the angle of the manipulator arm end, reducing the size of the robot end operating part, while avoiding interference with the patient's anatomical structure or the doctor's line of sight when the operating part moves. The end of the connecting bracket 12 is integrated with a sensor guide interface 13 (the structure in the figure is only schematic), which is a mounting interface with positioning features and is used to rigidly mount the bracket interface 33 of the visual-tactile sensor guide assembly 3, so that the position of the visual-tactile sensor guide assembly 3 relative to the manipulator 2 is known. Figure 2 The connecting bracket 12 shown in the figure is an optional structure. Optionally, the structure, size and angle can be changed according to the actual robot configuration and surgical requirements.

[0074] In other embodiments of the visual-tactile sensing guide device for an orthopedic surgical robot provided by the present invention, Figure 4 and Figure 5 As shown, the main module 31 includes a shell 311, a light-guiding lens 312, a light source 313 and a camera 314, wherein the shell 311 is a wrapping structure with an open end, the elastic module 32 is installed at the open end of the shell 311, and a bracket interface 33 is provided on the outer wall of the shell 311; the light-guiding lens 312 is fixed inside the shell 311 and is located on the side of the elastic module 32 facing the shell 311; the light source 313 is located inside the shell 311, and is used to emit monochromatic stable light toward the elastic module 32; the camera 314 is located inside the shell 311, and is used to capture deformation image information of the elastic module 32.

[0075] It is understandable that the internal structure of the visual tactile sensor guide assembly 3 is as shown in the exploded view. Figure 4 With cross-section Figure 5 As shown. The main module 31 consists of a housing 311, a light guide lens 312, a light source 313, and a camera 314. The housing 311 is the outermost rigid wrapping structure of the main module 31 and has the required waterproof and insulating properties. The light guide lens 312 is a fully transparent light-guiding structure, rigidly fixed within the housing 311 and located above the elastic module 32. The light guide lens 312 supports the elastic module 32 and refracts the light it guides into the field of view of the camera 314. The light source 313 is a ring-shaped LED light board with a circuit board, rigidly fixed above the light guide lens 312, and emits monochromatic, stable light toward the elastic module 32. The camera 314 is a miniature camera used to capture the light refracted by the light guide lens 312. Optionally, the number of cameras 314 can be one or more depending on the shape, structure, and size of the visual-tactile sensor guide assembly 3.

[0076] Furthermore, the elastic module 32 is a light-transmitting solid colloid 321 in the shape of a spherical head (with a curved end) having biocompatibility characteristics that meet the requirements. One end of the light-transmitting solid colloid 321 is rigidly mounted in the shell 311, and the spherical head at the other end is exposed outside the shell 311. The spherical head of the light-transmitting solid colloid 321 is arranged in an array with a plurality of opaque characteristic protrusions 322. The characteristic protrusions 322 are constructed from the surface of the light-transmitting solid colloid 321 toward the interior of the light-transmitting solid colloid 321. When the light-transmitting solid colloid 321 contacts the target surface, the light source 313 illuminates the contact surface through the light-transmitting solid colloid 321, but the characteristic protrusions 322 remain dark. The light-guiding lens 312 refracts the light it guides through the light-transmitting solid colloid 321 into the field of view of the camera 314, which captures the corresponding deformation of the contact side of the light-transmitting solid colloid 321 with the bone surface due to the shape and texture of the target bone surface.

[0077] It should be understood that in the above embodiment, the elastic module 32 is a spherical (with a curved end) translucent solid colloid 321, but the elastic module 32 can also be other colloid shapes. The shape of the elastic module 32 is selected to suit different usage positions and bone surface shapes of the human body, so as to more accurately obtain the corresponding deformation of the contact side of the translucent solid colloid 321 and the bone surface due to the shape and texture of the target bone surface.

[0078] Specifically, according to the design three-dimensional model of the visual-tactile sensor guide assembly 3, the light guide lens 312, the transparent solid colloid 321 and the characteristic protrusions 322 on its surface are manufactured by an integrated 3D printing process with adjustable hardness. Compared with silicone mold manufacturing, its geometric requirements (no bubbles in the transparent solid colloid 321, the color and position of the characteristic protrusions 322, the thickness of the light guide lens 312, etc.) can be better guaranteed. At the same time, based on the input of the patient's preoperative CT image, the patient-customized shape of the transparent solid colloid 321 (Patient Specific Instrumentation) can be designed to optimize the incomplete information capture due to severe wear of the patient's bone surface.

[0079] It should be understood that the visual-tactile sensing guidance device relies on the computer system's extraction of image feature information and pattern learning. During operation, the exposed surface of the elastic module 32 fully contacts the target bone surface and applies stress. The contact side of the transparent solid colloid 321 with the bone surface is squeezed by the shape and texture of the target bone surface and undergoes corresponding deformation, causing the characteristic points (characteristic protrusions 322) to each shift in different directions and distances. The camera 314 captures the two-dimensional image information of the characteristic points after the changes and tracks the offset of the characteristic point cloud, including the size and position changes of the characteristic protrusion 322. Before the surgical system goes online, it is necessary to select a large number of tooling with known surface shape and texture information. The surgical system is required to collect tooling surface information multiple times from multiple angles and forces. The corresponding known surface information and force information are used as the reference truth (ground truth) for neural network-based machine learning. The computer system is required to establish a corresponding mapping relationship between the characteristic point cloud offset two-dimensional image and the source three-dimensional shape. Ultimately, the surgical system obtains the ability to accurately calculate the actual three-dimensional deformation of different areas based on the captured characteristic point cloud offset two-dimensional image, obtain the shape and texture characteristics of the entire contact surface, and achieve three-dimensional reconstruction.

[0080] The present invention provides a visual-tactile sensing guide device for an orthopedic surgical robot, wherein the shape, size, and configuration of the visual-tactile sensing guide assembly 3 can be changed according to the requirements of the robot and the surgical procedure, such as circular, donut-shaped, semicircular, etc.; the size can be freely adjusted according to the bones and the surgical field.

[0081] Optionally, the visual tactile sensor guide assembly 3 can have different shapes and sizes according to the robot trajectory planning and procedure requirements. Figure 6 、 Figure 7 、 Figure 8 and Figure 9 , provides possible embodiments including but not limited to the following during the glenoid needle placement operation, where the surgical instrument 4 is a Kirschner wire and the surgical instrument interface 34 is a Kirschner wire guide cylinder. Figure 6 The main body module 31 and the surgical instrument interface 34 are both axially closed. In this configuration, after the surgical instrument 4 is drilled into the target bone along the long axis of the surgical instrument interface 34, the visual-tactile sensor guide assembly 3 can be withdrawn axially away from the bone. Figure 7 The main module 31 and the surgical instrument interface 34 are depicted with an evacuation slot 341 in the same direction, slightly wider than the diameter of the Kirschner wire and a possible transition sleeve. With this structure, after the surgical instrument 4 is inserted, the visual-tactile sensor guide assembly 3 can be laterally withdrawn from the evacuation slot 341, shortening the motion trajectory of the robotic arm. Figure 8 It is described that for smaller joints (such as shoulder joints, or patients who are children), the surgical field will also be reduced accordingly. In this case, the overall size of the visual-tactile sensor guide assembly 3 can be reduced accordingly, and the proportion of the diameter of the surgical instrument 4 in the overall width will increase, so the shape and size of the main body module 31 can be further adjusted; Figure 9 The structure of the surgical instrument interface 34 outside the main body module 31 is described, and the size of the main body module 31 is greatly reduced when it is desired to expose more critical bone surface areas for surgical fields.

[0082] Optionally, the visual-tactile sensor guide assembly 3 can have different configurations and usage positions according to the intraoperative operation and the adapted surgical instrument. According to the needs of different surgical instruments, corresponding surgical instrument interfaces are selected to couple the sensing and guiding functions, such as surgical instrument interfaces that can be used for drilling, grinding, cutting, etc. The structure of the surgical instrument interface 34 varies according to different surgical instruments, and the position is set at different positions as needed, such as through the interior of the sensor body ( Figure 10 ), attached externally ( Figure 11 ), extending from the outside ( Figure 12 ) etc. As shown in the schematic diagram Figure 10 、 Figure 11 、 Figure 12 , gives possible embodiments of the surgical instrument interface 34 under different usage requirements, including but not limited to.

[0083] In some specific examples, the surgical instrument interface 34 is constructed as a guide hole that passes through the main body module 31 and the elastic module 32 in sequence. Figure 10 The description describes the glenoid needle placement operation, in which the surgical instrument 4 is a Kirschner wire and the surgical instrument interface 34 is a Kirschner wire guide tube. At this time, the surgical instrument interface 34 passes through the elastic module 32, and the surgical instrument 4 drills into the glenoid along the long axis of the surgical instrument interface 34.

[0084] In other specific examples, the surgical instrument interface 34 is a hollow guide cylinder, which is arranged parallel to the outer periphery of the main body module 31 along the length direction of the main body module 31. Figure 11 It is described that in this case, the overall size of the visual-tactile sensor guide assembly 3 is further reduced. At this time, the surgical instrument interface 34 is placed on the outside of the elastic module 32, and the surgical instrument 4 is drilled into the glenoid cavity along the long axis of the surgical instrument interface 34, retaining a larger surgical field in the key area of ​​the bone surface.

[0085] In some other examples, the surgical instrument interface 34 is a guide block, which is connected to the outer peripheral surface of the main body module 31 through a connector. Figure 12 The humeral head osteotomy operation is described, in which the surgical instrument 4 is the saw blade at the end of the oscillating saw of the power tool, and the surgical instrument interface 34 is the guide block of the surgical instrument 4, allowing the surgical instrument 4 to cut along the tooth direction and perform small lateral translation within a limited plane. It is rigidly connected to the main body module 31, and the elastic module 32 continuously captures the information of the non-osteotomy area of ​​the humeral head, and inversely calculates the actual position of the oscillating saw blade, providing real-time guidance and limitation for the osteotomy operation.

[0086] Another aspect of the present invention provides a surgical method of an orthopedic surgical robot, which is adapted to the orthopedic surgical robot visual tactile sensing guide device provided in any of the above embodiments, see Figure 16 As shown, the surgical method of the orthopedic surgical robot includes the following steps S1 to S4.

[0087] S1. Mount the visual-tactile sensor guide device on the orthopedic surgical robot, move the orthopedic surgical robot to the operating position and start it, so that the surgical system of the orthopedic surgical robot intervenes in the surgical process.

[0088] S2. Use the visual-tactile sensor guidance device to reconstruct two-dimensional and three-dimensional data of the patient's target bones and target bone surfaces, and perform preoperative / intraoperative alignment of the visual-tactile sensor guidance device.

[0089] S3. Select a surgical instrument 4 suitable for the surgical operation and install it on the surgical instrument interface 34 of the visual-tactile sensing guide device, and perform orthopedic surgery robot-assisted intraoperative operation based on the data model reconstructed in step S2.

[0090] S4. After all operations requiring the assistance of the orthopedic surgical robot are completed, the orthopedic surgical robot is put on standby and withdraws from the surgical process, and the orthopedic surgical robot-assisted surgery part is completed.

[0091] It is understandable that after the operation begins, first, the doctor team completes the preparation work, places the orthopedic surgical robot equipped with a visual-tactile sensing guide device in a reasonable operating position, starts the orthopedic surgical robot system, and allows the orthopedic surgical robot system to intervene in the surgical process; secondly, the doctor moves the robotic arm in the free drag mode, and according to the instructions, fully contacts the exposed surface of the elastic module 32 with the target bone surface and applies stress to complete the data collection; repeat the collection operation until the collected data is sufficient, and then adjust and confirm the model reconstructed by the system to complete the alignment process; thirdly, the doctor selects the surgical instrument 4 and the surgical instrument interface 34 that are suitable for the surgical operation, and the robotic arm automatically moves to the target posture and adjusts according to the target bone surface information confirmed in the above steps; when the doctor is operating, the visual-tactile sensing guide device monitors and micro-tracks the target bone surface in real time to ensure the accuracy of the doctor's operation; finally, after all operations requiring the assistance of the orthopedic surgical robot are completed, the doctor folds the robotic arm away from the patient, the orthopedic surgical robot stands by, and the surgical process is withdrawn, and the orthopedic surgical robot-assisted surgery part ends.

[0092] The surgical method of the orthopedic surgical robot in the above embodiment consists of interventional surgery, a preoperative / intraoperative alignment process driven by a visual-tactile sensing guide device, an intraoperative operation process driven by a visual-tactile sensing guide device, and a withdrawal system. It is versatile and can be used for high-precision open orthopedic surgeries involving bone surface treatment, such as direct / reverse shoulder replacement, elbow replacement, hip replacement, unicompartmental / total knee replacement, and long bone fractures.

[0093] Specifically, in some examples, the process of performing pre-operative / intra-operative registration of the visual-tactile sensing guidance device in step S2 includes S21 to S29.

[0094] S21. Determine the area around the feature points on the target bone surface and the area involved in the intraoperative operation as the acquisition area.

[0095] S22, pulling the robotic arm to bring the visual-tactile sensing guide device closer to the target bone surface, and adjusting the posture of the visual-tactile sensing guide device so that the elastic module 32 is aligned with the collection area defined in step S21.

[0096] S23 , contacting the elastic module 32 with the acquisition area defined in step S21 , gradually applying stress until the elastic module 32 fully contacts the target bone surface, and acquiring surface data information of the target bone surface.

[0097] S24: Determine whether the target bone surface data information collected in step S23 meets the threshold for intraoperative model reconstruction. If the judgment is no, execute step S25; if the judgment is yes, execute step S26.

[0098] S25. Determine the target bone surface area with insufficient data, add it as a supplementary acquisition area, and repeat steps S22 to S24.

[0099] S26. Reconstruct the target bone and target bone surface of the patient in two dimensions and three dimensions according to the target bone surface data information acquired in step S23.

[0100] S27. The preoperative CT scan and the data model reconstructed in step S26 are simultaneously displayed on the main control trolley display screen, and key view angles and areas are superimposed for comparison to determine whether the reconstructed model result is reliable. If not, execute step S28; if yes, execute step S29.

[0101] S28: Adjust the weights of the reconstructed model and the preoperative CT model, correct the current reconstructed model, form a new fusion reconstructed model, and then re-execute step S27.

[0102] S29. Confirm to the surgical system of the orthopedic surgical robot that the current model is reliable and effective, and complete the preoperative / intraoperative alignment of the visual-tactile sensing guide device.

[0103] Understandably, see Figure 17 As shown in the figure, after the pre-operative / intraoperative registration process begins, first, based on the anatomical structure and preoperative planning, the system determines the areas around the feature points on the target bone surface and the areas involved in the intraoperative operation as the key acquisition areas. Figure 18As shown, a model 81 of the target bone and target bone surface reconstructed from CT will appear on the display screen of the system main console (which can also be an AR / VR display device worn by the doctor or other display device that can be used to display the control interface), and the above-mentioned areas will be highlighted one by one as recommended acquisition areas 82. The position 83 of each acquisition will be highlighted on the screen. When a recommended acquisition area is completed and exceeds the judgment threshold, the next area to be acquired 84 will be highlighted; secondly, in the free drag mode of the robotic arm, the doctor pulls the robotic arm to make the visual-tactile sensing guide device close to the target bone surface, and adjusts the posture of the visual-tactile sensing guide device so that the exposed surface of its elastic module 32 is aligned with the highlighted recommended acquisition area described above; secondly, the doctor holds the robotic arm and fully contacts the elastic module 32 with the recommended acquisition area described above, gradually applies stress until the elastic module 32 fully contacts the bone surface, and maintains it until the system prompts to withdraw after the acquisition is completed; secondly, the system determines whether the currently acquired surface data is sufficient for the intraoperative model reconstruction, that is, whether the amount of data collected meets the threshold for model reconstruction. If it is judged as no, the system determines the area with insufficient data and adds it as a supplementary recommended acquisition area, and then Return to the above steps and supplement data collection; if the judgment is yes, the system will reconstruct the patient's target bones and target bone surfaces in two dimensions and three dimensions based on the collected surface data; secondly, the main console display screen will simultaneously display the reconstructed models of the preoperative CT and intraoperative data, and superimpose and compare key perspectives and areas for the doctor to judge whether the intraoperative reconstruction results are reliable. If the judgment is yes, the doctor will confirm to the system through the system main console that the current model is reliable, and the model will take effect; if the judgment is no, the doctor will directly interact with the system through the system main console, manually adjust the weights of the intraoperative reconstruction model and the preoperative CT model, correct the current reconstruction model, and form a new fusion reconstruction model; secondly, make judgments until the current model is confirmed to be reliable. The doctor will confirm to the system through the system main console that the current model is reliable, and the model will take effect, and the preoperative / intraoperative registration process will end.

[0104] In other examples, the present invention provides an orthopedic surgical robot surgical method, in which the intraoperative operation process in step S3 includes S31 to S39.

[0105] S31 , selecting a surgical instrument 4 and assembling a visual-tactile sensing guide device according to the intraoperative operation to be performed.

[0106] S32. Based on the data model reconstructed in step S2, the surgical system of the orthopedic surgical robot records the posture of the target bone surface. The robotic arm automatically plans the trajectory and moves to the posture of the target bone surface suitable for intraoperative operation, so that the elastic module 32 of the visual-tactile sensing guide device fully contacts the target bone surface.

[0107] S33. The visual-tactile sensing guidance device collects surface information of the target bone surface, obtains the local pose of the collection area on the target bone surface, and calculates the current global pose of the target bone surface.

[0108] S34. Compare the current global posture of the target bone surface calculated in step S33 with the posture of the target bone surface recorded by the surgical system of the orthopedic surgical robot in step S32, and determine whether the two postures are consistent. If not, execute step S35; if yes, execute step S36.

[0109] S35: The robotic arm slightly tracks to a suitable operating posture under the current target bone surface posture, and then proceeds to step S36.

[0110] S36. The doctor begins the intraoperative operation.

[0111] S37. Determine whether the doctor has finished the intraoperative operation in this link. If the judgment is no, execute step S38; if the judgment is yes, execute step S39.

[0112] S38: The doctor has not completed the intraoperative operation in this link, and steps S33 to S37 are executed again.

[0113] S39. The doctor has completed the intraoperative operation of this link, confirmed that the operation is completed, and executed the end operation instruction. The robotic arm drives the visual and tactile sensing guide device to leave the target bone surface.

[0114] Understandably, see Figure 19As shown, after the intraoperative operation process begins, first, the doctor selects a suitable surgical instrument 4 according to the intraoperative operation to be performed (such as needle placement, grinding, osteotomy, etc.), connects it to the visual and tactile sensor guide assembly 3 through the surgical instrument interface 34, and verifies the necessary mobility (rigid connection, translation along the axis, rotation around the axis, etc.); secondly, the system has recorded the posture of the target bone surface at the last moment (if it is the first execution, it is the posture of the target bone surface recorded after the alignment process), the robot arm automatically plans the trajectory and moves to the posture of the above-mentioned target bone surface suitable for the intraoperative operation, so that the elastic module 32 fully contacts the target area; secondly, the sensor guide collects surface information, obtains the local posture of the collection area on the bone surface, and calculates the current global posture of the target bone surface; secondly, due to the influence of the patient himself or the doctor's operation, the target bone surface may move slightly, so The system will obtain the current posture of the target bone surface and compare it with the posture at the previous moment to determine whether the two postures are consistent (if it is the first execution, compare the posture recorded after alignment). If it is judged to be no, the robotic arm will slightly track to the appropriate operation posture under the current target bone surface posture, and then the doctor will start the intraoperative operation. If it is judged to be yes, the doctor will directly start the intraoperative operation; secondly, the system will determine whether the doctor has ended the intraoperative operation of this link. If it is judged to be no, the system has not received the doctor's end operation instruction, and the sensor guide will be used to collect surface information again, and the above steps will be performed in sequence. If it is judged to be yes, the doctor has completed the intraoperative operation of this link, and confirms to the system through the system main control cart that the operation has been completed, executes the end operation instruction, and the intraoperative operation process ends. The robotic arm drives the visual and tactile sensor guide device to automatically or manually leave the bone surface.

[0115] On the other hand, the present invention also provides a surgical system of an orthopedic surgical robot, which is applicable to any one of the above-mentioned embodiments of the orthopedic surgical robot visual tactile sensing guide device and the surgical method of the orthopedic surgical robot. In some embodiments of the surgical system of the orthopedic surgical robot of the present invention, see Figure 13 As shown, the surgical system of the orthopedic surgical robot includes an input information unit 5, a system information processing unit 6 and an output information unit 7, wherein the input information unit 5 is used to obtain external input data that has not been processed by the intraoperative computer system; the system information processing unit 6 is connected to the input information unit 5, and is used to collect and process the data of the input information unit 5; the output information unit 7 is connected to the system information processing unit 6, and is used to obtain the data processing results of the system information processing unit 6 and output a control signal to guide the doctor's decision-making and control the orthopedic surgical robot to assist in completing the operation.

[0116] It can be understood that the surgical system of the orthopedic surgical robot in this embodiment can mainly be adapted to steps S2 and S3 in the surgical method of the above-mentioned orthopedic surgical robot. The data of the input information unit 5 is collected and processed by the system information processing unit 6 and converted into the output information unit 7 to guide the doctor's decision-making and the movement of the surgical robot.

[0117] The external input data obtained by the input information unit 5 refers to the external input data that has not been processed by the intraoperative computer system [I n ]Includes: [I1] Image data obtained from the patient's preoperative CT scan, [I2] Orthopedic surgical robot definition data (kinematics and dynamic parameters), [I3] The patient's actual intraoperative bone surface structure data (key structure exposure, cartilage stripping, and blood influence), [I4] Preoperative planning result data: local posture of key points on the target bone, posture planning of the surgical robot (end) for key points, and [I5] Patient Specific Instrumentation (PSI) customized design results of the patient's anatomical structure obtained by the visual-tactile sensor guide.

[0118] The system information processing unit 6 collects and processes the data of the input information unit 5, which refers to data collection [C n ] and data processing operations [P n ], including: [C1] reconstructing the target bone 3D model from the preoperative CT image and obtaining the local pose of the target bone surface, [C2] the visual-tactile sensor guide collects the surface data of the target bone and records the global pose of the collection point in combination with the robotic arm posture, [C3] determining the key collection area based on the preoperative planning and determining its local pose on the target bone, [P1] data smoothing, image stitching, and matching processing of the collected data with the preoperative CT image, [P2] overlay comparison processing between the reconstructed models, [P3] data visualization processing, [P4] direct model modification processing based on human-computer interaction, and [P5] finding the local pose of the collection area in the target bone surface.

[0119] The control signal output by the output information unit 7 refers to the direct data obtained after the input information unit 5 is converted by the system information processing unit 6 and is used to guide the doctor's decision-making and control the surgical robot to assist in completing the operation. n ], including: [O1] the global pose of the target bone including the target bone surface in the world coordinate system, [O2] the global pose of the bone feature position in the world coordinate system during the intraoperative operation and the trajectory planning results, [O3] the global pose and trajectory planning results of the robot arm during the intraoperative operation, [O p1 ]Intraoperative reconstruction model of target bone and bone surface model, [O p3 Visual overlay comparison results of target bone surface and key acquisition areas in CT reconstruction model and intraoperative reconstruction model, p4 ]The fusion reconstruction model of the target bone and bone surface.

[0120] Specifically, when S2 performs intraoperative / preoperative registration in the surgical method of the orthopedic surgical robot, the surgical systems involved are as follows: Figure 14As shown. The external input data obtained by the input information unit 5 of this part involves [I1] to [I5]. The system information processing unit 6 collects and processes the data of the input information unit 5, involving [C1] to [C3] and [P1] to [P4]. The control signal output by the output information unit 7 involves [O1], [O2], [O p1 ]、[O p3 ] and [O p4 ].

[0121] For detailed procedures, see Figure 14 As shown in the figure: after the registration process starts, the system performs three-dimensional reconstruction of the target bone based on the preoperative CT image [I1], and determines the local position of the target bone surface on the model [C1] based on the preoperative planned surgical operation area [I4]; when executing step S21, the system determines the acquisition area of ​​the sensor guide according to the preoperative planned operation area and the required surgical instruments [I4], determines it as the key area, and determines its local position on the model [C3]; when executing step S23, the doctor uses the patient-customized sensor guide [I5] on the key acquisition area determined by [C3] to perform surface data acquisition [C2] on the patient's actual bone surface [I3], and records the global position [C2] of the acquisition point through the robot's kinematic parameters [I3]; when executing step S26, the system uses the acquisition results of [C1] and [C2], and after data smoothing, adjacent area stitching, and matching with the CT reconstruction model [P1], performs intraoperative reconstruction of the target bone and bone surface model as the intermediate output [O p1 ]; When executing step S27, the system superimposes and compares the CT reconstruction model [C1] and the intraoperative reconstruction model [O p1 ] and the key acquisition area [P2]; visualize the comparison results of [P2] [P3], and output the visualized comparison results as the intermediate output [O p3 ] for the doctor to select and modify; if the comparison results are not negligible, the doctor executes step S28, in [O p3 ] and directly interact (such as dragging) in the model to modify it until the subjective model is determined to be reliable [P4], and obtain the fused reconstruction model [O p4 ]; When executing step S29, based on the fusion reconstruction model [O p4 ], calculate the global pose of the target skeleton [O1], and further obtain the global pose of the operated part and the trajectory planning of the robotic arm [O2], and the registration process is completed.

[0122] Specifically, when S3 performs the intraoperative operation process in the surgical method of the orthopedic surgical robot, the surgical systems involved are as follows: Figure 15The system information processing unit 6 of this part collects and processes the data of the input information unit 5, involving [C2] and [P5], and the control signal output by the output information unit 7 involves [O1], [O2], [O3] and [O p4 ], no input information is involved.

[0123] For detailed procedures, see Figure 15 As shown in the figure: After the operation process starts, when executing step S32, the robot arm moves to the operation posture [O3] according to the global posture of the target bone [O1] and the local posture of the operated part, and the trajectory planning [O2] of the robot arm; when executing step S33, the system collects and obtains the global posture of the collection point [C2], and at the same time reconstructs the model [O p4 ], find the local pose [P5] of the acquisition point in the target bone surface; when executing step S34, the global pose of the acquisition point + the local pose of the acquisition point on the bone surface = the global pose of the target bone surface ([C2]+[P5]=[O1]), and the output information of [O1] and [O2] is updated; when executing step S35, the system re-executes the updated [O3]; the above steps are repeated until the intraoperative operation process ends (step S39), realizing the real-time position tracking of the sensor guide, and the intraoperative operation process ends.

[0124] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A visual-tactile sensing and guiding device for an orthopedic surgical robot, characterized in that: include: A robotic arm connecting member (1), wherein a first end of the robotic arm connecting member (1) is connected to a robotic arm (2) of an orthopedic surgical robot; A visual-tactile sensor guide assembly (3) is used to directly obtain the shape and texture information of a target bone surface so as to complete the calculation of the target bone posture; the visual-tactile sensor guide assembly (3) comprises a main body module (31), an end of the main body module (31) is provided with a light-transmitting elastic module (32), the elastic module (32) is a light-transmitting solid colloid (321), one end of the light-transmitting solid colloid (321) is rigidly installed in the main body module (31), and the other end is exposed outside the main body module (31), the light-transmitting solid colloid (321) 21) The end portion exposed outside the main body module (31) is provided with a plurality of opaque characteristic protrusions (322) distributed in an array, and the characteristic protrusions (322) are constructed from the surface of the light-transmitting solid colloid (321) toward the inside of the light-transmitting solid colloid (321); a support interface (33) is provided on the side of the main body module (31), and the support interface (33) is connected to the second end of the robot arm connecting member (1); and a surgical instrument interface (34) for connecting a surgical instrument (4) is also provided on the main body module (31).

2. The visual-tactile sensing and guiding device for an orthopedic surgical robot according to claim 1, characterized in that: The robot arm connecting member (1) comprises an end flange (11), a connecting bracket (12) and a sensor guide interface (13); the first end face of the end flange (11) is rigidly connected to the robot arm (2); the second end face of the end flange (11) is connected to the first end of the connecting bracket (12); the connecting bracket (12) is provided with a bending portion to change the connection angle of the robot arm (2); the second end of the connecting bracket (12) constructs the sensor guide interface (13); and the sensor guide interface (13) is rigidly matched with the bracket interface (33).

3. The visual-tactile sensing and guiding device for an orthopedic surgical robot according to claim 1, characterized in that: The main body module (31) comprises: A shell (311), the shell (311) being a wrapping structure with one end open, the elastic module (32) being mounted on the open end of the shell (311), and the bracket interface (33) being arranged on the outer wall of the shell (311); A light-guiding lens (312) is fixed inside the housing (311) and is located on a side of the elastic module (32) facing the housing (311); A light source (313) is located inside the housing (311) and is used to emit monochromatic and stable light in the direction of the elastic module (32); A camera (314) is located inside the housing (311) and is used to capture deformation image information of the elastic module (32).

4. The visual-tactile sensing and guiding device for an orthopedic surgical robot according to any one of claims 1 to 3, characterized in that: The surgical instrument interface (34) is constructed as a guide hole that sequentially passes through the main body module (31) and the elastic module (32); the main body module (31) and the elastic module (32) are provided with an evacuation groove (341) in the same side direction, and the surgical instrument interface (34) is constructed at the inner end of the evacuation groove (341); or, The surgical instrument interface (34) is a hollow guide cylinder, and the hollow guide cylinder is arranged parallel to the outer periphery of the main body module (31) along the length direction of the main body module (31); or, The surgical instrument interface (34) is a guide block, and the guide block is connected to the outer peripheral surface of the main body module (31) via a connecting piece.

5. A surgical system of an orthopedic surgical robot, characterized in that: The visual-tactile sensing and guiding device for an orthopedic surgical robot according to any one of claims 1 to 4 comprises: An input information unit (5) is used to obtain external input data that has not been processed by the intraoperative computer system; A system information processing unit (6), connected to the input information unit (5), and used to collect and process data from the input information unit (5); An output information unit (7) is connected to the system information processing unit (6) and is used to obtain the data processing result of the system information processing unit (6) and output a control signal to control the orthopedic surgical robot to assist in completing the surgery.

6. The orthopedic surgical robot surgical system according to claim 5, characterized in that: The external input data acquired by the input information unit (5) includes: image data obtained by CT scanning of the patient before surgery, orthopedic surgical robot definition data, actual intraoperative bone surface structure data of the patient, preoperative planning result data, and customized design results of the patient's anatomical structure acquired by the visual-tactile sensor guide; The system information processing unit (6) collects and processes the data of the input information unit (5), including: reconstructing a target bone 3D model from a preoperative CT image and obtaining the local posture of the target bone surface therein, collecting target bone surface data with a visual tactile sensor guide and recording the global posture of the collection location in combination with the posture of the robotic arm, determining a key collection area based on preoperative planning and determining its local posture on the target bone, data smoothing, image stitching, matching processing of the collected data with the preoperative CT image, superposition and comparison processing between the reconstructed models, data visualization processing, direct model modification processing based on human-computer interaction, and finding the local posture of the collection area on the target bone surface; The control signal output by the output information unit (7) includes: the global position of the target bone including the target bone surface in the world coordinate system, the global position of the bone feature position in the world coordinate system during the intraoperative operation and the trajectory planning result, the global position and trajectory planning result of the robot arm when performing the intraoperative operation, the intraoperative reconstruction model of the target bone and bone surface model, the visual superposition comparison result of the target bone surface and the key acquisition area in the CT reconstruction model and the intraoperative reconstruction model, and the fusion reconstruction model of the target bone and bone surface.

7. The orthopedic surgical robot surgical system according to claim 5, characterized in that: The surgical system of the orthopedic surgical robot is suitable for performing the following surgical methods: S1. Mount the visual-tactile sensor guide device on the orthopedic surgical robot, move the orthopedic surgical robot to the operating position and start it; S2, using the visual-tactile sensor guidance device to perform two-dimensional and three-dimensional data reconstruction of the target bone and the target bone surface to perform registration of the visual-tactile sensor guidance device; S3, selecting a surgical instrument suitable for the surgical operation and installing it on the surgical instrument interface of the visual-tactile sensing guide device, and performing orthopedic surgical robot-assisted operation based on the data model reconstructed in step S2; S4. After all operations requiring the assistance of the orthopedic surgical robot are completed, the orthopedic surgical robot is put on standby and withdraws from the surgical process.

8. The orthopedic surgical robot surgical system according to claim 7, characterized in that: The process of performing registration of the visual-tactile sensing guide device in step S2 includes: S21, determining the area around the feature points on the target bone surface as a collection area; S22, pulling the mechanical arm to bring the visual-tactile sensor guide device close to the target bone surface, and adjusting the posture of the visual-tactile sensor guide device to align the elastic module with the collection area defined in step S21; S23, contacting the elastic module with the collection area defined in step S21, gradually applying stress until the elastic module fully contacts the target bone surface, and collecting surface data information of the target bone surface; S24, judging whether the target bone surface data information collected in step S23 meets the threshold value of model reconstruction, if judged as no, then executing step S25, if judged as yes, then executing step S26; S25, determine the target bone surface area of ​​insufficient data, and increase it as a supplementary acquisition area, re-steps S22 ~ S24; S26, according to the target bone surface data information collected in step S23, the target skeleton and the target bone surface are carried out two-dimensional and three-dimensional data reconstruction; S27, the display screen of the main console simultaneously displays the preoperative CT and the data model reconstructed in step S26, and superimposes and compares the key viewing angles and areas to determine whether the reconstructed model result is reliable. If it is not, execute step S28; if it is yes, execute step S29; S28, adjusting the weights of the reconstructed model and the preoperative CT model, correcting the current reconstructed model, forming a new fusion reconstructed model, and then re-performing step S27; S29. Confirm to the surgical system of the orthopedic surgical robot that the current model is reliable and effective, and complete the alignment of the visual-tactile sensing guide device.

9. The orthopedic surgical robot surgical system according to claim 7, characterized in that: In step S3, the orthopedic surgery robot-assisted operation based on the data model reconstructed in step S2 includes: S31, selecting a surgical instrument to assemble a visual-tactile sensor guide device; S32, based on the data model reconstructed in step S2, the surgical system of the orthopedic surgical robot records the position and posture of the target bone surface, and the robotic arm automatically plans the trajectory and moves to the position and posture of the target bone surface suitable for intraoperative operation, so that the elastic module of the visual-tactile sensing guide device fully contacts the target bone surface; S33, the visual-tactile sensor guidance device collects surface information of the target bone surface, obtains the local position and posture of the collection area on the target bone surface, and calculates the current global position and posture of the target bone surface; S34, comparing the current global posture of the target bone surface calculated in step S33 with the posture of the target bone surface recorded by the surgical system of the orthopedic surgical robot in step S32, and determining whether the two postures are consistent. If the judgment is no, executing step S35; if the judgment is yes, the doctor starts the intraoperative operation; S35, the robotic arm micro-moves and tracks to a suitable operating posture under the current target bone surface posture, and then the doctor starts the intraoperative operation.

Images