Registration method between 3D sensor of spinal robot and DICOM image

Abstract

The present invention relates to the field of image registration technology, and in particular to a registration method between a spinal robot 3D sensor and a DICOM image. The method comprises the following steps: obtaining a DICOM image of a patient's lesion site; performing laser scanning on a calibration plate on the patient's body through a laser radar fusion sensor to obtain calibration plate three-dimensional point cloud data; performing calibration plate spatial fusion on the DICOM image of the patient's lesion site and the calibration plate three-dimensional point cloud data to generate calibration plate fusion three-dimensional point cloud data; obtaining patient posture perception data; performing point cloud division on the calibration plate fusion three-dimensional point cloud data and the site 3D point cloud data to generate source point cloud data and target point cloud data. The present invention improves the accuracy and reliability of registration by utilizing DICOM image processing, laser radar fusion sensor, patient posture perception, robotics technology, and registration optimization model.

Description

Technical Field

[0001] The present invention relates to the technical field of image registration, and in particular to a registration method between a spinal robot 3D sensor and a DICOM image. Background Art

[0002] Advances in the field of medical imaging have laid the foundation for registration technology. From the earliest X-rays to CT (computed tomography), MRI (magnetic resonance imaging) and the latest PET (positron emission tomography) imaging, the resolution and quality of medical images have continued to improve, providing a data basis for accurate registration. Subsequently, the development of robotics technology has made precise manipulation and control possible. Spinal robots, as a special type of medical robot, have the ability to perform precise manipulations on the spine. 3D sensors play a key role in this process, providing data support for subsequent registration by capturing the morphology and position information of the spine in real time. Existing orthopedic robot navigation methods mostly use Canadian NDI cameras. Using a near-infrared camera, this system usually includes a small ball or sensor with an infrared transmitter and receiver on it. The ball is usually fixed to the patient's bone part or surgical instrument. The infrared transmitter emits infrared light signals, which are emitted at a specific frequency and pattern so that they can be captured and processed by the receiver. The robot is then aligned with the CT image by receiving the infrared signal information. However, the current traditional methods are easily affected by factors such as changes in the patient's body shape and robot hand shaking during the alignment process, resulting in inaccurate and unstable alignment results. At the same time, manual intervention or multiple parameter adjustments are required to achieve the ideal alignment effect, which consumes time and human resources, and leads to low alignment accuracy and reliability. Summary of the invention

[0003] Based on this, it is necessary to provide a registration method between a spinal robot 3D sensor and DICOM images to solve at least one of the above technical problems.

[0004] A registration method between a spinal robot 3D sensor and a DICOM image comprises the following steps:

[0005] Step S1: Obtain a DICOM image of the patient's lesion; perform laser scanning on the calibration plate on the patient through a laser radar fusion sensor to obtain the calibration plate's three-dimensional point cloud data; perform calibration plate spatial fusion on the DICOM image of the patient's lesion and the calibration plate's three-dimensional point cloud data to generate calibration plate fused three-dimensional point cloud data;

[0006] Step S2: Acquire the patient's posture perception data; perform point cloud division on the calibration plate fused three-dimensional point cloud data and the part 3D point cloud data to generate source point cloud data and target point cloud data; perform IPC coarse registration on the source point cloud data and the target point cloud data to generate point cloud coarse registration data; perform registration jitter analysis on the point cloud coarse registration data through the patient's posture perception data to generate registration jitter data; compare the registration jitter data with a preset registration jitter threshold, and when the registration jitter data is greater than the preset registration jitter threshold, perform jitter registration optimization on the point cloud coarse registration data until the registration jitter data is less than or equal to the preset registration jitter threshold, and output the point cloud optimized registration data;

[0007] Step S3: acquiring the position data of the spinal robot manipulator; constructing the trajectory to be punctured on the calibration plate based on the position data of the spinal robot manipulator to generate the trajectory data to be punctured;

[0008] Step S4: Use the 3D vision sensor on the spinal robot to analyze the puncture trajectory data and generate spinal robot manipulator posture data; convert the spinal robot manipulator posture data into needle insertion navigation instructions based on the jitter alignment optimization model to execute the spinal robot needle insertion navigation operation.

[0009] The present invention integrates information from different data sources through laser scanning and three-dimensional point cloud technology, establishes a unified spatial reference frame, and provides accurate spatial positioning for subsequent operations. The data is processed and optimized by using registration and optimization algorithms to ensure accurate matching and accuracy of the data, thereby improving the accuracy and efficiency of the operation. Based on the robot position data, a specific operation path is constructed to provide guidance for the robot in subsequent operations, thereby improving the automation and accuracy of the operation. The data is analyzed by a visual sensor to generate the robot's posture data, and the posture data is converted by using an optimization model to ensure the accuracy and stability of the robot in operation. By integrating different data sources, the comprehensive utilization rate of the data and the integrity of the information are improved. The registration and optimization algorithms are used to improve the accuracy and accuracy of the data and reduce operational errors. The optimized data is more in line with the actual situation and helps to improve the success rate and efficiency of subsequent operations. A specific operation path is constructed, and the robot's action trajectory is planned in advance, which helps to reduce unnecessary movement and time waste. The automation of the operation is improved, the burden of manual operation is reduced, and the risk of human error is reduced. The data is analyzed in real time by a visual sensor to ensure timely response to environmental changes during the operation, thereby improving the flexibility and adaptability of the operation. The optimization model is used to convert the posture data, which improves the accuracy and stability of the robot operation and reduces the operation risk and the possibility of failure. Therefore, the present invention improves the accuracy and reliability of registration by using DICOM image processing, laser radar fusion sensor, patient posture perception, robotics technology and registration optimization model.

[0010] Preferably, step S1 comprises the following steps:

[0011] Step S11: using a CT machine to obtain a dicom image of the patient's lesion;

[0012] Step S12: performing laser scanning on the calibration plate on the patient through a laser radar fusion sensor to obtain three-dimensional point cloud data of the calibration plate;

[0013] Step S13: Perform three-dimensional reconstruction on the DICOM image of the patient's lesion site to generate 3D point cloud data of the site;

[0014] Step S14: spatially fuse the calibration plate with the 3D point cloud data of the part and the 3D point cloud data of the calibration plate to generate fused 3D point cloud data of the calibration plate.

[0015] The present invention can accurately determine the position and size of the lesion by fusing the three-dimensional reconstruction data of the CT image with the calibration plate data obtained by laser scanning. The point cloud data of the laser radar can provide additional spatial information, enhance the contrast and details of the image, and the fused three-dimensional point cloud data can be used to generate a more detailed and accurate model. Fusion of CT data with laser radar data can reduce the error caused by a single imaging technology and improve the accuracy and reliability of the image. Through the fusion of the three-dimensional reconstruction data of the CT image and the laser radar scanning data, high-precision three-dimensional point cloud data can be generated, which improves the accuracy and integrity of the overall data. The efficient integration of CT machines and laser radar sensors is achieved, demonstrating the advanced nature of multi-sensor data fusion technology, which helps to promote more cross-domain data integration applications. Compared with data from a single source, the fused three-dimensional point cloud data can more comprehensively reflect the three-dimensional structure of the object and improve the ability and effect of three-dimensional reconstruction. Through the step-by-step data acquisition and processing process, an efficient data processing and fusion process is achieved, which improves work efficiency and the automation level of data processing.

[0016] Preferably, step S13 includes the following steps:

[0017] Step S131: segmenting the lesion area of ​​the patient's DICOM image to generate a lesion area segmentation mask;

[0018] Step S132: using the lesion area segmentation mask to mark the image level of the DICOM image of the patient's lesion part, and generating the patient's lesion part marked image level;

[0019] Step S133: stacking the marked image layers of the patient's lesion site to generate three-dimensional volume data of the patient's lesion site; meshing the surface of the patient's lesion site to generate surface mesh data;

[0020] Step S134: extracting point cloud features from the surface mesh data to generate surface mesh point cloud feature data; reconstructing point cloud features from the surface mesh point cloud feature data to generate part 3D point cloud data.

[0021] The present invention achieves a high degree of automation in the entire process through steps such as segmentation, labeling and fault stacking, reduces manual intervention, and improves data processing efficiency and consistency. From lesion area segmentation to three-dimensional point cloud data generation, each step ensures the accuracy and detail retention of three-dimensional modeling, and provides a high-quality three-dimensional model. The image level labeling and fault stacking steps enable the data to be detailed labeled and combined at different levels and dimensions, which helps to generate more detailed and rich three-dimensional volume data. Through surface meshing and point cloud feature extraction, the data is optimized, making the final generated 3D point cloud data more compact and orderly in structure, which is convenient for subsequent processing and analysis. The feature extraction step can identify and retain important features in the surface mesh data, improving the accuracy and detail expression of point cloud reconstruction. The generated 3D point cloud data format is standardized, easy to exchange and integrate with other systems or software, and improves the compatibility and availability of data. Through precise image segmentation and reconstruction technology, image data is effectively utilized, the processing requirements of redundant information are reduced, and the use of computing resources is optimized.

[0022] Preferably, step S14 comprises the following steps:

[0023] Step S141: photographing the calibration plate with an infrared 2D camera based on the 3D point cloud data of the calibration plate to obtain a 2D image of the calibration plate;

[0024] Step S142: extracting calibration plate morphology information from the calibration plate 2D image to obtain calibration plate morphology information data, wherein the calibration plate morphology information data includes calibration plate height data and calibration plate shape data;

[0025] Step S143: segmenting the calibration plate imaging area of ​​the calibration plate 3D point cloud data using the calibration plate height data and the calibration plate shape data to generate calibration plate area data;

[0026] Step S144: Perform CT coordinate fusion comparison on the calibration plate area data through the part 3D point cloud data to generate calibration plate fused 3D point cloud data.

[0027] The present invention realizes the fusion of multi-dimensional data by combining the images taken by the infrared 2D camera and the three-dimensional point cloud data, and provides a richer and more accurate data representation. By extracting the morphological information (height and shape) of the calibration plate, the calibration plate area can be accurately identified and segmented, thereby improving the accuracy and reliability of the overall data processing. The use of morphological information to segment the imaging area of ​​the calibration plate ensures the accuracy and consistency of the segmentation results, laying a solid foundation for subsequent data fusion. Through CT coordinate fusion and comparison, point cloud data from different sources can be efficiently compared and integrated, improving the efficiency and accuracy of data processing. Fusion of data information from different sources (such as 2D image data and 3D point cloud data) helps to generate more complete and detailed three-dimensional point cloud data. By extracting and utilizing the morphological information of the calibration plate, the accuracy of the three-dimensional point cloud data of the calibration plate is significantly improved, providing high-precision basic data for subsequent applications.

[0028] Preferably, step S2 comprises the following steps:

[0029] Step S21: Acquire patient posture perception data;

[0030] Step S22: performing data preprocessing on the calibration plate fused 3D point cloud data and the part 3D point cloud data to generate standard calibration plate fused 3D point cloud data and standard part 3D point cloud data, wherein the data preprocessing includes data normalization, data denoising and data enhancement;

[0031] Step S23: performing point cloud division on the fused 3D point cloud data of the standard calibration plate and the 3D point cloud data of the standard part to generate source point cloud data and target point cloud data;

[0032] Step S24: performing IPC coarse registration on the source point cloud data and the target point cloud data to generate point cloud coarse registration data; performing registration jitter analysis on the point cloud coarse registration data through the patient's body posture perception data to generate registration jitter data;

[0033] Step S25: comparing the registration jitter data with a preset registration jitter threshold, and when the registration jitter data is greater than the preset registration jitter threshold, performing jitter registration optimization according to the source point cloud coarse registration data and the target point cloud coarse registration data to generate a jitter registration optimization model;

[0034] Step S26: importing the point cloud coarse registration data into the jitter registration optimization model to perform jitter registration optimization until the registration jitter data is less than or equal to a preset registration jitter threshold, and outputting the point cloud optimized registration data.

[0035] The present invention ensures high-precision registration of source point cloud data and target point cloud data and reduces errors by acquiring patient posture perception data and fine jitter registration analysis. Through data normalization, denoising and data enhancement, the standardized processing of data is ensured, the quality and consistency of point cloud data are improved, and a solid foundation is laid for subsequent processing. The denoising step in the data preprocessing process effectively reduces the noise in the point cloud data and improves the clarity and reliability of the overall data. Through data enhancement technology, more sample data can be generated, and the robustness and adaptability of point cloud data under different conditions are enhanced. The point cloud division and rough registration steps ensure the refined processing of point cloud data, refine each step of data processing, and improve the overall processing effect. The jitter in the registration process is analyzed in real time through posture perception data, ensuring the dynamic adjustment and optimization of data in the registration process. The introduction of registration jitter analysis and jitter registration optimization model makes the registration algorithm more optimized and can effectively deal with various jitter problems in the registration process. Through standardization and preprocessing steps, the amount and complexity of calculation can be reduced, the overall calculation efficiency can be improved, and the data processing time can be shortened.

[0036] Preferably, performing IPC coarse registration on the source point cloud data and the target point cloud data includes:

[0037] Extract corner feature of source point cloud data and target point cloud data to obtain source point cloud corner feature points and target point cloud corner feature points;

[0038] Match the feature points of the source point cloud corner points with the target point cloud corner points to generate an initial point cloud transformation matrix, where the initial point cloud transformation matrix includes a point cloud rotation matrix and a point cloud translation vector;

[0039] Perform preliminary translation transformation on the source point cloud data and the target point cloud data through the point cloud rotation matrix and the point cloud translation vector to generate preliminary point cloud registration transformation data;

[0040] Performing registration distance measurement on the preliminary point cloud registration transformation data to generate preliminary point cloud registration distance data; performing ICP registration parameter setting according to the preliminary point cloud registration distance data to generate ICP registration parameter data;

[0041] Based on the ICP algorithm, the registration position of the preliminary point cloud registration transformation data is adjusted according to the ICP registration parameter data to generate point cloud coarse registration data.

[0042] The present invention can effectively capture key feature points in the source point cloud and the target point cloud through corner feature extraction, ensuring the accuracy of subsequent matching and registration. The initial point cloud transformation matrix is ​​generated by feature point matching, so that the preliminary translation and rotation transformation of the source point cloud data and the target point cloud data is more efficient, and the preliminary alignment effect is better. The preliminary translation transformation step is roughly aligned before registration, which reduces the complexity of subsequent iterative calculations of the ICP algorithm and improves the overall calculation efficiency. The preliminary point cloud registration distance data generated by the registration distance metric can be used to dynamically adjust the ICP registration parameters to ensure the accuracy and stability of the ICP algorithm. The ICP registration parameters are set according to the preliminary point cloud registration distance data to ensure the parameter optimization of the ICP algorithm during fine registration and improve the final registration effect. The ICP algorithm iteratively optimizes the preliminary point cloud registration transformation data to make the registration position adjustment more accurate, and the point cloud rough registration data finally generated has higher registration accuracy. Through the preliminary translation transformation of the rotation matrix and the translation vector, the spatial consistency of the source point cloud data and the target point cloud data is significantly enhanced, which is convenient for subsequent processing. The introduction of the preliminary registration step reduces the error accumulation caused by the direct application of the ICP algorithm and improves the stability and reliability of the overall registration process.

[0043] Preferably, performing registration jitter analysis on the point cloud coarse registration data using the patient posture perception data includes:

[0044] Performing patient fluctuation curve analysis on the patient's posture perception data to generate a patient fluctuation curve;

[0045] Performing ups and downs curvature analysis on the patient's ups and downs curve to generate the patient's ups and downs curvature data; performing change cycle analysis on the patient's ups and downs curvature data to generate the patient's ups and downs change cycle data;

[0046] The patient's breathing pattern is divided according to the patient's fluctuation cycle data to obtain the patient's breathing pattern data; the patient's breathing pattern data is collected in time series point cloud to obtain time series point cloud data;

[0047] The time series point cloud data and the patient's fluctuation curve are aligned with the respiratory phase to generate the respiratory phase aligned point cloud data; the registration difference of the point cloud coarse registration data is calculated through the respiratory phase aligned point cloud data to generate the registration jitter data.

[0048] The present invention can more accurately understand and align the patient's body changes under different respiratory phases by analyzing the patient's undulation curve and breathing pattern, thereby improving the overall accuracy of point cloud registration. Time series point cloud acquisition and alignment are performed according to the patient's breathing pattern, ensuring that the registration process can dynamically adapt to the patient's body changes and reducing the registration error caused by breathing. By aligning the point cloud data by respiratory phase to calculate the registration difference, the registration jitter caused by the patient's breathing can be effectively identified and corrected, thereby reducing the registration error and improving the registration quality. The introduction of registration jitter analysis improves the robustness of the system, enabling it to cope with dynamic changes caused by breathing and body changes, and ensuring the stability of the registration results. Steps such as undulation curvature analysis and change cycle analysis help identify and process subtle body changes during breathing, ensuring that the registration process can capture and adjust these details. By aligning the point cloud data by respiratory phase, the registration process is optimized, enabling it to efficiently process and align point cloud data in different time series, thereby improving the overall registration efficiency. The respiratory phase alignment step ensures that the data collected under different respiratory phases are consistent, thereby improving the coherence and consistency of the data during the registration process. The collection and processing of time series point cloud data enables the system to more accurately track and analyze the patient's body posture changes, improving the timeliness and accuracy of the registration data.

[0049] Preferably, step S25 includes the following steps:

[0050] Step S251: comparing the registration jitter data with a preset registration jitter threshold, and when the registration jitter data is greater than the preset registration jitter threshold, performing normal vector calculation on the source point cloud coarse registration data and the target point cloud coarse registration data according to the point feature histogram to obtain normal vector data;

[0051] Step S252: performing neighborhood search on the source point cloud coarse registration data and the target point cloud coarse registration data to generate a neighborhood point set; performing PFH calculation on the source point cloud coarse registration data and its corresponding neighborhood point set and the target point cloud coarse registration data and its corresponding neighborhood point set through normal vector data to obtain a neighborhood feature descriptor;

[0052] Step S253: construct a matrix based on the source point cloud coarse registration data and the target point cloud coarse registration data to obtain a 4k-dimensional neighborhood matrix; perform a descriptor low-dimensional transformation on the 4k-dimensional neighborhood matrix through a preset PointNet model to generate a 32-dimensional feature descriptor;

[0053] Step S254: Optimize the preset PointNet model through the 32-dimensional feature descriptor and the neighborhood feature descriptor to generate a jitter registration optimization model.

[0054] The present invention can identify and refine the errors existing in the registration process by comparing the registration jitter data with the preset threshold and calculating the normal vector, thereby improving the overall registration accuracy. The point feature histogram and the normal vector calculation steps ensure that more accurate and representative feature information is extracted from the point cloud data, providing a solid foundation for subsequent registration optimization. The neighborhood search generates a neighborhood point set, so that the registration process can better capture the local features of the point cloud data and improve the accuracy of feature matching. The neighborhood feature descriptor is obtained by PFH calculation, which provides detailed local geometric information and enhances the richness and accuracy of feature description. The 4k-dimensional neighborhood matrix is ​​transformed into a low-dimensional matrix using the preset PointNet model to generate a 32-dimensional feature descriptor, which reduces the computational complexity and improves the feature matching efficiency. The PointNet model is optimized by the 32-dimensional feature descriptor and the neighborhood feature descriptor, so that the model can more accurately identify and correct the registration jitter and generate a better jitter registration optimization model. The 32-dimensional feature descriptor after feature dimensionality reduction occupies less computing resources in the feature matching process, thereby improving the matching speed and efficiency. The introduction and optimization of the PointNet model makes the system adaptable to different scenarios and data, and can handle various complex registration problems.

[0055] Preferably, step S3 comprises the following steps:

[0056] Step S31: Acquire the position data of the spinal robot manipulator;

[0057] Step S32: coordinate mapping the calibration plate based on the manipulator position data of the spine robot to generate manipulator-calibration plate mapping coordinate data;

[0058] Step S33: construct the trajectory to be punctured based on the DICOM image of the patient's lesion site and the coordinate data of the manipulator-calibration plate mapping to generate the trajectory data to be punctured.

[0059] The present invention can achieve real-time monitoring and positioning of the manipulator by acquiring the position data of the spinal robot manipulator, thereby achieving accurate navigation of the surgical operation. By using the spinal robot manipulator position data to map the coordinates of the calibration plate, the spatial position relationship between the manipulator and the calibration plate can be accurately converted into coordinate data, providing a basis for subsequent operations. According to the DICOM image of the patient's lesion site, combined with the manipulator-calibration plate mapping coordinate data, the puncture trajectory can be constructed and planned to ensure the accuracy and safety of the surgical operation. During the construction of the puncture trajectory, it can be adjusted and optimized according to the real-time manipulator position data to cope with changes in the patient's anatomical structure or the uncertainty of the surgical environment. By pre-constructing the puncture trajectory, misoperation during surgery and puncture deviation from the target can be avoided.

[0060] Preferably, step S4 comprises the following steps:

[0061] Step S41: using the 3D visual sensor on the spinal robot to perform spinal robot manipulator posture analysis on the puncture trajectory data to be processed, and generating spinal robot manipulator posture data;

[0062] Step S42: performing hand-eye calibration on the spinal robot manipulator posture data based on the jitter registration optimization model to generate spinal robot hand-eye calibration data;

[0063] Step S43: converting the puncture trajectory data into needle insertion navigation instructions according to the spinal robot hand-eye calibration data to execute the spinal robot needle insertion navigation operation.

[0064] The present invention analyzes the puncture trajectory data through the 3D visual sensor on the spinal robot, and can obtain the position data of the spinal robot manipulator in real time, helping doctors monitor the position and posture of the manipulator during the operation. Based on the jitter registration optimization model, the manipulator position data is calibrated by hand and eye, and the hand-eye relationship of the spinal robot can be accurately calculated, which improves the accuracy and stability of surgical navigation. The spinal robot hand-eye calibration data is used to convert the puncture trajectory data into needle insertion navigation instructions, and the navigation instructions can be accurately converted into motion instructions for the manipulator, which improves the precision and accuracy of navigation. The spinal robot performs the needle insertion navigation operation according to the converted navigation instructions, realizes the automation of surgical navigation, reduces the interference and errors of human operation, and improves the safety of the operation. Through accurate hand-eye calibration and navigation instruction conversion, the adjustment and correction of navigation during the operation can be reduced, and the complexity of the operation can be reduced.

[0065] The beneficial effect of the present invention is to obtain the 3D model data of the target object by using a 3D scanning device, and use a laser radar or other sensor to scan the calibration plate fixed in the working area to generate the 3D point cloud data of the calibration plate. The 3D model data of the target object and the 3D point cloud data of the calibration plate are spatially fused to generate the fused 3D point cloud data of the calibration plate. The goal of this step is to align the virtual model of the target object with the actual physical calibration data to ensure accurate positioning. The sensor is used to obtain the data of the working environment, such as the position of the robot's workbench and the position of the surrounding objects. The fused 3D point cloud data of the calibration plate and the 3D point cloud data of the target object are divided to generate source point cloud data and target point cloud data. The source point cloud data and the target point cloud data are roughly registered to generate point cloud coarse registration data. The point cloud coarse registration data is jitter analyzed using the working environment data to generate registration jitter data. The registration jitter data is compared with a preset jitter threshold. If the registration jitter data exceeds the threshold, the jitter registration optimization is performed until the jitter data is within the threshold range. Output the final point cloud optimization registration data. Obtain the current position data of the spinal robot manipulator. Based on the position data of the manipulator, construct the trajectory to be operated on the calibration plate to generate the trajectory data to be operated. Use the 3D visual sensor on the spinal robot to analyze the manipulator's posture on the trajectory data to be operated, and generate the manipulator's posture data. Through accurate 3D scanning data and point cloud data fusion, jitter optimization and navigation command conversion, ensure that the spinal robot can accurately locate and perform operation tasks. Accurate registration and navigation reduce the operational risks caused by errors. Automated processes and precise navigation reduce operation time and improve efficiency. Through the perception and analysis of working environment data, adjustments can be made for different working environments to improve the personalization and accuracy of surgery. Therefore, the present invention improves the accuracy and reliability of registration by utilizing DICOM image processing, lidar fusion sensors, patient posture perception, robotics technology, and registration optimization models. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 A schematic diagram of the steps of a registration method between a spinal robot 3D sensor and a DICOM image;

[0067] Figure 2 for Figure 1 Detailed implementation steps of step S2 in the flowchart;

[0068] Figure 3 for Figure 2 Detailed implementation steps of step S25;

[0069] Figure 4 for Figure 1 Detailed implementation steps of step S3 in FIG.

[0070] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0071] The technical method of the present invention is described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by technicians in this field without creative work are within the scope of protection of the present invention.

[0072] In addition, the accompanying drawings are only schematic illustrations of the present invention and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. The functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor methods and / or microcontroller methods.

[0073] It should be understood that, although the terms "first", "second", etc. may be used herein to describe various units, these units should not be limited by these terms. These terms are used only to distinguish one unit from another unit. For example, without departing from the scope of the exemplary embodiments, the first unit may be referred to as the second unit, and similarly the second unit may be referred to as the first unit. The term "and / or" used herein includes any and all combinations of one or more of the listed associated items.

[0074] To achieve this, please refer to Figures 1 to 4 , a registration method between a spinal robot 3D sensor and a DICOM image, comprising the following steps:

[0075] Step S1: Obtain a DICOM image of the patient's lesion; perform laser scanning on the calibration plate on the patient through a laser radar fusion sensor to obtain the calibration plate's three-dimensional point cloud data; perform calibration plate spatial fusion on the DICOM image of the patient's lesion and the calibration plate's three-dimensional point cloud data to generate calibration plate fused three-dimensional point cloud data;

[0076] Step S2: Acquire the patient's posture perception data; perform point cloud division on the calibration plate fused three-dimensional point cloud data and the part 3D point cloud data to generate source point cloud data and target point cloud data; perform IPC coarse registration on the source point cloud data and the target point cloud data to generate point cloud coarse registration data; perform registration jitter analysis on the point cloud coarse registration data through the patient's posture perception data to generate registration jitter data; compare the registration jitter data with a preset registration jitter threshold, and when the registration jitter data is greater than the preset registration jitter threshold, perform jitter registration optimization on the point cloud coarse registration data until the registration jitter data is less than or equal to the preset registration jitter threshold, and output the point cloud optimized registration data;

[0077] Step S3: acquiring the position data of the spinal robot manipulator; constructing the trajectory to be punctured on the calibration plate based on the position data of the spinal robot manipulator to generate the trajectory data to be punctured;

[0078] Step S4: Use the 3D vision sensor on the spinal robot to analyze the puncture trajectory data and generate spinal robot manipulator posture data; convert the spinal robot manipulator posture data into needle insertion navigation instructions based on the jitter alignment optimization model to execute the spinal robot needle insertion navigation operation.

[0079] The present invention integrates information from different data sources through laser scanning and three-dimensional point cloud technology, establishes a unified spatial reference frame, and provides accurate spatial positioning for subsequent operations. The data is processed and optimized by using registration and optimization algorithms to ensure accurate matching and accuracy of the data, thereby improving the accuracy and efficiency of the operation. Based on the robot position data, a specific operation path is constructed to provide guidance for the robot in subsequent operations, thereby improving the automation and accuracy of the operation. The data is analyzed by a visual sensor to generate the robot's posture data, and the posture data is converted by using an optimization model to ensure the accuracy and stability of the robot in operation. By integrating different data sources, the comprehensive utilization rate of the data and the integrity of the information are improved. The registration and optimization algorithms are used to improve the accuracy and accuracy of the data and reduce operational errors. The optimized data is more in line with the actual situation and helps to improve the success rate and efficiency of subsequent operations. A specific operation path is constructed, and the robot's action trajectory is planned in advance, which helps to reduce unnecessary movement and time waste. The automation of the operation is improved, the burden of manual operation is reduced, and the risk of human error is reduced. The data is analyzed in real time by a visual sensor to ensure timely response to environmental changes during the operation, thereby improving the flexibility and adaptability of the operation. The optimization model is used to convert the posture data, which improves the accuracy and stability of the robot operation and reduces the operation risk and the possibility of failure. Therefore, the present invention improves the accuracy and reliability of registration by using DICOM image processing, laser radar fusion sensor, patient posture perception, robotics technology and registration optimization model.

[0080] In the embodiment of the present invention, reference Figure 1 The above is a schematic diagram of the steps of a registration method between a spinal robot 3D sensor and a DICOM image of the present invention. In this example, the registration method between a spinal robot 3D sensor and a DICOM image includes the following steps:

[0081] Step S1: Obtain a DICOM image of the patient's lesion; perform laser scanning on the calibration plate on the patient through a laser radar fusion sensor to obtain the calibration plate's three-dimensional point cloud data; perform calibration plate spatial fusion on the DICOM image of the patient's lesion and the calibration plate's three-dimensional point cloud data to generate calibration plate fused three-dimensional point cloud data;

[0082] In an embodiment of the present invention, the lesion of the patient is scanned by using a CT (computed tomography) or MRI (magnetic resonance imaging) device to obtain image data in dicom format. The dicom image is preprocessed using professional software (such as OsiriX, 3D Slicer, etc.), including operations such as image denoising and contrast enhancement. The calibration plate is fixed at a specified position on the patient's body to ensure that the calibration plate remains stable and motionless during the scanning process. The calibration plate is three-dimensionally scanned using a laser radar fusion sensor (such as LiDAR) to obtain the three-dimensional point cloud data of the calibration plate. The acquired three-dimensional point cloud data is stored in a computer for subsequent processing. The preprocessed dicom image and three-dimensional point cloud data are respectively imported into three-dimensional image processing software (such as MATLAB, MeshLab, CloudCompare, etc.). In the three-dimensional point cloud data, the point cloud information of the calibration plate is detected and extracted. This is usually achieved by identifying specific marking points on the calibration plate. The respective coordinate systems of the dicom image and the three-dimensional point cloud data of the calibration plate are determined, and the spatial alignment of the two is achieved through specific marking points on the calibration plate. Through the spatial transformation matrix, the DICOM image and the 3D point cloud data of the calibration plate are fused in the same coordinate system to generate fused 3D point cloud data containing the calibration plate information. Check the fusion result to ensure that the positional relationship between the calibration plate and the lesion is accurate. If there is a deviation, adjust the transformation parameters to correct it. Export the fused 3D point cloud data into common 3D data formats (such as PLY, OBJ, STL, etc.) for subsequent analysis and processing.

[0083] Step S2: Acquire the patient's posture perception data; perform point cloud division on the calibration plate fused three-dimensional point cloud data and the part 3D point cloud data to generate source point cloud data and target point cloud data; perform IPC coarse registration on the source point cloud data and the target point cloud data to generate point cloud coarse registration data; perform registration jitter analysis on the point cloud coarse registration data through the patient's posture perception data to generate registration jitter data; compare the registration jitter data with a preset registration jitter threshold, and when the registration jitter data is greater than the preset registration jitter threshold, perform jitter registration optimization on the point cloud coarse registration data until the registration jitter data is less than or equal to the preset registration jitter threshold, and output the point cloud optimized registration data;

[0084] In an embodiment of the present invention, a body data of a patient is collected by using a body sensing device (such as a 3D human body scanner, a motion capture system, etc.) to generate a three-dimensional body model. The collected body sensing data is preprocessed, including operations such as denoising, alignment, and simplification, for subsequent processing. The calibration plate is imported to fuse the three-dimensional point cloud data and the 3D point cloud data of the patient's lesion site. The point cloud data is divided using an algorithm, and the point cloud data related to the calibration plate is used as the source point cloud data, and the point cloud data of the patient's lesion site is used as the target point cloud data. The initial parameters of the ICP (Iterative Closest Point) algorithm are set, including the maximum number of iterations, the convergence tolerance, etc. The source point cloud data and the target point cloud data are coarsely registered by ICP to generate point cloud coarse registration data. The patient's body sensing data is used to perform jitter analysis on the coarsely registered point cloud data to calculate the registration jitter data. The calculated registration jitter data is compared with the preset registration jitter threshold. If the registration jitter data is greater than a preset threshold, the point cloud coarse registration data is optimized for jitter registration until the registration jitter data is less than or equal to the threshold, and the point cloud optimized registration data is output.

[0085] Step S3: acquiring the position data of the spinal robot manipulator; constructing the trajectory to be punctured on the calibration plate based on the position data of the spinal robot manipulator to generate the trajectory data to be punctured;

[0086] In an embodiment of the present invention, the real-time position data of the manipulator during the operation is obtained by using the interface or API of the spinal robot system. The position data generally includes the 3D coordinates and posture information (such as direction and angle) of the manipulator. The position data of the manipulator is formatted into a suitable structure (such as an array or a data frame) for subsequent processing. The specific position and coordinate system of the calibration plate during the operation is determined, which can be obtained through the previous point cloud data registration step. The position data of the manipulator is converted into the coordinate system of the calibration plate to ensure that the trajectory construction is based on the same reference frame. Based on the position data of the manipulator, the trajectory to be punctured is constructed. The trajectory to be punctured can be designed according to the surgical plan and the location of the lesion to ensure the safety and effectiveness of the puncture path. The data of the trajectory to be punctured is saved in a suitable format (such as JSON, CSV or a custom format) for use by the robot control system.

[0087] Step S4: Use the 3D vision sensor on the spinal robot to analyze the puncture trajectory data and generate spinal robot manipulator posture data; convert the spinal robot manipulator posture data into needle insertion navigation instructions based on the jitter alignment optimization model to execute the spinal robot needle insertion navigation operation.

[0088] In an embodiment of the present invention, a 3D vision sensor (such as a stereo camera, a structured light sensor, etc.) installed on a spinal robot is used to scan the puncture trajectory in real time to obtain the 3D posture data of the manipulator. The data of the 3D vision sensor is processed using a computer vision algorithm to extract the feature points of the manipulator and the target position. This can be achieved through libraries such as OpenCV and PCL. According to the data of the feature points, the posture (including position and posture) of the manipulator is calculated. The PnP (Perspective-n-Point) algorithm or other posture estimation algorithms are usually used for calculation. The calculated posture data is formatted into a suitable structure (such as an array or a data frame) for subsequent processing. The jitter registration optimization model generated in the previous step is loaded to optimize the posture data. The optimized posture data is converted into a needle insertion navigation instruction. This step usually involves converting the posture data into an instruction format that can be understood by the robot control system. The generated needle insertion navigation instruction is sent to the control system of the spinal robot to guide the manipulator to perform the needle insertion operation. During the needle insertion process, the position and posture of the manipulator are monitored in real time, and adjusted as needed to ensure the accuracy and safety of the needle insertion.

[0089] Preferably, step S1 comprises the following steps:

[0090] Step S11: using a CT machine to obtain a dicom image of the patient's lesion;

[0091] Step S12: performing laser scanning on the calibration plate on the patient through a laser radar fusion sensor to obtain three-dimensional point cloud data of the calibration plate;

[0092] Step S13: Perform three-dimensional reconstruction on the DICOM image of the patient's lesion site to generate 3D point cloud data of the site;

[0093] Step S14: spatially fuse the calibration plate with the 3D point cloud data of the part and the 3D point cloud data of the calibration plate to generate fused 3D point cloud data of the calibration plate.

[0094] In an embodiment of the present invention, the patient is placed on a CT scanning bed to ensure that the lesion is within the scanning range. The CT scanning instrument scans the patient according to preset parameters (such as scanning layer thickness, interval, etc.). After the scan is completed, the CT machine generates image data in DICOM format, including cross-sectional images and related marking information. A calibration plate is placed near the lesion on the patient to ensure that it can be captured by the CT scan and the laser radar scan at the same time. The calibration plate is scanned using a laser radar fusion sensor, and the laser beam is reflected back to the sensor to generate three-dimensional point cloud data on the surface of the calibration plate. It is ensured that the point cloud data on the surface of the calibration plate accurately captures the shape and position of the calibration plate. The DICOM image of the patient is processed using medical image processing software or a custom algorithm. The DICOM image can be converted into a three-dimensional model using methods such as volume rendering and voxel filling. The finally generated three-dimensional model can be represented as point cloud data composed of a large number of points, which represent the surface morphology of the patient's lesion. The three-dimensional point cloud data of the patient's lesion is registered with the three-dimensional point cloud data of the calibration plate using a registration algorithm. The registration process includes the rigid or non-rigid transformation of the point cloud to make the coordinate system and spatial position of the two consistent. The final generated three-dimensional point cloud data fused with the calibration plate contains the spatial information of the patient's lesion and the calibration plate in the same coordinate system.

[0095] Preferably, step S13 includes the following steps:

[0096] Step S131: segmenting the lesion area of ​​the patient's DICOM image to generate a lesion area segmentation mask;

[0097] Step S132: using the lesion area segmentation mask to mark the image level of the DICOM image of the patient's lesion part, and generating the patient's lesion part marked image level;

[0098] Step S133: stacking the marked image layers of the patient's lesion site to generate three-dimensional volume data of the patient's lesion site; meshing the surface of the patient's lesion site to generate surface mesh data;

[0099] Step S134: extracting point cloud features from the surface mesh data to generate surface mesh point cloud feature data; reconstructing point cloud features from the surface mesh point cloud feature data to generate part 3D point cloud data.

[0100] In an embodiment of the present invention, the DICOM image of the patient's lesion site is segmented by using medical image processing software or a custom algorithm. The segmentation process includes image processing methods such as threshold segmentation, region growing, and edge detection to determine the boundary of the lesion area and generate a segmentation mask. The DICOM image of the patient's lesion site is hierarchically marked using the lesion area segmentation mask. The hierarchical marking can be graded according to the degree, density and other attributes of the lesion for subsequent analysis and processing. The marked image levels are stacked to obtain three-dimensional volume data of the patient's lesion site. The three-dimensional volume data is surface meshed to generate surface mesh data of the patient's lesion site. Surface meshing includes steps such as voxel-to-surface conversion and meshing algorithm to generate surface mesh data with a certain density and geometric shape. Point cloud feature extraction is performed on the surface mesh data, including calculation of surface normals, curvatures, feature points, etc. Based on the extracted point cloud feature data, point cloud reconstruction is performed to generate 3D point cloud data of the patient's lesion site. The point cloud reconstruction process includes steps such as filtering, sampling, and reconstruction algorithms of the point cloud to obtain 3D point cloud data of the part with a certain accuracy and density. More specifically, image segmentation algorithms such as threshold segmentation, region growing, or convolutional neural networks are used to identify the lesion area. The segmentation result can be represented as a binary mask, in which the lesion area is marked as white and the non-lesion area is marked as black. The lesion area segmentation mask is superimposed on the original DICOM image. According to the properties such as the location, shape, and density of the lesion area, the lesion area is divided into different levels or marks. The level marks can be represented by different colors or grayscale values ​​for subsequent analysis and visualization. The marked image levels are stacked to generate three-dimensional volume data along the Z-axis direction of the DICOM image. The three-dimensional volume data is converted into mesh data representing the surface of the patient's lesion area using a volume rendering algorithm or a voxel filling method. The meshing process needs to take into account parameters such as voxel size and surface smoothness to obtain surface mesh data with sufficient details and accuracy. Point cloud feature extraction is performed on the surface mesh data, including calculation of surface normals, curvatures, surface feature points, etc. According to the extracted point cloud feature data, a point cloud reconstruction algorithm (such as a grid-based point cloud reconstruction algorithm or a voxel-to-point cloud conversion algorithm) is used to convert the surface mesh data into 3D point cloud data of the part.

[0101] Preferably, step S14 comprises the following steps:

[0102] Step S141: photographing the calibration plate with an infrared 2D camera based on the 3D point cloud data of the calibration plate to obtain a 2D image of the calibration plate;

[0103] Step S142: extracting calibration plate morphology information from the calibration plate 2D image to obtain calibration plate morphology information data, wherein the calibration plate morphology information data includes calibration plate height data and calibration plate shape data;

[0104] Step S143: segmenting the calibration plate imaging area of ​​the calibration plate 3D point cloud data using the calibration plate height data and the calibration plate shape data to generate calibration plate area data;

[0105] Step S144: Perform CT coordinate fusion comparison on the calibration plate area data through the part 3D point cloud data to generate calibration plate fused 3D point cloud data.

[0106] In an embodiment of the present invention, by placing a calibration plate on the patient, ensure that its position and posture are the same as during CT scanning. Use an infrared 2D camera to shoot the calibration plate, and maintain appropriate lighting and camera conditions. Ensure that the camera can capture the entire surface of the calibration plate to obtain complete image information. Preprocess the captured calibration plate image, including removing the background, enhancing the contrast, etc. Use image processing algorithms or computer vision technology to extract morphological information of the calibration plate, such as height and shape. According to the appearance features of the calibration plate, such as boundaries and corners, calculate the height and shape data of the calibration plate. Combined with the height and shape data of the calibration plate, the three-dimensional point cloud data of the calibration plate is divided into calibration plate areas. According to the shape information of the calibration plate, a region surrounding the calibration plate is defined in three-dimensional space. The point cloud data falling within the region is identified as calibration plate region data, and other point cloud data is excluded. The 3D point cloud data of the patient's lesion site is matched and aligned with the 3D point cloud data of the calibration plate area. Using the CT coordinate system, the point cloud data of the calibration plate area is converted to the coordinate system of the patient's lesion site. Through the matching algorithm, the spatial position and posture of the calibration plate are fused with the point cloud data of the patient's lesion. The final generated calibration plate fused 3D point cloud data contains the spatial information of the patient's lesion and the calibration plate, which can be used for subsequent analysis and processing.

[0107] As an example of the present invention, refer to Figure 2 As shown, in this example, step S2 includes:

[0108] Step S21: Acquire patient posture perception data;

[0109] Step S22: performing data preprocessing on the calibration plate fused 3D point cloud data and the part 3D point cloud data to generate standard calibration plate fused 3D point cloud data and standard part 3D point cloud data, wherein the data preprocessing includes data normalization, data denoising and data enhancement;

[0110] Step S23: performing point cloud division on the fused 3D point cloud data of the standard calibration plate and the 3D point cloud data of the standard part to generate source point cloud data and target point cloud data;

[0111] Step S24: performing IPC coarse registration on the source point cloud data and the target point cloud data to generate point cloud coarse registration data; performing registration jitter analysis on the point cloud coarse registration data through the patient's body posture perception data to generate registration jitter data;

[0112] Step S25: comparing the registration jitter data with a preset registration jitter threshold, and when the registration jitter data is greater than the preset registration jitter threshold, performing jitter registration optimization according to the source point cloud coarse registration data and the target point cloud coarse registration data to generate a jitter registration optimization model;

[0113] Step S26: importing the point cloud coarse registration data into the jitter registration optimization model to perform jitter registration optimization until the registration jitter data is less than or equal to a preset registration jitter threshold, and outputting the point cloud optimized registration data.

[0114] In an embodiment of the present invention, a body posture sensing device (such as a depth camera or a three-dimensional camera system) is used to scan or photograph the patient. These devices can capture the patient's body posture information, including posture, movement, and body shape. The calibration plate fused three-dimensional point cloud data and the part 3D point cloud data are normalized and converted into a unified coordinate system. The point cloud data is denoised, and a filtering algorithm (such as Gaussian filtering, median filtering, etc.) is used to remove noise points. For data enhancement, data augmentation techniques (such as rotation, translation, scaling, etc.) can be used to increase the diversity and quantity of data. The calibration plate fused three-dimensional point cloud data and the part 3D point cloud data are divided into source point cloud data and target point cloud data. The source point cloud data generally refers to the calibration plate fused three-dimensional point cloud data, and the target point cloud data generally refers to the part 3D point cloud data. A coarse registration algorithm (such as an ICP algorithm) is used to preliminarily register the source point cloud data and the target point cloud data to obtain coarse registration data. Combined with the patient's body perception data, the coarse registration data is subjected to jitter analysis to evaluate the stability and accuracy of the registration results and generate registration jitter data. The registration jitter data is compared with the preset registration jitter threshold. If the jitter data exceeds the threshold, the registration needs to be further optimized. Based on the source point cloud coarse registration data and the target point cloud coarse registration data, a jitter registration optimization model is constructed. The source point cloud coarse registration data and the target point cloud coarse registration data are input into the jitter registration optimization model for registration optimization. Through iterative optimization, the registration parameters are continuously adjusted until the registration jitter data meets the preset threshold requirements. The optimized point cloud registration data is output to improve the accuracy and stability of the registration results.

[0115] Preferably, performing IPC coarse registration on the source point cloud data and the target point cloud data includes:

[0116] Extract corner feature of source point cloud data and target point cloud data to obtain source point cloud corner feature points and target point cloud corner feature points;

[0117] Match the feature points of the source point cloud corner points with the target point cloud corner points to generate an initial point cloud transformation matrix, where the initial point cloud transformation matrix includes a point cloud rotation matrix and a point cloud translation vector;

[0118] Perform preliminary translation transformation on the source point cloud data and the target point cloud data through the point cloud rotation matrix and the point cloud translation vector to generate preliminary point cloud registration transformation data;

[0119] Performing registration distance measurement on the preliminary point cloud registration transformation data to generate preliminary point cloud registration distance data; performing ICP registration parameter setting according to the preliminary point cloud registration distance data to generate ICP registration parameter data;

[0120] Based on the ICP algorithm, the registration position of the preliminary point cloud registration transformation data is adjusted according to the ICP registration parameter data to generate point cloud coarse registration data.

[0121] In an embodiment of the present invention, corner feature extraction is performed on source point cloud data and target point cloud data by using a feature extraction algorithm (such as Harris corner detector, FAST corner detector, etc.). Corner feature points are usually significant local extreme points in the point cloud, with obvious angle changes and high curvature. The source point cloud corner feature points and the target point cloud corner feature points are matched using a feature point matching algorithm (such as the nearest neighbor matching algorithm, the RANSAC algorithm, etc.). The matched feature point pairs can be used to calculate the initial transformation matrix between the point clouds, including the rotation matrix and the translation vector. According to the obtained initial point cloud transformation matrix, the source point cloud data is rotated and translated to be roughly aligned with the target point cloud data. The quality of the preliminary point cloud registration is measured using a registration distance measurement method (such as the least squares method, the distance calculation between point clouds, etc.) to generate registration distance data. According to the registration distance data, the parameters of the ICP registration algorithm, such as the number of iterations, the threshold, etc., are adjusted to generate ICP registration parameter data. The ICP registration algorithm (Iterative Closest Point Algorithm) is used to iteratively optimize the preliminary point cloud registration transformation data to better align it with the target point cloud data. The parameters of the ICP algorithm, such as the number of iterations, convergence threshold, etc., are set according to the ICP registration parameter data to achieve accurate point cloud registration.

[0122] Preferably, performing registration jitter analysis on the point cloud coarse registration data using the patient posture perception data includes:

[0123] Performing patient fluctuation curve analysis on the patient's posture perception data to generate a patient fluctuation curve;

[0124] Performing ups and downs curvature analysis on the patient's ups and downs curve to generate the patient's ups and downs curvature data; performing change cycle analysis on the patient's ups and downs curvature data to generate the patient's ups and downs change cycle data;

[0125] The patient's breathing pattern is divided according to the patient's fluctuation cycle data to obtain the patient's breathing pattern data; the patient's breathing pattern data is collected in time series point cloud to obtain time series point cloud data;

[0126] The time series point cloud data and the patient's fluctuation curve are aligned with the respiratory phase to generate the respiratory phase aligned point cloud data; the registration difference of the point cloud coarse registration data is calculated through the respiratory phase aligned point cloud data to generate the registration jitter data.

[0127] In an embodiment of the present invention, a patient is scanned or photographed by using a posture sensing device (such as a depth camera or a three-dimensional camera system) to capture changes in his posture. Through image processing and feature extraction algorithms, the patient's undulating curve on the time axis is extracted from the posture sensing data to reflect the changes in his posture. The curvature analysis of the patient's undulating curve is performed to calculate the curvature value. The curvature value reflects the rate and amplitude of the patient's posture change, which can be used to evaluate the degree of fluctuation of the posture. Through the time series analysis method, the patient's undulating curvature data is periodically analyzed to find out its change cycle. The change cycle can reflect the rhythmicity and periodicity of the patient's posture change, and provide a basis for the subsequent division of breathing patterns. According to the characteristics of the undulating change cycle, the patient's posture is divided into different breathing phases or breathing patterns, such as the inhalation phase, the exhalation phase, etc. The divided breathing pattern data describes the periodicity and regularity of the patient's posture changes. According to the patient's breathing pattern and the time information of the posture sensing data, the point cloud data at the corresponding time point is collected. The time series point cloud data reflects the changes in the patient's posture over time, and can be used for subsequent respiratory phase alignment and registration difference calculation. The time series point cloud data are aligned according to the patient's respiratory phase so that each point cloud data corresponds to a specific respiratory phase. Aligning the point cloud data with the respiratory phase can eliminate the time difference during the respiratory process, so that the point cloud data are in the same respiratory phase. Compare the respiratory phase aligned point cloud data with the coarse registration data and calculate the registration difference between them. The registration difference data reflects the jitter and deviation in the point cloud registration process and can be used to evaluate the stability and accuracy of the registration.

[0128] As an example of the present invention, refer to Figure 3 As shown, in this example, step S25 includes:

[0129] Step S251: comparing the registration jitter data with a preset registration jitter threshold, and when the registration jitter data is greater than the preset registration jitter threshold, performing normal vector calculation on the source point cloud coarse registration data and the target point cloud coarse registration data according to the point feature histogram to obtain normal vector data;

[0130] Step S252: performing neighborhood search on the source point cloud coarse registration data and the target point cloud coarse registration data to generate a neighborhood point set; performing PFH calculation on the source point cloud coarse registration data and its corresponding neighborhood point set and the target point cloud coarse registration data and its corresponding neighborhood point set through normal vector data to obtain a neighborhood feature descriptor;

[0131] Step S253: construct a matrix based on the source point cloud coarse registration data and the target point cloud coarse registration data to obtain a 4k-dimensional neighborhood matrix; perform a descriptor low-dimensional transformation on the 4k-dimensional neighborhood matrix through a preset PointNet model to generate a 32-dimensional feature descriptor;

[0132] Step S254: Optimize the preset PointNet model through the 32-dimensional feature descriptor and the neighborhood feature descriptor to generate a jitter registration optimization model.

[0133] In an embodiment of the present invention, the registration jitter of the coarse registration data of the source point cloud and the coarse registration data of the target point cloud is calculated. If the registration jitter is greater than the preset registration jitter threshold, the next step of processing is performed. The normal vectors of the source point cloud and the target point cloud can be calculated using a normal vector estimation algorithm (such as least squares method, normal convolution, etc.). A neighborhood search is performed on the source point cloud and the target point cloud to find the neighborhood point set of each point. For each point and its neighborhood point set, the point feature histogram (PFH) is calculated using the normal vector data to describe the local geometric shape features of the point. According to the PFH feature descriptors of the source point cloud and the target point cloud, a 4k-dimensional neighborhood matrix is ​​constructed. The 4k-dimensional neighborhood matrix is ​​subjected to feature extraction and dimensionality reduction processing using a preset PointNet model to generate a 32-dimensional feature descriptor. The 32-dimensional feature descriptor and the neighborhood feature descriptor are input into the preset PointNet model. Through model training and optimization, the parameters of the PointNet model are adjusted to maximize the extraction and utilization of feature information of the point cloud data. The optimized PointNet model will be used as a jitter registration optimization model for the subsequent registration jitter optimization process. More specifically, the registration jitter between the source point cloud and the target point cloud can be calculated. The Euclidean distance or other metric methods can be used to quantify the difference between the two point clouds. The calculated registration jitter is compared with the preset jitter threshold. If the registration jitter exceeds the threshold, the next step is required; otherwise, it is considered that the registration has met the requirements and no further processing is required. The normal vectors are calculated for the source point cloud and the target point cloud. The normal vector can be obtained by fitting the surface or the neighborhood point method. A neighborhood search is performed on each point cloud to find the neighboring point set around it. Using the normal vector data and the neighborhood point set, the point feature histogram (PFH) is calculated to describe the local geometric features of the point. Based on the PFH features of the source point cloud and the target point cloud, a 4k-dimensional neighborhood matrix is ​​constructed. Using the pre-trained PointNet model, the 4k-dimensional neighborhood matrix is ​​subjected to feature extraction and dimensionality reduction to generate a 32-dimensional feature descriptor. The generated 32-dimensional feature descriptor and neighborhood feature descriptor are input into the preset PointNet model. Through the model training and optimization process, the parameters of the PointNet model are adjusted to better extract the feature information of the point cloud. The optimized PointNet model will be used as the jitter registration optimization model for the subsequent registration jitter optimization process.

[0134] As an example of the present invention, refer to Figure 4 As shown, in this example, step S3 includes:

[0135] Step S31: Acquire the position data of the spinal robot manipulator;

[0136] Step S32: coordinate mapping the calibration plate based on the manipulator position data of the spine robot to generate manipulator-calibration plate mapping coordinate data;

[0137] Step S33: construct the trajectory to be punctured based on the DICOM image of the patient's lesion site and the coordinate data of the manipulator-calibration plate mapping to generate the trajectory data to be punctured.

[0138] In an embodiment of the present invention, the position data of the manipulator is obtained in real time by using sensors or encoders in the spinal robot system. The manipulator position data generally includes the three-dimensional coordinates and posture information of the manipulator, which is used to determine the position and orientation of the manipulator in three-dimensional space. The position data of the manipulator is matched and mapped with the position of the calibration plate that has been calibrated in advance. According to the position relationship between the known calibration points on the calibration plate and the manipulator, the mapping coordinates of the manipulator to each calibration point on the calibration plate are calculated. The generated manipulator-calibration plate mapping coordinate data describes the position information of the manipulator in the calibration plate coordinate system, which is used for subsequent trajectory construction. According to the patient's lesion site and the doctor's puncture plan, the target area and puncture path to be punctured are determined. The position of the target area in the calibration plate coordinate system is converted to the position in the manipulator-calibration plate mapping coordinate system. According to the position of the target area and the current position of the manipulator, one or more puncture trajectories are constructed. The trajectory data to be punctured describes the puncture path that the manipulator needs to follow, and is used to guide the manipulator to accurately position and puncture in the patient's body. More specifically, sensors or encoders in the spinal robot system, such as photoelectric encoders or inertial measurement units (IMUs), are used to collect the position and posture data of the manipulator in real time. These sensors are usually installed on each joint of the manipulator to obtain the angle information of the joints, thereby calculating the position and posture of the manipulator. The manipulator position data is represented in a three-dimensional coordinate system, including the position (x, y, z coordinates) and posture (direction, rotation angle) information of the manipulator. Before the operation, the calibration plate is calibrated in advance to determine the three-dimensional coordinates of specific points on the calibration plate. When the position data of the manipulator is available, the current position of the manipulator is matched with the known points of the calibration plate. The position data of the manipulator is mapped to the calibration plate coordinate system through a specific algorithm (such as the least squares method or the transformation matrix), so as to obtain the position relationship of the manipulator relative to the calibration plate. The generated manipulator-calibration plate mapping coordinate data describes the position and posture information of the manipulator in the calibration plate coordinate system. Before the operation, the doctor determines the lesion site and puncture path that need to be punctured through the patient's CT or MRI images. The position of the patient's lesion in the calibration plate coordinate system is converted to the position in the robot-calibration plate mapping coordinate system. According to the position of the target area and the current position of the robot, the puncture path to be executed is calculated. The trajectory data to be punctured includes the path and action that the robot needs to perform, such as where the robot needs to move and how to rotate and tilt it.

[0139] Preferably, step S4 comprises the following steps:

[0140] Step S41: using the 3D visual sensor on the spinal robot to perform spinal robot manipulator posture analysis on the puncture trajectory data to be processed, and generating spinal robot manipulator posture data;

[0141] Step S42: performing hand-eye calibration on the spinal robot manipulator posture data based on the jitter registration optimization model to generate spinal robot hand-eye calibration data;

[0142] Step S43: converting the puncture trajectory data into needle insertion navigation instructions according to the spinal robot hand-eye calibration data to execute the spinal robot needle insertion navigation operation.

[0143] In an embodiment of the present invention, the three-dimensional information of the surgical area is obtained in real time by using a 3D visual sensor (such as a stereo camera or a laser radar) carried on the spinal robot system. Based on the trajectory data to be punctured, the morphology and structure of the surgical area are analyzed to determine the optimal posture of the manipulator. By analyzing the image or point cloud data captured by the visual sensor, the posture of the spinal robot manipulator in the surgical area is calculated, including position and posture information. The spinal robot manipulator posture data is further optimized and calibrated using the previously constructed jitter registration optimization model. According to the requirements of the spinal robot hand-eye calibration, the spatial relationship between the surgical tool (manipulator) and the visual sensor (eye) is determined. By monitoring and recording the manipulator movement and visual sensor data during the actual operation, the precise position and posture relationship between the manipulator and the visual sensor is obtained. The spinal robot hand-eye calibration data is generated to describe the accurate position and posture of the manipulator in the visual sensor coordinate system. The trajectory data to be punctured is converted to the coordinate system of the spinal robot system so that the spinal robot can understand and execute. According to the spinal robot hand-eye calibration data, the position and posture information in the trajectory data to be punctured is converted into the motion instructions of the manipulator in the robot system. Generate the final needle insertion navigation instructions to guide the spinal robot manipulator to accurately navigate and perform the puncture operation according to the trajectory to be punctured during the operation. More specifically, use the 3D visual sensor (such as a stereo camera or a lidar) on the spinal robot system to capture the three-dimensional data of the surgical area in real time. Process the acquired image or point cloud data, such as denoising, filtering, and feature extraction, to obtain accurate morphological information of the surgical area. According to the data of the trajectory to be punctured, match and align the real-time surgical area data with the trajectory to be punctured by image registration or point cloud registration, so as to calculate the position and posture of the spinal robot manipulator in the surgical area. Generate the spinal robot manipulator posture data, including the position (x, y, z coordinates) and posture (direction, rotation angle) information of the manipulator. Use the previously constructed jitter registration optimization model to further optimize and calibrate the spinal robot manipulator posture data to improve the accuracy and stability of positioning. In the actual surgical environment, prepare the surgical tools (manipulator) and visual sensors (eyes). By recording the movement of the manipulator and the data captured by the visual sensor during the operation, the calibration algorithm is used to calculate the exact position and posture relationship between the manipulator and the visual sensor. The spinal robot hand-eye calibration data is generated, which describes the exact position and posture of the manipulator in the visual sensor coordinate system. The trajectory data to be punctured is converted from the surgical area coordinate system to the coordinate system of the spinal robot system so that the spinal robot can understand and execute it. According to the spinal robot hand-eye calibration data, the position and posture information in the trajectory data to be punctured is converted into motion instructions for the manipulator in the robot system, including movement, rotation, tilt and other actions.The generated needle insertion navigation instructions are input into the spinal robot system to guide the manipulator to accurately navigate and perform the puncture operation along the puncture trajectory to complete the surgical operation.

[0144] Therefore, the embodiments should be regarded as illustrative and non-restrictive from all points, and the scope of the present invention is limited by the appended claims rather than the above description, and it is therefore intended that all changes falling within the meaning and range of equivalent elements of the application documents are included in the present invention.

[0145] The above description is only a specific embodiment of the present invention, so that those skilled in the art can understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but should conform to the widest scope consistent with the principles and novel features invented herein.

Claims

1. A registration method between a spinal robot 3D sensor and a DICOM image, characterized in that: The following steps are involved: Step S1: Obtain a DICOM image of the patient's lesion; perform laser scanning on the calibration plate on the patient through a laser radar fusion sensor to obtain the calibration plate's three-dimensional point cloud data; perform calibration plate spatial fusion on the DICOM image of the patient's lesion and the calibration plate's three-dimensional point cloud data to generate calibration plate fused three-dimensional point cloud data; Step S2: Acquire the patient's posture perception data; perform point cloud division on the calibration plate fused three-dimensional point cloud data and the part 3D point cloud data to generate source point cloud data and target point cloud data; perform IPC coarse registration on the source point cloud data and the target point cloud data to generate point cloud coarse registration data; perform registration jitter analysis on the point cloud coarse registration data through the patient's posture perception data to generate registration jitter data; compare the registration jitter data with a preset registration jitter threshold, and when the registration jitter data is greater than the preset registration jitter threshold, perform jitter registration optimization on the point cloud coarse registration data until the registration jitter data is less than or equal to the preset registration jitter threshold, and output the point cloud optimized registration data; Step S2 includes the following steps: Step S21: Acquire patient posture perception data; Step S22: performing data preprocessing on the calibration plate fused 3D point cloud data and the part 3D point cloud data to generate standard calibration plate fused 3D point cloud data and standard part 3D point cloud data, wherein the data preprocessing includes data normalization, data denoising and data enhancement; Step S23: performing point cloud division on the fused 3D point cloud data of the standard calibration plate and the 3D point cloud data of the standard part to generate source point cloud data and target point cloud data; Step S24: performing IPC coarse registration on the source point cloud data and the target point cloud data to generate coarse point cloud registration data; performing registration jitter analysis on the coarse point cloud registration data using the patient posture perception data to generate registration jitter data; performing registration jitter analysis on the coarse point cloud registration data using the patient posture perception data includes: Performing patient fluctuation curve analysis on the patient's posture perception data to generate a patient fluctuation curve; Performing ups and downs curvature analysis on the patient's ups and downs curve to generate the patient's ups and downs curvature data; performing change cycle analysis on the patient's ups and downs curvature data to generate the patient's ups and downs change cycle data; The patient's breathing pattern is divided according to the patient's fluctuation cycle data to obtain the patient's breathing pattern data; the patient's breathing pattern data is collected in time series point cloud to obtain time series point cloud data; Perform respiratory phase alignment on the time series point cloud data and the patient's undulation curve to generate respiratory phase aligned point cloud data; perform registration difference calculation on the point cloud rough registration data through the respiratory phase aligned point cloud data to generate registration jitter data; Step S25: Compare the registration jitter data with the preset registration jitter threshold. When the registration jitter data is greater than the preset registration jitter threshold, perform jitter registration optimization according to the source point cloud coarse registration data and the target point cloud coarse registration data to generate a jitter registration optimization model. Step S25 includes the following steps: Step S251: comparing the registration jitter data with a preset registration jitter threshold, and when the registration jitter data is greater than the preset registration jitter threshold, performing normal vector calculation on the source point cloud coarse registration data and the target point cloud coarse registration data according to the point feature histogram to obtain normal vector data; Step S252: performing neighborhood search on the source point cloud coarse registration data and the target point cloud coarse registration data to generate a neighborhood point set; performing PFH calculation on the source point cloud coarse registration data and its corresponding neighborhood point set and the target point cloud coarse registration data and its corresponding neighborhood point set through normal vector data to obtain a neighborhood feature descriptor; Step S253: construct a matrix based on the source point cloud coarse registration data and the target point cloud coarse registration data to obtain a 4k-dimensional neighborhood matrix; perform a descriptor low-dimensional transformation on the 4k-dimensional neighborhood matrix through a preset PointNet model to generate a 32-dimensional feature descriptor; Step S254: optimizing the preset PointNet model by using a 32-dimensional feature descriptor and a neighborhood feature descriptor, thereby generating a jitter registration optimization model; Step S26: importing the point cloud coarse registration data into the jitter registration optimization model to perform jitter registration optimization until the registration jitter data is less than or equal to a preset registration jitter threshold, and outputting the point cloud optimized registration data; Step S3: acquiring the position data of the spinal robot manipulator; constructing the trajectory to be punctured on the calibration plate based on the position data of the spinal robot manipulator to generate the trajectory data to be punctured; Step S4: Use the 3D vision sensor on the spinal robot to analyze the puncture trajectory data and generate spinal robot manipulator posture data; convert the spinal robot manipulator posture data into needle insertion navigation instructions based on the jitter alignment optimization model to execute the spinal robot needle insertion navigation operation.

2. The registration method between the spinal robot 3D sensor and the DICOM image according to claim 1, characterized in that: Step S1 includes the following steps: Step S11: using a CT machine to obtain a dicom image of the patient's lesion; Step S12: performing laser scanning on the calibration plate on the patient through a laser radar fusion sensor to obtain three-dimensional point cloud data of the calibration plate; Step S13: Perform three-dimensional reconstruction on the DICOM image of the patient's lesion site to generate 3D point cloud data of the site; Step S14: spatially fuse the calibration plate with the 3D point cloud data of the part and the 3D point cloud data of the calibration plate to generate fused 3D point cloud data of the calibration plate.

3. The registration method between the spinal robot 3D sensor and the DICOM image according to claim 2, characterized in that: Step S13 includes the following steps: Step S131: segmenting the lesion area of ​​the patient's DICOM image to generate a lesion area segmentation mask; Step S132: using the lesion area segmentation mask to mark the image level of the DICOM image of the patient's lesion part, and generating the patient's lesion part marked image level; Step S133: stacking the marked image layers of the patient's lesion site to generate three-dimensional volume data of the patient's lesion site; meshing the surface of the patient's lesion site to generate surface mesh data; Step S134: extracting point cloud features from the surface mesh data to generate surface mesh point cloud feature data; reconstructing point cloud features from the surface mesh point cloud feature data to generate part 3D point cloud data.

4. The registration method between the spinal robot 3D sensor and the DICOM image according to claim 2, characterized in that: Step S14 includes the following steps: Step S141: photographing the calibration plate with an infrared 2D camera based on the 3D point cloud data of the calibration plate to obtain a 2D image of the calibration plate; Step S142: extracting calibration plate morphology information from the calibration plate 2D image to obtain calibration plate morphology information data, wherein the calibration plate morphology information data includes calibration plate height data and calibration plate shape data; Step S143: segmenting the calibration plate imaging area of ​​the calibration plate 3D point cloud data using the calibration plate height data and the calibration plate shape data to generate calibration plate area data; Step S144: Perform CT coordinate fusion comparison on the calibration plate area data through the part 3D point cloud data to generate calibration plate fused 3D point cloud data.

5. The registration method between the spinal robot 3D sensor and the DICOM image according to claim 1, characterized in that: The IPC coarse registration of the source point cloud data and the target point cloud data includes: Extract corner feature of source point cloud data and target point cloud data to obtain source point cloud corner feature points and target point cloud corner feature points; Match the feature points of the source point cloud corner points with the target point cloud corner points to generate an initial point cloud transformation matrix, where the initial point cloud transformation matrix includes a point cloud rotation matrix and a point cloud translation vector; Perform preliminary translation transformation on the source point cloud data and the target point cloud data through the point cloud rotation matrix and the point cloud translation vector to generate preliminary point cloud registration transformation data; Performing registration distance measurement on the preliminary point cloud registration transformation data to generate preliminary point cloud registration distance data; performing ICP registration parameter setting according to the preliminary point cloud registration distance data to generate ICP registration parameter data; Based on the ICP algorithm, the registration position of the preliminary point cloud registration transformation data is adjusted according to the ICP registration parameter data to generate point cloud coarse registration data.

6. The registration method between the spinal robot 3D sensor and the DICOM image according to claim 1, characterized in that: Step S3 includes the following steps: Step S31: Acquire the position data of the spinal robot manipulator; Step S32: coordinate mapping the calibration plate based on the manipulator position data of the spine robot to generate manipulator-calibration plate mapping coordinate data; Step S33: construct the trajectory to be punctured based on the DICOM image of the patient's lesion site and the coordinate data of the manipulator-calibration plate mapping to generate the trajectory data to be punctured.

7. The registration method between the spinal robot 3D sensor and the DICOM image according to claim 1, characterized in that: Step S4 includes the following steps: Step S41: using the 3D visual sensor on the spinal robot to perform spinal robot manipulator posture analysis on the puncture trajectory data to be processed, and generating spinal robot manipulator posture data; Step S42: performing hand-eye calibration on the spinal robot manipulator posture data based on the jitter registration optimization model to generate spinal robot hand-eye calibration data; Step S43: converting the puncture trajectory data into needle insertion navigation instructions according to the spinal robot hand-eye calibration data to execute the spinal robot needle insertion navigation operation.

Images