A spinal lamina decompression surgery positioning method based on point cloud feature extraction

Abstract

The application discloses a spine lamina decompression surgery positioning method based on point cloud feature extraction, comprising the following steps: obtaining the 3D point cloud of the open back of a person to be positioned; converting the 3D point cloud into a correction coordinate system to complete posture correction; performing feature segmentation on the corrected 3D point cloud to separate it into two blocks containing target regions; for each block, small batch sampling and interpolation mapping are used to obtain the position of the articular process; and based on the position of the articular process, the spine lamina decompression surgery positioning result of the person to be positioned is obtained. The spine surgery area and the surrounding surface contour three-dimensional model are obtained through a 3D structured light camera, then the target position of the surgery is directly identified and positioned in the three-dimensional model, and the method has the advantages of no radiation, high speed, low cost and the like, and solves the problem that the current spine robot surgery must be implanted with a mark and multiple CTs are performed to position.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a method for locating spinal laminectomy surgery based on point cloud feature extraction. Background Technology

[0002] Currently, laminectomy still requires an open back. The human back and spine are not rigidly connected, making it difficult for minimally invasive surgery to accurately locate the invisible spine. Open back surgery relies heavily on experienced surgeons and places significant physical and psychological challenges on them. To alleviate this situation, the introduction of assistive robots to help surgeons is being considered.

[0003] Using robots to assist or even replace doctors in surgery can improve surgical safety and consistency. With the development of perception technology, robots can gradually adapt to perception tasks in complex environments, such as in autonomous driving scenarios. Traditional 2D vision cannot effectively distinguish different tissues and structures in open spinal surgery, while 3D vision can solve the challenge of insignificant color differences in the surgical area during open spinal surgery. Point clouds are also the most commonly used data structure in 3D vision, so point cloud analysis can more effectively extract structural features of the surgical area and complete the task of surgical area localization.

[0004] Current point cloud recognition algorithms have limited application scenarios, mostly targeting specific 3D scenes or 3D models, and cannot effectively solve the problem of recognizing complex structures in the surgical area of ​​open spinal surgery. Summary of the Invention

[0005] This invention provides a method for locating spinal laminectomy surgery based on point cloud feature extraction, which solves the problem that current point cloud recognition application algorithms are limited to specific 3D scenes or 3D models and cannot effectively address the technical issue of recognizing complex structures in the surgical area during open spinal surgery.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0007] On one hand, the present invention provides a method for surgical localization of spinal laminectomy based on point cloud feature extraction, the method comprising:

[0008] The surface data of the open back of the subject to be located is obtained, and a 3D point cloud in the corresponding camera coordinate system is generated based on the surface data. The 3D point cloud is then preprocessed. The open back is the surgical area of ​​the exposed part of the spine, and the surface data is a 3D contour map of the surface of the open back surgical area.

[0009] A correction coordinate system is constructed, and the preprocessed 3D point cloud is transformed into the correction coordinate system to complete the pose correction of the preprocessed 3D point cloud and obtain the corrected 3D point cloud.

[0010] The corrected 3D point cloud is segmented into two blocks containing the target region.

[0011] For each block, the articular process position is obtained by mini-batch sampling and interpolation mapping.

[0012] Based on the articular process position, the coordinates of the target area in the correction coordinate system are obtained, and the obtained coordinates are mapped back to the camera coordinate system to obtain the positioning result of the spinal laminectomy surgery for the patient to be located.

[0013] Further, the step of acquiring surface data of the open back of the person to be located, generating a 3D point cloud in the corresponding camera coordinate system based on the surface data, and preprocessing the 3D point cloud includes:

[0014] Utilize sensor devices to collect surface data of the open back of the person to be located;

[0015] Generate a 3D point cloud in the corresponding camera coordinate system based on the surface data;

[0016] Invalid and zero points in the 3D point cloud are removed, and then the 3D point cloud is filtered.

[0017] Furthermore, the sensor device is a sensor with a three-dimensional structure capture function.

[0018] Furthermore, the filtering of the 3D point cloud includes voxel downsampling, isolated point removal, and redundant region removal; wherein, the voxel size used for voxel downsampling is 0.8mm, reducing the number of points to 1 / 10 of the original; isolated point removal uses a statistical filtering method to calculate the average distance of 50 points and remove points that are less than a certain threshold.

[0019] Further, the construction of the correction coordinate system, which transforms the preprocessed 3D point cloud into the correction coordinate system to complete the pose correction of the preprocessed 3D point cloud and obtain the corrected 3D point cloud, includes:

[0020] Principal component analysis was performed on the preprocessed 3D point cloud to obtain the first three principal vectors;

[0021] A correction coordinate system is constructed using the obtained principal vectors. The preprocessed 3D point cloud is then transformed into the correction coordinate system to complete the pose correction of the preprocessed 3D point cloud, resulting in a corrected 3D point cloud.

[0022] Furthermore, when performing principal component analysis on the preprocessed 3D point cloud, singular value decomposition is used to calculate the covariance matrix to obtain the principal vector.

[0023] Furthermore, the step of performing feature segmentation on the corrected 3D point cloud to separate it into two blocks containing the target region includes:

[0024] Using parametric projection, the corrected 3D point cloud is projected into 2D from a top-down view.

[0025] Based on 2D projection, Gaussian clustering is used to separate the surgical area into two blocks containing the target region, with the spinous process as the boundary. The randomness brought by Gaussian clustering is eliminated, and the two blocks are named the left block and the right block.

[0026] The 2D projections that have completed block separation are mapped back to the 3D point cloud to obtain the 3D point cloud after feature segmentation.

[0027] Furthermore, the articular process positions are obtained by using mini-batch sampling and interpolation mapping, including:

[0028] The 3D point cloud of the area to be processed is obtained by parametric projection in 2D projection from the side view.

[0029] In the obtained 2D projection, the bottom region of the surgical area is sampled in batches along the direction of the maximum principal component, and a fixed number of points are collected in each batch.

[0030] All sampling points were fitted using a curve fitting algorithm;

[0031] Find the location of each trough in the curve;

[0032] Using the nearest neighbor algorithm, multiple 3D regions are obtained by sampling from the 3D point cloud after feature segmentation, with multiple valleys as the center; wherein, the 3D regions describe the positions of articular processes.

[0033] Furthermore, when fitting all sampling points using the curve fitting algorithm, the fitting scheme is divided into two steps: first, pre-fitting to reduce the impact of data instability, and then performing a second fitting on the pre-fitted curve to obtain the final curve.

[0034] Further, based on the articular process position, the coordinates of the target region in the correction coordinate system are obtained, and the obtained coordinates are mapped back to the camera coordinate system to obtain the positioning result for the spinal laminectomy surgery of the patient to be located, including:

[0035] The target region is estimated based on the 3D region; wherein, the target region is the position between the two articular processes near the spinous process.

[0036] Based on the estimated point location, the nearest neighbor algorithm is used again to extract the target region from the 3D point cloud after feature segmentation, and the coordinates of the target region in the corrected coordinate system are obtained.

[0037] The obtained coordinates are mapped back to the camera coordinate system to obtain the positioning results for spinal laminectomy.

[0038] In another aspect, the present invention also provides an electronic device comprising a processor and a memory; wherein the memory stores at least one instruction, which is loaded and executed by the processor to implement the above-described method.

[0039] In another aspect, the present invention also provides a computer-readable storage medium storing at least one instruction that is loaded and executed by a processor to implement the above-described method.

[0040] The beneficial effects of the technical solution provided by this invention include at least the following:

[0041] Current technologies generally utilize deep learning or feature descriptor techniques for surgical localization in spinal laminectomy. The former relies on massive amounts of data, while physical therapy data is scarce; the latter requires extensive computation and is unsuitable for recognizing complex structures. To address this, this invention employs a projection method to identify projected contours, effectively recognizing complex structures in the surgical area while reducing data dependence and improving computational speed. Experimental results demonstrate the invention's effectiveness in open spinal surgery. Furthermore, this invention requires minimal investment, boasts high performance, robustness, requires no additional labeling, is non-radioactive, and is highly portable. Attached Figure Description

[0042] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0043] Figure 1 This is a schematic diagram illustrating the implementation principle of the spinal laminectomy surgical localization method based on point cloud feature extraction provided in this embodiment of the invention.

[0044] Figure 2 This is an execution flowchart of the spinal laminectomy surgical localization method based on point cloud feature extraction provided in an embodiment of the present invention;

[0045] Figure 3 This is a schematic diagram of a 3D point cloud of an open back provided in an embodiment of the present invention. Detailed Implementation

[0046] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.

[0047] First Embodiment

[0048] To achieve complex and variable structural recognition in the surgical area and effectively locate the target region, providing a more reliable perception solution for spinal robotic surgery, this embodiment provides a spinal laminectomy surgical localization method based on point cloud feature extraction. Its implementation principle is as follows: Figure 1 As shown, this method obtains the original point cloud from a 3D structured light camera. After preprocessing and segmentation, it first calculates the principal vector using PCA and then uses the principal vector for pose correction. Next, Gaussian clustering is used to separate the two sides of the spike. Then, the point cloud is projected onto a 2D plane, and small-batch sampling is performed near the target region to fit the bottom curve. Finally, the target location is found by determining geometric structural features. Specifically, the execution flow of this method is as follows: Figure 2 As shown, it includes the following steps:

[0049] S1, acquire the surface data of the open back of the person to be located, generate a 3D point cloud in the corresponding camera coordinate system based on the surface data, and preprocess the 3D point cloud.

[0050] The open back refers to the surgical area of ​​the back where the spine is partially exposed. This area exposes the complete spinous processes and some articular processes of the spine, with the remaining surrounding area consisting of muscle tissue. The surface data is a 3D contour map of the open back surgical area obtained by scanning with devices such as structured light sensors.

[0051] Specifically, in this example, the implementation process of S1 is as follows:

[0052] S11, using sensor devices to collect surface data of the open back of the person to be located; wherein, the sensor devices use sensors with stereo structure capture functions such as structured light cameras or binocular depth cameras;

[0053] S12, Generate a 3D point cloud in the corresponding camera coordinate system based on the surface data, such as... Figure 3 As shown; the obtained three-dimensional model should have a certain degree of integrity on both sides with the spinous process as the horizontal axis.

[0054] S13, remove invalid points and zero points in the 3D point cloud; where invalid points contain NAN in their coordinates, and zero points are a large number of points with coordinate values ​​of zero. Both types of points are caused by sensor errors.

[0055] S14, Filter the 3D point cloud; the filtering includes voxel downsampling, isolated point removal and redundant region removal; the voxel size used for voxel downsampling is 0.8mm, reducing the number of points to 1 / 10 of the original; isolated point removal uses a statistical filtering method to calculate the average distance of 50 points and remove points that are less than a certain threshold.

[0056] S2, Construct a correction coordinate system, transform the preprocessed 3D point cloud into the correction coordinate system to complete the posture correction of the preprocessed 3D point cloud and obtain the corrected 3D point cloud;

[0057] Specifically, in this example, the implementation process of S2 is as follows:

[0058] S21, perform principal component analysis (PCA) on the preprocessed 3D point cloud to obtain the first three principal vectors;

[0059] In this embodiment, when performing principal component analysis on the preprocessed 3D point cloud, singular value decomposition (SVD) is used to calculate the covariance matrix to obtain the principal vectors. The formula used is as follows:

[0060] A=U∑V T

[0061] S22, Construct a correction coordinate system using the obtained principal vector, and transform the preprocessed 3D point cloud into the correction coordinate system to complete the pose correction of the preprocessed 3D point cloud, obtaining the corrected 3D point cloud. S3, Perform feature segmentation on the corrected 3D point cloud, separating it into two blocks containing the target region;

[0062] It should be noted that in this embodiment, the complete surgical area is divided into two parts along the spinous process, and the target is extracted separately. The segmentation process first projects the point cloud onto a two-dimensional plane for processing, and then maps it back into the three-dimensional point cloud.

[0063] Specifically, in this example, the implementation process of S3 is as follows:

[0064] S31, using parametric projection to obtain the 2D projection of the corrected 3D point cloud from a top-down view.

[0065] S32 uses Gaussian clustering to separate the surgical area into two blocks containing the target region, with the spinous process as the boundary;

[0066] S33, eliminate the randomness introduced by Gaussian clustering, and name the two blocks as the left block and the right block.

[0067] S34, map the 2D projection of the completed block separation back to the 3D point cloud to obtain the 3D point cloud after feature segmentation. S4, for each block, use mini-batch sampling and interpolation mapping to obtain the articular process position;

[0068] The mini-batch sampling and interpolation mapping step requires first projecting the point cloud onto a two-dimensional plane for processing, and then mapping it back into a three-dimensional point cloud. Specifically, in this example, the implementation process of S4 is as follows:

[0069] S41, Use parametric projection to obtain the 2D projection of the 3D point cloud of the block to be processed in the side view.

[0070] S42, sample the bottom region of the surgical area in batches along the direction of the maximum principal component in the obtained 2D projection, and collect a fixed number of points in each batch;

[0071] S43, use a curve fitting algorithm to fit all sampling points; the fitting scheme is divided into two steps: first, pre-fitting to reduce the impact of data instability, and then performing a second fitting on the pre-fitted curve to obtain the final curve.

[0072] S44, find the location of each trough of the curve; where the trough location is actually the articular process of the spine, which can be used as a reference location for the target area;

[0073] S45, using the nearest neighbor algorithm, sample from the 3D point cloud obtained in S34 with multiple valleys as centers to obtain multiple 3D regions; wherein, the 3D regions describe the positions of articular processes.

[0074] S5. Based on the articular process position, the coordinates of the target area in the correction coordinate system are obtained, and the obtained coordinates are mapped back to the camera coordinate system to obtain the positioning results of the spinal laminectomy surgery for the patient to be located.

[0075] Specifically, in this example, the implementation process of S5 is as follows:

[0076] S51, Estimate the target region based on the 3D region obtained in S45; wherein the target region is the location between two articular processes near the spinous process, where a surgical robot can be used for drilling. A point cloud field of view contains 3 or more pairs (both sides of the spinous process) of target regions;

[0077] S52, based on the estimated point position, the nearest neighbor algorithm is used again to extract the target region from the 3D point cloud obtained in S34, and the coordinates of the target region in the corrected coordinate system are obtained.

[0078] S53 maps the obtained coordinates back to the camera coordinate system to obtain the positioning results for spinal laminectomy.

[0079] In summary, this embodiment provides a method for surgical localization of spinal laminectomy based on point cloud feature extraction. This method effectively identifies complex structures in the surgical area, reduces data dependency, and improves computational speed by employing a projection method to identify the projected contour. Experimental results demonstrate that the spinal laminectomy localization method based on point cloud feature extraction provided in this embodiment has good efficacy for open spinal surgery. Furthermore, this method requires minimal investment, has high performance, good robustness, requires no additional labeling, is non-radioactive, and is highly portable.

[0080] Second Embodiment

[0081] This embodiment provides an electronic device, which includes a processor and a memory; wherein the memory stores at least one instruction, which is loaded and executed by the processor to implement the method of the first embodiment.

[0082] The electronic device can vary considerably depending on its configuration or performance, and may include one or more processors (central processing units, CPUs) and one or more memories, wherein the memories store at least one instruction that is loaded by the processor and executed in accordance with the above method.

[0083] Third Embodiment

[0084] This embodiment provides a computer-readable storage medium storing at least one instruction, which is loaded and executed by a processor to implement the method of the first embodiment described above. The computer-readable storage medium may be a ROM, random access memory, CD-ROM, magnetic tape, floppy disk, or optical data storage device, etc. The instruction stored therein can be loaded and executed by a processor in a terminal.

[0085] Furthermore, it should be noted that the present invention can be provided as a method, apparatus, or computer program product. Therefore, embodiments of the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, embodiments of the present invention can take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code.

[0086] Embodiments of the present invention are described with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0087] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The functions specified in one or more boxes. These computer program instructions may also be loaded onto a computer or other programmable data processing terminal equipment to cause a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0088] It should also be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0089] Finally, it should be noted that the above description represents a preferred embodiment of the present invention. It should be pointed out that although preferred embodiments have been described, those skilled in the art, once they understand the basic inventive concept of the present invention, can make various improvements and modifications without departing from the principles described herein. These improvements and modifications should also be considered within the scope of protection of the present invention. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the embodiments of the present invention.

Claims

1. A spinal laminectomy decompression surgery positioning method based on point cloud feature extraction, characterized in that, The method comprises the following steps: acquiring surface data of an open back of a person to be positioned, generating a corresponding 3D point cloud in a camera coordinate system according to the surface data, and pre-processing the 3D point cloud; wherein the open back is a back surgical area of a bare part of a spine, and the surface data is a 3D contour map of a surface of the open back surgical area; constructing a correction coordinate system, converting the pre-processed 3D point cloud into the correction coordinate system to complete the posture correction of the pre-processed 3D point cloud, and obtaining a corrected 3D point cloud, comprising: performing principal component analysis on the pre-processed 3D point cloud, calculating a covariance matrix by singular value decomposition, and obtaining the first three principal vectors; constructing a correction coordinate system using the obtained principal vectors, converting the pre-processed 3D point cloud into the correction coordinate system to complete the posture correction of the pre-processed 3D point cloud, and obtaining a corrected 3D point cloud; performing feature segmentation on the corrected 3D point cloud to separate it into two blocks containing target areas, comprising: obtaining a 2D projection of the corrected 3D point cloud under a top-down view by using parameter projection; based on the 2D projection, using Gaussian clustering to separate the surgical area into two blocks containing target areas with the spinous process as the boundary, and eliminating the randomness caused by Gaussian clustering, and naming the two blocks as left block and right block; mapping the 2D projection after completing the block separation back to the 3D point cloud to obtain the 3D point cloud after feature segmentation; for each block, a joint process position is obtained by using small batch sampling and interpolation mapping, comprising: obtaining a 2D projection of the 3D point cloud of the block to be processed under a side view by using parameter projection; sampling the bottom area of the surgical area in batches in the maximum principal component direction of the obtained 2D projection, and collecting a fixed number of points in each batch; fitting all sampling points by using a curve fitting algorithm; finding the position of each wave trough of the curve; using the nearest neighbor algorithm to sample multiple 3D regions in the 3D point cloud after feature segmentation with multiple wave troughs as the center to obtain multiple 3D regions; wherein the 3D region describes the joint process position; wherein when fitting all sampling points by using the curve fitting algorithm, the fitting scheme is divided into two steps, first pre-fitting to reduce the influence of data instability, and then twice fitting the pre-fitting curve to obtain the final curve; based on the joint process position, obtaining the coordinates of the target area in the correction coordinate system, and mapping the obtained coordinates back to the camera coordinate system to obtain the spinal lamina decompression surgery positioning result of the person to be positioned, comprising: estimating the target area according to the 3D region; wherein the position between the two joint processes close to the spinous process is the target area; using the nearest neighbor algorithm to extract the target area in the 3D point cloud after feature segmentation again according to the estimated point position to obtain the coordinates of the target area in the correction coordinate system; mapping the obtained coordinates back to the camera coordinate system to obtain the spinal lamina decompression surgery positioning result.

2. The method of claim 1, wherein, The method comprises the following steps: acquiring surface data of an open back of a person to be positioned, generating a corresponding 3D point cloud in a camera coordinate system according to the surface data, and pre-processing the 3D point cloud; wherein the open back is a back surgical area of a bare part of a spine, and the surface data is a 3D contour map of a surface of the open back surgical area; acquiring surface data of an open back of a person to be positioned by using a sensor device; According to the surface data, a 3D point cloud in a corresponding camera coordinate system is generated; Invalid points and zero-value points existing in the 3D point cloud are removed, and then the 3D point cloud is filtered.

3. The method of claim 2, wherein the method further comprises: The sensor device is a sensor with a three-dimensional structure capturing function.

4. The spine laminectomy decompression surgery positioning method based on point cloud feature extraction of claim 2, wherein, The filtering of the 3D point cloud includes voxel downsampling, isolated point removal and redundant region removal; wherein the voxel size used in the voxel downsampling is 0.8mm, and the point number is reduced to 1 / 10 of the original; the statistical filtering method is used for isolated point removal to count the average distance of 50 points, and points less than a certain threshold are removed.

Images