Positioning method of lung puncture surgical robot based on point cloud registration

Abstract

The invention discloses a lung puncture surgical robot positioning method based on point cloud registration, and relates to the field of surgical robots. The problem that in the existing lung puncture operation process, chest and lung positioning is difficult to achieve rapidly and accurately is solved. Comprising the following steps: 1, registration preparation work: performing CT scanning on thoracic and lung parts of a patient before an operation to obtain a three-dimensional image, marking an operation target point in the thoracic and lung three-dimensional image, and reconstructing thoracic and lung three-dimensional point cloud data, namely source point cloud; in the operation, a binocular camera is used for obtaining three-dimensional point cloud, namely target point cloud, of the thoracic and lung surfaces; 2, noise reduction and down-sampling are carried out on the point cloud data, and features are extracted; 3, performing coarse registration on the target point cloud and the source point cloud by using an improved RANSAC algorithm; 4, performing fine registration of the target point cloud and the source point cloud by using an improved ICP algorithm; and 5, obtaining a rigid body transformation matrix from the source point cloud to the target point cloud, and completing registration positioning. The method is used for point cloud-based registration of the lung puncture surgical robot.

Description

Technical Field

[0001] The present invention relates to the technical field of chest and lung robots, and in particular to a positioning method for a lung puncture surgery robot based on point cloud registration. Background Art

[0002] In recent years, surgical robots have made significant progress and are widely used in various clinical departments. With their exceptional stability, high precision, and intelligent features, surgical robots can effectively prevent operator hand tremors during surgery, reduce postoperative complications, and shorten operative times. Consequently, surgical robots have become a hot topic in the field of medical and engineering research, garnering increasing attention due to their significant economic benefits and outstanding performance. In actual surgeries, these high-precision surgical robots often require high-precision chest and lung positioning technology.

[0003] Chinese patent CN118453137A discloses a lung puncture surgical robot, the terminal device of which can achieve precise and autonomous puncture. The key to this method is to extract the target point cloud through a laser sensor, and reconstruct the source point cloud based on the CT image, and then perform coarse registration through the point cloud bounding box, and then use the ICP algorithm to finely align the target point cloud and the source point cloud data, so as to solve the registration matrix, that is, the relative position relationship between the optical tracking image space coordinate system and the patient's lesion coordinate system, and finally achieve accurate positioning of the patient's chest and lungs. Although this positioning method uses expensive laser sensors, its coarse registration method cannot provide an ideal initial position, and the ICP algorithm for fine registration is time-consuming and has the problem of failing to converge or falling into a local optimal solution, making it difficult to achieve accurate and fast positioning.

[0004] Therefore, technicians in this field are committed to developing a positioning method for a lung puncture surgery robot based on point cloud registration. Summary of the Invention

[0005] In view of the above-mentioned defects of the prior art, the technical problem to be solved by the present invention is to solve the problem that it is difficult to quickly and accurately locate the chest and lungs during the existing lung puncture surgery.

[0006] To achieve the above objectives, the present invention provides a positioning method for a lung puncture surgical robot based on point cloud registration, characterized by comprising the following steps: registration preparation: preoperatively, a CT scan of the patient's chest and lungs is performed to obtain a three-dimensional image, the surgical target points are marked in the three-dimensional chest and lung image, and the three-dimensional point cloud data of the chest and lungs, i.e., the source point cloud, is reconstructed; during the operation, a binocular camera is used to obtain a three-dimensional point cloud of the chest and lung surface, i.e., the target point cloud; step two: noise reduction, downsampling, and feature extraction of the point cloud data; step three: coarse registration of the target point cloud and the source point cloud using an improved RANSAC algorithm; step four: fine registration of the target point cloud and the source point cloud using an improved ICP algorithm; and step five: obtaining a rigid body transformation matrix from the source point cloud to the target point cloud to complete registration and positioning. The present invention is used for point cloud registration of a lung puncture surgical robot.

[0007] Technical effects:

[0008] The present invention provides a positioning method for a lung puncture surgical robot based on point cloud registration, which has a reasonable design and strong practicality, and can effectively and quickly achieve chest and lung positioning during existing lung puncture surgery.

[0009] The concept, specific structure and technical effects of the present invention will be further described below in conjunction with the accompanying drawings to fully understand the purpose, characteristics and effects of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 It is a flowchart of the lung puncture positioning method of the present invention;

[0011] Figure 2 It is a schematic diagram of the process of preprocessing point cloud data according to the present invention;

[0012] Figure 3 It is a schematic diagram of the registration process of the present invention. DETAILED DESCRIPTION

[0013] The following describes several preferred embodiments of the present invention with reference to the accompanying drawings to make its technical content clearer and easier to understand. The present invention can be embodied in many different forms of embodiments, and the scope of protection of the present invention is not limited to the embodiments mentioned herein.

[0014] In the drawings, components with identical structures are denoted by the same reference numerals, and components with similar structures or functions are denoted by similar reference numerals. The size and thickness of each component shown in the drawings are arbitrary and are not limited by the present invention. For clarity, the thickness of components in some places in the drawings is appropriately exaggerated.

[0015] Specific implementation method 1: Combination Figures 1 to 3 To illustrate this embodiment, the embodiment includes the following steps:

[0016] Step 1. Registration preparation:

[0017] Before surgery, a CT scan of the patient's chest and lungs is performed to obtain three-dimensional images, such as Figure 2 As shown in (d), the surgical target point is marked in the 3D image of the chest and lungs, and the 3D point cloud data of the chest and lungs, namely the source point cloud, is reconstructed. Figure 2 (e)

[0018] During the operation, a binocular camera is used to obtain the three-dimensional point cloud of the patient's chest and lung surface, namely the target point cloud, such as Figure 2 As shown in (a).

[0019] Step 2: Preprocess the point cloud data:

[0020] Denoise, downsample, and extract features from the source point cloud obtained before surgery and the target point cloud obtained during surgery;

[0021] Step 3: Use the improved RANSAC algorithm to perform rough registration of the target point cloud and the source point cloud;

[0022] Step 4: Use the improved ICP algorithm to perform precise registration of the target point cloud and the source point cloud;

[0023] Step 5: Obtain the rigid body transformation matrix from the source point cloud to the target point cloud to complete the registration and positioning.

[0024] Specific implementation method 2: Combination Figures 1 to 2 To illustrate this embodiment, the step 2 of this embodiment of pre-processing the point cloud data includes the following steps:

[0025] Step 2.1: Filter out points whose coordinate values in the three dimensions of x, y, and z are not within the given value range, and perform preliminary interference removal on the target point cloud acquired by the binocular camera, removing the point cloud that is not on the chest and lung surface in the target point cloud, such as Figure 2 (b)

[0026] Step 2.2, downsampling: Use the fast uniform sampling method to downsample the target point cloud and the source point cloud at the same time, sampling a point every certain number of points to retain the shape information of the point cloud, such as Figure 2 (c)(f);

[0027] Step 2.3: Use statistical filtering to further reduce the noise of the target point cloud;

[0028] Step 2.4: further downsample the target point cloud and the source point cloud;

[0029] Step 2.5 extracts feature points of the target point cloud and source point cloud based on the Harris 3D corner point feature extraction method.

[0030] Specific implementation method three: Combination Figures 1 to 2 To illustrate this embodiment, step 2.2 of this embodiment further includes extracting source point cloud information from the binocular camera scan target point cloud and the CT scan image, and downsampling the source point cloud and the target point cloud simultaneously.

[0031] Specific implementation method four: Combination Figure 2 To illustrate this embodiment, the process of further denoising the source point cloud in step 2.3 of this embodiment is as follows:

[0032] Step 2.3.1: When the binocular camera collects point clouds, sparse outliers will be generated. Outliers are caused by noise and are characterized by sparse distribution in space. Therefore, if the point cloud at a certain point is less than a certain density, the point cloud is invalid.

[0033] Step 2.3.2. Calculate the average distance from each point to its nearest points. The distances of all points in the point cloud should form a Gaussian distribution. Given the mean and variance, remove points outside the range.

[0034] Specific implementation method five: Combination Figure 2 To illustrate this embodiment, in step 2.5 of this embodiment, extracting feature points of the target point cloud and the source point cloud based on the Harris 3D corner point feature extraction method includes the following steps:

[0035] Step 2.5.1. Calculate the covariance matrix:

[0036] For each point p i , select a set of neighborhood points near it, and calculate the covariance matrix based on the local geometry of these points. The covariance matrix can be calculated using the following formula:

[0037]

[0038] Among them, p i are the coordinates of the neighborhood points, is the centroid of the neighborhood points.

[0039] Step 2.5.2, calculate the Harris response function:

[0040] For each point, the Harris response function is calculated using the eigenvalues of the covariance matrix. The response function reflects the local surface change of the point, and the formula is as follows:

[0041] R = det(M) - α (trace(M)) 2

[0042] Here, α is an empirical parameter, usually ranging from 0.04 to 0.06.

[0043] Step 2.5.3, corner detection:

[0044] According to the value of Harris response function, the point with the largest response value is selected as the corner point. A larger response value indicates that the local geometric change of the point is larger, which is usually a feature point.

[0045] Specific implementation method six: combination Figure 3 To illustrate this embodiment, in step 3 of this embodiment, using the improved RANSAC algorithm to perform coarse registration of the target point cloud and the source point cloud includes the following steps:

[0046] Step 3.1, randomly select m matching point pairs from the Harris3D corner point set of the reference point cloud and the Harris3D corner point set Q_{Harris3D} of the point cloud to be registered, and obtain P compute Point set. Typically, m point sets are used to estimate the transformation relationship of 6 degrees of freedom.

[0047] Step 3.2: Perform triangle similarity judgment on the three selected sampling points. For each sampling point on the point cloud set, calculate the similarity between the triangle formed by the three points and the triangle formed by the sampling points on the other two point clouds, and find the maximum value Rate diffmax If this value is less than the set threshold Rate diffthreshold , then the point is considered to be a correct matching point, otherwise return to the first step and reselect samples until a matching point pair that meets the criteria is obtained. If the conditions are not met, the Rate diffthreshold Set to 0.01.

[0048] Step 3.3, use the SVD method to solve the transformation matrix through the selected matching points, and calculate Q k Harris3D Perform a rigid transformation to obtain Q k+1 Harris3D , represents the kth transformation.

[0049] Step 3.4, calculate P Harris3D Each point to Q Harris3D The nearest neighbor distance is compared with the threshold d threshold Compare and confirm the matching points.

[0050] Step 3.5: Perform j random samplings of three matching points, determine their triangle similarity, and remove matching points that do not meet the conditions.

[0051] Step 3.6: Count the number N of valid matching point pairs.

[0052] Step 3.7: When N is greater than the set threshold N threshold When , all matching points are used to perform SVD solution to obtain a more accurate transformation matrix.

[0053] Step 3.8: Apply the formula Calculate P_{Harris3D} and Q compute The error E k .

[0054] Step 3.9: Repeat steps 3.1 and 3.7 until the preset maximum number of iterations k is reached. max or error E k Less than the set threshold E threshold , the algorithm stops at this point. The final result of the rough registration is as follows Figure 3 (b) shown.

[0055] Specific implementation method seven: combination Figure 3 To illustrate this embodiment, in step 4 of this embodiment, the use of the improved ICP algorithm to perform precise registration of the target point cloud and the source point cloud includes the following steps:

[0056] Step 4.1: Input the reference point cloud P and the point cloud to be registered Q.

[0057] Step 4.2: Construct a KDTree data structure for the reference point cloud P and the point cloud to be registered Q to quickly find the nearest neighbor points.

[0058] Step 4.3: Extract linear features for point clouds P and Q respectively, and calculate their weighting factors f i :

[0059] f i =12detM i -0.3(traceM i ) 2

[0060] Step 4.4, according to each point p in point P i , using the constructed KDTree and formula

[0061] Search for the closest point q i As matching points, and obtain the matching point set S i .

[0062] Step 4.5: Calculate matching point S i The average characteristic distance d between midpoints mean And standard deviation σ, if a feature distance d si Satisfy condition d si <d mean +3σ, the matching point is retained; otherwise, the matching points that do not meet the conditions are eliminated.

[0063] Step 4.6, according to the feature distance d si Matching point set S from small to large iSort and select the first 70% of matching points for calculation.

[0064] Step 4.7: According to the method proposed by Low, assuming that the rotation matrix R and translation vector T after rough registration are close to the ideal value, calculate the angles α, β, γ and the translation amount t x ,t y ,t z , and perform an approximate rotation transformation based on these parameters, using the formula for calculation:

[0065]

[0066] The SVD method is used to solve the rotation matrix R and translation vector T, and then the point cloud transformation is performed.

[0067] Step 4.8: Calculate the error evaluation function E of this registration k (R, T), and the error value D of the previous round k =E k -E k-1 Compare. If the difference between the first two errors is D k If the value is less than the preset threshold, the registration is considered to have reached convergence; otherwise, the iteration step continues.

[0068] Step 4.9: If the error does not converge, repeat steps 4.1 and 4.8 until the preset maximum number of iterations is reached or the error meets the convergence condition. The final result of the fine registration is as follows: Figure 3 (c) shown.

[0069] The preferred embodiments of the present invention have been described in detail above. It should be understood that numerous modifications and variations based on the concepts of the present invention are possible without inventive effort by those skilled in the art. Therefore, any technical solution that can be derived by one skilled in the art through logical analysis, reasoning, or limited experimentation based on the concepts of the present invention and the prior art should be within the scope of protection defined by the claims.

Claims

1. A positioning method for a lung puncture surgery robot based on point cloud registration, characterized in that: The method comprises the following steps: Step 1. Registration preparation: Before surgery, a 3D image of the patient's chest and lungs is obtained by CT scanning. The surgical target points are marked in the 3D image of the chest and lungs, and the 3D point cloud data of the chest and lungs, i.e., the source point cloud, is reconstructed. During the operation, a binocular camera is used to obtain a three-dimensional point cloud of the patient's chest and lung surface, namely the target point cloud; Step 2: Preprocess the point cloud data: Denoise, downsample, and extract features from the source point cloud obtained before surgery and the target point cloud obtained during surgery; Step 3: Use the improved RANSAC algorithm to perform rough registration of the target point cloud and the source point cloud; Step 4: Use the improved ICP algorithm to perform precise registration of the target point cloud and the source point cloud; Step 5: Obtain the rigid body transformation matrix from the source point cloud to the target point cloud to complete the registration and positioning.

2. The positioning method of a lung puncture surgery robot based on point cloud registration according to claim 1 is characterized in that: The step 2 of pre-processing the point cloud data includes the following steps: Step 2.1: Filter out points whose coordinate values in the x, y, and z dimensions are not within the given range, perform preliminary interference removal on the target point cloud acquired by the binocular camera, and remove point clouds that are not on the chest and lung surfaces in the target point cloud; Step 2.2, downsampling: Use the fast uniform sampling method to downsample the target point cloud and the source point cloud at the same time, sampling a point every certain number of points to preserve the shape information of the point cloud; Step 2.3: Use statistical filtering to further reduce the noise of the target point cloud; Step 2.4: further downsample the target point cloud and the source point cloud; Step 2.5: Extract feature points of the target point cloud and source point cloud based on the Harris 3D corner point feature extraction method.

3. The positioning method for a lung puncture surgery robot based on point cloud registration according to claim 2 is characterized in that: The step 2.2 further includes extracting source point cloud information from the binocular camera scan target point cloud and the CT scan image, and simultaneously downsampling the source point cloud and the target point cloud.

4. The positioning method for a lung puncture surgery robot based on point cloud registration according to claim 3 is characterized in that: The process of further denoising the source point cloud in step 2.3 is as follows: Step 2.3.1: When the binocular camera collects point clouds, sparse outliers will be generated. Outliers are caused by noise and are characterized by sparse distribution in space. Therefore, if the point cloud at a certain point is less than a certain density, the point cloud is invalid. Step 2.3.

2. Calculate the average distance from each point to its nearest points. The distances of all points in the point cloud should form a Gaussian distribution. Given the mean and variance, remove points outside the range.

5. The positioning method for a lung puncture surgery robot based on point cloud registration according to claim 4 is characterized in that: Extracting feature points of the target point cloud and the source point cloud based on the Harris3D corner feature extraction method in step 2.5 includes the following steps: Step 2.5.

1. Calculate the covariance matrix: For each point p_i, select a set of neighboring points near it and calculate the covariance matrix based on the local geometry of these points. The covariance matrix can be calculated using the following formula: Among them, p i are the coordinates of the neighborhood points, is the centroid of the neighborhood points; Step 2.5.2, calculate the Harris response function: For each point, the Harris response function is calculated using the eigenvalues of the covariance matrix. The response function reflects the local surface change of the point, and the formula is as follows: R=det(M)-α·(trace(M)) 2 Among them, α is an empirical parameter, usually ranging from 0.04 to 0.06; Step 2.5.3, corner detection: Based on the value of the Harris response function, the point with the largest response value is selected as the corner point. A larger response value indicates that the local geometric change of the point is larger, and it is usually a feature point.

6. A positioning method for a lung puncture surgical robot based on point cloud registration according to claim 1 or 5, characterized in that: The rough registration of the target point cloud and the source point cloud using the improved RANSAC algorithm in step 3 includes the following steps: Step 3.1, from the Harris3D corner point set of the reference point cloud and the Harris3D corner point set Q of the point cloud to be registered Harris3D Randomly select m matching point pairs and get P compute Point set. Usually, m point sets are used to estimate the transformation relationship of 6 degrees of freedom; Step 3.2: Perform triangle similarity judgment on the three selected sampling points. For each sampling point on the point cloud set, calculate the similarity between the triangle formed by the three points and the triangle formed by the sampling points on the other two point clouds, and find the maximum value Rate diffmax If this value is less than the set threshold Rate diffthreshold , then the point is considered to be a correct matching point, otherwise return to the first step and reselect samples until a matching point pair that meets the criteria is obtained. If the conditions are not met, the Rate diffthreshold Set to 0.01; Step 3.3, use the SVD method to solve the transformation matrix through the selected matching points, and calculate Q k Harris3D Perform a rigid transformation to obtain Q k+1 Harris3D , represents the kth transformation; Step 3.4, calculate P Harris3D Each point to Q Harris3D The nearest neighbor distance is compared with the threshold d threshold Compare and confirm matching points; Step 3.5: Perform j random sampling of three matching points, determine their triangle similarity, and remove matching points that do not meet the conditions; Step 3.6, count the number of valid matching point pairs N; Step 3.7: When N is greater than the set threshold N threshold When , all matching points are used to perform SVD solution to obtain a more accurate transformation matrix; Step 3.8: Apply the formula Calculate P Harris3D With Q compute The error E k ; Step 3.9: Repeat steps 3.1 and 3.7 until the preset maximum number of iterations k is reached. max or error E k Less than the set threshold E threshold , then stop the algorithm.

7. The positioning method for a lung puncture surgery robot based on point cloud registration according to claim 1 is characterized in that: In step 4, the use of the improved ICP algorithm to perform precise registration of the target point cloud and the source point cloud includes the following steps: Step 4.1: Input the reference point cloud P and the point cloud to be registered Q; Step 4.2: Construct a KDTree data structure for the reference point cloud P and the point cloud to be registered Q to quickly find the nearest neighbor points. Step 4.3: Extract linear features for point clouds P and Q respectively, and calculate their weighting factors f i : f i =12detM i -0.3(traceM i ) 2 Step 4.4, according to each point p in point P i , using the constructed KDTree and formula Search for the closest point q i As matching points, and obtain the matching point set S i ; Step 4.5: Calculate matching point S i The average characteristic distance d between midpoints mean And standard deviation σ, if a feature distance d si Satisfy condition d si <d mean +3σ, then the matching point is retained; otherwise, the matching points that do not meet the conditions are eliminated; Step 4.6, according to the feature distance d si Matching point set S from small to large i Sort and select the first 70% of matching points for calculation; Step 4.7: According to the method proposed by Low, assuming that the rotation matrix R and translation vector T after coarse registration are close to the ideal value, calculate the angles αβ, γ and the translation amount t x ,t y ,tz, and perform an approximate rotation transformation based on these parameters, using the formula for calculation: The SVD method is used to solve the rotation matrix R and translation vector T, and then the point cloud transformation is performed; Step 4.8: Calculate the error evaluation function E of this registration k (R, T), and the error value D of the previous round k =E k -E k-1 Compare, if the difference between the first two errors D k If it is less than the preset threshold, the registration is considered to have reached convergence; otherwise, the iterative steps are continued; Step 4.9: If the error does not converge, repeat steps 4.1 to 4.8 until the preset maximum number of iterations is reached or the error meets the convergence condition.

Images