Deep learning-based point cloud registration method from liver preoperative point cloud to liver intraoperative point cloud

Abstract

The invention discloses a point cloud registration method from a liver preoperative point to an intraoperative point based on deep learning. The method comprises the following steps: acquiring existing real three-dimensional liver model data; the method comprises the following steps: preprocessing existing real three-dimensional liver model data to obtain preoperative complete point clouds and intra-operative partial point clouds of the liver, and taking the corresponding preoperative complete point clouds and intra-operative partial point clouds of the liver as samples to obtain a sample data set; based on a double-branch network framework, constructing an initial liver point cloud registration model; training the initial liver point cloud registration model by using the sample data set to obtain a liver point cloud registration model; and performing actual liver point cloud registration by using the liver point cloud registration model. According to the method, the registration precision is greatly improved, the registration speed is increased, the real-time performance is achieved, and compared with an existing method, the method has the advantages of being high in registration precision and high in registration speed.

Description

Technical Field

[0001] The present invention relates to a three-dimensional point cloud registration method, and in particular to a point cloud registration method from pre-operative to intra-operative liver surgery based on deep learning. Background Art

[0002] Point cloud registration involves determining a rigid transformation to align one point cloud with another. Traditional point cloud registration methods first extract local features, then determine the correspondence between points in the source point cloud and the target point cloud through feature matching, and finally estimate the rigid transformation matrix.

[0003] In liver surgery, the complete liver surface point cloud can be segmented in the CT image before the operation. However, during the operation, due to occlusion problems, the reconstructed liver surface point cloud will have a large number of omissions, and only a partial liver surface point cloud can be obtained. Because the liver model is created differently before and after the operation, it is generally difficult to determine the one-to-one correspondence between points. The loss of the liver surface during the operation and the deformation of the organ caused by the patient's breathing or the squeezing of surgical instruments during the operation make feature matching very difficult. Therefore, a better method is needed to complete the registration method from the preoperative complete liver surface point cloud to the intraoperative partial surface point cloud.

[0004] In recent years, deep learning has achieved considerable success in point cloud registration tasks. Learning-based methods can obtain knowledge of biomechanical constraints in training data and solve the problems caused by organ deformation and occlusion to a certain extent. In non-rigid organ registration tasks, it is difficult to obtain a large amount of real intraoperative data. The existing solution is to use time-consuming biomechanical simulations to simulate the deformation of real liver models to obtain a large amount of training data.

[0005] Most of the existing deep learning registration network frameworks are based on fully convolutional networks. However, convolution-based neural networks have local inductive biases. In the registration task, the source point cloud and the target point cloud are generally far apart, and the convolutional network cannot complete the information transfer. Therefore, there is an urgent need for a method that can better perform point cloud registration on non-rigid organs. Summary of the invention

[0006] The purpose of the present invention is to provide a point cloud registration method from preoperative to intraoperative liver surgery based on deep learning, which improves the registration accuracy and speed by constructing a new neural network framework for point cloud registration.

[0007] The present invention provides a point cloud registration method from preoperative to intraoperative liver surgery based on deep learning, comprising the following steps:

[0008] S1. Acquire existing real three-dimensional liver model data;

[0009] S2. Preprocessing the existing real three-dimensional liver model data obtained in step S2 to obtain a complete liver point cloud before surgery and a partial liver point cloud during surgery, and using the corresponding complete liver point cloud before surgery and the partial liver point cloud during surgery as samples to obtain a sample data set;

[0010] S3. Based on the dual-branch network framework, the initial liver point cloud registration model was constructed;

[0011] S4. Using the sample data set obtained in step S2 to train the initial liver point cloud registration model obtained in step S3, a liver point cloud registration model is obtained;

[0012] S5. Use the liver point cloud registration model obtained in step S4 to perform actual liver point cloud registration.

[0013] In step S1, the real three-dimensional liver model data is the complete liver surface model data segmented in the abdominal CT scan.

[0014] Step S2 specifically includes the following steps:

[0015] S21. Randomly sample the existing real three-dimensional liver model data obtained in step S1 to obtain a complete point cloud of the liver before surgery as the source point cloud S;

[0016] S22. Filling the existing real three-dimensional liver model data obtained in step S1 with tetrahedral elements, and then performing biomechanical simulation, randomly adding a random force of a preset size in the biomechanical simulation, and calculating using a finite element solver to obtain a biomechanical simulation deformed liver model;

[0017] S23. randomly intercepting the surface on the biomechanical simulation deformed liver model, resampling and deleting random vertices, adding uniform random displacement to the position of each remaining vertex, and randomly rotating and translating several point clouds of the processed model to obtain a partial point cloud of the liver during surgery as the target point cloud T;

[0018] S24. Take the corresponding preoperative liver complete point cloud and intraoperative liver partial point cloud as samples, increase the number of samples by flipping the point cloud, and obtain a sample data set.

[0019] Step S22 is specifically as follows: using tetrahedral elements to fill the existing real three-dimensional liver model data surface grid obtained in step S1 to obtain a filled three-dimensional liver surface model; performing biomechanical simulation on the filled three-dimensional liver surface model, randomly selecting a number of surfaces to set a 0 displacement boundary condition, applying up to three random forces with a maximum of 1.5N to random parts of the surface, calculating with a finite element solver, and using a neo-Hookean hyperelastic material model with a random Young's modulus (2kPa to 5kPa) and a Poisson's ratio of 0.35 to obtain a biomechanical simulation deformed liver model;

[0020] Step S23 specifically includes: randomly intercepting a surface on the biomechanical simulation deformed liver model, resampling and deleting random vertices, adding uniform random displacement to the position of each remaining vertex, and performing random rotation of a maximum of 90° and a maximum translation of 15 cm on a number of point clouds of the processed model to obtain a point cloud of the liver during surgery;

[0021] Step S24 is specifically as follows: using the corresponding preoperative liver complete point cloud and intraoperative liver partial point cloud as samples to obtain an initial sample data set; flipping the initial sample data set along the X-axis, Y-axis and Z-axis respectively to increase the number of samples and obtain a sample data set.

[0022] Step S3 is specifically as follows:

[0023] Based on a dual-branch network framework, an initial liver point cloud registration model is constructed; the initial liver point cloud registration model includes a feature extraction module and an SVD decomposition module;

[0024] The feature extraction module is a dual-branch network, including a source point cloud branch and a target point cloud branch; the source point cloud branch and the target point cloud branch have the same structure; the source point cloud branch includes the first EdgeConv layer, the second EdgeConv layer, the third EdgeConv layer, the first cross attention submodule, and the second cross attention submodule connected in series; the target point cloud branch includes the fourth EdgeConv layer, the fifth EdgeConv layer, the sixth EdgeConv layer, the third cross attention submodule, and the fourth cross attention submodule connected in series; the first to sixth EdgeConv layers are normalized using InstanceNorm and use GELU is used as the activation function; the point cloud input by the source point cloud branch first extracts local features through the first to third EdgeConv layers, and then exchanges information with the corresponding cross-attention submodule in the target point cloud branch through the cross-attention submodule to obtain global features, wherein the first cross-attention submodule exchanges information with the third cross-attention submodule, and the second cross-attention submodule exchanges information with the fourth cross-attention submodule; after the outputs of the source point cloud branch and the target point cloud branch are connected in feature dimension, the result is used as the input of the SVD decomposition module; the SVD decomposition module calculates the translation vector and rotation matrix based on the input data as the output of the model.

[0025] The output of the first cross-attention submodule is expressed using the following formula:

[0026]

[0027] Among them, Y S is the output of the first cross-attention submodule; X Sis the output of the third EdgeConv layer in the source point cloud branch; d k For X S The dimension size of the feature channel; Q S For X S After linear transformation, K S is the first input of the third cross-attention submodule to the first cross-attention submodule by exchanging information, V S The second input of the third cross-attention submodule to the first cross-attention submodule by exchanging information is calculated using the following formula:

[0028] Q S =X S W Q

[0029] K S =X T W K

[0030] V S =X T W V

[0031] Among them, X T is the output of the sixth EdgeConv layer in the target point cloud branch; W Q is the first weight; W K is the second weight; W V is the third weight;

[0032] The output of the source point cloud branch is the output of the second cross-attention submodule, which is expressed using the following formula:

[0033]

[0034] Among them, Y S ' is the output of the second cross attention submodule; dd k Y S The dimension size of the feature channel; Q' S Y S After linear transformation, K' S is the first input of the fourth cross-attention submodule to the second cross-attention submodule by exchanging information, V S ' is the second input of the fourth cross attention submodule to the second cross attention submodule by exchanging information, and is calculated using the following formula:

[0035] Q′ S =Y S WW Q

[0036] K′ S=Y T WW K

[0037] V′ S =Y T WW V

[0038] Among them, Y T is the output of the third cross-attention submodule; WW Q The fourth weight; WW K The fifth weight; WW V is the sixth weight;

[0039] The output of the third cross-attention submodule is expressed using the following formula:

[0040]

[0041] Among them, Y T is the output of the third cross-attention submodule; d' k For X T The dimension size of the feature channel; Q T For X T After linear transformation, K T is the first input of the first cross-attention submodule to the third cross-attention submodule by exchanging information, V T The second input of the first cross-attention submodule to the third cross-attention submodule by exchanging information is calculated using the following formula:

[0042] Q T =X T W′ Q

[0043] K T =X S W′ K

[0044] V T =X S W′ V

[0045] Among them, W' Q is the seventh weight; W' K is the eighth weight; W' Y The ninth weight;

[0046] The output of the target point cloud branch is the output of the fourth cross-attention submodule, which is expressed using the following formula:

[0047]

[0048] Among them, Y T' is the output of the fourth cross attention submodule; dd' k Y T The dimension size of the feature channel; Q' T Y T After linear transformation, K' T is the first input of the second cross-attention submodule to the fourth cross-attention submodule by exchanging information, V T ' is the second input of the second cross-attention submodule to the fourth cross-attention submodule by exchanging information, and is calculated using the following formula:

[0049] Q′ T =Y T WW′ Q

[0050] K′ T =Y S WW′ K

[0051] V′ T =Y S WW′ V

[0052] Among them, WW′ Q is the tenth weight; WW′ K is the eleventh weight; WW′ V is the twelfth weight;

[0053] The SVD decomposition module calculates the point-to-point correlation m between the source point cloud and the target point cloud based on the input data; then the corresponding position of each point in the target point cloud in the source point cloud is calculated through the correlation m, and the corresponding relationship between the source point cloud and the target point cloud is obtained; finally, according to the corresponding relationship between the source point cloud and the target point cloud, the rotation matrix and translation vector are obtained through SVD decomposition;

[0054] The affinity vector is calculated using the following formula:

[0055] m(t i ,S)=softmax(Y S y ti T )

[0056] Among them, m(t i ,S) is point t i The correlation vector with all points in the source point cloud S; t i is the i-th point in the target point cloud; y ti Output Y of the target point cloud branch T The i-th row in ;

[0057] The correspondence between the source point cloud and the target point cloud is expressed using the following formula:

[0058] s i =S T m(t i ,S)

[0059] Among them, s i is the i-th point in the source point cloud.

[0060] In step S4, the initial liver point cloud registration model is trained using the quadratic registration error loss to obtain a liver point cloud registration model. The loss function Loss is expressed using the following formula:

[0061] Loss=∑(t i -t i ') 2

[0062] Among them, t i is the i-th point in the target point cloud; t i 'For i The coordinates transformed by the rotation matrix and translation vector output by the model.

[0063] Step S5 specifically includes the following steps:

[0064] a. Obtain the actual preoperative liver model point cloud and the intraoperative partial liver point cloud;

[0065] b. Processing the point cloud data obtained in step a to obtain a liver point cloud data set;

[0066] c. Input the paired point cloud data set obtained in step b into the liver point cloud registration model obtained in step S4 to obtain the transformation matrix and translation vector, and use the transformation matrix and translation vector to perform coordinate transformation on the preoperative liver model point cloud to complete the actual liver point cloud registration.

[0067] Step b specifically includes: resampling and denoising the point cloud data obtained in step a, and performing coordinate normalization processing.

[0068] The present invention discloses a deep learning-based registration method for preoperative complete liver point cloud to intraoperative partial point cloud, which not only greatly improves the registration accuracy, but also improves the registration speed, making it real-time. Compared with the existing methods, it has the advantages of high registration accuracy and fast registration speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Figure 1 It is a schematic diagram of the process of the present invention. DETAILED DESCRIPTION

[0070] The present invention provides a point cloud registration method from preoperative to intraoperative liver surgery based on deep learning, and its flow chart is as follows: Figure 1As shown, the following steps are included:

[0071] S1. Acquire existing real three-dimensional liver model data;

[0072] In step S1, the real three-dimensional liver model data is the complete liver surface model data segmented in the abdominal CT scan.

[0073] S2. Preprocess the existing real 3D liver model data obtained in step S2 to obtain a complete liver point cloud before surgery and a partial liver point cloud during surgery, and use the corresponding complete liver point cloud before surgery and the partial liver point cloud during surgery as samples to obtain a sample data set, specifically including the following steps:

[0074] S21. Randomly sample the existing real three-dimensional liver model data obtained in step S1 to obtain a complete point cloud of the liver before surgery as the source point cloud S;

[0075] S22. Fill the existing real three-dimensional liver model data obtained in step S1 with tetrahedral elements, and then perform biomechanical simulation, randomly add a random force of a preset size in the biomechanical simulation, and use a finite element solver to calculate to obtain a biomechanically simulated deformed liver model, specifically: fill the surface mesh of the existing real three-dimensional liver model data obtained in step S1 with tetrahedral elements to obtain a filled three-dimensional liver surface model; perform biomechanical simulation on the filled three-dimensional liver surface model, randomly select a number of surfaces to set a 0 displacement boundary condition, apply up to three random forces with a maximum of 1.5N to random parts of the surface, calculate with a finite element solver, and adopt a neo-Hookean hyperelastic material model with a random Young's modulus (2kPa-5kPa) and a Poisson's ratio of 0.35 to obtain a biomechanically simulated deformed liver model;

[0076] S23. Randomly intercept the surface on the biomechanical simulation deformable liver model, resample and delete random vertices, add uniform random displacement to the position of each remaining vertex, and randomly rotate and translate several point clouds of the processed model to obtain a partial point cloud of the liver during the operation as the target point cloud T, specifically: randomly intercept the surface on the biomechanical simulation deformable liver model, resample and delete random vertices, add uniform random displacement to the position of each vertex, and randomly rotate and translate several point clouds of the processed model by a maximum of 90° and a maximum of 15 cm to obtain a partial point cloud of the liver during the operation;

[0077] S24. Using the corresponding preoperative complete liver point cloud and intraoperative partial liver point cloud as samples, increasing the number of samples by flipping the point cloud, and obtaining a sample data set. Specifically, using the corresponding preoperative complete liver point cloud and intraoperative partial liver point cloud as samples, obtaining an initial sample data set; flipping the initial sample data set along the X-axis, Y-axis, and Z-axis, respectively, increasing the number of samples, and obtaining a sample data set.

[0078] S3. Based on the dual-branch network framework, the initial liver point cloud registration model was constructed;

[0079] Based on a dual-branch network framework, an initial liver point cloud registration model is constructed; the initial liver point cloud registration model includes a feature extraction module and an SVD decomposition module;

[0080] The feature extraction module is a dual-branch network, including a source point cloud branch and a target point cloud branch; the source point cloud branch and the target point cloud branch have the same structure; the source point cloud branch includes the first EdgeConv layer, the second EdgeConv layer, the third EdgeConv layer, the first cross attention submodule, and the second cross attention submodule connected in series; the target point cloud branch includes the fourth EdgeConv layer, the fifth EdgeConv layer, the sixth EdgeConv layer, the third cross attention submodule, and the fourth cross attention submodule connected in series; the first to sixth EdgeConv layers are normalized using InstanceNorm and use GELU is used as the activation function; the point cloud input by the source point cloud branch first extracts local features through the first to third EdgeConv layers, and then exchanges information with the corresponding cross-attention submodule in the target point cloud branch through the cross-attention submodule to obtain global features, wherein the first cross-attention submodule exchanges information with the third cross-attention submodule, and the second cross-attention submodule exchanges information with the fourth cross-attention submodule; after the outputs of the source point cloud branch and the target point cloud branch are connected in feature dimension, the result is used as the input of the SVD decomposition module; the SVD decomposition module calculates the translation vector and rotation matrix based on the input data as the output of the model.

[0081] The output of the first cross-attention submodule is expressed using the following formula:

[0082]

[0083] Among them, Y S is the output of the first cross-attention submodule; X S is the output of the third EdgeConv layer in the source point cloud branch; d k For X S The dimension size of the feature channel; Q S For X S After linear transformation, K Sis the first input of the third cross-attention submodule to the first cross-attention submodule by exchanging information, V S The second input of the third cross-attention submodule to the first cross-attention submodule by exchanging information is calculated using the following formula:

[0084] Q S =X S W Q

[0085] K S =X T W K

[0086] V S =X T W V

[0087] Among them, X T is the output of the sixth EdgeConv layer in the target point cloud branch; W Q is the first weight; W K is the second weight; W V is the third weight;

[0088] The output of the source point cloud branch is the output of the second cross-attention submodule, which is expressed using the following formula:

[0089]

[0090] Among them, Y S ' is the output of the second cross attention submodule; dd k Y S The dimension size of the feature channel; Q' S Y S After linear transformation, K' S is the first input of the fourth cross-attention submodule to the second cross-attention submodule by exchanging information, V S ' is the second input of the fourth cross attention submodule to the second cross attention submodule by exchanging information, and is calculated using the following formula:

[0091] Q′ S =Y S WW Q

[0092] K′ S =Y T WW K

[0093] V′ S =Y T WW V

[0094] Among them, Y T is the output of the third cross-attention submodule; WW Q The fourth weight; WW K The fifth weight; WW V is the sixth weight;

[0095] The output of the third cross-attention submodule is expressed using the following formula:

[0096]

[0097] Among them, Y T is the output of the third cross-attention submodule; d' k For X T The dimension size of the feature channel; Q T For X T After linear transformation, K T is the first input of the first cross-attention submodule to the third cross-attention submodule by exchanging information, V T The second input of the first cross-attention submodule to the third cross-attention submodule by exchanging information is calculated using the following formula:

[0098] Q T =X T W′ Q

[0099] K T =X S W′ K

[0100] V T =X S W′ V

[0101] Among them, W' Q is the seventh weight; W' K is the eighth weight; W' Y The ninth weight;

[0102] The output of the target point cloud branch is the output of the fourth cross-attention submodule, which is expressed using the following formula:

[0103]

[0104] Among them, Y T ' is the output of the fourth cross attention submodule; dd' k Y T The dimension size of the feature channel; Q' T Y T After linear transformation, K' Tis the first input of the second cross-attention submodule to the fourth cross-attention submodule by exchanging information, V T ' is the second input of the second cross-attention submodule to the fourth cross-attention submodule by exchanging information, and is calculated using the following formula:

[0105] Q′ T =Y T WW′ Q

[0106] K′ T =Y S WW′ K

[0107] V′ T =Y S WW′ V

[0108] Among them, WW′ Q is the tenth weight; WW′ K is the eleventh weight; WW′ V is the twelfth weight;

[0109] The SVD decomposition module calculates the point-to-point correlation m between the source point cloud and the target point cloud based on the input data; then the corresponding position of each point in the target point cloud in the source point cloud is calculated through the correlation m, and the corresponding relationship between the source point cloud and the target point cloud is obtained; finally, according to the corresponding relationship between the source point cloud and the target point cloud, the rotation matrix and translation vector are obtained through SVD decomposition;

[0110] The affinity vector is calculated using the following formula:

[0111] m(t i ,S)=softmax(Y S y ti T )

[0112] Among them, m(t i ,S) is point t i The correlation vector with all points in the source point cloud S; t i is the i-th point in the target point cloud; y ti Output Y of the target point cloud branch T The i-th row in ;

[0113] The correspondence between the source point cloud and the target point cloud is expressed using the following formula:

[0114] s i =S T m(t i ,S)

[0115] Among them, s iis the i-th point in the source point cloud.

[0116] S4. Using the sample data set obtained in step S2 to train the initial liver point cloud registration model obtained in step S3, a liver point cloud registration model is obtained;

[0117] The initial liver point cloud registration model is trained using the quadratic registration error loss to obtain the liver point cloud registration model. The loss function Loss is expressed using the following formula:

[0118] Loss=∑(t i -t i ') 2

[0119] Among them, t i is the i-th point in the target point cloud; t i 'For i The coordinates transformed by the rotation matrix and translation vector output by the model.

[0120] S5. Using the liver point cloud registration model obtained in step S4, actual liver point cloud registration is performed, which specifically includes the following steps:

[0121] a. Obtain the actual preoperative liver model point cloud and the intraoperative partial liver point cloud;

[0122] b. Processing the point cloud data obtained in step a to obtain a liver point cloud data set;

[0123] c. Input the paired point cloud data set obtained in step b into the liver point cloud registration model obtained in step S4 to obtain the transformation matrix and translation vector, and use the transformation matrix and translation vector to perform coordinate transformation on the preoperative liver model point cloud to complete the actual liver point cloud registration.

[0124] Step b specifically includes: resampling and denoising the point cloud data obtained in step a, and performing coordinate normalization processing.

[0125] The method of the present invention is further described below in conjunction with an embodiment:

[0126] Using the method provided by the present invention, the liver surface point cloud registration model was constructed using the public liver model dataset 3D-IRCADb-01. The existing methods DCP, LiverMatch, and REGTR were then used to perform point cloud registration on the above dataset. The registration result error was calculated using MAE, MSE, and RMSE, and the results are shown in Table 1.

[0127] Table 1 Comparison of registration accuracy between the method of the present invention and the existing method

[0128] Method MAE MSE RMSE DCP 0.023387 0.001187 0.034447 LiverMatch 0.021624 0.001059 0.032542 REGTR 0.019736 0.000940 0.030653 Ours 0.015206 0.000628 0.025056

[0129] It can be seen from the data in the table that the method disclosed in the present invention performs better than the existing methods on this data set.

Claims

1. A deep learning-based point cloud registration method for liver surgery from preoperative to intraoperative, characterized in that: The following steps are involved: S1. Acquire existing real three-dimensional liver model data; S2. Preprocessing the existing real three-dimensional liver model data obtained in step S2 to obtain a complete liver point cloud before surgery and a partial liver point cloud during surgery, and using the corresponding complete liver point cloud before surgery and the partial liver point cloud during surgery as samples to obtain a sample data set; S3. Based on the dual-branch network framework, the initial liver point cloud registration model was constructed; S4. Using the sample data set obtained in step S2 to train the initial liver point cloud registration model obtained in step S3, a liver point cloud registration model is obtained; S5. Use the liver point cloud registration model obtained in step S4 to perform actual liver point cloud registration.

2. The deep learning-based point cloud registration method for liver surgery from preoperative to intraoperative according to claim 1, characterized in that: In step S1, the real three-dimensional liver model data is the complete liver surface model data segmented in the abdominal CT scan.

3. The deep learning-based point cloud registration method for liver surgery from preoperative to intraoperative according to claim 1, characterized in that: Step S2 specifically includes the following steps: S21. Randomly sample the existing real three-dimensional liver model data obtained in step S1 to obtain a complete point cloud of the liver before surgery as the source point cloud S; S22. Filling the existing real three-dimensional liver model data obtained in step S1 with tetrahedral elements, and then performing biomechanical simulation, randomly adding a random force of a preset size in the biomechanical simulation, and calculating using a finite element solver to obtain a biomechanical simulation deformed liver model; S23. randomly intercepting the surface on the biomechanical simulation deformed liver model, resampling and deleting random vertices, adding uniform random displacement to the position of each remaining vertex, and randomly rotating and translating several point clouds of the processed model to obtain a partial point cloud of the liver during surgery as the target point cloud T; S24. Take the corresponding preoperative liver complete point cloud and intraoperative liver partial point cloud as samples, increase the number of samples by flipping the point cloud, and obtain a sample data set.

4. The method for point cloud registration from preoperative to intraoperative liver surgery based on deep learning according to claim 3, characterized in that: Step S22 specifically includes: using tetrahedral elements to fill the existing real three-dimensional liver model data surface grid obtained in step S1 to obtain a filled three-dimensional liver surface model; performing biomechanical simulation on the filled three-dimensional liver surface model, randomly selecting a number of surfaces to set a 0 displacement boundary condition, applying up to three random forces with a maximum of 1.5N to random parts of the surface, calculating with a finite element solver, and using a neo-Hookean hyperelastic material model with a random Young's modulus ranging from 2KPa to 5KPa and a Poisson's ratio of 0.35 to obtain a biomechanically simulated deformed liver model; Step S23 specifically includes: randomly intercepting a surface on the biomechanical simulation deformed liver model, resampling and deleting random vertices, adding uniform random displacement to the position of each remaining vertex, and performing random rotation of a maximum of 90° and a maximum translation of 15 cm on a number of point clouds of the processed model to obtain a point cloud of the liver during surgery; Step S24 is specifically as follows: using the corresponding preoperative liver complete point cloud and intraoperative liver partial point cloud as samples to obtain an initial sample data set; flipping the initial sample data set along the X-axis, Y-axis and Z-axis respectively to increase the number of samples and obtain a sample data set.

5. The method for point cloud registration from preoperative to intraoperative liver surgery based on deep learning according to claim 1, characterized in that: Step S3 is specifically as follows: Based on a dual-branch network framework, an initial liver point cloud registration model is constructed; the initial liver point cloud registration model includes a feature extraction module and an SVD decomposition module connected in series; The feature extraction module is a dual-branch network, including a source point cloud branch and a target point cloud branch; the source point cloud branch and the target point cloud branch have the same structure; the source point cloud branch includes the first EdgeConv layer, the second EdgeConv layer, the third EdgeConv layer, the first cross attention submodule, and the second cross attention submodule connected in series; the target point cloud branch includes the fourth EdgeConv layer, the fifth EdgeConv layer, the sixth EdgeConv layer, the third cross attention submodule, and the fourth cross attention submodule connected in series; the first to sixth EdgeConv layers are normalized using InstanceNorm and use GELU is used as the activation function; the point cloud input by the source point cloud branch first extracts local features through the first to third EdgeConv layers, and then exchanges information with the corresponding cross-attention submodule in the target point cloud branch through the cross-attention submodule to obtain global features, wherein the first cross-attention submodule exchanges information with the third cross-attention submodule, and the second cross-attention submodule exchanges information with the fourth cross-attention submodule; after the outputs of the source point cloud branch and the target point cloud branch are connected in feature dimension, the result is used as the input of the SVD decomposition module; the SVD decomposition module calculates the translation vector and rotation matrix based on the input data as the output of the model.

6. The method for point cloud registration from preoperative to intraoperative liver surgery based on deep learning according to claim 5, characterized in that: In the feature extraction module, the output of the first cross-attention submodule is expressed using the following formula: Among them, Y S is the output of the first cross-attention submodule; X S is the output of the third EdgeConv layer in the source point cloud branch; d k For X S The dimension size of the feature channel; Q S For X S After linear transformation, K S is the first input of the third cross-attention submodule to the first cross-attention submodule by exchanging information, V S The second input of the third cross-attention submodule to the first cross-attention submodule by exchanging information is calculated using the following formula: Q S =X S W Q K S =X T W K V S =X T W V Among them, X T is the output of the sixth EdgeConv layer in the target point cloud branch; W Q is the first weight; W K is the second weight; W V is the third weight; The output of the source point cloud branch is the output of the second cross-attention submodule, which is expressed using the following formula: Among them, Y' S is the output of the second cross-attention submodule; dd k Y S The dimension size of the feature channel; Q' S Y S After linear transformation, K' S The fourth cross-attention submodule is the first input of the second cross-attention submodule by exchanging information, V' S The second input of the fourth cross-attention submodule to the second cross-attention submodule by exchanging information is calculated using the following formula: Q' S =Y S WW Q K' S =Y T WW K V' S =Y T WW V Among them, Y T is the output of the third cross-attention submodule; WW Q The fourth weight; WW K The fifth weight; WW V is the sixth weight; The output of the third cross-attention submodule is expressed using the following formula: Among them, Y T is the output of the third cross-attention submodule; d' k For X T The dimension size of the feature channel; Q T For X T After linear transformation, K T is the first input of the first cross-attention submodule to the third cross-attention submodule by exchanging information, V T The second input of the first cross-attention submodule to the third cross-attention submodule by exchanging information is calculated using the following formula: Q T =X T W' Q K T =X S W' K V T =X S W' V Among them, W' Q is the seventh weight; W' K is the eighth weight; W' Y The ninth weight; The output of the target point cloud branch is the output of the fourth cross-attention submodule, which is expressed using the following formula: Among them, Y' T is the output of the fourth cross-attention submodule; dd' k Y T The dimension size of the feature channel; Q' T Y T After linear transformation, K' T V' is the first input of the second cross-attention submodule to the fourth cross-attention submodule by exchanging information. T The second input of the second cross-attention submodule to the fourth cross-attention submodule through the exchange of information is calculated using the following formula: Q' T =Y T WW' Q K' T =Y S WW' K V' T =Y S WW' V Among them, WW' Q The tenth weight; WW' K WW' is the eleventh weight; V It is the twelfth weight.

7. The method for point cloud registration from preoperative to intraoperative liver surgery based on deep learning according to claim 5, characterized in that: The SVD decomposition module calculates the point-to-point correlation m between the source point cloud and the target point cloud based on the input data; then the corresponding position of each point in the target point cloud in the source point cloud is calculated through the correlation m, and the corresponding relationship between the source point cloud and the target point cloud is obtained; finally, according to the corresponding relationship between the source point cloud and the target point cloud, the rotation matrix and translation vector are obtained through SVD decomposition; The affinity vector is calculated using the following formula: m(t i ,S)=softmax(Y S and ti T ) Among them, m(t i ,S) is point t i The correlation vector with all points in the source point cloud S; t i is the i-th point in the target point cloud; y ti Output Y of the target point cloud branch T The i-th row in ; The correspondence between the source point cloud and the target point cloud is expressed using the following formula: s i =S T m(t i ,S) Among them, s i is the i-th point in the source point cloud.

8. The deep learning-based point cloud registration method for liver surgery from preoperative to intraoperative according to claim 1, characterized in that: In step S4, the initial liver point cloud registration model is trained using the quadratic registration error loss to obtain a liver point cloud registration model. The loss function Loss is expressed using the following formula: Loss=∑(t i -t' i ) 2 Among them, t i is the i-th point in the target point cloud; t' i For i The coordinates transformed by the rotation matrix and translation vector output by the model.

9. The method for point cloud registration from preoperative to intraoperative liver surgery based on deep learning according to claim 1, characterized in that: Step S5 specifically includes the following steps: a. Obtain the actual preoperative liver model point cloud and the intraoperative partial liver point cloud; b. Processing the point cloud data obtained in step a to obtain a liver point cloud data set; c. Input the paired point cloud data set obtained in step b into the liver point cloud registration model obtained in step S4 to obtain the transformation matrix and translation vector, and use the transformation matrix and translation vector to perform coordinate transformation on the preoperative liver model point cloud to complete the actual liver point cloud registration.

10. The deep learning-based point cloud registration method for liver surgery from preoperative to intraoperative according to claim 9, characterized in that: Step b specifically includes: resampling and denoising the point cloud data obtained in step a, and performing coordinate normalization processing.

Images