A deep learning-based point cloud registration method for liver surgery from preoperative to intraoperative

Abstract

The present invention discloses a method for preoperative to intraoperative liver point cloud registration based on deep learning, comprising the following steps: obtaining existing real three-dimensional liver model data; preprocessing the existing real three-dimensional liver model data to obtain a complete preoperative liver point cloud and a partial intraoperative liver point cloud, using the corresponding complete preoperative liver point cloud and partial intraoperative liver point cloud as samples to obtain a sample data set; constructing an initial liver point cloud registration model based on a dual-branch network framework; using the sample data set to train the initial liver point cloud registration model to obtain a liver point cloud registration model; and using the liver point cloud registration model to perform actual liver point cloud registration. The method of the present invention not only greatly improves the registration accuracy, but also increases the registration speed, making it real-time. Compared with existing methods, it has the advantages of high registration accuracy and fast registration speed.

Description

Technical Field

[0001] The present invention relates to a three-dimensional point cloud registration method, and in particular to a point cloud registration method from pre-operative to intra-operative liver surgery based on deep learning. Background Art

[0002] Point cloud registration involves determining a rigid transformation to align one point cloud with another. Traditional point cloud registration methods first extract local features, then determine the point-to-point correspondences between the source and target point clouds through feature matching, and finally estimate the rigid transformation matrix.

[0003] During liver surgery, a complete liver surface point cloud can be segmented from CT images before the operation. However, during the operation, due to occlusion, the reconstructed liver surface point cloud will have a large number of omissions, and only a partial liver surface point cloud can be obtained. Because the liver model is created differently before and after surgery, it is generally difficult to determine the one-to-one correspondence between points. The loss of the liver surface during surgery and the deformation of the organ caused by the patient's breathing or squeezing of surgical instruments during the operation make feature matching very difficult. Therefore, a better method is needed to complete the registration method from the preoperative complete liver surface point cloud to the intraoperative partial surface point cloud.

[0004] In recent years, deep learning has achieved considerable success in point cloud registration. Learning-based methods can capture knowledge of biomechanical constraints in training data, addressing challenges associated with organ deformation and occlusion to a certain extent. However, in non-rigid organ registration, obtaining large amounts of real intraoperative data is difficult. Existing solutions rely on time-consuming biomechanical simulations to simulate deformation of a real liver model in order to obtain a large amount of training data.

[0005] Most existing deep learning registration frameworks are based on fully convolutional networks. However, convolutional neural networks suffer from local inductive biases. In registration tasks, the source and target point clouds are often far apart, making it difficult for convolutional networks to transfer information. Therefore, a method is urgently needed to better perform point cloud registration on non-rigid organs. Summary of the Invention

[0006] The purpose of the present invention is to provide a point cloud registration method from preoperative to intraoperative liver surgery based on deep learning, which improves the registration accuracy and speed by constructing a new neural network framework for point cloud registration.

[0007] The present invention provides a deep learning-based point cloud registration method for liver surgery from preoperative to intraoperative, comprising the following steps:

[0008] S1. Obtain existing real 3D liver model data;

[0009] S2. Preprocess the existing real 3D liver model data obtained in step S2 to obtain a complete preoperative liver point cloud and a partial intraoperative liver point cloud, and use the corresponding complete preoperative liver point cloud and partial intraoperative liver point cloud as samples to obtain a sample data set;

[0010] S3. Construct an initial liver point cloud registration model based on a dual-branch network framework;

[0011] S4. Using the sample data set obtained in step S2, the initial liver point cloud registration model obtained in step S3 is trained to obtain a liver point cloud registration model;

[0012] S5. Use the liver point cloud registration model obtained in step S4 to perform actual liver point cloud registration.

[0013] In step S1, the real three-dimensional liver model data is the complete liver surface model data segmented from the abdominal CT scan.

[0014] Step S2 specifically includes the following steps:

[0015] S21. Randomly sample the existing real 3D liver model data obtained in step S1 to obtain a complete point cloud of the liver before surgery as the source point cloud S;

[0016] S22. Filling the existing real 3D liver model data obtained in step S1 with tetrahedral elements, performing a biomechanical simulation, randomly adding a random force of a preset magnitude to the biomechanical simulation, and calculating using a finite element solver to obtain a biomechanically simulated deformed liver model;

[0017] S23. Randomly intercept the surface of the biomechanically simulated deformed liver model, resample and delete random vertices, add a uniform random displacement to the position of each remaining vertex, and randomly rotate and translate several point clouds of the processed model to obtain a partial point cloud of the liver during surgery, which is used as the target point cloud T;

[0018] S24. Use the corresponding preoperative liver complete point cloud and intraoperative liver partial point cloud as samples, increase the number of samples by flipping the point cloud, and obtain a sample dataset.

[0019] Step S22 specifically comprises: filling the surface mesh of the existing real three-dimensional liver model data obtained in step S1 with tetrahedral elements to obtain a filled three-dimensional liver surface model; performing a biomechanical simulation on the filled three-dimensional liver surface model, randomly selecting a number of surfaces to set zero-displacement boundary conditions, applying up to three random forces with a maximum value of 1.5 N to random locations on the surface, and calculating with a finite element solver using a neo-Hookean hyperelastic material model with a random Young's modulus (2 kPa to 5 kPa) and a Poisson's ratio of 0.35 to obtain a biomechanically simulated deformable liver model;

[0020] Step S23 specifically includes: randomly intercepting the surface of the biomechanically simulated deformed liver model, resampling and deleting random vertices, adding uniform random displacement to the position of each remaining vertex, and performing random rotation of up to 90° and translation of up to 15 cm on several point clouds of the processed model to obtain a point cloud of the liver during surgery;

[0021] Step S24 is specifically as follows: using the corresponding preoperative liver complete point cloud and intraoperative liver partial point cloud as samples to obtain an initial sample data set; flipping the initial sample data set along the X-axis, Y-axis, and Z-axis respectively to increase the number of samples and obtain a sample data set.

[0022] Step S3 is specifically as follows:

[0023] Based on a dual-branch network framework, an initial liver point cloud registration model is constructed; the initial liver point cloud registration model includes a feature extraction module and an SVD decomposition module;

[0024] The feature extraction module is a dual-branch network, including a source point cloud branch and a target point cloud branch. The source point cloud branch and the target point cloud branch have the same structure. The source point cloud branch includes the first EdgeConv layer, the second EdgeConv layer, the third EdgeConv layer, the first cross attention submodule, and the second cross attention submodule connected in series. The target point cloud branch includes the fourth EdgeConv layer, the fifth EdgeConv layer, the sixth EdgeConv layer, the third cross attention submodule, and the fourth cross attention submodule connected in series. The first to sixth EdgeConv layers are normalized using InstanceNorm and are used. GELU is used as the activation function; the point cloud input by the source point cloud branch first extracts local features through the first to third EdgeConv layers, and then exchanges information with the corresponding cross-attention sub-module in the target point cloud branch through the cross-attention sub-module to obtain global features, among which the first cross-attention sub-module exchanges information with the third cross-attention sub-module, and the second cross-attention sub-module exchanges information with the fourth cross-attention sub-module; after the outputs of the source point cloud branch and the target point cloud branch are connected in the feature dimension, the result is used as the input of the SVD decomposition module; the SVD decomposition module calculates the translation vector and rotation matrix based on the input data as the output of the model.

[0025] The output of the first cross-attention submodule is expressed as follows:

[0026]

[0027] Among them, Y S is the output of the first cross-attention submodule; X Sis the output of the third EdgeConv layer in the source point cloud branch; d k For X S The dimension of the feature channel; Q S For X S After linear transformation, K S The third cross attention submodule is the first input of the first cross attention submodule by exchanging information, V S The second input of the third cross-attention submodule to the first cross-attention submodule through the exchange of information is calculated using the following formula:

[0028] Q S =X S W Q

[0029] K S =X T W K

[0030] V S =X T W V

[0031] Among them, X T is the output of the sixth EdgeConv layer in the target point cloud branch; W Q is the first weight; W K is the second weight; W V is the third weight;

[0032] The output of the source point cloud branch is the output of the second cross-attention submodule, which is expressed using the following formula:

[0033]

[0034] Among them, Y S ' is the output of the second cross attention submodule; dd k Y S The dimension of the feature channel; Q' S Y S After linear transformation, K' S The fourth cross attention submodule exchanges information with the second cross attention submodule through the first input, V S ' is the second input of the fourth cross attention submodule to the second cross attention submodule by exchanging information, and is calculated using the following formula:

[0035] Q′ S =Y S WW Q

[0036] K′ S=Y T WW K

[0037] V′ S =Y T WW V

[0038] Among them, Y T is the output of the third cross-attention submodule; WW Q The fourth weight; WW K The fifth weight; WW V is the sixth weight;

[0039] The output of the third cross-attention submodule is expressed as follows:

[0040]

[0041] Among them, Y T is the output of the third cross-attention submodule; d' k For X T The dimension of the feature channel; Q T For X T After linear transformation, K T V is the first input of the first cross attention submodule to the third cross attention submodule by exchanging information. T The second input of the first cross-attention submodule to the third cross-attention submodule through the exchange of information is calculated using the following formula:

[0042] Q T =X T W′ Q

[0043] K T =X S W′ K

[0044] V T =X S W′ V

[0045] Among them, W' Q is the seventh weight; W' K is the eighth weight; W' Y The ninth weight;

[0046] The output of the target point cloud branch is the output of the fourth cross-attention submodule, which is expressed using the following formula:

[0047]

[0048] Among them, Y T' is the output of the fourth cross attention submodule; dd' k Y T The dimension of the feature channel; Q' T Y T After linear transformation, K' T The second cross attention submodule is the first input of the fourth cross attention submodule by exchanging information, V T ' is the second input of the second cross attention submodule to the fourth cross attention submodule by exchanging information, and is calculated using the following formula:

[0049] Q′ T =Y T WW′ Q

[0050] K′ T =Y S WW′ K

[0051] V′ T =Y S WW′ V

[0052] Among them, WW′ Q is the tenth weight; WW′ K is the eleventh weight; WW′ V is the twelfth weight;

[0053] The SVD decomposition module calculates the point-to-point correlation m between the source point cloud and the target point cloud based on the input data; then, the corresponding position of each point in the target point cloud in the source point cloud is calculated using the correlation m, and the corresponding relationship between the source point cloud and the target point cloud is obtained; finally, based on the corresponding relationship between the source point cloud and the target point cloud, the rotation matrix and translation vector are obtained through SVD decomposition;

[0054] The affinity vector is calculated using the following formula:

[0055] m(t i ,S)=softmax(Y S y ti T )

[0056] Among them, m(t i ,S) is point t i The correlation vector with all points in the source point cloud S; t i is the i-th point in the target point cloud; y ti Output Y of the target point cloud branch T The i-th row in ;

[0057] The correspondence between the source point cloud and the target point cloud is expressed using the following formula:

[0058] s i =S T m(t i ,S)

[0059] Among them, s i is the i-th point in the source point cloud.

[0060] In step S4, the initial liver point cloud registration model is trained using the quadratic registration error loss to obtain the liver point cloud registration model. The loss function Loss is expressed using the following formula:

[0061] Loss=∑(t i -t i ') 2

[0062] Among them, t i is the i-th point in the target point cloud; t i 'for s i The coordinates after transformation by the rotation matrix and translation vector output by the model.

[0063] Step S5 specifically includes the following steps:

[0064] a. Obtain the actual preoperative liver model point cloud and the intraoperative liver point cloud;

[0065] b. Processing the point cloud data obtained in step a to obtain a liver point cloud dataset;

[0066] c. Input the paired point cloud dataset obtained in step b into the liver point cloud registration model obtained in step S4 to obtain the transformation matrix and translation vector. Use the transformation matrix and translation vector to perform coordinate transformation on the preoperative liver model point cloud to complete the actual liver point cloud registration.

[0067] Step b is specifically as follows: resampling and denoising the point cloud data obtained in step a, and performing coordinate normalization processing.

[0068] The present invention discloses a deep learning-based registration method for preoperative complete liver point cloud to intraoperative partial point cloud, which not only greatly improves the registration accuracy, but also improves the registration speed, making it real-time. Compared with existing methods, it has the advantages of high registration accuracy and fast registration speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Figure 1 Schematic diagram of the process of the present invention. DETAILED DESCRIPTION

[0070] The present invention provides a point cloud registration method from preoperative to intraoperative liver surgery based on deep learning, the flow chart of which is as follows: Figure 1As shown, the following steps are included:

[0071] S1. Obtain existing real 3D liver model data;

[0072] In step S1, the real three-dimensional liver model data is the complete liver surface model data segmented from the abdominal CT scan.

[0073] S2. Preprocess the existing real 3D liver model data obtained in step S2 to obtain a complete preoperative liver point cloud and a partial intraoperative liver point cloud. Use the corresponding complete preoperative liver point cloud and partial intraoperative liver point cloud as samples to obtain a sample dataset, specifically comprising the following steps:

[0074] S21. Randomly sample the existing real 3D liver model data obtained in step S1 to obtain a complete point cloud of the liver before surgery as the source point cloud S;

[0075] S22. Filling the existing real three-dimensional liver model data obtained in step S1 with tetrahedral elements, then performing a biomechanical simulation, randomly adding random forces of a preset magnitude to the biomechanical simulation, and calculating using a finite element solver to obtain a biomechanically simulated deformed liver model. Specifically, filling the surface mesh of the existing real three-dimensional liver model data obtained in step S1 with tetrahedral elements to obtain a filled three-dimensional liver surface model; performing a biomechanical simulation on the filled three-dimensional liver surface model, randomly selecting several surfaces and setting zero-displacement boundary conditions, applying up to three random forces with a maximum of 1.5 N to random locations on the surface, and calculating using a finite element solver using a neo-Hookean hyperelastic material model with a random Young's modulus (2 kPa to 5 kPa) and a Poisson's ratio of 0.35 to obtain a biomechanically simulated deformed liver model.

[0076] S23. Randomly intercept the surface of the biomechanically simulated deformable liver model, resample and delete random vertices, add a uniform random displacement to the position of each remaining vertex, and randomly rotate and translate several point clouds of the processed model to obtain a partial point cloud of the liver during surgery, which serves as the target point cloud T. Specifically, randomly intercept the surface of the biomechanically simulated deformable liver model, resample and delete random vertices, add a uniform random displacement to the position of each vertex, and randomly rotate and translate several point clouds of the processed model by a maximum of 90 degrees and a maximum of 15 cm to obtain a partial point cloud of the liver during surgery.

[0077] S24. Using the corresponding preoperative complete liver point cloud and intraoperative partial liver point cloud as samples, the number of samples is increased by flipping the point cloud to obtain a sample data set. Specifically, using the corresponding preoperative complete liver point cloud and intraoperative partial liver point cloud as samples to obtain an initial sample data set; flipping the initial sample data set along the X-axis, Y-axis, and Z-axis respectively to increase the number of samples to obtain a sample data set.

[0078] S3. Construct an initial liver point cloud registration model based on a dual-branch network framework;

[0079] Based on a dual-branch network framework, an initial liver point cloud registration model is constructed; the initial liver point cloud registration model includes a feature extraction module and an SVD decomposition module;

[0080] The feature extraction module is a dual-branch network, including a source point cloud branch and a target point cloud branch. The source point cloud branch and the target point cloud branch have the same structure. The source point cloud branch includes the first EdgeConv layer, the second EdgeConv layer, the third EdgeConv layer, the first cross attention submodule, and the second cross attention submodule connected in series. The target point cloud branch includes the fourth EdgeConv layer, the fifth EdgeConv layer, the sixth EdgeConv layer, the third cross attention submodule, and the fourth cross attention submodule connected in series. The first to sixth EdgeConv layers are normalized using InstanceNorm and are used. GELU is used as the activation function; the point cloud input by the source point cloud branch first extracts local features through the first to third EdgeConv layers, and then exchanges information with the corresponding cross-attention sub-module in the target point cloud branch through the cross-attention sub-module to obtain global features, among which the first cross-attention sub-module exchanges information with the third cross-attention sub-module, and the second cross-attention sub-module exchanges information with the fourth cross-attention sub-module; after the outputs of the source point cloud branch and the target point cloud branch are connected in the feature dimension, the result is used as the input of the SVD decomposition module; the SVD decomposition module calculates the translation vector and rotation matrix based on the input data as the output of the model.

[0081] The output of the first cross-attention submodule is expressed as follows:

[0082]

[0083] Among them, Y S is the output of the first cross-attention submodule; X S is the output of the third EdgeConv layer in the source point cloud branch; d k For X S The dimension of the feature channel; Q S For X S After linear transformation, K SThe third cross attention submodule is the first input of the first cross attention submodule by exchanging information, V S The second input of the third cross-attention submodule to the first cross-attention submodule through the exchange of information is calculated using the following formula:

[0084] Q S =X S W Q

[0085] K S =X T W K

[0086] V S =X T W V

[0087] Among them, X T is the output of the sixth EdgeConv layer in the target point cloud branch; W Q is the first weight; W K is the second weight; W V is the third weight;

[0088] The output of the source point cloud branch is the output of the second cross-attention submodule, which is expressed using the following formula:

[0089]

[0090] Among them, Y S ' is the output of the second cross attention submodule; dd k Y S The dimension of the feature channel; Q' S Y S After linear transformation, K' S The fourth cross attention submodule exchanges information with the second cross attention submodule through the first input, V S ' is the second input of the fourth cross attention submodule to the second cross attention submodule by exchanging information, and is calculated using the following formula:

[0091] Q′ S =Y S WW Q

[0092] K′ S =Y T WW K

[0093] V′ S =Y T WW V

[0094] Among them, Y T is the output of the third cross-attention submodule; WW Q The fourth weight; WW K The fifth weight; WW V is the sixth weight;

[0095] The output of the third cross-attention submodule is expressed as follows:

[0096]

[0097] Among them, Y T is the output of the third cross-attention submodule; d' k For X T The dimension of the feature channel; Q T For X T After linear transformation, K T V is the first input of the first cross attention submodule to the third cross attention submodule by exchanging information. T The second input of the first cross-attention submodule to the third cross-attention submodule through the exchange of information is calculated using the following formula:

[0098] Q T =X T W′ Q

[0099] K T =X S W′ K

[0100] V T =X S W′ V

[0101] Among them, W' Q is the seventh weight; W' K is the eighth weight; W' Y The ninth weight;

[0102] The output of the target point cloud branch is the output of the fourth cross-attention submodule, which is expressed using the following formula:

[0103]

[0104] Among them, Y T ' is the output of the fourth cross attention submodule; dd' k Y T The dimension of the feature channel; Q' T Y T After linear transformation, K' TThe second cross attention submodule is the first input of the fourth cross attention submodule by exchanging information, V T ' is the second input of the second cross attention submodule to the fourth cross attention submodule by exchanging information, and is calculated using the following formula:

[0105] Q′ T =Y T WW′ Q

[0106] K′ T =Y S WW′ K

[0107] V′ T =Y S WW′ V

[0108] Among them, WW′ Q is the tenth weight; WW′ K is the eleventh weight; WW′ V is the twelfth weight;

[0109] The SVD decomposition module calculates the point-to-point correlation m between the source point cloud and the target point cloud based on the input data; then, the corresponding position of each point in the target point cloud in the source point cloud is calculated using the correlation m, and the corresponding relationship between the source point cloud and the target point cloud is obtained; finally, based on the corresponding relationship between the source point cloud and the target point cloud, the rotation matrix and translation vector are obtained through SVD decomposition;

[0110] The affinity vector is calculated using the following formula:

[0111] m(t i ,S)=softmax(Y S y ti T )

[0112] Among them, m(t i ,S) is point t i The correlation vector with all points in the source point cloud S; t i is the i-th point in the target point cloud; y ti Output Y of the target point cloud branch T The i-th row in ;

[0113] The correspondence between the source point cloud and the target point cloud is expressed using the following formula:

[0114] s i =S T m(t i ,S)

[0115] Among them, s iis the i-th point in the source point cloud.

[0116] S4. Using the sample data set obtained in step S2, the initial liver point cloud registration model obtained in step S3 is trained to obtain a liver point cloud registration model;

[0117] The initial liver point cloud registration model is trained using the quadratic registration error loss to obtain the liver point cloud registration model. The loss function Loss is expressed using the following formula:

[0118] Loss=∑(t i -t i ') 2

[0119] Among them, t i is the i-th point in the target point cloud; t i 'for s i The coordinates after transformation by the rotation matrix and translation vector output by the model.

[0120] S5. Using the liver point cloud registration model obtained in step S4, actual liver point cloud registration is performed, specifically comprising the following steps:

[0121] a. Obtain the actual preoperative liver model point cloud and the intraoperative liver point cloud;

[0122] b. Processing the point cloud data obtained in step a to obtain a liver point cloud dataset;

[0123] c. Input the paired point cloud dataset obtained in step b into the liver point cloud registration model obtained in step S4 to obtain the transformation matrix and translation vector. Use the transformation matrix and translation vector to perform coordinate transformation on the preoperative liver model point cloud to complete the actual liver point cloud registration.

[0124] Step b is specifically as follows: resampling and denoising the point cloud data obtained in step a, and performing coordinate normalization processing.

[0125] The method of the present invention is further described below with reference to an embodiment:

[0126] Using the method provided by this invention, a liver surface point cloud registration model was constructed using the publicly available liver model dataset 3D-IRCADb-01. This dataset was then subjected to point cloud registration using existing methods: DCP, LiverMatch, and REGTR. The registration error was calculated using MAE, MSE, and RMSE. The results are compared in Table 1.

[0127] Table 1 Comparison of registration accuracy between the method of the present invention and the existing method

[0128] Method MAE MSE RMSE DCP 0.023387 0.001187 0.034447 LiverMatch 0.021624 0.001059 0.032542 REGTR 0.019736 0.000940 0.030653 Ours 0.015206 0.000628 0.025056

[0129] It can be seen from the data in the table that the method disclosed in the present invention performs better than the existing methods on this data set.

Claims

1. A deep learning-based point cloud registration method for liver surgery from preoperative to intraoperative, characterized by: The following steps are involved: S1. Obtain existing real 3D liver model data; S2. Preprocess the existing real 3D liver model data obtained in step S1 to obtain a complete preoperative liver point cloud and a partial intraoperative liver point cloud. Use the corresponding complete preoperative liver point cloud and partial intraoperative liver point cloud as samples to obtain a sample dataset. S3. Construct an initial liver point cloud registration model based on a dual-branch network framework. S4 uses the sample data set obtained in step S2 to train the initial liver point cloud registration model obtained in step S3 to obtain a liver point cloud registration model; S5. Perform actual liver point cloud registration using the liver point cloud registration model obtained in step S4; Step S3 is specifically as follows: Based on a dual-branch network framework, an initial liver point cloud registration model is constructed; the initial liver point cloud registration model includes a feature extraction module and an SVD decomposition module connected in series; The feature extraction module is a dual-branch network, including a source point cloud branch and a target point cloud branch. The source point cloud branch and the target point cloud branch have the same structure. The source point cloud branch includes the first EdgeConv layer, the second EdgeConv layer, the third EdgeConv layer, the first cross attention submodule, and the second cross attention submodule connected in series. The target point cloud branch includes the fourth EdgeConv layer, the fifth EdgeConv layer, the sixth EdgeConv layer, the third cross attention submodule, and the fourth cross attention submodule connected in series. The first to sixth EdgeConv layers are normalized using InstanceNorm and GELU is used as the activation function; the point cloud input by the source point cloud branch first extracts local features through the first to third EdgeConv layers, and then exchanges information with the corresponding cross-attention submodule in the target point cloud branch through the cross-attention submodule to obtain global features. Among them, the first cross-attention submodule exchanges information with the third cross-attention submodule, and the second cross-attention submodule exchanges information with the fourth cross-attention submodule; the outputs of the source point cloud branch and the target point cloud branch are concatenated in the feature dimension, and the result is used as the input of the SVD decomposition module; the SVD decomposition module calculates the translation vector and rotation matrix based on the input data as the output of the model; In the feature extraction module, the output of the first cross-attention submodule is expressed using the following formula: ;in, is the output of the first cross-attention submodule; is the output of the third EdgeConv layer in the source point cloud branch; for The dimension of the feature channel; for After linear transformation, The third cross attention submodule exchanges the first input of the first cross attention submodule with information, The second input of the third cross-attention submodule to the first cross-attention submodule through the exchange of information is calculated using the following formula: , , ;in, is the output of the sixth EdgeConv layer in the target point cloud branch; is the first weight; is the second weight; is the third weight; The output of the source point cloud branch is the output of the second cross-attention submodule, which is expressed using the following formula: ;in, is the output of the second cross-attention submodule; for The dimension of the feature channel; for After linear transformation, The fourth cross attention submodule exchanges the first input of the second cross attention submodule with information, The second input of the fourth cross-attention submodule to the second cross-attention submodule by exchanging information is calculated using the following formula: , , ;in, is the output of the third cross-attention submodule; is the fourth weight; is the fifth weight; is the sixth weight; The output of the third cross-attention submodule is expressed as follows: ;in, is the output of the third cross-attention submodule; for The dimension of the feature channel; for After linear transformation, is the first input of the first cross attention submodule to the third cross attention submodule by exchanging information, The second input of the first cross-attention submodule to the third cross-attention submodule through the exchange of information is calculated using the following formula: , , ;in, is the seventh weight; is the eighth weight; The ninth weight; The output of the target point cloud branch is the output of the fourth cross-attention submodule, which is expressed using the following formula: ;in, is the output of the fourth cross-attention submodule; for The dimension of the feature channel; for After linear transformation, The second cross attention submodule exchanges the first input of the fourth cross attention submodule with the information, The second input of the second cross-attention submodule to the fourth cross-attention submodule through the exchange of information is calculated using the following formula: , , ;in, is the tenth weight; is the eleventh weight; It is the twelfth weight.

2. The deep learning-based point cloud registration method for liver surgery from preoperative to intraoperative according to claim 1, characterized in that: In step S1, the real three-dimensional liver model data is the complete liver surface model data segmented from the abdominal CT scan.

3. The deep learning-based point cloud registration method for liver surgery from preoperative to intraoperative according to claim 1, characterized in that: Step S2 specifically includes the following steps: S21. Randomly sample the existing real 3D liver model data obtained in step S1 to obtain a complete point cloud of the liver before surgery as the source point cloud S; S22. Filling the existing real 3D liver model data obtained in step S1 with tetrahedral elements, performing a biomechanical simulation, randomly adding random forces of a preset magnitude to the biomechanical simulation, and calculating using a finite element solver to obtain a biomechanically simulated deformed liver model; S23. Randomly intercept the surface of the biomechanically simulated deformed liver model, resample and delete random vertices, add uniform random displacements to the positions of each remaining vertex, and randomly rotate and translate several point clouds of the processed model to obtain a partial point cloud of the liver during surgery, which serves as the target point cloud T. S24. Use the corresponding preoperative liver complete point cloud and intraoperative liver partial point cloud as samples, and increase the number of samples by flipping the point clouds to obtain a sample dataset.

4. The method for point cloud registration from preoperative to intraoperative liver surgery based on deep learning according to claim 3, characterized in that: Step S22 specifically comprises: filling the surface mesh of the existing real three-dimensional liver model data obtained in step S1 with tetrahedral elements to obtain a filled three-dimensional liver surface model; performing a biomechanical simulation on the filled three-dimensional liver surface model, randomly selecting a number of surfaces and setting zero-displacement boundary conditions, applying up to three random forces with a maximum value of 1.5 N to random locations on the surface, and calculating with a finite element solver using a neo-Hookean hyperelastic material model with a random Young's modulus ranging from 2 kPa to 5 kPa and a Poisson's ratio of 0.35 to obtain a biomechanically simulated deformable liver model; Step S23 specifically includes: randomly intercepting the surface of the biomechanically simulated deformed liver model, resampling and deleting random vertices, adding uniform random displacement to the position of each remaining vertex, and performing random rotation of up to 90° and translation of up to 15 cm on several point clouds of the processed model to obtain a point cloud of the liver during surgery; Step S24 is specifically as follows: using the corresponding preoperative liver complete point cloud and intraoperative liver partial point cloud as samples to obtain an initial sample data set; flipping the initial sample data set along the X-axis, Y-axis, and Z-axis respectively to increase the number of samples and obtain a sample data set.

5. The deep learning-based point cloud registration method for liver surgery from pre-operative to intra-operative according to claim 1, characterized in that: The SVD decomposition module calculates the point-to-point correlation m between the source point cloud and the target point cloud based on the input data; then, the corresponding position of each point in the target point cloud in the source point cloud is calculated using the correlation m, and the corresponding relationship between the source point cloud and the target point cloud is obtained; finally, based on the corresponding relationship between the source point cloud and the target point cloud, the rotation matrix and translation vector are obtained through SVD decomposition; The affinity vector is calculated using the following formula: ;in, for point The correlation vector with all points in the source point cloud S; is the i-th point in the target point cloud; Output of the target point cloud branch The i-th row in ; The correspondence between the source point cloud and the target point cloud is expressed using the following formula: ;in, is the i-th point in the source point cloud.

6. The deep learning-based point cloud registration method for liver surgery from preoperative to intraoperative according to claim 1, characterized in that: In step S4, the initial liver point cloud registration model is trained using the quadratic registration error loss to obtain the liver point cloud registration model. The loss function Use the following formula to express it: ;in, is the i-th point in the target point cloud; for The coordinates after transformation by the rotation matrix and translation vector output by the model.

7. The method for point cloud registration from preoperative to intraoperative liver surgery based on deep learning according to claim 1, characterized in that: Step S5 specifically includes the following steps: a. Obtain the actual preoperative liver model point cloud and the intraoperative liver point cloud; b. Processing the point cloud data obtained in step a to obtain a liver point cloud dataset; c. Input the paired point cloud dataset obtained in step b into the liver point cloud registration model obtained in step S4 to obtain the transformation matrix and translation vector. Use the transformation matrix and translation vector to perform coordinate transformation on the preoperative liver model point cloud to complete the actual liver point cloud registration.

8. The deep learning-based point cloud registration method for liver surgery from pre-operative to intra-operative according to claim 7, characterized in that: Step b is specifically as follows: resampling and denoising the point cloud data obtained in step a, and performing coordinate normalization processing.

Images