Lung airway tree model reconstruction method and system based on implicit expression network

Abstract

The invention provides a lung airway tree model reconstruction method and system based on a hidden expression network, and the method comprises the steps: obtaining a three-dimensional medical image; carrying out volume rendering contour surface algorithm processing on the three-dimensional medical image to determine a first part data set; determining a second partial data set according to the first partial data set; training a preset implicit expression network based on the depth geometry corresponding relation by adopting the second part of data set, and determining the implicit expression network based on the depth geometry corresponding relation after model training is completed; and inputting the three-dimensional medical image containing the lung airway tree into the implicit expression network which completes model training and is based on the depth geometric corresponding relation, and determining a reconstructed lung airway tree model. According to the method and the device, the lung airway tree model is modeled in the continuous space based on the implicit expression network of the depth geometry corresponding relation, so that the integrity and fidelity of reconstruction of the lung airway tree model are improved.

Description

Technical Field

[0001] The present disclosure relates to the technical field of medical three-dimensional reconstruction, and in particular, to a method and system for reconstructing a lung airway tree model based on a latent expression network. Background Art

[0002] Detailed lung airway tree modeling is essential for navigation of endobronchial interventional procedures, which have shown great advantages in the treatment of chronic obstructive pulmonary disease, lung cancer, and the pandemic COVID-19 pneumonia. Due to the elaborate and complex branching topology of the lung airway tree, manual annotation is prone to errors, time-consuming, and highly dependent on the expertise of clinicians.

[0003] To obtain the airway tree shape for surgical path planning, labeling is first performed on the discrete voxel grid of the CT scan, and then a surface reconstruction algorithm, such as the Marching Cubes algorithm, is performed. Since the labeling is performed layer by layer in the image stack, the reconstructed shape is inevitably affected by the resolution of the discrete voxels. In addition, as the bronchi become thinner, the discrimination of CT images gradually decreases. These properties lead to a rough depiction of the airway tree structure accompanied by spike-like noise interference. Traditional Laplace smoothing algorithms cannot ensure high-fidelity airway modeling while eliminating spike-like noise interference. Noise is very unfavorable for central axis extraction, and it is extremely sensitive to small changes in the boundaries. Skeletonization algorithms based on reconstructed airways are prone to produce incorrect branches and connections, which will destroy the topological structure of the airway and mislead surgical path planning.

[0004] The refined pulmonary airway tree requires painstaking manual delineation by experienced clinicians, which has prompted the development of automatic segmentation algorithms. With the advancement of deep learning, especially the convolutional neural network-based methods, automatic airway tree modeling has become possible, but it also encounters the aforementioned problems because it still models airway structures in discrete voxel space. The convolutional neural network-based method can be trained with preoperatively annotated data to make inferences on the intraoperative data of the same patient, reducing the burden of repeated labeling for clinicians. However, convolutional neural networks show relatively weak performance in extracting complex shape prior knowledge, in which case breakage problems are prone to occur, resulting in interruptions in the trajectory of intraoperative navigation. Summary of the invention

[0005] To achieve the above objectives, in view of the defects in the prior art, the purpose of the present invention is to provide a method and system for reconstructing a lung airway tree model based on a latent expression network.

[0006] According to a first aspect of the present disclosure, a method for reconstructing a pulmonary airway tree model based on a latent expression network is provided, comprising:

[0007] Acquire a three-dimensional medical image data set, wherein the three-dimensional medical image data set includes three-dimensional medical image images;

[0008] Performing volume rendering isosurface algorithm processing on the three-dimensional medical image to determine a first part of the data set;

[0009] The first part of the data set is sequentially subjected to coordinate transformation processing based on physical information, shape alignment transformation processing within the data set, and point cloud sampling processing to determine the second part of the data set;

[0010] The second part of the data set is used to train the preset latent expression network based on deep geometric correspondence, and determine the latent expression network based on deep geometric correspondence that completes the model training;

[0011] The target three-dimensional medical image to be reconstructed is input into the latent expression network based on deep geometric correspondence that has completed model training to determine the reconstructed lung airway tree model.

[0012] Optionally, the method further comprises:

[0013] The reconstructed pulmonary airway tree model is transformed into a discrete voxel space through coordinates using the model-trained latent expression network based on deep geometric correspondence, and the reconstructed shape of the pulmonary airway tree in the discrete three-dimensional voxel space is determined.

[0014] Optionally, the data in the first part of the data set includes three-dimensional grid data.

[0015] Optionally, the performing volume rendering isosurface algorithm processing on the three-dimensional medical image to determine the first part of the data set includes:

[0016] Performing volume data extraction processing on the three-dimensional medical image to determine volume data of the three-dimensional medical image, wherein the volume data includes three-dimensional medical data and annotated organ true value data;

[0017] The volume data is processed by a volume rendering isosurface algorithm to determine the three-dimensional grid data.

[0018] Optionally, the step of sequentially performing coordinate transformation processing based on physical information, shape alignment transformation processing within the data set, and point cloud sampling processing on the first part of the data set to determine the second part of the data set includes:

[0019] According to the voxel interval, voxel origin and voxel arrangement direction of the pixels of the three-dimensional medical image, the image coordinates corresponding to the three-dimensional grid data are converted into world coordinates to determine the three-dimensional grid data having world coordinates;

[0020] Performing point cloud registration processing on the three-dimensional mesh data with world coordinates based on a preset reference system, determining a rigid transformation matrix in the world coordinate system, applying the rigid transformation matrix in the world coordinate system to each instance, and determining three-dimensional mesh data that has undergone shape alignment transformation processing in the data set;

[0021] Point cloud sampling processing is performed on the three-dimensional mesh data of the pulmonary airway tree in the three-dimensional mesh data that has been processed by the shape alignment transformation in the data set, and random sampling processing is performed in the coordinate space of the three-dimensional mesh data of the pulmonary airway tree that has been processed by the shape alignment transformation in the data set, and a signed distance function value between the point cloud that has been processed by the point cloud sampling processing and the point cloud that has been processed by the random sampling processing is obtained to determine the second part of the data set.

[0022] Optionally, the second part of the data set includes a second training set, a second validation set and a second test set.

[0023] Optionally, the using the second part of the data set to train a preset latent expression network based on deep geometric correspondence to determine the latent expression network based on deep geometric correspondence that has completed model training includes:

[0024] Using the second training set and the second validation set to train the preset latent expression network based on deep geometric correspondence, and determine the latent expression network based on deep geometric correspondence that has been trained by the model;

[0025] Fine-tune the latent expression network based on deep geometric correspondence that has been trained by the model to determine the latent expression network based on deep geometric correspondence that has been fine-tuned;

[0026] The second test set is used to test the latent expression network based on deep geometric correspondence that has been fine-tuned to determine the latent expression network based on deep geometric correspondence that has completed model training.

[0027] Optionally, the preset implicit expression network based on deep geometric correspondence includes an encoder module shared by each instance and driven by an implicit template vector, a decoder module unique to each instance and driven by a latent shape vector of each instance, and a signed distance function prediction module driven by the latent shape vector.

[0028] Optionally, the using the second part of the data set to train a preset latent expression network based on deep geometric correspondence to determine the latent expression network based on deep geometric correspondence that has completed model training also includes:

[0029] A preset overall loss function is used to perform parameter optimization processing on the model-trained latent expression network based on deep geometric correspondence.

[0030] According to a second aspect of the present disclosure, a pulmonary airway tree model reconstruction system based on a latent expression network is provided, comprising:

[0031] An image acquisition module, used to acquire a three-dimensional medical image dataset, wherein the three-dimensional medical image dataset includes three-dimensional medical images;

[0032] A first data processing module is used to perform volume rendering isosurface algorithm processing on the three-dimensional medical image to determine a first part of the data set;

[0033] A second data processing module is used to sequentially perform coordinate transformation processing based on physical information, shape alignment transformation processing within the data set, and point cloud sampling processing on the first part of the data set to determine a second part of the data set;

[0034] A model training module, used to train a preset latent expression network based on deep geometric correspondence using the second part of the data set, and determine the latent expression network based on deep geometric correspondence that completes the model training;

[0035] The reconstruction module is used to input the target three-dimensional medical image to be reconstructed into the latent expression network based on deep geometric correspondence that has completed model training, and determine the reconstructed lung airway tree model.

[0036] According to a third aspect of the present disclosure, a non-temporary computer-readable storage medium is provided, on which a computer program is stored, characterized in that when the program is executed by a processor, the steps of any one of the methods provided in the first aspect of the present disclosure are implemented.

[0037] According to a fourth aspect of the present disclosure, a terminal is provided, including:

[0038] a memory having a computer program stored thereon;

[0039] A processor is used to execute the computer program in the memory to implement the steps of any one of the methods provided in the first aspect of the present disclosure.

[0040] Compared with the prior art, the embodiments of the present disclosure have at least one of the following beneficial effects:

[0041] Through the above technical solution, an implicit expression network based on deep geometric correspondence is adopted to model the lung airway tree model in continuous space, and the intrinsic topological features shared between different airway cases are explored on the basis of implicit neural representation. Implicit geometric regularization is used to promote the smooth and high-fidelity implicit representation of small airway structures. By going beyond the representation of discrete voxel space, an airway tree structure with advanced topological fidelity is generated, and the lung airway tree model is reconstructed. The completeness and fidelity of the reconstruction of the lung airway tree model are improved, which is conducive to improving the robustness of the skeleton extraction algorithm, alleviating the fracture phenomenon of the lung airway tree model, and assisting doctors in quantitative analysis of airway-related diseases and clinical application of preoperative path planning. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Other features, objects and advantages of the present disclosure will become more apparent by reading the detailed description of non-limiting embodiments with reference to the following drawings:

[0043] Figure 1 The present invention is a flowchart of a method for reconstructing a lung airway tree model based on a latent expression network according to an exemplary embodiment.

[0044] Figure 2 It is a schematic diagram showing the role of a latent expression network based on deep geometric correspondence in the reconstruction of a lung airway tree model according to an exemplary embodiment.

[0045] Figure 3 It is a schematic diagram showing a comparison of effects of reconstruction of a lung airway tree model according to an exemplary embodiment.

[0046] Figure 4 The figure is a schematic diagram showing a comparison of skeleton extraction effects of a lung airway tree model according to an exemplary embodiment.

[0047] Figure 5 It is a schematic diagram showing a comparison of the repair effects of a pulmonary airway tree model fracture according to an exemplary embodiment.

[0048] Figure 6 It is a block diagram of a lung airway tree model reconstruction system based on a latent expression network according to an exemplary embodiment. DETAILED DESCRIPTION

[0049] The present disclosure is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present disclosure, but are not intended to limit the present disclosure in any form. It should be noted that, for those of ordinary skill in the art, several variations and improvements may be made without departing from the concept of the present disclosure. These all fall within the scope of protection of the present disclosure.

[0050] Figure 1The present invention is a flowchart of a method for reconstructing a lung airway tree model based on a latent expression network according to an exemplary embodiment.

[0051] like Figure 1 As shown, the present disclosure provides a method for reconstructing a pulmonary airway tree model based on a latent expression network, including S11 to S15.

[0052] S11, obtaining a three-dimensional medical image.

[0053] The three-dimensional medical image includes an image containing a pulmonary airway tree and labeled organ truth data. In the present disclosure, the labeled organ truth data is labeled pulmonary airway tree truth data.

[0054] S12, performing volume rendering isosurface algorithm processing on the three-dimensional medical image to determine the first part of the data set.

[0055] Among them, the discrete voxel space of the three-dimensional medical image is processed by volume rendering isosurface algorithm.

[0056] The data in the first part of the data set includes three-dimensional mesh data, wherein the three-dimensional mesh data is mesh data of a three-dimensional shape surface.

[0057] The first part of the data set includes a first training set, a first validation set, and a first test set.

[0058] S13, performing coordinate transformation processing based on physical information, shape alignment transformation processing within the data set, and point cloud sampling processing on the first part of the data set in sequence to determine the second part of the data set.

[0059] The second part of the data set includes a second training set, a second validation set and a second test set.

[0060] S14, using the second part of the data set to train the preset latent expression network based on deep geometric correspondence, and determine the latent expression network based on deep geometric correspondence that completes the model training.

[0061] Among them, the second training set and the second validation set in the second part of the data set are used to perform model training on the preset latent expression network based on deep geometric correspondence, and the second test set is used to test the latent expression network based on deep set correspondence that has been trained by the model.

[0062] S15, inputting the target three-dimensional medical image to be reconstructed into the latent expression network based on deep geometric correspondence that has completed model training, and determining the reconstructed lung airway tree model.

[0063] In a possible embodiment, a three-dimensional medical image including a pulmonary airway tree is preprocessed to extract volume data of the three-dimensional medical image including the pulmonary airway tree, wherein the volume data of the three-dimensional medical image including the pulmonary airway tree includes three-dimensional image data of the three-dimensional medical image including the pulmonary airway tree and annotated true value data of the pulmonary airway tree.

[0064] The volume data of the three-dimensional medical image containing the pulmonary airway tree is processed by a volume rendering isosurface algorithm to obtain three-dimensional mesh data of the three-dimensional medical image containing the pulmonary airway tree.

[0065] The three-dimensional mesh data of the three-dimensional medical image containing the pulmonary airway tree and the annotated pulmonary airway tree true value data are input into the latent expression network based on deep geometric correspondence that completes the model training to determine the reconstructed pulmonary airway tree model.

[0066] Through the above technical solution, an implicit expression network based on deep geometric correspondence is adopted to model the lung airway tree model in continuous space, and the intrinsic topological features shared between different airway cases are explored on the basis of implicit neural representation. Implicit geometric regularization is used to promote the smooth and high-fidelity implicit representation of small airway structures. By going beyond the representation of discrete voxel space, an airway tree structure with advanced topological fidelity is generated, and the lung airway tree model is reconstructed. The completeness and fidelity of the reconstruction of the lung airway tree model are improved, which is conducive to improving the robustness of the skeleton extraction algorithm, alleviating the fracture phenomenon of the lung airway tree model, and assisting doctors in quantitative analysis of airway-related diseases and clinical application of preoperative path planning.

[0067] In a possible embodiment, a method for reconstructing a pulmonary airway tree model based on a latent expression network may further include S16.

[0068] The reconstructed pulmonary airway tree model is transformed into a discrete voxel space through coordinate transformation using a model-trained latent expression network based on deep geometric correspondence, and the reconstructed shape of the pulmonary airway tree in the discrete three-dimensional voxel space is determined.

[0069] Among them, a latent expression network based on deep geometric correspondence that has been trained by a model is used to transform the reconstructed pulmonary airway tree model into a discrete voxel space through coordinates, including: using a latent expression network based on deep geometric correspondence that has been trained by a model to perform an inverse processing of the shape alignment transformation processing in the data set and an inverse processing of the coordinate transformation processing based on physical information on the reconstructed pulmonary airway tree model, so as to transform the reconstructed pulmonary airway tree model into a discrete voxel space through coordinates.

[0070] In a possible embodiment, S12 may include S21.

[0071] S21, performing volume data extraction processing on the three-dimensional medical image to determine the volume data of the three-dimensional medical image.

[0072] Among them, volume data includes three-dimensional medical data and labeled organ true value data.

[0073] S22, performing volume rendering isosurface algorithm processing on the volume data to determine three-dimensional grid data.

[0074] Volume rendering isosurface algorithm processing, also known as marching cubes algorithm processing.

[0075] As an example, the three-dimensional medical data and the annotated lung airway tree true value data are processed by volume rendering isosurface algorithm to obtain mesh data of the three-dimensional shape surface.

[0076] In a possible embodiment, S13 may include S31 to S33.

[0077] S31, converting the image coordinates corresponding to the three-dimensional grid data into world coordinates according to the voxel interval, voxel origin and voxel arrangement direction of the pixels of the three-dimensional medical image, and determining the three-dimensional grid data with world coordinates.

[0078] The coordinate transformation process T1 based on physical information is represented as follows: using the voxel interval, voxel origin and voxel arrangement direction of the pixels in the three-dimensional medical image to transform the image coordinates corresponding to the three-dimensional grid data into world coordinates.

[0079] S32, performs point cloud registration processing on the three-dimensional mesh data with world coordinates based on a preset reference system, determines the rigid transformation matrix in the world coordinate system, applies the rigid transformation matrix in the world coordinate system to each instance, and determines the three-dimensional mesh data processed by the shape alignment transformation in the data set.

[0080] Among them, the preset reference system is: constructing a standard reference system according to a preset template shape as the preset reference system.

[0081] As an example, the shape alignment transformation processing T2 in the data set is represented as: performing point cloud registration processing on the three-dimensional mesh data of the lung airway tree of each instance and the mesh data of a preset template shape under a preset reference system, that is, the shape alignment transformation processing in the data set.

[0082] As another example, applying a rigid transformation matrix in a world coordinate system to each instance includes: applying the rigid transformation matrix in a world coordinate system to the three-dimensional mesh data of the lung airway tree of each instance using a coherent point drift algorithm.

[0083] S33, performing point cloud sampling processing on the three-dimensional mesh data of the pulmonary airway tree in the three-dimensional mesh data that has been processed by the shape alignment transformation in the data set, and performing random sampling processing in the coordinate space of the three-dimensional mesh data of the pulmonary airway tree that has been processed by the shape alignment transformation in the data set, obtaining the signed distance function value between the point cloud that has been processed by the point cloud sampling processing and the point cloud that has been processed by the random sampling processing, and determining the second part of the data set.

[0084] The three-dimensional mesh data that has undergone the shape alignment transformation process in the data set is three-dimensional mesh data that has undergone a coordinate transformation process T1 based on physical information and a shape alignment transformation process T2 in the data set in sequence.

[0085] For example, point cloud sampling is performed on the surface of the three-dimensional mesh data of the pulmonary airway tree that has been processed by the shape alignment transformation in the data set, the three-dimensional coordinates (x, y, z) of the point cloud data are determined, and the same number of point cloud data as that sampled on the surface of the three-dimensional mesh data is randomly sampled in the entire space, as well as the related attribute data of the randomly sampled point cloud data. The normal vector of the current point cloud sampled is (n x ,n y ,n z ), the signed distance function value between the current point cloud and the randomly sampled power source is s.

[0086] The second part of the data set includes point cloud data sampled from the surface point cloud of the three-dimensional grid data, point cloud data randomly sampled in the entire space with the same number of point cloud sampling, and related data of the randomly sampled point cloud data.

[0087] Figure 2 It is a schematic diagram showing the role of a latent expression network based on deep geometric correspondence in the reconstruction of a lung airway tree model according to an exemplary embodiment.

[0088] like Figure 2 As shown, in a possible embodiment, the preset implicit expression network based on deep geometric correspondence includes an encoder module E driven by an implicit template vector shared by each instance, a decoder module D driven by a potential shape vector of each instance unique to each instance, and a signed distance function prediction module F driven by a potential shape vector. The preset implicit expression network based on deep geometric correspondence is expressed as:

[0089]

[0090] Among them, M1 represents the preset implicit expression network based on deep geometric correspondence. represents the encoder module driven by the implicit template vector shared by each instance, β represents the K-dimensional implicit template latent vector, φ βrepresents a multi-layer perceptron weight prediction network driven by β, represents a decoder module unique to each instance driven by the latent shape vector of each instance, α i represents the K-dimensional latent shape vector of the i-th instance, Indicated by α i The multi-layer perceptron weight prediction network driven by , p represents the three-dimensional coordinates of the point cloud, represents the signed distance function prediction module driven by the latent shape vector, and s represents the true value of the signed distance function corresponding to the 3D coordinates of the point cloud.

[0091] Among them, the multi-layer perceptron weight prediction network φ driven by β β For prediction The weight of each layer of the neural network in i Driven multi-layer perceptron weight prediction network For prediction The weights of each layer of the neural network.

[0092] like Figure 2 As shown, the encoder module E shared by each instance and driven by the implicit template vector, the decoder module D unique to each instance and driven by the latent shape vector of each instance are located in the shared space, and the signed distance function prediction module F driven by the latent shape vector is located in the instance space.

[0093] As an example, in a signed distance function prediction module F driven by a latent shape vector, represents the signed distance function prediction module driven by the latent shape vector, Indicated by α i Driven multi-layer perceptron weight prediction network to generate The weights of each layer of the neural network in The weights of the signed distance function prediction module driven by the latent shape vector The input is the three-dimensional coordinates p = {x, y, z} of the point cloud sampled, and the output is the predicted signed distance function value.

[0094] As another example, in the encoder module E driven by the implicit template vector, the encoder module E driven by the implicit template vector is composed of a multi-layer perceptron, represents the encoder module driven by the implicit template vector shared by each instance, β represents the K-dimensional implicit template latent vector, φ β represents a multi-layer perceptron weight prediction network driven by β, and an encoder module driven by an implicit template vector The input of the point cloud is the three-dimensional coordinates p = {x, y, z} of the point cloud sampled and the signed distance function prediction module driven by the potential shape vector. The predicted signed distance function value in , the output is the K-dimensional intermediate implicit feature z i .

[0095] Encoder module driven by implicit template vector The operation expression is as follows:

[0096]

[0097] As another example, in a decoder module D unique to each instance driven by the latent shape vector of each instance, represents a decoder module unique to each instance driven by the latent shape vector of each instance, α i represents the K-dimensional latent shape vector of the i-th instance, Indicated by α i A multi-layer perceptron weight prediction network driven by each instance, a decoder module unique to each instance driven by the latent shape vector of each instance The input is the K-dimensional intermediate implicit feature z i The output is the three-dimensional coordinate information, which is then input into the signed distance function prediction module driven by the potential shape vector. , predict the final signed distance function value s.

[0098] A decoder module unique to each instance driven by the latent shape vector of each instance The encoder module shared by each instance is driven by the implicit template vector The operation expression is as follows:

[0099]

[0100] In a possible embodiment, S14 may include S41 to S43.

[0101] S41, using the second training set and the second validation set to train the preset latent expression network based on deep geometric correspondence, to determine the latent expression network based on deep geometric correspondence that has been trained by the model.

[0102] Among them, the second training set is used to train the preset latent expression network based on deep set correspondence; the second verification set is used to verify the preset latent expression network based on deep set correspondence, and the performance of the latent expression network based on deep geometric correspondence on the verification set is evaluated.

[0103] S42, fine-tuning the latent expression network based on deep geometric correspondence that has been trained by the model, and determining the latent expression network based on deep geometric correspondence that has been fine-tuned.

[0104] Among them, in the model training stage, the latent expression network based on deep geometric correspondence that has been trained by the model is fine-tuned, the parameters of the latent expression network based on deep geometric correspondence are adjusted, the generalization ability of the model is optimized, and the latent expression network based on deep geometric correspondence that has been fine-tuned is determined.

[0105] The fine-tuned latent expression network based on deep geometric correspondence can be used to repair the broken lung airway model.

[0106] S43, using the second test set to test the latent expression network based on deep geometric correspondence that has been fine-tuned, to determine the latent expression network based on deep geometric correspondence that has completed model training.

[0107] The fine-tuned deep geometric correspondence based latent expression network is tested by using the second test set to conduct a final model performance evaluation on the trained and fine-tuned deep geometric correspondence based latent expression network to evaluate the accuracy, generalization ability and stability of the model.

[0108] The latent expression network based on deep set correspondence that has completed model training after testing can be used to reconstruct the lung airway tree model, and the reconstructed lung airway tree model is transformed into a discrete three-dimensional voxel space through coordinate transformation and then skeleton extraction is performed to obtain the target shape.

[0109] In a possible embodiment, S14 may also include S44.

[0110] The preset overall loss function is used to optimize the parameters of the model-trained latent expression network based on deep geometric correspondence. The preset overall loss function is expressed as:

[0111] L total =L sdf +L reg +L cor

[0112] Among them, L totaf represents the overall loss function, L sdf represents the optimization objective function of the signed distance function, L reg represents the regularization constraint on the potential shape vector and implicit template vector of each instance, L cor Represents the constraints on the correspondence of the latent representation network based on deep geometric correspondence.

[0113] Optimization objective function L of the signed distance function sdf Defined as:

[0114]

[0115] Among them, in the first term, Ω represents the entire point set space, p represents the three-dimensional coordinates of the point cloud, α represents the K-dimensional potential shape vector of each instance, β represents the K-dimensional implicit template potential vector, s represents the true value of the signed distance function corresponding to the three-dimensional coordinates of the point cloud, and ω s Represents the weight coefficient of the first term.

[0116] The weight coefficient of the first term is ω s 3e3 is adopted in this disclosure.

[0117] In a possible embodiment, the optimization objective function L of the signed distance function is sdf The first term uses the mean absolute error to determine the loss, namely L1 loss.

[0118] Among them, in the second term, ρ=exp(-δ*|s|), δ>>1, ω ρ Represents the weight coefficient of the second term.

[0119] The second term represents the penalty term for points on the edge surface of the reconstructed non-3D shape, which is used to impose a penalty on points that are incorrectly predicted to be on the edge.

[0120] The weight coefficient of the second term ω ρ 5e2 is adopted in this disclosure.

[0121] Among them, the third Represents the edge surfaces of a three-dimensional shape, Represents the first-order derivative of the output of the latent expression network based on the deep geometric correspondence to the input, that is, the normal vector of the predicted point, n = {n x ,n y ,n z} represents the true value of the normal vector of the predicted point, <·> represents the vector inner product, ω n Represents the weight coefficient of the third term.

[0122] The third term represents the constraint term on the normal vector of the reconstructed organ shape, in which the cosine similarity between the normal vector of the latent expression network based on deep geometric correspondence and the true normal vector is maintained at a high level.

[0123] The weight coefficient of the third term ω n 1e2 is adopted in this disclosure.

[0124] Among them, the fourth term ω Eik Represents the weight coefficient of the fourth term.

[0125] The weight coefficient of the fourth term is 5e1 in the present disclosure.

[0126] The fourth term represents a constraint on the gradient of the reconstructed signed distance function value.

[0127] In a possible embodiment, the optimization objective function L of the signed distance function is sdr The fourth term can be determined by the minimum square error method, that is, the L2 loss, which is determined by the properties of the signed distance function. The modulus of the gradient of each point on the signed distance function is 1.

[0128] The regularization constraint L for each instance’s latent shape vector α and the implicit template latent vector β is reg Defined as the intra-class regularization term and the inter-class regularization term:

[0129] L reg =ω intra L intra-reg +ω inter L inter-reg ,

[0130] Among them, L intra-reg Defined as:

[0131]

[0132] Among them, L inter-reg Defined as:

[0133]

[0134] Among them, ω intra represents the first weight coefficient, ω inter Represents the second weight coefficient.

[0135] In the present disclosure, the first weight coefficient ω inter Using 1e6, the second weight coefficient ω inter Use 1e2.

[0136] The constraint term L on the correspondence of the latent representation network based on deep geometric correspondence cor Defined as:

[0137]

[0138] Among them, ω cor Represents the constraint L cor The weight coefficient of .

[0139] In the present disclosure, the constraint L cor The weight coefficient ω cor 5e4 can be used.

[0140] The constraint term L of the correspondence relationship of the latent representation network based on deep geometric correspondence cor For constrained decoder modules The decoded coordinates and original coordinates ensure the accuracy of the depth correspondence.

[0141] In a possible embodiment, the constraint term L of the correspondence relationship of the latent expression network based on the deep geometric correspondence relationship is cor The loss can be determined by the method of minimum square error, that is, L2 loss.

[0142] Step S44 of the present disclosure is performed after step S41 in the model training phase.

[0143] In a possible embodiment, 90 three-dimensional chest CT scan images are used as a three-dimensional medical image training set to train a preset latent expression network based on deep set correspondence, and the network is divided into a training set, a validation set and a test set, wherein the training set includes 50 three-dimensional chest CT scan images, the validation set includes 20 three-dimensional chest CT scan images, and the test set includes 20 three-dimensional chest CT scan images.

[0144] The size of the target three-dimensional medical image to be reconstructed is set to z×512×512 pixels, where z represents the number of 2D slices.

[0145] The preset latent expression network based on deep geometric correspondence is built using the Pytorch 1.12.0 platform, and the computing hardware is 2 Nvidia GTX 3090GPUs.

[0146] In the process of training the latent expression network based on deep geometric correspondence, Adam is used as the optimizer, the initial learning rate is 1e-4, the batch is set to 16, the sampling points of each batch of each instance are 8K, and the model training process is 500 generations.

[0147] Figure 3 It is a schematic diagram showing a comparison of effects of reconstruction of a lung airway tree model according to an exemplary embodiment.

[0148] like Figure 3 As shown, under the same experimental conditions, the lung airway tree model reconstruction method based on latent expression network provided by the present invention and the cube marching algorithm (MC), cube marching algorithm plus Laplace smoothing post-processing (MC w / Lap), Occ, Con-OCC, DeepSDF, and DIF are used to reconstruct the lung airway tree model of the target three-dimensional medical image to be reconstructed in the test set.

[0149] Depend on Figure 3It can be seen that the lung airway tree model reconstruction method based on latent expression network provided by the present disclosure can reconstruct the lung airway tree model with fidelity and minimal reconstruction error.

[0150] In a possible embodiment, the pulmonary airway tree model reconstruction method based on latent expression network provided by the present disclosure and the legislative body marching algorithm (MC), cube marching algorithm plus Laplace smoothing post-processing (MC w / Lap), Occ, Con-OCC, DeepSDF, and DIF are used to perform quantitative error analysis on 20 three-dimensional chest CT scan images in the test set, and the evaluation indicators are: the number of non-manifold vertices (NM-V), the number of non-manifold edges (NM-E), the number of non-manifold faces (NM-F), dice score coefficient (DSC (%)), tracheal tree length detection rate (tree length detected rate, TD (%)), and tracheal tree branch detection rate (branch detected rate, BD (%)).

[0151] Among them, the number of non-manifold vertices (NM-V) represents three or more non-coplanar connected vertices, the number of non-manifold edges (NM-E) represents the edges shared by three or more faces, and the number of non-manifold faces (NM-F) represents the faces that cannot form a closed continuum. Fewer non-manifold vertices (NM-V), non-manifold edges (NM-E), and non-manifold faces (NM-F) indicate better connectivity and smoothness of the reconstructed pulmonary airway tree model.

[0152] The dice score coefficient (DSC (%)) is defined as follows:

[0153]

[0154] Where i represents the pixel index, N represents the total number of pixels, and p i It represents the binary segmentation result of a specific point in the prediction result, and gi represents the binary segmentation result of a specific point manually marked.

[0155] The value range of Dice score coefficient (DSC(%)) is 0-1, and the closer the DSC value is to 1, the better the effect of the method.

[0156] The tracheal tree length detection rate (tree length detected rate, TD (%)) is defined as follows:

[0157]

[0158] Among them, T det represents the total length of all branches detected in the prediction, T ref Represents the length of the entire tree in the true data.

[0159] The tracheal tree branch detection rate (BD (%)) is defined as follows:

[0160]

[0161] Among them, B det Indicates the number of correctly detected branches of the tracheal tree, B ref Indicates the total number of branches in the tracheal tree.

[0162] The test results of the 20 three-dimensional chest CT scan images in the test set using the above method are shown in Table 1:

[0163] method NM-V NM-E NM-F TD BD DSC MC 30.9 120.1 2102.4 -- -- -- MCw / Lap 15.0 450.7 1656.9 -- -- -- Occ 4.7 12.6 338.3 28.7 27.2 73.5 Con-Occ 2.1 20.4 1309.7 43.2 43.6 75.1 DeepSDF 1.5 7.1 276.5 82.3 84.2 79.8 DIFNet 0.7 5.1 265.9 89.4 91.5 82.5 The disclosed method 0.0 4.6 235.3 98.6 99.7 91.2

[0164] Table 1

[0165] As shown in Table 1, the lung airway tree model reconstruction method based on the hidden expression network provided by the present disclosure has achieved good results in various indicators of the lung airway tree model reconstruction, and achieved an average of 0.0 NM-V, 4.6 NM-E, 235.3 NM-F, 98.6% TD, 99.7% BD, and 91.2% Dice on the test set. Compared with other methods, the lung airway tree model reconstruction method based on the hidden expression network provided by the present disclosure can better improve the accuracy of the lung airway tree model reconstruction.

[0166] Figure 4 The figure is a schematic diagram showing a comparison of skeleton extraction effects of a lung airway tree model according to an exemplary embodiment.

[0167] like Figure 4 As shown, in a possible embodiment, the lung airway tree model reconstruction method based on latent expression network provided by the present disclosure and skel, cldice, and gbo-skel are used to extract the skeleton of the lung airway tree model.

[0168] Depend on Figure 4 It can be seen that the lung airway tree model reconstruction method based on latent expression network provided by the present disclosure can improve the robustness of the lung airway tree model skeleton extraction.

[0169] Figure 5 It is a schematic diagram showing a comparison of the repair effects of a pulmonary airway tree model fracture according to an exemplary embodiment.

[0170] like Figure 5 As shown, the repair effect of the broken pulmonary airway tree model obtained by the hidden expression network based on the deep geometric correspondence after training and fine-tuning the pulmonary airway tree model reconstruction method based on the hidden expression network provided by the present disclosure is better than the repair effect of the broken pulmonary airway tree model obtained by fine-tuning the hidden expression network based on the deep set correspondence without the training of the present disclosure, and is closer to the true value of the pulmonary airway tree model that is not broken. It can be obtained that the pulmonary airway tree model reconstruction method based on the hidden expression network provided by the present disclosure can be used to repair the pulmonary airway tree model, and the repair effect of the broken pulmonary airway tree is fidelity.

[0171] Figure 6 It is a block diagram of a lung airway tree model reconstruction system based on a latent expression network according to an exemplary embodiment.

[0172] Based on the same concept, the present disclosure also provides a pulmonary airway tree model reconstruction system 100 based on latent expression network, such as Figure 6 As shown, it includes: an image acquisition module 110, a first data processing module 120, a second data processing module 130, a model training module 140, and a reconstruction module 150.

[0173] An image acquisition module 110 is used to acquire a three-dimensional medical image dataset, wherein the three-dimensional medical image dataset includes three-dimensional medical images;

[0174] A first data processing module 120 is used to perform volume rendering isosurface algorithm processing on the three-dimensional medical image to determine a first part of the data set;

[0175] The second data processing module 130 is used to sequentially perform coordinate transformation processing based on physical information, shape alignment transformation processing within the data set, and point cloud sampling processing on the first part of the data set to determine a second part of the data set;

[0176] A model training module 140 is used to train a preset latent expression network based on deep geometric correspondence using the second part of the data set to determine a latent expression network based on deep geometric correspondence that has completed model training;

[0177] The reconstruction module 150 is used to input the target three-dimensional medical image to be reconstructed into the latent expression network based on deep geometric correspondence that has completed model training, and determine the reconstructed lung airway tree model.

[0178] Through the above technical solution, an implicit expression network based on deep geometric correspondence is adopted to model the lung airway tree model in continuous space, and the intrinsic topological features shared between different airway cases are explored on the basis of implicit neural representation. Implicit geometric regularization is used to promote the smooth and high-fidelity implicit representation of small airway structures. By going beyond the representation of discrete voxel space, an airway tree structure with advanced topological fidelity is generated, and the lung airway tree model is reconstructed. The completeness and fidelity of the reconstruction of the lung airway tree model are improved, which is conducive to improving the robustness of the skeleton extraction algorithm, alleviating the fracture phenomenon of the lung airway tree model, and assisting doctors in quantitative analysis of airway-related diseases and clinical application of preoperative path planning.

[0179] Regarding the embodiment of the above system, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0180] Based on the same concept as above, in another embodiment of the present disclosure, an electronic device is also provided, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor, and the processor is used to execute a method for reconstructing a pulmonary airway tree model based on an implicit expression network when executing the program.

[0181] Optionally, the memory is used to store programs; the memory may include volatile memory (English: volatile memory), such as random-access memory (English: random-access memory, abbreviated: RAM), such as static random-access memory (English: static random-access memory, abbreviated: SRAM), double data rate synchronous dynamic random access memory (English: Double Data Rate Synchronous Dynamic Random Access Memory, abbreviated: DDR SDRAM), etc.; the memory may also include non-volatile memory (English: non-volatile memory), such as flash memory (English: flash memory). The memory is used to store computer programs (such as applications, functional modules, etc. that implement the above method), computer instructions, etc., and the above computer programs, computer instructions, etc. can be partitioned and stored in one or more memories. And the above computer programs, computer instructions, data, etc. can be called by the processor.

[0182] The above-mentioned computer programs, computer instructions, etc. may be stored in one or more memories in partitions, and the above-mentioned computer programs, computer instructions, data, etc. may be called by a processor.

[0183] The processor is used to execute the computer program stored in the memory to implement the various steps in the method involved in the above embodiment. For details, please refer to the relevant description in the above method embodiment.

[0184] The processor and the memory may be independent structures or integrated structures. When the processor and the memory are independent structures, the memory and the processor may be coupled and connected via a bus.

[0185] In an embodiment of the present disclosure, a non-temporary computer-readable storage medium is also provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of a method for reconstructing a pulmonary airway tree model based on an implicit expression network in any of the above embodiments are implemented.

[0186] Those skilled in the art will appreciate that the embodiments of the present disclosure may be provided as methods, systems, or computer program products. Therefore, the present disclosure may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present disclosure may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0187] The present disclosure is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present disclosure. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0188] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0189] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0190] Although the preferred embodiments of the present disclosure have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present disclosure.

[0191] Obviously, those skilled in the art can make various changes and modifications to the present disclosure without departing from the spirit and scope of the present disclosure. Thus, if these modifications and variations of the present disclosure fall within the scope of the claims of the present disclosure and their equivalents, the present disclosure is also intended to include these modifications and variations.

Claims

1. A method for reconstructing a lung airway tree model based on a latent expression network, characterized in that: include: Acquire a three-dimensional medical image data set, wherein the three-dimensional medical image data set includes three-dimensional medical image images; Performing volume rendering isosurface algorithm processing on the three-dimensional medical image to determine a first part of the data set; The first part of the data set is sequentially subjected to coordinate transformation processing based on physical information, shape alignment transformation processing within the data set, and point cloud sampling processing to determine the second part of the data set; The second part of the data set is used to train the preset latent expression network based on deep geometric correspondence, and determine the latent expression network based on deep geometric correspondence that completes the model training; The target three-dimensional medical image to be reconstructed is input into the latent expression network based on deep geometric correspondence that has completed model training to determine the reconstructed lung airway tree model.

2. The method according to claim 1, characterized in that The method further comprises: The reconstructed pulmonary airway tree model is transformed into a discrete voxel space through coordinates using the model-trained latent expression network based on deep geometric correspondence, and the reconstructed shape of the pulmonary airway tree in the discrete three-dimensional voxel space is determined.

3. The method according to claim 1, characterized in that The data in the first part of the data set includes three-dimensional grid data; The performing volume rendering isosurface algorithm processing on the three-dimensional medical image to determine the first part of the data set includes: Performing volume data extraction processing on the three-dimensional medical image to determine volume data of the three-dimensional medical image, wherein the volume data includes three-dimensional medical data and annotated organ true value data; The volume data is processed by a volume rendering isosurface algorithm to determine the three-dimensional grid data.

4. The method according to claim 3, characterized in that The step of sequentially performing coordinate transformation processing based on physical information, shape alignment transformation processing within the data set, and point cloud sampling processing on the first part of the data set to determine the second part of the data set includes: According to the voxel interval, voxel origin and voxel arrangement direction of the pixels of the three-dimensional medical image, the image coordinates corresponding to the three-dimensional grid data are converted into world coordinates to determine the three-dimensional grid data having world coordinates; Performing point cloud registration processing on the three-dimensional mesh data with world coordinates based on a preset reference system, determining a rigid transformation matrix in the world coordinate system, applying the rigid transformation matrix in the world coordinate system to each instance, and determining three-dimensional mesh data that has undergone shape alignment transformation processing in the data set; Point cloud sampling processing is performed on the three-dimensional mesh data of the pulmonary airway tree in the three-dimensional mesh data that has been processed by the shape alignment transformation in the data set, and random sampling processing is performed in the coordinate space of the three-dimensional mesh data of the pulmonary airway tree that has been processed by the shape alignment transformation in the data set, and a signed distance function value between the point cloud that has been processed by the point cloud sampling processing and the point cloud that has been processed by the random sampling processing is obtained to determine the second part of the data set.

5. The method according to claim 1, characterized in that The second part of the data set includes a second training set, a second validation set and a second test set; The step of using the second part of the data set to train a preset latent expression network based on deep geometric correspondence to determine the latent expression network based on deep geometric correspondence that completes the model training includes: Using the second training set and the second validation set to train the preset latent expression network based on deep geometric correspondence, and determine the latent expression network based on deep geometric correspondence that has been trained by the model; Fine-tune the latent expression network based on deep geometric correspondence that has been trained by the model to determine the latent expression network based on deep geometric correspondence that has been fine-tuned; The second test set is used to test the latent expression network based on deep geometric correspondence that has been fine-tuned to determine the latent expression network based on deep geometric correspondence that has completed model training.

6. The method according to claim 5, characterized in that The preset implicit expression network based on deep geometric correspondence includes an encoder module shared by each instance and driven by an implicit template vector, a decoder module unique to each instance and driven by a potential shape vector of each instance, and a signed distance function prediction module driven by the potential shape vector. The preset implicit expression network based on deep geometric correspondence is expressed as: Wherein, M1 represents the preset implicit expression network based on deep geometric correspondence, represents the encoder module driven by the implicit template vector shared by each instance, β represents the K-dimensional implicit template potential vector, φ β represents a multi-layer perceptron weight prediction network driven by β, represents the decoder module unique to each instance driven by the latent shape vector of each instance, α i represents the K-dimensional latent shape vector of the i-th instance, Indicated by α i The multi-layer perceptron weight prediction network driven by , p represents the three-dimensional coordinates of the point cloud, represents the signed distance function prediction module driven by the latent shape vector and s represents the signed distance function value.

7. The method according to claim 5, characterized in that The method of using the second part of the data set to train a preset latent expression network based on deep geometric correspondence to determine the latent expression network based on deep geometric correspondence that has completed model training also includes: The preset overall loss function is used to perform parameter optimization processing on the model-trained latent expression network based on deep geometric correspondence, and the preset overall loss function is expressed as: L total =L sdf +L reg +L cor Among them, L total Denotes the overall loss function, L sdf represents the optimization objective function of the signed distance function, L reg represents the regularization constraint term for the potential shape vector α of each instance and the implicit template potential vector β, L cor Represents a constraint item on the correspondence of the latent expression network based on deep geometric correspondence.

8. A pulmonary airway tree model reconstruction system based on latent expression network, characterized in that: include: An image acquisition module, used to acquire a three-dimensional medical image dataset, wherein the three-dimensional medical image dataset includes three-dimensional medical images; A first data processing module is used to perform volume rendering isosurface algorithm processing on the three-dimensional medical image to determine a first part of the data set; A second data processing module is used to sequentially perform coordinate transformation processing based on physical information, shape alignment transformation processing within the data set, and point cloud sampling processing on the first part of the data set to determine a second part of the data set; A model training module, used to train a preset latent expression network based on deep geometric correspondence using the second part of the data set, and determine the latent expression network based on deep geometric correspondence that completes the model training; The reconstruction module is used to input the target three-dimensional medical image to be reconstructed into the latent expression network based on deep geometric correspondence that has completed model training, and determine the reconstructed lung airway tree model.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method described in any one of claims 1 to 7 are implemented.

10. A terminal, characterized in that: include: a memory having a computer program stored thereon; A processor, configured to execute the computer program in the memory to implement the steps of the method according to any one of claims 1 to 7.

Images