Artificial intelligence chest multi-organ three-dimensional reconstruction method

Abstract

The invention discloses an artificial intelligence chest multi-organ three-dimensional reconstruction method, and belongs to the technical field of medical image processing. Comprising the following steps: acquiring a plurality of preprocessed target CT images, and performing AI organ recognition and labeling on each target CT image; performing chest multi-organ segmentation model training according to the marked CT image to obtain a segmentation model for performing organ segmentation on the marked CT image, and obtaining multi-layer cross section data of each chest organ for performing three-dimensional reconstruction of a single chest organ; acquiring relative position information of each chest organ, and combining with the three-dimensional reconstruction model of the single chest organ to complete three-dimensional reconstruction of multiple chest organs to obtain an initial three-dimensional model; and converting the initial three-dimensional model into discrete point cloud data, constructing a point cloud anomaly detection network model to perform anomaly recognition on the point cloud data, and analyzing an anomaly recognition result by using a pathology basis large model to obtain a pathology result for performing corresponding marking on the initial three-dimensional model to obtain a target three-dimensional reconstruction model.

Description

Technical Field

[0001] This invention belongs to the field of medical image processing technology, specifically relating to an artificial intelligence method for three-dimensional reconstruction of multiple organs in the chest. Background Technology

[0002] With the continuous development of CT technology, multi-slice spiral CT scans can reconstruct high-resolution thin-slice images down to the millimeter level in real time, making them an important tool for doctors to qualitatively and quantitatively assess the function of various tissues in the human body. CT images allow for independent, intuitive, and repeatable observation of local areas, precise measurement of indicators such as volume and density, and non-invasive virtual endoscopic examinations; they can also guide surgery and facilitate disease screening.

[0003] In the current field of medical imaging, the three-dimensional reconstruction technology of multiple organs in the chest is of great significance for the analysis of lesions, surgical planning and prognostic assessment of organs such as the lungs, heart and mediastinum. However, the existing three-dimensional reconstruction methods of the chest are usually based on the segmentation and reconstruction of single organs based on CT images, which makes it difficult to achieve the collaborative localization and accurate modeling of multiple organs, resulting in limited integrity of the final three-dimensional model of multiple organs in the chest. Meanwhile, after the existing three-dimensional reconstruction of multiple organs in the chest, the pathological inference of the organs relies entirely on the doctor's clinical experience. However, due to the common problems of blurring and distortion in three-dimensional reconstruction, relying solely on the doctor to make pathological inferences of organs based on the three-dimensional reconstruction results is not only time-consuming, but also requires a high level of clinical experience from the doctor, thus limiting the clinical application value of the final three-dimensional model of multiple organs in the chest. Summary of the Invention

[0004] In view of this, the purpose of this invention is to provide an artificial intelligence method for three-dimensional reconstruction of multiple organs in the chest, in order to solve the above-mentioned technical problems.

[0005] To achieve the above objectives, the present invention provides the following technical solution: An artificial intelligence-based three-dimensional reconstruction method for multiple organs in the chest includes: Acquire multiple preprocessed target CT images, and perform AI organ identification and annotation for each target CT image; A multi-organ segmentation model for the chest is trained based on labeled CT images. The resulting segmentation model is used to segment organs in labeled CT images, and multi-layer cross-sectional data of each chest organ is obtained for three-dimensional reconstruction of a single chest organ. The relative position information of each thoracic organ is obtained, and the 3D reconstruction model of a single thoracic organ is combined to complete the 3D reconstruction of multiple thoracic organs, thus obtaining an initial 3D model. The initial 3D model of a single thoracic organ was converted into discrete point cloud data. A point cloud anomaly detection network model was constructed to identify anomalies in the point cloud data. The pathological basis large model was used to analyze the anomaly identification results to obtain pathological results, which were then used to label the initial 3D model to obtain the target 3D reconstruction model.

[0006] Furthermore, multiple preprocessed target CT images are acquired, and AI organ identification and annotation are performed on each target CT image, including: A deep learning-based organ recognition model is constructed; the recognition process uses grayscale features combined with contour features for comprehensive similarity recognition. Multiple preprocessed target CT images are acquired, and chest organ identification is performed on each target CT image using an organ recognition model to obtain target identification results; the target identification results include the specific identified organ types and the pseudo-identified organ types. Each target CT image is labeled based on the target recognition results. After labeling, all target CT images are filtered to select target CT images that meet preset conditions as labeled CT images. The preset filtering conditions include: the number of organs labeled in the labeled CT images meets the preset standard number of chest multi-organs, and there are no duplicate organ labeling results in the labeled CT images.

[0007] Furthermore, an artificial intelligence method for three-dimensional reconstruction of multiple organs in the chest also includes: setting a first threshold and a second threshold for organ similarity recognition, wherein the first threshold is greater than the second threshold; When using the organ recognition model to identify organs in a target CT image, if the similarity of any organ to any target region is greater than the first threshold, the identification result is the organ type associated with the corresponding organ similarity, which is used as the specific organ type to be identified. If the identification result for any target region is that the similarity of any organ is greater than the second threshold and less than or equal to the first threshold, then the identification result is the organ type associated with the corresponding organ similarity, which is used as the pseudo-identified organ type; wherein, the similarity of any organ used for comparison is the highest organ similarity in the corresponding target region.

[0008] Furthermore, an artificial intelligence-based method for three-dimensional reconstruction of multiple organs in the chest also includes: Based on the relative position information of each thoracic organ, the initial three-dimensional model of the thoracic multi-organ three-dimensional reconstruction is completed by combining the three-dimensional reconstruction model of a single thoracic organ. After corresponding spatial transformation, the model meets the preset organ registration accuracy with the organ image in each target CT image.

[0009] Furthermore, the 3D reconstruction model of a single thoracic organ in the initial 3D model is converted into discrete point cloud data. A point cloud anomaly detection network model is constructed to identify anomalies in the point cloud data, including: Obtain the physiological information corresponding to the target individuals from all target CT image scan sources, and filter the preset point cloud anomaly detection network model database based on the physiological information to obtain a set of point cloud anomaly detection network models that conform to the current target individual's physiological information; A three-dimensional reconstruction model of a single thoracic organ is obtained, converted into discrete point cloud data, and input into the point cloud anomaly detection network model corresponding to the thoracic organ in the point cloud anomaly detection network model set. The point cloud anomaly detection result is then output. Based on the point cloud anomaly detection results, the abnormal areas in the corresponding 3D reconstruction model are located, and the abnormal areas are identified by the AI ​​lesion recognition model to obtain lesion data of a single thoracic organ. Obtain lesion data for all thoracic organs as the anomaly identification results for the current initial 3D model.

[0010] Furthermore, an artificial intelligence-based method for three-dimensional reconstruction of multiple organs in the chest also includes: The pre-set point cloud anomaly detection network model database includes several point cloud anomaly detection network model sets; each point cloud anomaly detection network model set corresponds to a set of physiological information, and each point cloud anomaly detection network model set contains point cloud anomaly detection network models corresponding to all chest organs.

[0011] Furthermore, an artificial intelligence-based method for three-dimensional reconstruction of multiple organs in the chest also includes: The point cloud anomaly detection network model uses an improved convolutional neural network as the initial network to classify and identify fused data, and is trained and generated through supervised learning using a preset loss function. The fused data is the data obtained by fusing known healthy chest organ features with pseudo lesion features generated based on point cloud deep learning data.

[0012] Furthermore, an artificial intelligence-based method for three-dimensional reconstruction of multiple organs in the chest also includes: Obtain the preliminary treatment plan formulated by the doctor based on the pathological feedback in the target 3D reconstruction model; By using a clinical efficacy database, clinical efficacy data corresponding to the initial treatment plan is obtained, and a 3D model completion model is trained based on the clinical efficacy data. The 3D model completion model includes a classifier, an encoder, a model completion network, and a decoder. By removing abnormal regions from the initial 3D reconstruction model, an incomplete 3D model is obtained. The incomplete 3D model is completed using a 3D model completion model to obtain a 3D model for effect prediction.

[0013] Furthermore, the training process for the 3D model completion model includes: Construct an initial 3D model completion model, including an initial classifier, an initial encoder, an initial model completion network, and an initial decoder; Based on the standard model data of the lesion corresponding to the clinical effect data, a target three-dimensional model sample is generated, and the initial classifier is trained based on the target three-dimensional model sample to obtain the target classifier; The loss function for constructing the target 3D model sample and the initial 3D model completion model is used to train the initial encoder, the initial model completion network and the initial decoder to obtain the temporary 3D model completion model; The target classifier and the temporary 3D model completion model are concatenated to obtain the final 3D model completion model; the 3D model completion model includes a target classifier, a target encoder, a target model completion network, and a target decoder. The completion operation includes: The model data of the incomplete 3D model is extracted as the data to be completed; each data to be completed corresponds to an abnormal region and each data to be completed corresponds to a type of lesion data. Obtain the target classifier to classify any data to be completed, and determine the lesion type of the target data to be completed; The corresponding data to be completed is input into the target encoder corresponding to the lesion type to obtain incomplete model data. The incomplete model data is then completed using the target model completion network to obtain complete model data, which is then input into the target decoder corresponding to the lesion type to obtain target model data, which is used to reconstruct a 3D model for effect prediction.

[0014] The beneficial effects of this invention are as follows: This invention proposes an AI-based method for 3D reconstruction of multiple organs in the chest. It utilizes AI recognition to automatically segment multiple organs in the chest, obtaining single-organ CT image data, and simultaneously acquiring the relative positions of multiple organs to complete 3D reconstruction. This solves the isolation problem of traditional single-organ reconstruction and provides a complete 3D reconstruction model of multiple organs in the chest that conforms to real physiological structures. Simultaneously, this application accurately identifies abnormal regions using a point cloud anomaly recognition method and uses a large-scale pathological model to analyze and infer pathological results. Based on the pathological results, it automatically annotates the 3D reconstruction model, providing a reliable and objective basis for clinical diagnosis. This addresses the problem that existing 3D reconstruction models of multiple organs in the chest rely entirely on doctors' clinical experience, limiting their clinical application value.

[0015] Other advantages, objectives, and features of the invention will be set forth in the following description and will be apparent to those skilled in the art in some respects, or may be learned by practice of the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the written description and the accompanying drawings.

[0016] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0017] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart of an artificial intelligence-based three-dimensional reconstruction method for multiple organs in the chest, as described in an embodiment of the present invention. Figure 2 This is a flowchart of the AI ​​organ identification and annotation process in an artificial intelligence-based three-dimensional reconstruction method for multiple organs in the chest, as described in an embodiment of the present invention. Figure 3 This is a flowchart illustrating the point cloud data anomaly identification process in a 3D reconstruction model of an artificial intelligence-based multi-organ 3D reconstruction method for the chest, as described in an embodiment of the present invention. Figure 4 This is a flowchart illustrating the process of constructing a 3D model for predicting the effect of an artificial intelligence-based 3D reconstruction method for multiple organs in the chest, as described in an embodiment of the present invention. Figure 5 This is a flowchart illustrating the training process of a three-dimensional model completion model in an artificial intelligence-based three-dimensional reconstruction method for multiple organs in the chest, as described in an embodiment of the present invention. Detailed Implementation

[0018] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0019] like Figure 1 As shown, this invention proposes an artificial intelligence-based three-dimensional reconstruction method for multiple organs in the chest, comprising: S101. Acquire multiple preprocessed target CT images, and perform AI organ recognition and annotation for each target CT image; S102. Train a multi-organ segmentation model for the chest based on labeled CT images. The resulting segmentation model is used to segment organs in labeled CT images and obtain multi-layer cross-sectional data of each chest organ for three-dimensional reconstruction of a single chest organ. S103. Obtain the relative position information of each chest organ, and combine the three-dimensional reconstruction model of a single chest organ to complete the three-dimensional reconstruction of multiple chest organs, thus obtaining an initial three-dimensional model. The initial three-dimensional model obtained by completing the three-dimensional reconstruction of multiple chest organs based on the relative position information of each chest organ and the three-dimensional reconstruction model of a single chest organ, after corresponding spatial transformation, satisfies the preset organ registration accuracy with the organ images in each target CT image. S104. Convert the three-dimensional reconstruction model of a single chest organ in the initial three-dimensional model into discrete point cloud data, construct a point cloud anomaly detection network model to identify anomalies in the point cloud data, and use a large pathology-based model to analyze the anomaly identification results to obtain pathological results, which are used to label the initial three-dimensional model accordingly to obtain the target three-dimensional reconstruction model. The working principle and beneficial effects of the above technical solution are as follows: In order to solve the problem that the accuracy of the three-dimensional reconstruction of multiple organs in the chest is insufficient and the clinical application value is limited in the existing technology, this application proposes an artificial intelligence three-dimensional reconstruction method for multiple organs in the chest, which is used to combine artificial intelligence to perform organ AI identification and segmentation of multiple organs in the chest, complete high-precision three-dimensional reconstruction of multiple organs in the chest, and use a large pathological model to automatically label the pathological results of abnormal identification, so as to solve the problem of limited clinical application value. Specifically, in practical applications, the method provided in this application first requires acquiring multiple preprocessed target CT images. This preprocessing includes conventional image preprocessing techniques such as denoising to ensure the accuracy of the acquired CT images. Then, for each target CT image, an organ recognition model is used for automatic recognition, and the recognition results are annotated on the target CT image accordingly, so that each target CT image has corresponding organ annotations, providing reliable data support for subsequent organ segmentation. Compared with existing manual annotation, this method eliminates the need for manual recognition and annotation of each target CT image, reducing the tedious process of manual operation. At the same time, the use of a fully automatic AI recognition method helps to avoid errors that may occur during manual annotation and reduces the dependence on manual annotation experience. After organ identification is completed on all target CT images, a multi-organ segmentation model for the chest is trained based on the labeled CT images. After training, a segmentation model that meets the preset training conditions is obtained. During training, labeled CT images are used as training samples. Various methods exist in existing technologies for the specific training process, which will not be elaborated here. It is worth noting that since the labeled target CT images all carry full markers of chest organs, the data requirements during actual segmentation training are reduced, increasing the model generation speed. Finally, this segmentation model is used to segment organs in the labeled CT images, obtaining multi-layer cross-sectional data for each chest organ, which is then used for 3D reconstruction of a single chest organ. This 3D reconstruction model preferably uses mesh data and utilizes multimodal medical image information and prior anatomical knowledge to optimize the reconstructed 3D model, resulting in a more natural and physiologically accurate 3D organ model. Because the single-organ 3D model is generated by completely segmenting the corresponding organ data in the target CT image using the trained segmentation model during data acquisition, the accuracy of the single-organ 3D model generated by this method is higher than that of existing technologies. After all individual chest organs have been reconstructed in 3D, the initial relative position of each individual chest organ is obtained synchronously using prior anatomical knowledge. Based on this initial relative position, and combined with the 3D reconstruction model of the individual chest organ, preliminary reconstruction is completed. Then, multiple target CT images are used to perform organ registration with the preliminary reconstructed 3D model to fine-tune the organ positions. Finally, after meeting the preset registration accuracy, the 3D reconstruction of multiple chest organs is completed, and an initial 3D model is obtained. This method solves the isolation problem of traditional single organ reconstruction and provides a complete 3D reconstruction model of multiple chest organs that conforms to the real physiological structure. It provides doctors with a reliable and complete model basis for extracting pathological information based on the reconstruction results. In existing technologies, the initial 3D model is typically output immediately after acquisition, without considering its practical clinical value. Furthermore, since chest CT scans involve numerous organs, doctors relying solely on clinical experience cannot accurately account for all pathological conditions when making pathological inferences based on the 3D reconstruction model. Therefore, this application proposes a method for intelligently labeling abnormal pathological features in the 3D model, enabling the reconstructed 3D model to better meet practical needs, providing doctors with auxiliary diagnostic markers, and improving the clinical application value of CT image 3D reconstruction. Specifically, this includes: The three-dimensional reconstruction model of a single thoracic organ in the initial three-dimensional model is converted into discrete point cloud data. It is worth noting that since point cloud data is simpler to generate than grid data and can directly represent each element in the three-dimensional model, point cloud data is selected as the basic data for anomaly identification in this application, which facilitates the accurate identification of each information element in the initial three-dimensional model. In addition, before converting point cloud data, it is necessary to pre-build a point cloud anomaly detection network model to identify anomalies in the subsequently converted point cloud data, thereby determining the abnormal areas. Compared with the existing technology that directly performs full-domain anomaly detection on the initial 3D model, this method can directly lock the abnormal areas by comparing point cloud data anomalies and then perform anomaly identification on the abnormal areas, which can reduce the time required for full-coverage anomaly detection. At the same time, performing point detection of abnormal areas can also improve the accuracy of anomaly identification. Finally, the pathological foundation model is used to analyze the abnormality identification results of all organs to obtain pathological results, which are then used to label the initial 3D model to obtain the target 3D reconstruction model. The pathological results include the inference of pathological results for single organs, as well as the inference of pathological results for multiple organs in the chest by combining the abnormality identification results of all single organs. This provides a reliable and objective basis for clinical diagnosis and solves the problem that the existing 3D models for multiple organs in the chest rely entirely on the clinical experience of doctors, which limits their clinical application value. It is worth noting that there are corresponding training methods for the pathological foundation model in the existing technology. After limiting the training data to the abnormality identification results and corresponding pathological results mentioned in this method, the specific training methods will not be elaborated here.

[0020] like Figure 2 As shown, in one embodiment, multiple preprocessed target CT images are acquired, and AI organ identification and annotation are performed on each target CT image, including: S201. Construct an organ recognition model based on deep learning; wherein, the recognition process uses grayscale features combined with contour features for comprehensive similarity recognition; S202. Acquire multiple preprocessed target CT images, and use an organ recognition model to identify chest organs in each target CT image to obtain target recognition results; wherein, the target recognition results include specific identified organ types and pseudo-identified organ types; S203. Each target CT image is labeled according to the target recognition results, and after labeling, all target CT images are filtered to select target CT images that meet the preset conditions as labeled CT images; wherein, the preset filtering conditions include: the number of organs labeled in the labeled CT images meets the preset standard number of chest multi-organs, and there is no duplication of organ labeling results in the labeled CT images. The working principle of the above technical solution is as follows: Construct an organ recognition model based on deep learning. Specifically, the organ recognition model based on deep learning uses ResNet50 as the base network. The input is a preprocessed CT image (size 512×512, HU value normalized to [-1000, 400]), and the output is an 'organ type probability vector' (covering 12 core thoracic organs such as lung, heart, mediastinum, trachea, and esophagus). The training data consists of 500-1000 clinical chest CT images (confirmed by annotation by 3 associate chief physicians). The training parameters are: learning rate 0.001, Adam optimizer, cross-entropy loss function, 100 iterations, and batch size set to 16. During the recognition process, grayscale features are extracted through convolutional layers 3-5 of ResNet50 (specifically, the local mean, variance, and histogram entropy of the CT value, dimension 256); contour features are extracted using the Canny edge detection algorithm (threshold [50, 150]), and then the Hu moments (7 invariant moments), perimeter, and area (dimensional 10) of the edges are calculated; the similarity calculation uses weighted Euclidean distance, preferably: Where P represents similarity, and 0.6 and 0.4 represent the weights of grayscale features and contour features, respectively. These weights can be modified according to actual needs, but since grayscale values ​​are more critical for distinguishing organ materials in CT images, the weight of grayscale features is preferably greater than the weight of contour features. The distance represents the Euclidean distance of the gray-level features, and htz represents the maximum value of the gray-level features. Represents the Euclidean distance of the contour features. This represents the maximum value of the contour feature. It is worth noting that the organ recognition model constructed in this application differs from existing technologies in that it preferentially uses grayscale features combined with contour features for comprehensive similarity recognition during training, which is beneficial for improving organ recognition accuracy. In practical application, multiple preprocessed target CT images are directly acquired, and then the pre-constructed organ recognition model is used to perform chest organ recognition on each target CT image to obtain the target recognition result. Each target CT image is then labeled. It is worth noting that due to the limitation of the organ similarity recognition threshold, the target recognition result includes both the specific identified organ type and the pseudo-identified organ type. Furthermore, to ensure subsequent... The segmentation accuracy of the segmentation model during training was utilized using the annotated CT images. Screening conditions were established to filter all target CT images, selecting those that met the preset criteria as annotated CT images. These criteria included: the number of organ annotations in the annotated CT images met the preset standard for multiple organs in the chest, and there were no duplicate organ annotations in the annotated CT images. Normally, all annotated target CT images met these criteria. If any did not meet these criteria, the unmet target CT images were not used for organ segmentation and data acquisition during subsequent organ segmentation, reducing the interference of abnormal data on the overall data. The beneficial effects of the above technical solution are as follows: Compared with the existing manual annotation, the above technical solution eliminates the need for manual identification and annotation of each target CT image, reducing the tedious process in manual operation. At the same time, the use of fully automatic AI recognition method helps to avoid errors that may occur during manual annotation and reduces the dependence on manual annotation experience when identifying organs.

[0021] In one embodiment, an AI-based three-dimensional reconstruction method for multiple organs in the chest further includes: A first threshold and a second threshold are set for organ similarity identification, wherein the first threshold is greater than the second threshold; When using the organ recognition model to identify organs in a target CT image, if the similarity of any organ to any target region is greater than the first threshold, the identification result is the organ type associated with the corresponding organ similarity, which is used as the specific organ type to be identified. If the identification result for any target region is that the similarity of any organ is greater than the second threshold and less than or equal to the first threshold, then the identification result is the organ type associated with the corresponding organ similarity, which is used as the pseudo-identified organ type; wherein, the similarity of any organ used for comparison is the highest organ similarity in the corresponding target region; The working principle of the above technical solution is as follows: To further disclose the identification and classification of specific organ types and pseudo-identified organ types in the organ identification model proposed in this application during the identification process, the following technical solution is proposed: First, during the training of the organ recognition model, a first threshold and a second threshold are pre-set for organ similarity recognition judgment, and the first threshold must be greater than or significantly greater than the second threshold. Then, when using the trained organ recognition model to identify organs in the target CT image, the target CT image is region-based based on grayscale features combined with contour features. If the result of identifying any target region is that the similarity of any organ is greater than the first threshold, the recognition result is the organ type associated with the corresponding organ similarity, which is used as the specific identified organ type. If the result of identifying any target region is that the similarity of any organ is greater than the second threshold but less than or equal to the first threshold, the recognition result is the organ type associated with the corresponding organ similarity, which is used as the pseudo-identified organ type. It is worth noting that during region recognition, the same target region may correspond to multiple organs, but this application imposes corresponding restrictions during the training of the organ recognition model, only taking the organ with the highest organ similarity as the main organ of the region, and using the similarity of the main organ as a parameter for threshold comparison to determine the recognition result. Furthermore, if there are manual annotations in the CT image, the manual annotation results are used to verify the recognition effect. The beneficial effects of the above technical solution are as follows: Currently, organ identification in clinical CT images is mainly obtained through manual delineation by doctors. However, when doctors delineate organs, they usually only delineate them according to their current needs, making it difficult to obtain fully labeled CT images when performing organ segmentation, or doctors have not yet labeled the CT images when they are first generated. Through the above technical solution, an organ identification model is used to replace manual labeling, providing fully labeled CT images and providing automated data support for three-dimensional reconstruction of multiple organs in the chest.

[0022] like Figure 3 As shown, in one embodiment, the 3D reconstruction model of a single chest organ in the initial 3D model is converted into discrete point cloud data, and a point cloud anomaly detection network model is constructed to identify anomalies in the point cloud data, including: S301. Obtain the physiological information corresponding to the target individuals from all target CT image scan sources, and filter the preset point cloud anomaly detection network model database based on the physiological information to obtain a set of point cloud anomaly detection network models that conform to the current target individual's physiological information. S302. Obtain the three-dimensional reconstruction model of a single chest organ, convert it into discrete point cloud data, and input it into the point cloud anomaly detection network model of the corresponding chest organ in the point cloud anomaly detection network model set, and output the point cloud anomaly detection result. S303. Based on the point cloud anomaly detection results, the abnormal areas in the corresponding three-dimensional reconstruction model are located, and the abnormal areas are identified by the AI ​​lesion recognition model to obtain the lesion data of a single thoracic organ. S304. Obtain lesion data for all thoracic organs as the anomaly identification results for the current initial 3D model; The preset point cloud anomaly detection network model database includes several point cloud anomaly detection network model sets; each point cloud anomaly detection network model set corresponds to a set of physiological information, and each point cloud anomaly detection network model set contains point cloud anomaly detection network models corresponding to all chest organs. For each point cloud anomaly detection network model, an improved convolutional neural network is used as the initial network to classify and identify the fused data, and supervised learning training is performed to generate the model through a preset loss function. The fused data is the data obtained by fusing known healthy chest organ features with pseudo lesion features generated based on point cloud deep learning data. The working principle of the above technical solution is as follows: Physiological information corresponding to the target individuals from all target CT image scan sources is obtained, and a preset point cloud anomaly detection network model database is filtered based on the physiological information to obtain a set of point cloud anomaly detection network models that conform to the physiological information of the current target individual. The preset point cloud anomaly detection network model database includes several sets of point cloud anomaly detection network models. Each set of point cloud anomaly detection network models corresponds to a set of physiological information, and each set contains point cloud anomaly detection network models corresponding to all chest organs. For each point cloud anomaly detection network model, an improved convolutional neural network is used as the initial network to classify and identify the fused data, and supervised learning training is performed using a preset loss function. The fused data is the data obtained by fusing known healthy chest organ features with pseudo-lesion features generated based on point cloud deep learning data. Specifically, PointNet++ is used as the base network, with the improvement being the addition of a Self-Attention module (8 attention heads, 256 dimensions) to the SetAbstraction (SA) layer to highlight the point cloud features of the lesion area (such as local dense points of lung nodules). The improved initial network, with an optimal network structure including 3 SA layers, 1 global pooling layer, and 2 fully connected layers, is then used for the generation and classification of the fused data. Extract features of healthy chest organs (from point cloud data of several lesion-free chest CT scans, including features such as local curvature, normal vector, density distribution, and dimensionality); Extract pseudo-lesion features (generated using GAN (generator is U-Net, discriminator is PatchGAN), taking the features of healthy chest organs as input, to generate pseudo-lesion features of 5 types of lesions such as lung nodules (e.g., diameter 3-30mm) and pericardial effusion (e.g. thickness 2-10mm), with dimensions consistent with the features of healthy chest organs); Feature fusion generates fused data (using feature-level concatenation to concatenate features of healthy chest organs with features of pseudolesions into high-dimensional fused data (if the dimension of the healthy chest organ features is 512, then the dimension of the fused data here is 1024)). The fused data is classified and identified (the fused data is input into the improved PointNet++, and after feature extraction by the SA layer, the fully connected layer outputs the binary classification result of 'normal / lesion' and the probability vector of 'lesion type' (5 categories)).

[0023] When using a pre-defined loss function for supervised learning training, the pre-defined loss function can be a weighted cross-entropy loss function, and the preferred function expression is: in, represents the weight of the lesion sample, used to balance the samples, y represents the true label, which can be 0 or 1 to represent normal and lesion respectively, and p represents the prediction probability. During training, the Adam optimizer is preferred, with a learning rate of 0.0005, until the preset number of iterations or preset training conditions are reached, and the training ends to generate a point cloud anomaly detection network model. In addition, the physiological information corresponding to the target individual includes, but is not limited to, age, gender, height, weight, history of underlying diseases (such as hypertension and diabetes), and smoking history. This information can be automatically extracted through the hospital's electronic medical record system. Then, based on the pre-trained point cloud anomaly detection network model database, the most matching model is selected according to the target individual's physiological information. For example, for men over 60 years old with a history of smoking, the system prioritizes calling the point cloud anomaly detection network model trained for lung lesions in elderly male smokers. Each model is trained using CT images and labeled data of the corresponding physiological characteristics population, making the model more targeted. Matching a dedicated detection model based on the target individual's physiological information overcomes the problem of poor adaptability of traditional "general models" to different populations. It is worth noting that in each set of physiological information, some data appear in the form of ranges, such as age, height, and weight. At the same time, when dealing with the history of underlying diseases, some comorbidities are preferably selected into the same set of physiological information to improve the applicability of the model. After determining the point cloud anomaly detection network model, a 3D reconstruction model of a single chest organ is obtained, converted into discrete point cloud data, and input into the corresponding chest organ's point cloud anomaly detection network model in the point cloud anomaly detection network model set. The output is the point cloud anomaly detection result. Taking the point cloud anomaly detection network model trained on the lung lesions of the aforementioned elderly male smoker as an example, the initial 3D model is converted into point cloud data through discretization. Each point contains 3D coordinates (x, y, z) and corresponding CT grayscale information. For example, for the lung model, the point cloud data can cover the surface and key internal locations of structures such as lung parenchyma, blood vessels, and bronchi. Then, the point cloud data is input into the selected point cloud anomaly detection network model (such as based on PointNet++ or DGCNN architecture). The model learns the local geometric features and global structural features in the point cloud data through multi-layer feature extraction and convolution operations, and finally outputs the anomaly probability value or classification result of each point, forming the point cloud anomaly detection result. Specifically, the three SA layers in the aforementioned point cloud anomaly detection network model are used to complete multi-layer feature extraction and convolution operations, including: The first SA layer is used to sample the input point cloud (single organ point cloud, 2048 points), and local geometric features are extracted by a 3×3 convolution kernel. The output feature dimension is 128, followed by BatchNorm (momentum 0.9) and ReLU activation function. Using the second SA layer, based on the features of the first layer, with a sampling radius of 1.0 mm, 512 points, and a 3×3 convolution kernel, local features are fused into regional features (such as the interlobular septum structure features of lung lobes), and the output dimension is 256. It is also connected to BatchNorm and ReLU. Using the third SA layer, with a sampling radius of 2.0 mm, 256 points, and a 3×3 convolution kernel, global structural features (such as the overall volume of the organ, the position of the center of gravity, and the morphological symmetry) are extracted, and the output dimension is 512. In the final convolution operation, the stride of all convolutional layers is 1, and the padding is 'same' to prevent the feature map size from shrinking. The output of the anomaly probability value or classification result for each point includes: The 512-dimensional features of the third SA layer in the aforementioned point cloud anomaly detection network model are compressed into 1-dimensional features using the global pooling layer. These features are then mapped to [0,1] using the sigmoid function to obtain the 'lesion probability value' for each point (preferably, a probability ≥ 0.8 indicates a lesion point). Alternatively, the DBSCAN clustering algorithm can be used to cluster lesion points into 'abnormal regions' (such as pulmonary nodule regions or pericardial effusion regions). A fully connected layer maps the 512-dimensional global features into probability vectors for five lesion types, and the 'lesion type' (i.e., classification result) is determined using the argmax function. Based on the point cloud anomaly detection results, anomaly points are filtered by setting a probability threshold (e.g., 0.8), and points with probability values ​​higher than the threshold are clustered into lesion regions. Simultaneously, combined with prior knowledge of chest anatomy, the geometric shape and spatial location of the initially located lesion regions are constrained and optimized to determine the final abnormal regions. After identifying the abnormal area, the AI ​​lesion recognition model is used to perform point-to-point identification of the abnormal area to obtain lesion data of a single thoracic organ. When the AI ​​lesion recognition model is working, it prefers to extract the target detection image of the abnormal area from the initial 3D model, including but not limited to multi-view images such as axial, coronal, and sagittal planes, to ensure that the lesion morphology is fully displayed in order to obtain lesion data of a single thoracic organ. The beneficial effects of the above technical solution are as follows: Compared with the existing technology of directly performing full-domain anomaly detection on the initial 3D model, the above technical solution can directly lock the abnormal area by comparing point cloud data anomalies, and then perform anomaly identification on the abnormal area. This can reduce the time required for full-coverage anomaly detection. At the same time, performing point detection on the abnormal area can also improve the accuracy of anomaly identification and provide reliable data support for subsequent pathological inference.

[0024] like Figure 4 As shown, in one embodiment, an artificial intelligence-based three-dimensional reconstruction method for multiple organs in the chest further includes: S401. Obtain the preliminary treatment plan formulated by the doctor based on the pathological feedback in the target 3D reconstruction model; S402. Obtain clinical effect data corresponding to the initial treatment plan through the clinical effect database, and train a 3D model completion model based on the clinical effect data; wherein, the 3D model completion model includes a classifier, encoder, model completion network and decoder; S403. Remove the model data of abnormal areas in the initial 3D reconstruction model to obtain an incomplete 3D model; S404. Use the 3D model to complete the incomplete 3D model and obtain the effect prediction 3D model. The working principle and beneficial effects of the above technical solution are as follows: To further improve the clinical application value of the three-dimensional reconstruction model of multiple organs in the chest, this application also proposes a method for reconstructing a three-dimensional model for effect prediction. By formulating an abnormal area completion strategy based on the existing three-dimensional reconstruction model and the preliminary treatment plan given by the doctor, the method completes the restoration of the treatment effect after the treatment plan is applied to the current organ three-dimensional model, thus promoting the development of AI in medicine. Specifically, in practical application, this method first obtains the preliminary treatment plan formulated by the doctor in response to the pathological feedback in the target three-dimensional reconstruction model, and obtains the clinical effect data corresponding to the preliminary treatment plan through a clinical effect database. The data in the clinical effect database includes the clinical effect data of each treatment plan in the face of the current pathological feedback. Finally, a three-dimensional model completion model is trained based on the clinical effect data. Based on the aforementioned abnormal areas, a removal operation is performed on each chest single organ model in the initial three-dimensional reconstruction model to remove the model data of the abnormal areas in the three-dimensional reconstruction model of each single organ, resulting in a fragmented three-dimensional model. Finally, the fragmented three-dimensional model is completed using the three-dimensional model completion model, and then the location of multiple organs is determined to obtain the effect prediction three-dimensional model.

[0025] like Figure 5 As shown, in one embodiment, the training process of the 3D model completion model includes: S501. Construct the initial 3D model completion model, including the initial classifier, initial encoder, initial model completion network, and initial decoder; S502. Generate target 3D model samples based on the standard lesion model data corresponding to the clinical effect data, and train the initial classifier based on the target 3D model samples to obtain the target classifier. This target classifier is used to classify the lesion types of the data to be completed, and to determine the appropriate target classifier, target encoder, and target model completion network. It is worth noting that in the prior art, the 3D model applied to CT image 3D reconstruction is usually expressed in the form of a mesh. Therefore, after initially defining the model data type, this solution proposes a target classifier that is different from the classification of model data types (point cloud, mesh, etc.) in the prior art. It is used to classify the lesion types of the data and to determine which lesion standard model data to use to complete the incomplete model. Specifically, the target classifier preferably uses the lightweight MobileNetV2 network adapted to real-time clinical needs. The input is 'abnormal region point cloud features in the standard lesion model data', and the output is 'probability vectors for 5 lesion types' (pulmonary nodules, pulmonary fibrosis, pericardial effusion, mediastinal lymphadenopathy, and esophageal stricture). The training data consists of 500-1000 standard lesion model data (100-200 cases per lesion type, all from CT reconstruction models followed up after clinical treatment). During training, the Adam optimizer is preferably used with a learning rate of 0.001, employing the conventional cross-entropy loss function, and a batch size of 32, until the preset number of iterations or preset training conditions are reached. It is worth noting that the 5 categories mentioned here do not encompass all lesion types; they are only used as examples. The classification results of this target classifier are directly used to 'match the standard lesion model data'. For example, when classifying as 'pulmonary nodules', 'healthy standard pulmonary nodule model data' (diameter 3-30mm) is used. The incomplete model was completed using point cloud features of healthy lung tissue with nodules. S503. Construct the loss function for the target 3D model sample and the initial 3D model completion model, which is used to train the initial encoder, the initial model completion network and the initial decoder to obtain the temporary 3D model completion model; the specific loss function is constructed according to actual needs. It is worth noting that when facing rare disease types, due to the scarcity of clinical effect data, they are given higher weights. S504. The target classifier and the temporary 3D model completion model are spliced ​​together to obtain the final 3D model completion model, so as to achieve dynamic adaptation to lesion types; wherein, the 3D model completion model includes a target classifier, a target encoder, a target model completion network and a target decoder; The completion operation includes: The model data of the incomplete 3D model is extracted as the data to be completed; each data to be completed corresponds to an abnormal region and each data to be completed corresponds to a type of lesion data. Obtain the target classifier to classify any data to be completed, and determine the lesion type of the target data to be completed; Input the corresponding data to be completed into the target encoder corresponding to the lesion type to obtain incomplete model data. Use the target model completion network to complete the incomplete model data to obtain complete model data, and input it into the target decoder corresponding to the lesion type to obtain target model data, which is used to reconstruct the three-dimensional model for effect prediction. The beneficial effects of the above technical solution are as follows: By using the three-dimensional image completion approach and combining it with actual treatment data from the clinical effect database, a three-dimensional model completion model is constructed to reconstruct a three-dimensional model for effect prediction. This is beneficial for early detection of potential problems in the preliminary treatment plan, further improving the clinical application value of the three-dimensional model of multiple organs in the chest and promoting the development of AI in medicine.

[0026] Finally, it should be noted that the above preferred embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail through the above preferred embodiments, those skilled in the art should understand that various changes can be made to it in form and detail without departing from the scope defined by the claims of the present invention.

Claims

1. A method for three-dimensional reconstruction of multiple organs in the chest using artificial intelligence, characterized in that, include: Acquire multiple preprocessed target CT images, and perform AI organ identification and annotation for each target CT image; A multi-organ segmentation model for the chest is trained based on labeled CT images. The resulting segmentation model is used to segment organs in labeled CT images, and multi-layer cross-sectional data of each chest organ is obtained for three-dimensional reconstruction of a single chest organ. The relative position information of each thoracic organ is obtained, and the 3D reconstruction model of a single thoracic organ is combined to complete the 3D reconstruction of multiple thoracic organs, thus obtaining an initial 3D model. The initial 3D model of a single thoracic organ was converted into discrete point cloud data. A point cloud anomaly detection network model was constructed to identify anomalies in the point cloud data. The pathological basis large model was used to analyze the anomaly identification results to obtain pathological results, which were then used to label the initial 3D model to obtain the target 3D reconstruction model.

2. The artificial intelligence method for three-dimensional reconstruction of multiple organs in the chest according to claim 1, characterized in that, Acquire multiple preprocessed target CT images, and perform AI organ identification and annotation on each target CT image, including: A deep learning-based organ recognition model is constructed; the recognition process uses grayscale features combined with contour features for comprehensive similarity recognition. Multiple preprocessed target CT images are acquired, and chest organ identification is performed on each target CT image using an organ recognition model to obtain target identification results; the target identification results include the specific identified organ types and the pseudo-identified organ types. Each target CT image is labeled based on the target recognition results. After labeling, all target CT images are filtered to select target CT images that meet preset conditions as labeled CT images. The preset filtering conditions include: the number of organs labeled in the labeled CT images meets the preset standard number of chest multi-organs, and there are no duplicate organ labeling results in the labeled CT images.

3. The artificial intelligence method for three-dimensional reconstruction of multiple organs in the chest according to claim 2, characterized in that, Also includes: A first threshold and a second threshold are set for organ similarity identification, wherein the first threshold is greater than the second threshold; When using the organ recognition model to identify organs in a target CT image, if the similarity of any organ to any target region is greater than the first threshold, the identification result is the organ type associated with the corresponding organ similarity, which is used as the specific organ type to be identified. If the identification result for any target region is that the similarity of any organ is greater than the second threshold and less than or equal to the first threshold, then the identification result is the organ type associated with the corresponding organ similarity, which is used as the pseudo-identified organ type; wherein, the similarity of any organ used for comparison is the highest organ similarity in the corresponding target region.

4. The artificial intelligence method for three-dimensional reconstruction of multiple organs in the chest according to claim 1, characterized in that, Also includes: Based on the relative position information of each thoracic organ, the initial three-dimensional model of the thoracic multi-organ three-dimensional reconstruction is completed by combining the three-dimensional reconstruction model of a single thoracic organ. After corresponding spatial transformation, the model meets the preset organ registration accuracy with the organ image in each target CT image.

5. The artificial intelligence method for three-dimensional reconstruction of multiple organs in the chest according to claim 1, characterized in that, The initial 3D model of a single thoracic organ is converted into discrete point cloud data. A point cloud anomaly detection network model is then constructed to identify anomalies in the point cloud data, including: Obtain the physiological information corresponding to the target individuals from all target CT image scan sources, and filter the preset point cloud anomaly detection network model database based on the physiological information to obtain a set of point cloud anomaly detection network models that conform to the current target individual's physiological information; A three-dimensional reconstruction model of a single thoracic organ is obtained, converted into discrete point cloud data, and input into the point cloud anomaly detection network model corresponding to the thoracic organ in the point cloud anomaly detection network model set. The point cloud anomaly detection result is then output. Based on the point cloud anomaly detection results, the abnormal areas in the corresponding 3D reconstruction model are located, and the abnormal areas are identified by the AI ​​lesion recognition model to obtain lesion data of a single thoracic organ. Obtain lesion data for all thoracic organs as the anomaly identification results for the current initial 3D model.

6. The artificial intelligence method for three-dimensional reconstruction of multiple organs in the chest according to claim 5, characterized in that, Also includes: The pre-set point cloud anomaly detection network model database includes several point cloud anomaly detection network model sets; each point cloud anomaly detection network model set corresponds to a set of physiological information, and each point cloud anomaly detection network model set contains point cloud anomaly detection network models corresponding to all chest organs.

7. The artificial intelligence method for three-dimensional reconstruction of multiple organs in the chest according to claim 6, characterized in that, Also includes: The point cloud anomaly detection network model uses an improved convolutional neural network as the initial network to classify and identify fused data, and is trained and generated through supervised learning using a preset loss function. The fused data is the data obtained by fusing known healthy chest organ features with pseudo lesion features generated based on point cloud deep learning data.

8. The artificial intelligence method for three-dimensional reconstruction of multiple organs in the chest according to claim 1, characterized in that, Also includes: Obtain the preliminary treatment plan formulated by the doctor based on the pathological feedback in the target 3D reconstruction model; By using a clinical efficacy database, clinical efficacy data corresponding to the initial treatment plan is obtained, and a 3D model completion model is trained based on the clinical efficacy data. The 3D model completion model includes a classifier, an encoder, a model completion network, and a decoder. By removing abnormal regions from the initial 3D reconstruction model, an incomplete 3D model is obtained. The incomplete 3D model is completed using a 3D model completion model to obtain a 3D model for effect prediction.

9. The artificial intelligence method for three-dimensional reconstruction of multiple organs in the chest according to claim 8, characterized in that, The training process for a 3D model completion model includes: Construct an initial 3D model completion model, including an initial classifier, an initial encoder, an initial model completion network, and an initial decoder; Based on the standard model data of the lesion corresponding to the clinical effect data, a target three-dimensional model sample is generated, and the initial classifier is trained based on the target three-dimensional model sample to obtain the target classifier; The loss function for constructing the target 3D model sample and the initial 3D model completion model is used to train the initial encoder, the initial model completion network and the initial decoder to obtain the temporary 3D model completion model; The target classifier and the temporary 3D model completion model are concatenated to obtain the final 3D model completion model; the 3D model completion model includes a target classifier, a target encoder, a target model completion network, and a target decoder. The completion operation includes: The model data of the incomplete 3D model is extracted as the data to be completed; each data to be completed corresponds to an abnormal region and each data to be completed corresponds to a type of lesion data. Obtain the target classifier to classify any data to be completed, and determine the lesion type of the target data to be completed; The corresponding data to be completed is input into the target encoder corresponding to the lesion type to obtain incomplete model data. The incomplete model data is then completed using the target model completion network to obtain complete model data, which is then input into the target decoder corresponding to the lesion type to obtain target model data, which is used to reconstruct a 3D model for effect prediction.

Images