Lung organ age prediction method based on relation learning

Abstract

The invention discloses a lung organ age prediction method based on relation learning, and relates to the technical field of computers, and the method comprises the steps: S1, constructing an age prediction model; s2, acquiring a training data set; s3, training and optimizing the age prediction model to obtain an optimized age prediction model; s4, performing prediction by using the optimized age prediction model to obtain age relation estimation of each image; s5, analyzing the age estimation of each image; and S6, analyzing the age of the lung organ. The age prediction model can capture the overall aging characteristics of organs, then age relation prediction is carried out, and the biological age of the lung CT images in the test set is inferred according to the age relation prediction results of the multiple lung CT images; based on a 3D sample age inference strategy of a reference sample and an optimization method, through ablation experiments with other strategies and contrast experiments with other model frameworks, the superiority of the method in multiple evaluation indexes is proved, and the method can effectively improve the accuracy of lung age prediction.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and particularly to a method for predicting the age of the lung organ based on relational learning. Background Art

[0002] With the aging of the population and the aggravation of air pollution, respiratory diseases have become a major challenge to global public health. Lung health is not only closely related to age, but also affected by various factors, such as smoking, air, genetic factors, etc. Traditional lung function tests (such as vital capacity tests) often focus on the measurement of lung function, while the concept of "lung biological age" has not been widely applied. The "lung age" prediction technology aims to predict the true biological age of an individual's lungs by analyzing the lung image data of the individual and combining machine learning algorithms, so as to evaluate the lung health status. This technology can provide a scientific basis for disease prediction, and is particularly important for the prevention of diseases such as chronic obstructive pulmonary disease (COPD) and lung cancer. In the prior art, age prediction is directly regarded as a regression problem, and the problem of data imbalance is difficult to handle and is greatly affected by outliers. If age is regarded as a classification problem, the order information of age will be ignored. Summary of the Invention

[0003] The purpose of the present invention is to design a method for predicting the age of the lung organ based on relational learning to solve the above problems.

[0004] The present invention realizes the above purpose through the following technical solutions:

[0005] A method for predicting the age of the lung organ based on relational learning, comprising:

[0006] S1. Construct an age prediction model. The age prediction model sequentially includes a feature extraction module, a splicing layer, and a relationship prediction module from input to output. The feature extraction module is used to extract key features in the lung image, the splicing layer is used to expand and splice the extracted key features into a sequence, and the relationship prediction module is used to predict age relationship estimation;

[0007] S2. Obtain a training data set;

[0008] S3. Import the training data set into the age prediction model, train and optimize it to obtain an optimized age prediction model;

[0009] S4. Obtain an image set of the person to be predicted, and use the optimized age prediction model to predict each image to obtain the age relationship estimation of each image;

[0010] S5. Analyze the age estimation of each image according to the age relationship estimation;

[0011] S6. Analyze the lung organ age of the person to be predicted based on the age estimates of all images and the age of the person to be predicted.

[0012] A lung organ age prediction device based on relational learning, comprising:

[0013] A storage; the storage is used to store programs;

[0014] An executor; the executor is used to execute the programs stored in the storage. When the executor executes the programs stored in the storage, it implements a lung organ age prediction method based on relational learning as described above.

[0015] The beneficial effects of the present invention are as follows: This age prediction model can capture the overall aging characteristics of organs, and then perform age relationship prediction. Based on the age relationship prediction results of multiple lung CT images, the biological age of the lung CT images in the test set is inferred; this method proposes a 3D sample age inference strategy based on reference samples and optimization methods. Through ablation experiments with other strategies and comparison experiments with other model frameworks, the superiority of this method in multiple evaluation indicators is demonstrated, and this method can effectively improve the accuracy of lung age prediction. Description of the Drawings

[0016] Figure 1 is a schematic structural diagram of the age prediction model in a lung organ age prediction method based on relational learning of the present invention;

[0017] Figure 2 is a schematic structural diagram of the feature extraction module in a lung organ age prediction method based on relational learning of the present invention;

[0018] Figure 3 is a schematic structural diagram of the relationship prediction module in a lung organ age prediction method based on relational learning of the present invention;

[0019] Figure 4 is a graph of the cumulative proportion of prediction error samples of different methods in the experimental example. Detailed Embodiments

[0020] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. The components of the embodiments of the present invention described and illustrated herein can generally be arranged and designed in various different configurations.

[0021] Accordingly, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the scope of protection of the present invention.

[0022] It should be noted that like reference numerals and letters denote like items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings.

[0023] In the description of the present invention, it should be understood that the orientation or positional relationship indicated by the terms "upper", "lower", "inner", "outer", "left", "right", etc. is based on the orientation or positional relationship shown in the accompanying drawings, or the orientation or positional relationship in which the product of the invention is usually placed during use, or the orientation or positional relationship commonly understood by those skilled in the art. It is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus should not be construed as a limitation of the present invention.

[0024] In addition, the terms "first", "second", etc. are only used for distinguishing descriptions and cannot be understood as indicating or implying relative importance.

[0025] In the description of the present invention, it should also be noted that unless otherwise clearly specified and defined, terms such as "set", "connect" should be understood in a broad sense. For example, "connect" can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, and it can be the communication inside two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0026] The following will specifically describe the embodiments of the present invention in conjunction with the accompanying drawings.

[0027] A lung organ age prediction method based on relational learning includes:

[0028] S1. Construct an age prediction model. As Figure 1 shown, the age prediction model sequentially includes a feature extraction module, a splicing layer, and a relationship prediction module from input to output. The feature extraction module is used to extract key features in the lung image. The splicing layer is used to expand and splice the extracted key features into a sequence. The relationship prediction module is used to predict age relationship estimation. The feature extraction module sequentially includes a first residual structure, a batch normalization layer, an activation layer, a max pooling layer, a second residual structure, a third residual structure, and a fourth residual structure from input to output.

[0029] S2. Obtain the training data set.

[0030] S3. Import the training data set into the age prediction model, train and optimize it to obtain the optimized age prediction model.

[0031] S4. Obtain the image set of the person to be predicted, and use the optimized age prediction model to predict each image to obtain the age relationship estimation of each image; the age relationship estimation of each image includes age sum estimation , difference estimation , maximum estimation and minimum estimation ; the age sum relationship is expressed as: r1 = τ x +τ y , the age difference relationship is expressed as: r2 = τ x -τ y , the maximum relationship is expressed as: r3 = MAX(τ x , τ y ), the minimum relationship is expressed as: r4 = MIN(τ x , τ y ), τ x and τ y are the ages of the lung organs corresponding to two CT images.

[0032] S5. Analyze the age estimation of each image according to the age relationship estimation;

[0033] For the reference sample y sampled from the image set of the person to be predicted, its known age is τ y , and its estimated age is . Based on the four defined relationships, for each input image x, when the age of the reference sample y is τ y , there are four different ways to obtain its estimation, expressed as , , and ; the average value of these four estimations gives the age estimation of the image x, expressed as: ;

[0034] The sampling of the reference sample is specifically: conduct a sample reference analysis on each sample, and select the reference sample y according to the sample reference; the sample reference γ(y) is expressed as: , where W is the set of input samples, and They are the estimated value and the actual value of the i-th relationship when y is used as a reference sample. The closer the estimated value and the actual value are, the better the prediction of the age of image x, the larger γ(y), and the stronger the referenceability. Extract the optimal k reference samples as the input image y of the model, and then k estimations of the age of the input image x will be obtained.

[0035] The significance of sample referenceability is that if this sample is used as a reference sample to predict all input images, then the age estimations of most input images are correct.

[0036] S6. Analyze the lung organ age of the person to be predicted based on the age estimations of all images and the age of the person to be predicted; Lung organ age It is expressed as: , where N is the number of images, is the age estimation of the i-th image, is the age of the person X to be predicted, and the correctness function is expressed as: , where [∙] is the indicator function and σ is the threshold for measuring correct prediction; The optimization goal is to determine the appropriate to maximize the sum of the correctness functions of all image age estimations, and the that makes this function reach the maximum value is the lung age estimation of the person X to be predicted.

[0037] Feature extraction module design: The residual network alleviates the problems of vanishing gradients and exploding gradients that occur as the network depth increases by introducing residual connections. The initial parameters of this network model are trained using transfer learning, which helps with fast convergence during the training process and improves the training speed of the model. However, this residual network does not need to finally output the age prediction, but only extracts the features of the image. Therefore, the subsequent fully connected layer, as well as the last convolutional block and pooling layer, need to be discarded. This network is a variant of the ResNet34 structure. As Figure 2 shown, after the image passes through the first convolutional kernel, batch normalization is performed and then it is input into the ReLU activation function and then into the max pooling layer, and finally it is input into three consecutive residual structures and then outputs the feature vector. The network parameters are shown in Table 1:

[0038] Table 1

[0039] The relationship prediction module is as Figure 3As shown, it mainly consists of layer normalization (LN), multi-head attention mechanism (MSA), and multi-layer perceptron layer MLP. Layer normalization LN (Layer Normalization) is a common normalization technique in Transformer, usually used before and after each self-attention layer or feed-forward neural network layer. Its purpose is to stabilize the training process and accelerate convergence by normalizing the input features. Another important feature of the Transformer model is the multi-head attention mechanism (Multi-head Attention). Different from traditional single-head attention, Transformer can learn different representations in multiple subspaces in parallel by calculating multiple attention heads simultaneously. Each attention head is calculated using different parameters, and finally, the outputs of each head are concatenated and passed through a linear transformation to obtain the final result. The multi-layer perceptron layer MLP is a core component of each encoder and decoder in Transformer, usually a feed-forward neural network layer following each self-attention module. The MLP layer is a fully connected layer network, mainly composed of a fully connected layer and an activation function. The fully connected layer performs feature transformation by mapping the input to a higher-dimensional space. The activation function usually uses ReLU (Rectified Linear Unit) or other non-linear activation functions to introduce non-linearity, thereby enhancing the expression ability of the network.

[0040] A lung organ age prediction device based on relational learning, comprising:

[0041] A storage; the storage is used to store programs;

[0042] An actuator; the actuator is used to execute the programs stored in the storage. When the actuator executes the programs stored in the storage, it implements a lung organ age prediction method based on relational learning as described above.

[0043] Experimental example

[0044] The dataset is from Quzhou People's Hospital, and there are 343 cases of lung scan data. Each patient has a DICOM file, which stores tomographic scan image data with different slice thicknesses. Select the image data of the series with the thinnest slice thickness because this series has the largest number of images. When converting it into jpg format image data, it is necessary to read the window width and window level information from the file to determine the conversion parameters. If this information is not available in the file, the default window width and window level are -400 and 1500 respectively. In order to maintain the sequential relationship between multiple images of the same patient, zero-padding naming method needs to be used to name them. In addition, it is also necessary to read the age information from the DICOM file for prediction in subsequent experiments. After processing, finally 339 cases of valid data of patients are obtained. The data of 64 patients are extracted as the test set, and the data of 275 patients are used as the training set. The number of images contained in the training set and the test set are 81988 and 18464 respectively.

[0045] The main coding language of this experiment is Python, the integrated development environment is Pycharm, and the deep learning framework is Pytorch. This patent experiment trains the neural network model on two NVIDIA GeForce GTX 1080 graphics cards with 8GB of memory.

[0046] The age prediction model is trained by the Adam optimizer built in Pytorch. The momentum is set to 0.99, and the weight decay is set to 5e-4. The Adam optimizer is used for gradient calculation and parameter update. This optimizer calculates the first and second moments of the gradient simultaneously and uses bias correction, so that the learning rate can be adjusted adaptively, thus accelerating convergence. The initial learning rate is 1e-4, and the decay rate of the learning rate is 0.98, that is, it is halved approximately every 35 epochs. The batch size is 20. The loss function of the model is the L1 loss, that is, the absolute value loss function.

[0047] This experiment compared five other deep learning-based organ age estimation methods, including CNN, 3D CNN, Transformer, MV, and Ranking-CNN. The results of these methods were obtained on the dataset, and the basic training parameters were consistent with the experimental parameters of the method of this patent. Among them, CNN is the ResNet34 network introduced before. 3D CNN uses a 5-layer 3×3×3 convolutional kernel, and each convolutional layer is followed by a ReLU activation function and a max pooling layer. The number of convolutional kernels in the first layer is 8 and doubles after each max pooling. Finally, a global average pooling layer and a fully connected layer are used for organ age estimation. Transformer is the basic architecture of Vision-transformer. The MV method proposed using a mean-variance loss function instead of a cross-entropy loss function. The backbone network of the model is VGG16. For the hyperparameters λ_1 and λ_2, λ_1 = 0.2 and λ_2 = 0.05 are taken from the original paper. Ranking-CNN needs to train multiple binary classifiers to perform binary classification on ages at different stages. Figure 4 The cumulative proportion graph of prediction error samples of different methods is drawn. The abscissa is the absolute value of age error, and the ordinate is the cumulative proportion of samples.

[0048] From Figure 4 it can be seen that for all methods, the age estimation errors of the vast majority of samples are within 14 years old. For basically all age error ranges, from 1 year old to 14 years old, the proposed relational learning method has the largest number of samples within these error ranges. That is to say, using the method of this patent, the prediction accuracy is mostly the highest. The results of 3DCNN and Transformer are the worst, probably because the model structure is relatively complex, and the results of CNN are average. After using the mean-variance loss function, the prediction accuracy of the model is better than that of the general CNN using cross-entropy. Ranking-CNN uses multiple binary classifiers to perform binary classification on each age group. Each binary classifier needs to be trained using all training sets, with a high computational cost, but it is also better than the results of a single CNN. Through the optimal reference samples and optimization methods, the organ age prediction model based on relational learning of this patent has achieved the optimal age prediction results. The age prediction metrics MAE and R2 of specific models are shown in Table 2.

[0049]

[0050] Table 2

[0051] This method proposes an ordinal learning model, namely a relational learning model. This model is in two stages. In the first stage, four age-related relationships between a pair of input images are predicted, including cumulative relationship, relative relationship, maximum relationship, and minimum relationship. In the second stage, the organ age of the test image is inferred based on the prediction results of multiple age relationships between multiple pairs of images.

[0052] This method proposes a network that combines a feature extraction module based on an improved residual network and an age relationship regression module based on Transformer to predict age relationships. This model combines the local feature extraction ability of convolutional neural networks and the global feature extraction ability of Transformer, and can capture the overall aging characteristics of organs.

[0053] This method proposes a 3D sample age inference strategy based on reference samples and optimization methods. Through ablation experiments with other strategies and comparative experiments with other model frameworks, the superiority of this method in multiple evaluation metrics is demonstrated. This method can effectively improve the accuracy of lung age prediction.

[0054] The technical solution of the present invention is not limited to the limitations of the above specific embodiments. Any technical deformation made according to the technical solution of the present invention falls within the protection scope of the present invention.

Claims

1. A method for predicting the age of the lung organ based on relationship learning, characterized in that, Including: S1. Construct an age prediction model. The age prediction model sequentially includes a feature extraction module, a splicing layer, and a relationship prediction module from input to output. The feature extraction module is used to extract key features in the lung image. The splicing layer is used to expand and splice the extracted key features into a sequence. The relationship prediction module is used to predict age relationship estimation. S2. Obtain a training data set. S3. Import the training data set into the age prediction model, train and optimize it to obtain an optimized age prediction model. S4. Obtain an image set of the person to be predicted, and use the optimized age prediction model to predict each image to obtain the age relationship estimation of each image. S5. Analyze the age estimation of each image according to the age relationship estimation. S6. Analyze the lung organ age of the person to be predicted according to the age estimations of all images and the age of the person to be predicted.

2. The lung organ age prediction method based on relationship learning according to claim 1, characterized in that In S4, the age relationship estimation for each image includes age and estimation , difference estimation , maximum estimation and minimum estimation .

3. The method for predicting the age of a lung organ based on relational learning according to claim 2, wherein In S5, the age estimate of each image x is expressed as: , where , , and , is the age of the reference sample y.

4. The method for predicting the age of a lung organ based on relational learning according to claim 3, wherein Conduct sample reference analysis on each sample, and select a reference sample y according to the sample reference.

5. The lung organ age prediction method based on relationship learning according to claim 4, characterized in that The sample reference γ(y) is expressed as: , where W is the set of input samples, and are the estimated value and the actual value of the i-th relationship when y is used as the reference sample, respectively.

6. A method for predicting the age of the lung organ based on relational learning according to claim 3, characterized in that, In S6, the lung organ age is expressed as: , where N is the number of images, is the age estimate of the i-th image, is the age of the person X to be predicted, and the correctness function is expressed as: , where [∙] is the indicator function and σ is the threshold for measuring the prediction correctness; the optimization goal is to determine the appropriate to maximize the sum of the correctness functions of all image age estimates, and the that makes this function reach the maximum value is the lung age estimate of the person X to be predicted.

7. The method for predicting the age of a lung organ based on relationship learning according to claim 1, wherein The feature extraction module sequentially includes a first residual structure, a batch normalization layer, an activation layer, a max pooling layer, a second residual structure, a third residual structure, and a fourth residual structure from input to output.

8. A lung organ age prediction device based on relational learning, characterized in that Including: A memory; The memory is used to store programs; An executor; The executor is used to execute the programs stored in the memory. When the executor executes the programs stored in the memory, it implements a method for predicting lung organ age based on relationship learning as described in any one of claims 1-7.

Images