Iterative learning-based hepatic tumor focal lesion prediction method

Abstract

The invention relates to a liver tumor focal lesion prediction method based on iterative learning, and belongs to the field of computer models, and the method comprises the steps: firstly, carrying out the manual marking of a lesion region in part of medical image data, obtaining ROI training data, and constructing an automatic lesion detection model based on YOLOv8 through the data; in the ViT classification model, the cut image is divided into image blocks with a fixed size, and the image blocks are embedded into a high-dimensional space. Then, location information is added to each image block using location coding to preserve their spatial relationship. And inputting the embedded sequence into a Transform module, carrying out classification processing through a full connection layer, and finally outputting a predicted category. According to the method, a dynamic learning strategy is introduced, sample weights in a loss function are dynamically adjusted in different training stages, accumulative errors of the model can be reduced, and uncertain samples can be effectively processed at the same time.

Description

Technical Field

[0001] The present invention relates to the field of computer models, and in particular to a method for predicting focal lesions of liver tumors based on iterative learning. Background Art

[0002] The high morbidity and mortality of liver cancer in focal lesions of the liver make it the sixth most commonly diagnosed cancer and the third leading cause of cancer death worldwide. Malignant liver tumors mainly include primary liver cancer (Hepatocellular Carcinoma, HCC), intrahepatic cholangiocarcinoma (Intrahepatic Cholangiocarcinoma, ICC), and secondary liver cancer (metastatic liver cancer, MET). In addition to liver cancer, benign tumors such as hemangioma (HEM), focal nodular hyperplasia (FNH) and liver cysts (CYST) are also often encountered in clinical treatment. Different types of tumors have different treatment strategies and prognosis. Accurate diagnosis is crucial for treatment planning and patient prognosis. Traditional methods for diagnosing these lesions rely heavily on manual work, which can be time-consuming and affected by differences between observers.

[0003] Medical images can be used to classify focal liver lesions (FLLs). Shiraishi et al. developed a computer-aided diagnosis (CAD) system to classify FLLs using an enhanced ultrasound microfluidic imaging particle analysis system. The diagnostic system first extracted 43 features including time, morphology, and grayscale images, and selected 16 features from the above 43 features as inputs to 6 cascaded artificial neural networks to achieve classification of five types of FLLs. Rognin et al. developed a new parametric imaging technique that maps different features into a single dynamic vascular pattern parameter image, which can be used as a diagnostic aid to characterize lesion types. Kondo et al. proposed a new method for extracting spatial and temporal features of ultrasound images, and then used the SVM method to classify FLLs into HCC, liver metastasis, and benign tumors. Gatos et al. also developed a non-enhanced T2-weighted MRI FLL classification system based on SVM. First, each FLL was segmented using a Markov random field, and 42 texture features and 12 morphological features were extracted respectively; secondly, stepwise multivariate linear regression analysis was used for feature selection, and the lesions were classified using an SVM classifier. However, there are limitations to using artificially designed features for FLL classification. First, artificially designed features rely on the experience and knowledge of experts and are limited to their understanding of lesion characteristics. Second, the effectiveness of classification is overly dependent on the selected features. Third, artificially designed features are fixed after design and cannot be dynamically optimized as the data set changes. With the development of pattern recognition and visual analysis technology, computer-assisted automatic analysis of liver cancer images has become possible. Compared with traditional artificially designed features, computer-assisted methods can select the most useful features from medical images. This automated feature extraction process not only improves efficiency, but also captures some subtle patterns that are difficult for the human eye to detect.

[0004] Current research mainly focuses on the diagnosis of hepatocellular carcinoma (HCC), while less attention is paid to the diagnosis of other common liver tumors. Although some studies have made initial progress in these areas, these methods still face certain uncertainties and challenges in clinical application due to the insufficient size of training datasets, limited scope of external validation, and poor stability of validation results. Summary of the invention

[0005] The purpose of the present invention is to overcome the shortcomings of the prior art and provide a method for predicting focal lesions of liver tumors based on iterative learning, thereby solving the deficiencies of the prior art.

[0006] The object of the present invention is achieved by the following technical solution: a method for predicting focal lesions of liver tumors based on iterative learning, the prediction method comprising:

[0007] Step 1: First, manually annotate the lesion regions (ROIs) in some medical image data to obtain ROI training data, and use these data to build an automatic lesion detection model based on YOLOv8. This model can automatically detect the ROI in medical images, and extract a 224×224 pixel square area based on the center point of the ROI, and crop it as the input of the ViT classification model;

[0008] Step 2: Use block embedding in the ViT classification model to divide the image into a series of fixed-size image blocks, and embed these image blocks into a high-dimensional space. The embedded image block sequence is positionally encoded to add position information to each image block to preserve the spatial relationship. The embedded sequence is input into the Transformer module. The features output by the Transformer module are classified through the fully connected layer, and finally the predicted category representing the classification result of liver tumor lesions is output.

[0009] A dynamic learning strategy is introduced during the training process to dynamically adjust the sample weights in the loss function according to different training stages to reduce the cumulative error of the model and effectively handle uncertain samples.

[0010] The step 1 of automatically acquiring the ROI image in the medical image through the model includes: inputting a CT image with a size of 512×512, using the model to automatically detect the ROI in the CT image, determining the coordinates of the ROI center in the CT image, extracting a 224×224 pixel square area based on the ROI center, and cropping the image of the square area as input to the classification model.

[0011] Data detection, dynamic learning strategy and self-supervised learning are added to the ViT classification model; the data detection is used to screen and clean the noise in the input data to improve the data quality; the dynamic learning strategy adjusts the sample weight according to the confidence level of the data detection module, emphasizes the clean samples and weakens the influence of the noise samples, thereby optimizing the model training process; the self-supervised learning provides the model with additional supervision information to enhance its understanding of data representation and characteristics, thereby significantly improving the accuracy of the model.

[0012] The data detection specifically includes the following contents:

[0013] The softmax layer representation of the ViT classification model is used to identify fuzzy data, and the information entropy of the predicted probability of sample x is expressed as , and set the information entropy threshold , by comparing the information entropy of sample x with the threshold, we determine whether the sample has uncertainty. Then the sample x is uncertain, otherwise the sample x is certain, where is the classification label of the data set. When a sample input x is given, p(y i |x) indicates that the classification network predicts that the sample is classified as label y i The probability of , H(x) is the information entropy of sample x;

[0014] pass represents samples that satisfy both prediction error and high information entropy. The model predicts the label for sample x, For the indicator function, if the condition of the prediction error is true, it is uncertain data, otherwise it is clean data to identify potential uncertain data.

[0015] The dynamic learning strategy module specifically includes the following contents:

[0016] The weight assigned to each sample is adjusted by using the confidence level from the data detection module. For clean samples, their weight is kept as 1, and for samples marked as uncertain, the weight is dynamically assigned according to their confidence level.

[0017] During the training process, by learning the strategy Gradually reduce the weight assigned to samples near the decision boundary, where L CE (f(x;θ),y) is the cross entropy loss function, w t (x,y) represents the weight applied in training step t, 1-p t represents the probability of the ViT classification model processing incorrect categories, t is the training round, k is the hyperparameter, and e -kt It controls the rate at which the weights decrease as the number of training rounds increases.

[0018] The self-supervised learning module specifically includes the following contents:

[0019] The input image is divided into n blocks, aligned with the structure of the ViT classification model, and a representation is generated image blocks, C is a data matrix, R represents a set of real numbers, n represents the number of rows of the matrix (i.e., the number of blocks into which the input image is divided), and d represents the number of columns of the matrix;

[0020] Randomly select (1-η)×n of the image patches as context image patches, where η (η>0) represents the masking ratio, and then modify the self-attention layer, in which only the context image patches participate in the calculation of keys and values;

[0021] A linear prediction head is added after the last attention layer, and the position prediction loss is calculated using the cross entropy between the position indicator and the prediction head output: , where L CE represents the cross entropy loss, Indicates that a fully connected layer with parameter u is applied to the feature , represents the distribution of the actual position labels, p(y i |x) is the probability distribution of the actual label at the i-th position in sample x, is the probability distribution of the label at the i-th position in sample x predicted by the ViT classification model, y i Represents the true label of sample x;

[0022] Finally, the final loss function is , where P r is the random probability, is the dynamic probability, t is the training round, T is the total training round, is the noise perception loss, Prediction loss for image patch locations.

[0023] The present invention has the following advantages: a method for predicting focal lesions of liver tumors based on iterative learning, which can automatically analyze radiological images, quickly screen and identify suspicious areas for further examination by radiologists; it introduces dynamic probability, dynamically adjusts the sample weights in the loss function at different training stages, can reduce the cumulative error of the model, and effectively handle uncertain samples. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 It is a schematic diagram of the structure of the automation model of YOLOv8 of the present invention;

[0025] Figure 2 Schematic diagram of the structure of the ViT classification model. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the detailed description of the embodiments of the present application provided below in conjunction with the drawings is not intended to limit the scope of protection of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application. The present invention is further described below in conjunction with the drawings.

[0027] The present invention specifically relates to a method for predicting focal lesions of liver tumors based on iterative learning. In large sample data, the existence of noise data is usually inevitable. Since the data collection process may be affected by various factors, such as the performance of the data acquisition equipment, environmental conditions, human operation, etc., various forms of noise may exist in the data set. This also brings difficulties to model training. In order to reduce the impact of noise samples and improve the robustness of the model, the present invention proposes a dynamic iterative learning framework, which includes noise data detection for identifying uncertain samples and a weighting strategy for adjusting the model's attention to uncertain samples. On this basis, dynamic probability is introduced to dynamically adjust the sample weights in the loss function at different training stages. Label accuracy and information entropy are used as key indicators for evaluating sample quality. Information entropy not only indicates the uncertainty associated with the sample, but also helps to identify samples with inaccurate labels and abnormal feature distribution. Its significant feature is that it fully considers the accuracy of the label and the distribution of the data, demonstrating its effectiveness in processing uncertain samples. A specially designed dynamic learning strategy is used to continuously adjust the suppression of uncertain samples.

[0028] Specifically include the following:

[0029] 1. Data collection: 4458 cases of data were collected from 7 centers, including lesions of HCC, ICC, MET, CYST, HEM, and FNH. The training dataset included CT images of 1580 patients in a hospital, including 703 HCC, 323 ICC, 149 MET, 120 HEM, 133 cysts, and 100 FNH patients. The test dataset included CT images of 1308 patients, including 946 HCC, 80 ICC, 133 MET, 35 HEM, 63 CYST, and 51 FNH. External validation included 682 HCC, 145 ICC, 122 MET, 39 FNH, 139 HEM, 222 CYST, and 121 normal data. According to the clinical diagnostic consensus, HCC, ICC, and MET were classified as malignant tumors, and HEM, FNH, and CYST were classified as benign tumors.

[0030] 2. If Figure 1As shown in the figure, several professional doctors manually delineated the lesion areas (regions of interest, ROI) of 500 patients. Then, using these manually annotated ROI data, an automated model based on YOLOv8 was developed to automatically obtain ROIs in medical images. The input of the YOLOv8 model is a 512×512 CT image, and the output is the ROI coordinates of the CT image. Each detected ROI is centered in a new image. The specific steps are as follows: (1) Determine the coordinates of the ROI center; (2) Extract a 224×224 pixel square area based on the ROI center coordinates. This step ensures that the ROI is in the center of the image and appropriately introduces the tissue or environment around the lesion as contextual information. The image within the above square area is cropped as the input of the classification model.

[0031] 3. Classification model: Figure 2 As shown in the figure, the classification model infrastructure is Vision Transformer (ViT). The input is a 224×224 square image containing the lesion and the surrounding tissue environment. After passing through the Transformer, the probability of each category is output through the last fully connected layer of the classification model. The classification model will give the final classification results based on these probabilities. Data detection, dynamic learning strategy and self-supervised learning are added to the original ViT architecture. The data detection module is used to filter and clean the noise in the input data to improve the data quality. The dynamic learning strategy module adjusts the sample weights according to the confidence level of the data detection module, emphasizing clean samples and reducing the influence of noise samples, thereby optimizing the model training process. The self-supervised learning module provides the model with additional supervision information to enhance its understanding of data representation and features, thereby significantly improving the accuracy of the model.

[0032] Furthermore, the patch embedding in the classification model ViT architecture divides the image into a series of fixed-size image patches and embeds these image patches into a high-dimensional space. The embedded image patch sequence is positionally encoded to add position information to each patch to preserve the spatial relationship. Next, the embedded sequence is input into the Transformer encoder module (obtained by L Transformers in series), which consists of a multi-layer stacked multi-head self-attention mechanism and a feedforward neural network (MLP module). The self-attention mechanism can capture the relationship between different areas in the image, while the feedforward neural network is responsible for nonlinear transformation and feature extraction. The features output by the encoder are passed through the fully connected layer to predict the input image, and finally the classification result of liver tumor lesions is obtained.

[0033] Data detection: Generally speaking, samples from the same category should have similar features, so the prediction results of machine learning models for them should also be consistent. However, incorrectly labeled samples may disrupt this consistency, causing the model to make inconsistent predictions on similar samples. In supervised learning tasks, especially image classification problems, the prediction results output by the model are often presented in the form of probability distributions. Through the Softmax layer, the model converts the original output into the probability distribution of the category. For samples of the same category, the model will ideally output similar probability distributions (i.e., intra-class consistency). However, if some samples are incorrectly labeled or they have specific features, these samples may result in inconsistent probability distributions with other samples of the same category, thus appearing "abnormal". Detecting these "abnormal" samples (uncertain samples) can further perform data quality checks during the training process of the classification model, thereby improving the performance of the classification model. The representation of the softmax layer of the classification model is used to identify uncertain samples. To achieve this, consider the information entropy of the predicted probability of sample x, denoted as H(x), defined as:

[0034] ;

[0035] in, Classify the dataset into labels. When a sample input x is given, Indicates that the classification network predicts that the sample is classified as label y i The probability of H(x) is the information entropy of sample x.

[0036] In order to identify uncertain samples, the information entropy threshold is set to , and determine whether the sample has uncertainty by comparing the information entropy of sample x with the threshold. Then the sample x is uncertain, otherwise the sample x is certain.

[0037] High information entropy samples are usually more challenging and difficult for the model to classify accurately. However, high information entropy does not necessarily mean that there is noise in the data. It reflects the uncertainty of the model's prediction of these samples. In order to better highlight the performance issues of the model on these samples, the present invention designs a prediction error condition. It represents the samples that satisfy both the prediction error condition and high information entropy. Its expression is:

[0038] ;

[0039] Among them, y i represents the true label of sample x, The model predicts the label for sample x, For the indicator function, if the condition of the prediction error is true, it is uncertain data, otherwise it is clean data. This condition helps to identify potential uncertain data more accurately.

[0040] Dynamic learning strategy: The sample weights are adjusted according to the confidence level of the data detection module, with emphasis on clean samples and reduction of the impact of noise samples, thereby optimizing the model training process. For clean samples, the sample weights are fixed to 1, and no further adjustment is required in the context of cross-entropy loss; on the contrary, for samples marked as uncertain, weights are dynamically assigned according to their confidence. At the beginning of training, the prediction ability of the classification model is weak. Samples that meet the prediction error conditions described in the data detection do not mean that these samples are noise samples. Imposing penalties early may lead to cumulative errors and reduce the predictive ability of the model. In order to proactively address potential error accumulation, the present invention designs a dynamic data detection and reweighting strategy. Specifically, with the iteration of training, the dynamic learning strategy sets the dynamic probability to To reduce the risk of false positives, we gradually reduce the weights assigned to samples near the decision boundary in successive training rounds. The learning strategy L NA It is expressed as:

[0041] ;

[0042] Among them, L CE (f(x;θ),y) is the cross entropy loss function, w t (x,y) are the weights of our reweighting strategy. 1-p t represents the probability of the classification model processing incorrect categories, t is the training round, k is a hyperparameter (default is 0.1), e -kt Controls the rate at which weights are reduced as training epochs increase.

[0043] Self-supervised learning: In this invention, the pretext task in self-supervised learning is applied to supervised learning, providing the model with additional supervision information to enhance its understanding of data representation and features, thereby significantly improving the accuracy of the model. The basic idea of ​​the pretext task of the present invention includes predicting the location based on the content. Specifically, the input image is divided into n blocks and aligned with the structure of the ViT classification model. This process generates a Represents an image block, where C is a data matrix, R represents a set of real numbers, each element of the matrix C belongs to a real number, n represents the number of rows of the matrix (i.e., the number of blocks into which the input image is divided), and d represents the number of columns of the matrix. Subsequently, (1-η)×n in the image block are randomly selected as the context image block, where η (η>0) represents the masking ratio. The self-attention layer is then modified, in which only the context image block participates in the calculation of the key and value. That is, the present invention transitions from considering all input image blocks n to focusing on the context image block (1-η)×n. When η>0, the Transformer encoder module must sort all input image blocks while only using a subset of them to form a robust representation of the input. This forces the model to infer the relationship between context image blocks when inferring the masked image blocks. A linear prediction head is added after the last attention layer, and the position prediction loss is calculated using the cross entropy between the position indicator and the prediction head output:

[0044] ;

[0045] Among them, L CE represents the cross entropy loss, To apply a fully connected layer with parameter u to the feature , represents the distribution of the actual position labels, p(y i |x) is the probability distribution of the actual label at the i-th position in sample x, is the probability distribution of the label at the i-th position in sample x predicted by the classification model.

[0046] In summary, the loss function of the present invention consists of two key components. The first component is the noise-aware loss to skillfully handle the weight distribution of noisy samples. The second component is the cross-entropy loss to facilitate network training through preliminary tasks. Combining these two losses to improve the performance of the model, we get the final loss function:

[0047] ;

[0048] Among them, P r is the random probability, is the dynamic probability, L NA (x,y) is the noise perception loss, Prediction loss for image patch locations.

[0049] The above is only a preferred embodiment of the present invention. It should be understood that the present invention is not limited to the form disclosed herein, and should not be regarded as excluding other embodiments, but can be used for various other combinations, modifications and improvements, and can be modified within the scope of the concept described herein through the above teachings or the technology or knowledge of the relevant field. The changes and modifications made by those skilled in the art do not deviate from the spirit and scope of the present invention, and should be within the scope of protection of the claims attached to the present invention.

Claims

1. A method for predicting focal lesions of liver tumors based on iterative learning, characterized in that: The prediction method comprises: Step 1: First, manually annotate the ROIs in some medical image data to obtain ROI training data, and use these data to build an automatic lesion detection model based on YOLOv8. This model can automatically detect the ROI in medical images, extract a 224×224 pixel square area based on the center point of the ROI, and crop the image of the square area as the input of the ViT classification model; Step 2: Use block embedding in the ViT classification model to divide the image into a series of fixed-size image blocks, and embed these image blocks into a high-dimensional space. The embedded image block sequence is positionally encoded to add position information to each image block to preserve the spatial relationship. The embedded sequence is input into the Transformer encoder module. The features output by the Transformer encoder module are classified through the fully connected layer, and finally the predicted category representing the classification result of liver tumor lesions is output.

2. The method for predicting focal lesions of liver tumors based on iterative learning according to claim 1, characterized in that: In the step 1, automatically acquiring the ROI image in the medical image through the model includes: inputting a CT image with a size of 512×512, using the model to automatically detect the ROI in the CT image, determining the center coordinates of the ROI in the CT image, extracting a 224×224 pixel square area based on the ROI center, and cropping the image of the square area as input to the classification model.

3. The method for predicting focal lesions of liver tumors based on iterative learning according to claim 1, characterized in that: Data detection, dynamic learning strategy and self-supervised learning are added to the ViT classification model; the data detection is used to screen and clean the noise in the input data to improve the data quality; the dynamic learning strategy adjusts the sample weight according to the confidence level of the data detection module, emphasizes the clean samples and weakens the influence of the noise samples, thereby optimizing the model training process; the self-supervised learning provides the model with additional supervision information to enhance its understanding of data representation and characteristics, thereby improving the accuracy of the model.

4. The method for predicting focal lesions of liver tumors based on iterative learning according to claim 3, characterized in that: The data detection specifically includes the following contents: The softmax layer representation of the ViT classification model is used to identify fuzzy data, and the information entropy of the predicted probability of sample x is expressed as , and set the information entropy threshold to , by comparing the samples The information entropy and threshold are used to determine whether the sample has uncertainty. If Then the sample x is uncertain, otherwise the sample x is certain, where is the classification label of the data set. When a sample input x is given, p(y i |x) indicates that the classification network predicts that the sample is classified as label y i The probability of , H(x) is the information entropy of sample x; pass represents samples that satisfy both prediction error and high information entropy. The model predicts the label for sample x, For the indicator function, if the condition of the prediction error is true, it is uncertain data, otherwise it is clean data to identify potential uncertain data.

5. The method for predicting focal lesions of liver tumors based on iterative learning according to claim 3, characterized in that: The dynamic learning strategy module specifically includes the following contents: The weight assigned to each sample is adjusted by using the confidence level from the data detection module. For clean samples, their weight is kept as 1, and for samples marked as uncertain, the weight is dynamically assigned according to their confidence level. During the training process, by learning the strategy Gradually reduce the weight assigned to samples near the decision boundary, where L CE (f(x;θ),y) is the cross entropy loss function, w t (x,y) represents the weight applied in training step t, 1-p t represents the probability of the ViT classification model processing incorrect categories, t is the training round, k is the hyperparameter, and e -kt It controls the rate at which the weights decrease as the number of training rounds increases.

6. The method for predicting focal lesions of liver tumors based on iterative learning according to claim 3, characterized in that: The self-supervised learning module specifically includes the following contents: The input image is divided into n blocks, aligned with the structure of the ViT classification model, and a representation is generated The image block is a data matrix, C is a data matrix, R is a real number set, n is the number of rows of the matrix, and d is the number of columns of the matrix; Randomly select (1-η)×n of the image patches as context image patches, where η represents the masking ratio, and then modify the self-attention layer, where only the context image patches participate in the calculation of keys and values, η>0; A linear prediction head is added after the last attention layer, and the position prediction loss is calculated using the cross entropy between the position indicator and the prediction head output: , where L CE represents the cross entropy loss, Indicates that a fully connected layer with parameter u is applied to the feature , represents the distribution of the actual position labels, p(y i |x) is the probability distribution of the actual label at the i-th position in sample x, is the probability distribution of the label at the i-th position in sample x predicted by the ViT classification model, y i Represents the true label of sample x; Finally, the final loss function is , where P r is the random probability, is the dynamic probability, t is the training round, T is the total training round, is the noise perception loss, Prediction loss for image patch locations.

7. The method for predicting focal lesions of liver tumors based on iterative learning according to claim 1, characterized in that: A dynamic learning strategy is introduced in the training process of the ViT classification model. The sample weights in the loss function are dynamically adjusted according to different training stages to reduce the cumulative error of the ViT classification model and effectively handle uncertain samples.

Images