A transfer prediction method and system based on resampling and three-dimensional attention convolution

Abstract

The embodiment of the application provides a transfer prediction method and system based on resampling and three-dimensional attention convolution, and relates to the technical field of medical image artificial intelligence. The method can perform resampling on PET images in PET image data after obtaining the PET image data, to obtain balanced image data, and then extract image key features from the balanced image data through a three-dimensional residual convolution neural network and based on a three-dimensional convolution attention mechanism. The prediction model is trained by using the image basic features and the image key features, so as to output the transfer probability by using the prediction model. The method can balance the positive and negative example sample distribution by resampling, and relieve the model deviation caused by the high imbalance of positive and negative samples of 3D PET image data. The three-dimensional attention convolution network is used to deeply mine spatial features, sufficiently extract three-dimensional spatial features of 3D PET, and then the weighted fusion loss function is combined to optimize the model training process, so as to improve the prediction accuracy of the lymph node transfer probability.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology in medical imaging, and in particular to a transfer prediction method and system based on resampling and three-dimensional attention convolution. Background Technology

[0002] In medical image analysis, computer-aided analysis and prediction of image content can be used. For example, in medical images of lung cancer cases, computer-aided determination can be made to determine whether lymph node metastasis (LNM) has occurred, especially the metastasis of mediastinal lymph nodes (MLN), enabling the calculation of the probability of lymph node metastasis in a non-invasive manner.

[0003] Medical imaging is crucial for performing computer-aided analysis. For example, medical images can be positron emission tomography-computed tomography (PET-CT) images. PET-CT images can integrate functional metabolic information such as glucose metabolism activity with anatomical information such as lymph node location and morphology, comprehensively reflecting medical characteristics and improving analytical accuracy. During computer-aided analysis, PET-CT images can be input into a deep learning model. The deep learning model extracts image features from the PET-CT images to predict metastasis probabilities.

[0004] However, current deep learning analysis methods based on PET-CT images often involve compressing 3D PET volume data into a two-dimensional maximum density projection or using layer-by-layer 2D slicing as input to the network for modeling. This results in the loss of 3D spatial context information and reduces the accuracy of metastasis probability prediction. While some 3D CNN-based methods can preserve spatial dimensional information, they fail to fully exploit the value of 3D space and do not sufficiently extract 3D image features. Furthermore, at the data characteristics level, PET image data suffers from severe class imbalance, with the number of positive samples (metastatic lymph nodes) being far lower than the number of negative samples (non-metastatic lymph nodes). This makes it highly susceptible to majority class bias during model training. Summary of the Invention

[0005] In view of this, embodiments of this application provide a transition prediction method and system based on resampling and three-dimensional attention convolution to solve the problem of low accuracy in computer-aided transition probability prediction.

[0006] According to a first aspect of this application, a transition prediction method based on resampling and three-dimensional attention convolution is provided, the method comprising:

[0007] Acquire positron emission tomography (PET) image data, wherein the PET image data includes multiple PET images;

[0008] The PET images are resampled to obtain balanced image data, which includes positive samples and negative samples. The positive samples are transfer target images obtained by oversampling the PET images, and the negative samples are non-transfer target images obtained by undersampling the PET images.

[0009] Using a three-dimensional residual convolutional neural network, basic image features are extracted from the balanced image data. These basic image features include local features in the balanced image data that are case-discriminatory.

[0010] Based on the three-dimensional convolutional attention mechanism, key image features are extracted from the balanced image data. The key image features include local information with transfer correlation in the balanced image data. The key image features are the feature maps output by modeling the importance of each channel based on the channel attention mechanism and then weighting the spatial position of the feature maps through the spatial attention mechanism.

[0011] A prediction model is trained based on the basic features and key features of the image. The prediction model is a neural network model with a weighted combination of binary cross-entropy loss and F1 loss as the loss function.

[0012] The predicted model is used to output the transition probability.

[0013] In some embodiments, resampling is performed on the PET images to obtain balanced image data, including:

[0014] By traversing the PET images in the PET image data, the original data structure information is obtained. The basic data structure information is used to characterize the ratio of the number of original positive samples to the number of original negative samples in the PET image data.

[0015] The resampling factor is calculated based on the original data structure information, and the resampling factor includes the oversampling factor and / or the undersampling factor; the resampling factor is positively correlated with the integer part of the quantity ratio;

[0016] The PET image is resampled based on the resampling factor.

[0017] In some embodiments, resampling of the PET image is performed based on the resampling factor, including:

[0018] The number of resampling times is calculated based on the resampling factor;

[0019] The PET images are resampled according to the number of resampling operations to obtain generated sample data, which includes generated positive samples obtained by oversampling the original positive samples and / or generated negative samples obtained by undersampling the original negative samples.

[0020] Obtain the number of original samples in the PET image data, wherein the number of original samples includes the number of original positive samples and the number of original negative samples;

[0021] Calculate the number of generated samples, which is equal to the product of the original number of samples and the resampling factor; the number of generated samples includes the number of positive samples after oversampling the original positive samples, and / or the number of negative samples after undersampling the original negative samples;

[0022] The positive-negative sample difference is calculated based on the number of positive samples and the number of negative samples, and the positive-negative sample difference is equal to the difference between the number of positive samples and the number of negative samples;

[0023] If the positive and negative sample difference is greater than a preset deviation threshold, the number of resampling times is reset based on the positive and negative sample difference;

[0024] If the positive and negative sample difference is less than or equal to a preset deviation threshold, the balanced image data is output based on the generated sample data and the PET image data.

[0025] In some embodiments, the method further includes:

[0026] Extract the ratio of the number of original positive samples to the number of original negative samples from the original data structure information;

[0027] If the quantity ratio is within a preset quantity ratio range, the PET image data is marked as the balanced image data;

[0028] If the quantity ratio is not within the preset quantity ratio range, the PET image is resampled to obtain balanced image data.

[0029] In some embodiments, the extraction of basic image features from the balanced image data using a three-dimensional residual convolutional neural network includes:

[0030] The input features are extracted from the balanced image data using a feature extraction network;

[0031] The input features are input into a three-dimensional residual convolutional neural network, which includes a first branch path and a second branch path.

[0032] A first feature map is generated through the first branch path. The first feature map is generated by the first branch path performing a three-dimensional convolution operation on the input features, normalizing the convolution result, and then using a non-linear activation function.

[0033] A second feature map is generated through the second branch path. The second feature map is a feature map generated by normalizing the convolution result after the second branch path performs a three-dimensional convolution operation on the input features.

[0034] The image basic features are generated based on the first feature map and the second feature map.

[0035] In some embodiments, the three-dimensional residual convolutional neural network further includes a downsampling path and a feature fusion path; generating the image basic features based on the first feature map and the second feature map includes:

[0036] The input features are downsampled using the downsampling path to obtain a downsampling result;

[0037] The second feature map and the downsampling result are added element by element through the feature fusion path to obtain the first addition result;

[0038] The first summation result is nonlinearly activated using a nonlinear activation function to generate a third feature map, which is a feature map that integrates multipath information.

[0039] Output the first feature map and the third feature map to obtain the basic features of the image.

[0040] In some embodiments, key image features are extracted from the balanced image data based on a three-dimensional convolutional attention mechanism, including:

[0041] The three-dimensional convolutional attention module is invoked, which includes channel attention units and spatial attention units;

[0042] The third feature map is input into the three-dimensional convolutional attention module;

[0043] The importance of each channel corresponding to the third feature map is modeled using the channel attention unit to obtain modeling data;

[0044] The spatial position of the third feature map is weighted by a spatial attention unit to obtain a weighted result;

[0045] The modeling data and the weighted results are fused to obtain key fusion features;

[0046] Calculate the second summation result, which is the sum obtained by adding the fused key features and the downsampling result element by element;

[0047] The second summation result is subjected to nonlinear activation using a nonlinear activation function to generate the key features of the image.

[0048] In some embodiments, training a prediction model based on the image's basic features and key image features includes:

[0049] Obtain the weighting parameters, which are used to control the ratio of binary cross-entropy loss to F1 loss;

[0050] The loss function of the trained model is set according to the weighting parameters;

[0051] The basic features and key features of the image are input into the trained model to obtain the training prediction results output by the trained model.

[0052] Based on the loss function, the training loss is calculated according to the training labels of the PET images and the training prediction results;

[0053] If the training loss is greater than the loss threshold, the model parameters of the trained model are adjusted according to the training loss.

[0054] If the training loss is less than or equal to the loss threshold, the model parameters of the trained model are output to obtain the prediction model.

[0055] In some embodiments, using the prediction model to output transition probabilities includes:

[0056] Acquire PET images to be predicted;

[0057] The basic features of the PET image to be predicted are extracted using a three-dimensional residual convolutional neural network.

[0058] Based on a three-dimensional convolutional attention mechanism, key image features are extracted from the PET image to be predicted.

[0059] The image's basic features and key features are input into the prediction model to obtain the transition probability output by the prediction model.

[0060] According to a second aspect of this application, a transition prediction system based on resampling and three-dimensional attention convolution is provided, the system comprising:

[0061] The data acquisition module is used to acquire positron emission tomography (PET) image data, which includes multiple PET images.

[0062] A resampling module is used to perform resampling on the PET image to obtain balanced image data, the balanced image data including positive samples and negative samples, the positive samples being transfer target images obtained by oversampling the PET image; the negative samples being non-transfer target images obtained by undersampling the PET image;

[0063] The basic feature extraction module is used to extract basic image features from the balanced image data through a three-dimensional residual convolutional neural network. The basic image features include local features in the balanced image data that have case-discriminatory properties.

[0064] The key feature extraction module is used to extract key image features from the balanced image data based on the three-dimensional convolutional attention mechanism. The key image features include local information with transfer correlation in the balanced image data. The key image features are the feature maps output by modeling the importance of each channel based on the channel attention mechanism and weighting the spatial position of the feature maps through the spatial attention mechanism.

[0065] The model training module is used to train a prediction model based on the basic features and key features of the image. The prediction model is a neural network model with a weighted combination of binary cross-entropy loss and F1 loss as the loss function.

[0066] A probabilistic prediction model is used to output transition probabilities using the prediction model.

[0067] According to a third aspect of this application, a computer device is provided, including a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein the processor executes the program to implement the above-described transfer prediction method based on resampling and three-dimensional attention convolution.

[0068] According to a fourth aspect of this application, a storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the above-described transfer prediction method based on resampling and three-dimensional attention convolution.

[0069] By employing the above technical solutions, this application provides a method and system for predicting lymph node metastasis based on resampling and 3D attention convolution. The method involves resampling PET images after acquiring them to obtain balanced image data. Then, a 3D residual convolutional neural network and a 3D convolutional attention mechanism are used to extract key image features from the balanced image data. A prediction model is trained using the basic image features and key image features to output the metastasis probability. This method can balance the distribution of positive and negative samples through resampling, mitigating model bias caused by the high imbalance between positive and negative samples in 3D PET image data. The 3D attention convolutional network is used to deeply mine spatial features, fully extracting the 3D spatial features of 3D PET, and then combined with a weighted fusion loss function to optimize the model training process, improving the prediction accuracy of lymph node metastasis probability.

[0070] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0071] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0072] Figure 1 A schematic diagram of the transition prediction method based on resampling and 3D attention convolution provided in the embodiments of this application;

[0073] Figure 2 This is a schematic diagram of the transfer prediction process provided in an embodiment of this application;

[0074] Figure 3 This is a flowchart illustrating the preprocessing data provided in an embodiment of this application.

[0075] Figure 4 This is a schematic diagram of the process for extracting basic image features provided in an embodiment of this application;

[0076] Figure 5 This is a schematic diagram of the prediction model training process provided in the embodiments of this application;

[0077] Figure 6 This is a schematic diagram illustrating the process of using a predictive model to output transition probabilities, as provided in an embodiment of this application.

[0078] Figure 7 This is a schematic diagram of the transition prediction system structure based on resampling and three-dimensional attention convolution provided in an embodiment of this application. Detailed Implementation

[0079] The present application will be described in detail below with reference to the accompanying drawings and embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in the embodiments of the present application can be combined with each other.

[0080] In this embodiment, the transfer prediction method based on resampling and three-dimensional attention convolution can be applied to transfer prediction systems in the field of medical imaging artificial intelligence technology, specifically to medical devices, teaching devices, and experimental devices that predict transfer probabilities based on medical image data.

[0081] For example, the method described can be used to help predict the probability of lymph node metastasis in lung cancer. Lung cancer is one of the leading causes of cancer-related deaths worldwide, with non-small cell lung cancer (NSCLC) accounting for 80-85% of primary lung cancers and being a major cause of cancer-related deaths. Once lymph node metastasis (LNM) occurs, especially mediastinal lymph node (MLN) metastasis, the survival and prognosis of lung cancer patients deteriorate.

[0082] In some embodiments, computer-aided diagnosis can be used to determine whether lymph node metastasis has occurred, enabling the non-invasive calculation of the probability of lymph node metastasis and generating key radiological information. Positron emission tomography-computed tomography (PET-CT) integrates functional metabolic information (such as glucose metabolism activity) with anatomical structural information (such as lymph node location and morphology), overcoming the limitations of single morphological assessment and becoming the most comprehensive imaging method currently available, widely used in lung cancer staging and distant metastasis detection.

[0083] However, current deep learning-based diagnostic techniques for PET still have significant limitations. Specifically, in terms of 3D information processing, most studies compress 3D PET volume data into a 2D maximum density projection or use layer-by-layer 2D slicing as input to the network for modeling. This inevitably leads to the loss of 3D spatial context information, directly affecting prediction accuracy. While a few methods based on 3D CNNs can preserve spatial dimensional information, they fail to fully exploit the value of 3D space and do not sufficiently extract 3D image features.

[0084] At the data characteristics level, PET image data suffers from a severe class imbalance problem. That is, the number of positive samples (metastatic lymph nodes) in PET image data is far less than the number of negative samples (non-metastatic lymph nodes), which makes it very easy for the model to produce prediction bias towards the majority class during training.

[0085] To address the low accuracy of computer-aided transition probability prediction, this application provides a transition prediction method based on resampling and 3D attention convolution in some embodiments. This method addresses the dual problems of data imbalance and insufficient feature extraction faced by 3D PET image data. It balances the distribution of positive and negative samples through resampling technology, mitigating model bias caused by data imbalance. Furthermore, it employs a 3D attention convolution module to deeply mine spatial features, overcoming the limitations of insufficient 3D image feature extraction. By combining a weighted fusion loss function to optimize the model training process, prediction accuracy is effectively improved.

[0086] The method can be applied to electronic devices with data processing capabilities. These electronic devices include, but are not limited to, computers, servers, mobile terminals, smart wearable devices, and industrial control machines. For ease of description, this application embodiment uses an electronic device as the execution subject of the method. It should be understood that the method can also be applied to other types of execution subjects, which are not illustrated in this application embodiment. Figure 1 As shown, the method includes:

[0087] S101. Acquire positron emission tomography (PET) image data.

[0088] The PET image data includes multiple PET images. The electronic device can acquire PET image data through real-time image acquisition or from a specific database. For example, the electronic device selects the following three datasets, and the statistical information is shown in Table 1 below:

[0089] Table 1. Statistical information on the image data used in this scheme;

[0090] Dataset <![CDATA[N 1-3 (n)]]> <![CDATA[N0(n)]]> Data partitioning Lung-PET-CT-Dx 44 80 Training set: 74; Test set: 50 NSCLC Radiogenomics 31 122 Training set: 91; Test set: 62 ACRIN-NSCLC-FDG-PET 30 87 Training set: 70; Test set: 47

[0091] The NSCLC Radiogenomics study included data from 211 patients retrospectively identified from institutional databases at Stanford University School of Medicine and the Palo Alto Veterans Affairs Healthcare System. Inclusion criteria were based on the diagnosis of NSCLC, preoperative PET scans, and subsequent surgery. Cases with histological types other than small cell lung cancer, and those without N stage and PET scans, were excluded. Ultimately, 155 patients were included, of whom 122 were diagnosed with positive lymph node metastasis and identified as N. 1-3 31 cases were diagnosed without regional lymph node metastasis, denoted as N0.

[0092] Lung-PET-CT-Dx collected retrospective data from 355 NSCLC patients who underwent standard lung biopsy and PET. Similar to NSCLC Radiogenomics, the data only included non-small cell lung cancer (NSCLC) cases, with PET images corresponding to 80 N0 and 24 N1 cases. 1-3 Case.

[0093] ACRIN 6668 / RTOG 0235 is a multicenter prospective study funded by the National Cancer Institute, which included PET images of 242 patients aged 18 years and older with AJCC standard clinical stage IIB / III non-small cell lung cancer before and after chemoradiotherapy. After detailed inclusion and exclusion criteria were established, 117 patients were screened, including 30 N0 patients and 87 N20 patients. 1-3 patient.

[0094] like Figure 2 As shown, in some embodiments, after acquiring PET image data, the electronic device can clean the acquired PET image data. For example, data cleaning can address issues such as whether the PET image data contains lung images, whether there are missing data, and whether it contains lymph node metastasis.

[0095] S102. Resample the PET image to obtain balanced image data.

[0096] like Figure 3 As shown, after acquiring PET image data, the electronic device can perform standardized preprocessing on the PET image data. Specifically, the electronic device can load and read metadata, such as the subject ID, image path, and label from the PET image data. Based on the read metadata, the electronic device can perform interpolation, spatial alignment, region segmentation (lung segmentation), artifact removal, and region cropping (lung region cropping). After the above preprocessing, a standardized result can be obtained, and the preprocessed image data can be saved and output in a standardized image format, such as ".nrrd" format.

[0097] For standardized preprocessed PET image data, electronic devices can use resampling techniques to address class imbalance in the dataset. Specifically, PET images are resampled to obtain balanced image data, thereby balancing the distribution of positive and negative samples and mitigating model bias caused by data imbalance. Therefore, the balanced image data includes positive and negative samples. The positive samples are transfer target images obtained by oversampling PET images; the negative samples are non-transfer target images obtained by undersampling PET images.

[0098] In some embodiments, after the electronic device performs resampling on the PET images to obtain balanced image data, it can obtain the original data structure information by traversing the PET images in the PET image data. The basic data structure information is used to characterize the ratio of the number of original positive samples to the number of original negative samples in the PET image data.

[0099] For example, to avoid excessively large differences in the distribution of positive and negative samples, which could lead to model overfitting, and to address the problem of an imbalanced ratio of positive and negative samples in the training data, such as a significant increase in the number of negative samples compared to positive samples, electronic devices can first acquire basic data structure information. That is, by traversing the PET images in the PET image data, the number of positive samples N can be obtained. neg and the number of negative samples N pos And by calculating the number of positive samples N neg and the number of negative samples N pos The ratio of these values ​​yields the basic data structure information.

[0100] After obtaining the original data structure information, the electronic device calculates the resampling factor based on the original data structure information, and then performs resampling on the PET image based on the resampling factor. The resampling factor includes an oversampling factor and / or an undersampling factor; the resampling factor is positively correlated with the integer part of the quantity ratio.

[0101] For example, when the number of negative samples is much greater than the number of positive samples, the electronic device can calculate the maximum oversampling factor k. max Oversampling factor k max The number of positive and negative samples in the original sample is determined by the following formula:

[0102]

[0103] Where, N neg N represents the number of negative samples; pos Represents the number of positive samples; trunc[·] represents the truncation function for the integer part, indicating k. max Take (N) neg / N pos The integer part of ) ensures that the number of positive samples after oversampling does not exceed the number of original negative samples.

[0104] Determine the maximum oversampling factor k max Subsequently, the electronic device can oversample based on the maximum oversampling factor to increase the number of positive samples in the PET image data, making the ratio of positive to negative samples in the training set close to 1:1, and avoiding the model being biased towards the majority class (negative samples).

[0105] Similarly, when the number of positive samples is much greater than the number of negative samples, the electronic device can calculate the maximum undersampling factor. The calculation process is the same as the calculation process of the maximum oversampling factor mentioned above, and will not be repeated here.

[0106] As can be seen, electronic devices can employ different resampling methods depending on the ratio of positive to negative samples in PET image data. For PET image data with a positive-to-negative sample ratio close to 1:1, the electronic device can skip resampling and directly perform subsequent calculations. Therefore, in some embodiments, the electronic device can extract the ratio of the original positive to the original negative samples from the original data structure information and compare this ratio with a preset ratio range. If the ratio is within the preset ratio range, it indicates that the difference in the number of positive and negative samples is small, and therefore the PET image data can be directly marked as the balanced image data. If the ratio is not within the preset ratio range, it indicates that the difference in the number of positive and negative samples is large, and in this case, resampling can be performed on the PET image to obtain balanced image data.

[0107] In some embodiments, when an electronic device performs resampling on the PET image based on the resampling factor, it may first calculate the number of resampling operations based on the resampling factor, and then perform resampling on the PET image according to the number of resampling operations to obtain generated sample data.

[0108] The generated sample data includes generated positive samples obtained by oversampling the original positive samples, and / or generated negative samples obtained by undersampling the original negative samples. For example, when the number of negative samples is much greater than the number of positive samples, the positive samples can be repeated k times. max Next, a new number of positive samples is generated, that is:

[0109] N pos,new =N pos ×k max ;

[0110] Where, N pos,new This represents the positive sample data after resampling preprocessing; N pos k represents the number of positive samples. max This indicates the maximum oversampling factor.

[0111] Similarly, when undersampling negative samples, N samples can be randomly drawn from the original negative samples. pos,new N samples are used to ensure that the number of positive and negative samples is equal. neg,new =N pos,new .

[0112] After obtaining the generated sample data, the electronic device can inspect the resampled PET image data to determine whether the ratio of positive to negative samples in the resampled image data is reasonable. Therefore, in some embodiments, the electronic device can obtain the original sample count in the PET image data, that is, obtain the original positive sample count and the original negative sample count. Then, the generated sample count is calculated, which is equal to the product of the original sample count and the resampling factor; the generated sample count includes the number of positive samples after oversampling the original positive samples, and / or the number of negative samples after undersampling the original negative samples.

[0113] Next, the positive and negative sample difference is calculated based on the number of positive and negative samples. This difference is then compared to a preset deviation threshold. If the positive and negative sample difference is greater than the preset threshold, the resampling count is reset based on this difference, and resampling is performed again according to the method described in the above embodiment until the positive and negative sample difference is less than or equal to the preset deviation threshold. When the positive and negative sample difference is less than or equal to the preset deviation threshold, balanced image data can be output based on the generated sample data and PET image data.

[0114] As can be seen, through the preprocessing and resampling processes described in the above embodiments, the electronic device can standardize the preprocessed PET image data and use resampling to handle the class imbalance of the dataset, so as to balance the distribution of positive and negative samples through resampling technology and alleviate the model bias caused by data imbalance.

[0115] S103. Extract basic image features from the balanced image data using a three-dimensional residual convolutional neural network.

[0116] After obtaining balanced image data, the electronic device can extract basic image features from the balanced image data using a three-dimensional residual convolutional neural network. These basic image features include local features in the balanced image data that are case-discriminatory. The electronic device can construct a three-dimensional residual convolutional neural network module, which can employ a three-dimensional attention convolutional network to deeply mine spatial features, thereby achieving basic feature extraction from PET images and fully extracting 3D image features.

[0117] To extract fundamental image features, in some embodiments, when an electronic device extracts fundamental image features from balanced image data using a three-dimensional residual convolutional neural network, it can use a feature extraction network to extract input features from the balanced image data, and then input the input features into the three-dimensional residual convolutional neural network. The three-dimensional residual convolutional neural network includes a first branch path and a second branch path. Therefore, the electronic device can generate a first feature map through the first branch path, which is a feature map generated by performing a three-dimensional convolution operation on the input features through the first branch path, normalizing the convolution result, and then applying a non-linear activation function. And it can generate a second feature map through the second branch path, which is a feature map generated by performing a three-dimensional convolution operation on the input features through the second branch path and then normalizing the convolution result.

[0118] For example, such as Figure 4 As shown, based on the class-balanced PET image data X obtained after resampling preprocessing, let the input feature be x. Electronic devices can extract local spatial features of PET images by constructing a three-dimensional residual convolutional neural network module. During feature extraction, the three-dimensional residual convolutional neural network module can generate a first feature map z1 through the first branch path, i.e.:

[0119] z1 = ReLU(BN1(3DConv1(x)));

[0120] In generating the first feature map, the first branch path can first perform a 3D convolution operation on the input feature x based on 3D convolution (3DConv1) to extract local features in the spatial-temporal (or depth) dimensions. The output size of the 3D convolution can be dynamically adjusted according to the kernel size and stride. After the 3D convolution operation, the first branch path can perform 3D batch normalization (BN1), that is, normalize the convolution result to reduce internal covariate bias and stabilize the training process. Then, by introducing non-linear activation functions such as ReLU, positive features are preserved, i.e., ReLU(a) = max(0, a). Through the 3D convolution operation, normalization operation, and activation function, the first branch path can output the first feature map z1. It can be seen that the first feature map z1 is a feature map after non-linear activation, containing the initially extracted local features.

[0121] Similarly, when performing feature extraction, the 3D residual convolutional neural network module can generate a second feature map z2 through the second branch path, that is:

[0122] z2 = BN2(3DConv2(x));

[0123] The second branch path can first perform a 3D convolution (3DConv2) operation based on the operation sequence z2 = BN2(3DConv2(x)). Since the second branch path is independent of the first branch path, different convolution kernels can be used to further extract features during the 3D convolution operation. After obtaining the three-dimensional convolution result through the 3D convolution operation, the second branch path can perform 3D batch normalization (BN2), i.e., normalize the convolution output. Unlike the first branch path, the second branch path does not use an activation function during the operation. Therefore, the second feature map z2 output by the second branch path is a linearly transformed feature map, preserving the original numerical range, and is thus used for residual connections.

[0124] After obtaining the first and second feature maps through the above calculation method, the electronic device can generate the basic image features based on the first and second feature maps. In some embodiments, to generate the basic image features, the three-dimensional residual convolutional neural network further includes a downsampling path and a feature fusion path. When generating the basic image features based on the first and second feature maps, the electronic device can first use the downsampling path to downsample the input features to obtain a downsampling result. Then, the second feature map and the downsampling result are added element-wise through the feature fusion path to obtain a first summed result. A nonlinear activation function is then used to perform nonlinear activation on the first summed result to generate a third feature map, where the third feature map is a feature map that fuses multi-path information. Thus, by outputting the first and third feature maps, the basic image features are obtained.

[0125] For example, after obtaining the first feature map z1 and the second feature map z2 through the processing of the first branch path and the second branch path, the electronic device can call the downsampling path to downsample the input feature x to generate the downsampling result x′, that is:

[0126] x′=DownSample(x);

[0127] For the operation sequence x′ = DownSample(x), the electronic device can utilize a 1×1×1 3D convolution kernel to achieve downsampling (if the stride is >1) by adjusting the number of channels or spatial size, ensuring that the size downsampled result x′ matches the size of the second feature map z2. The output is then the downsampled result x′, which is a downsampled version of the input x, used for subsequent residual addition.

[0128] Residual summation is achieved by element-wise adding the output z2 of the second branch to the downsampled input x′ to obtain the first summation result (z2+x′). By adding the output z2 of the second branch to the downsampled input x′ element-wise, original information can be preserved (alleviating gradient vanishing) and feature representation ability can be enhanced. Then, a nonlinear activation is applied to the summation result to obtain the final output, i.e., the third feature map z3, which is:

[0129] z3 = ReLU(z2 + x');

[0130] Here, ReLU represents the activation function, z represents the intermediate feature map output, and x' represents the downsampling result, i.e., the output after downsampling. The third feature map z3 is a feature map that can fuse multi-path information, possessing both local features and global structure.

[0131] S104. Based on the three-dimensional convolutional attention mechanism, extract key image features from the balanced image data.

[0132] After extracting basic image features from balanced image data using a 3D residual convolutional neural network, electronic devices can extract key image features from the balanced image data based on a 3D convolutional attention mechanism. The key image features include local information with transfer correlation in the balanced image data. The key image features are feature maps output by modeling the importance of each channel based on a channel attention mechanism and then weighting the spatial position of the feature map using a spatial attention mechanism.

[0133] Electronic devices can enhance the extraction of key features from PET images along both spatial and channel dimensions by introducing a 3D CBAM attention mechanism. Therefore, in some embodiments, when the electronic device extracts key image features from the balanced image data based on the 3D convolutional attention mechanism, it can first invoke the 3D convolutional attention module and input the third feature map into the 3D convolutional attention module. The 3D convolutional attention module includes a channel attention unit and a spatial attention unit.

[0134] After the third feature map is input into the 3D convolutional attention module, the electronic device can model the importance of each channel corresponding to the third feature map through the channel attention unit to obtain modeling data, and then weight the spatial position of the third feature map through the spatial attention unit to obtain a weighted result. The fused key features are then obtained by fusing the modeling data and the weighted result. Next, the second summation result is calculated, which is the sum obtained by adding the fused key features to the downsampling result element-wise. A non-linear activation function is then applied to the second summation result to generate the image key features.

[0135] For example, based on the basic image features obtained from the above steps, electronic devices can introduce a 3D CBAM attention mechanism to enhance the extraction of key features from PET images along both spatial and channel dimensions, namely:

[0136] z′3=(3DCBAM(z3));

[0137] y = ReLU(z′3 + x′);

[0138] Here, y is the predicted output of the final model. 3DCBAM(·) represents the three-dimensional convolutional attention mechanism operation function. The module running this function is called the three-dimensional convolutional attention module, which can include channel attention and spatial attention. Therefore, electronic devices can first model the importance of each channel using the channel attention mechanism, and then weight the spatial location of the feature map using the spatial attention mechanism. The weighted feature map output by the three-dimensional convolutional attention module can significantly enhance information relevant to the classification task while suppressing redundant or irrelevant features.

[0139] As can be seen, compared with basic 3D convolution, the introduction of the attention mechanism can more effectively extract and enhance key features, further improving the model's expressive power and discriminative performance while maintaining the advantages of the original residual structure. Using the 3D residual attention module, full extraction of 3D spatial feature information can be achieved. It should be noted that electronic devices can also employ encoder-decoder, generative adversarial networks, and other methods for feature extraction, and combine them with ensemble learning, meta-learning, and other methods for optimizing model performance to adapt to different analytical needs.

[0140] S105. Train a prediction model based on the basic features and key features of the image.

[0141] After acquiring the basic and key features of the image, the electronic device can train a model based on these features, obtaining a prediction model with a certain output accuracy through iterative training. The prediction model is a neural network model using a weighted combination of binary cross-entropy loss and F1 loss as its loss function.

[0142] During model training, the electronic device can optimize the model using a weighted fusion loss of binary cross-entropy (BCE) and F1 loss. Therefore, in some embodiments, when the electronic device trains a prediction model based on the image's basic features and key features, it can first obtain the weighting parameters and set the loss function of the trained model according to the weighting parameters.

[0143] The weighting parameter controls the ratio of the binary cross-entropy loss (BCE) to the F1 loss. For example, the weighting parameter β is a value between [0, 1], used to control the ratio of BCE loss to F1 loss. Therefore, the loss function determined by the weighting parameter β is a weighted combination of the binary cross-entropy loss (BCE) and the F1 loss, i.e.:

[0144] L=β·L BCE +(1-β)·L F1 ;

[0145] In the formula, β is the weighting parameter, and L BCE It is a binary cross-entropy loss, L F1 It's an F1 loss.

[0146] like Figure 5 As shown, after setting the loss function, the electronic device can input the basic features and key features of the image into the trained model to obtain the training prediction result output by the trained model; then, based on the loss function, the training loss is calculated according to the training label of the PET image and the training prediction result. The training loss is then compared with a preset loss threshold. If the training loss is greater than the loss threshold, the model parameters of the trained model are adjusted according to the training loss, and iterative training continues based on the adjusted trained model until the training loss is less than or equal to the loss threshold. When the training loss is less than or equal to the loss threshold, it indicates that the model has converged, and therefore the model parameters of the trained model can be output to obtain the prediction model.

[0147] It is evident that using weighted fusion loss to optimize the prediction model and selecting BCE and F1 weighted loss can balance the advantages of BCE and F1. Specifically, it leverages the characteristics of BCE loss, which is good at optimizing probability distribution and sensitive to class imbalance, and the characteristics of F1 loss, which directly optimizes evaluation metrics and is more in line with the goals of classification tasks, thus achieving better model optimization.

[0148] In some embodiments, the electronic device may also employ a variety of evaluation metrics to comprehensively evaluate the model performance, including accuracy (Acc), area under the curve (Auc), sensitivity, specificity, precision, recall, F1 score, and geometric mean (G-mean).

[0149] Accuracy is used to measure the proportion of samples that the model correctly predicts out of the total samples, i.e.:

[0150]

[0151] In the formula, TP (True Positive) represents the number of positive classes correctly predicted as positive; TN (True Negative) represents the number of negative classes correctly predicted as negative; FP (False Positive) represents the number of negative classes incorrectly predicted as positive; and FN (False Negative) represents the number of positive classes incorrectly predicted as negative.

[0152] The area under the curve (Auc) is the area under the ROC curve (Receiving Operator Characteristic curve), used to represent the model's ability to distinguish between positive and negative samples. The value ranges from [0, 1], and the larger the value, the better the model performance.

[0153] Sensitivity, also known as recall or true positive rate, measures the proportion of positive samples that are correctly identified as positive, in order to evaluate the model's ability to identify positive samples.

[0154]

[0155] Specificity, or True Negative Rate, measures the proportion of negative samples that are correctly identified as negative, in order to evaluate the model's ability to identify negative samples.

[0156]

[0157] Precision measures the proportion of samples predicted as positive that are actually positive. It is used to evaluate the percentage of true positives in the model's predictions.

[0158]

[0159] The F1 score is the harmonic mean of precision and recall, used to comprehensively measure the model's precision and recall, balancing their performance.

[0160]

[0161] A higher F1 score indicates a better overall performance in both "accurately identifying positive cases" and "fully covering positive cases." Therefore, it can be used in scenarios with class imbalance where both missed and false positives need to be considered, such as medical diagnosis and information retrieval.

[0162] The Geometric Mean (Gmean) is an indicator that considers both Sensitivity and Specificity. It is used to comprehensively evaluate a model's ability to distinguish between positive and negative samples in imbalanced class scenarios.

[0163]

[0164] Gmeans balances the model’s performance on positive and negative samples by taking the geometric mean of sensitivity and specificity, thus avoiding the inability of a single metric to reflect the model’s true performance due to class imbalance.

[0165] S106. Use the prediction model to output the transition probability.

[0166] After obtaining a prediction model through training, electronic devices can predict lymph node metastasis based on this model, that is, use the prediction model to output the metastasis probability. For example... Figure 6 As shown, in some embodiments, when an electronic device uses the transition probability output by the prediction model, it can first acquire a PET image to be predicted, then extract basic image features from the PET image using a three-dimensional residual convolutional neural network, and extract key image features from the PET image based on a three-dimensional convolutional attention mechanism. The basic image features and the key image features are then input into the prediction model to obtain the transition probability output by the prediction model.

[0167] By applying the technical solutions of the above embodiments, the transfer prediction method based on resampling and 3D attention convolution provided in the above embodiments can predict the lymph node metastasis of lung cancer in 3D PET images, thereby improving prediction performance. To address the problem of highly imbalanced image data, the method can oversample positive samples and undersample negative samples using resampling technology. To address the challenge of insufficient 3D feature extraction, a 3D convolutional residual network and attention mechanism are proposed to extract basic and key features of PET images. A weighted loss function is then used to optimize the model, improving its prediction performance.

[0168] The transfer prediction method based on resampling and 3D attention convolution was tested on three datasets: Lung-PET-CT-Dx, NSCLCRadiogenomics, and ACRIN-NSCLC-FDG-PET. The experimental results are shown in Table 2 below.

[0169] Table 2. Evaluation metrics results on different PET datasets;

[0170]

[0171] Based on the above experimental results, the metastasis prediction method based on resampling and 3D attention convolution demonstrates performance advantages in predicting lung cancer lymph node metastasis in 3D PET images. Specifically, through resampling and balancing samples, feature extraction using the 3D attention convolution module, and weighted fusion loss optimization training, this scheme achieves good performance in terms of accuracy (Acc), area under the curve (Auc), and specificity, and can accurately predict lung cancer lymph node metastasis.

[0172] In some embodiments, as a specific implementation of the transfer prediction method based on resampling and 3D attention convolution described in the above embodiments, some embodiments of this application also provide a transfer prediction system based on resampling and 3D attention convolution, such as... Figure 7 As shown, the system includes:

[0173] The data acquisition module is used to acquire positron emission tomography (PET) image data, which includes multiple PET images.

[0174] A resampling module is used to perform resampling on the PET image to obtain balanced image data, the balanced image data including positive samples and negative samples, the positive samples being transfer target images obtained by oversampling the PET image; the negative samples being non-transfer target images obtained by undersampling the PET image;

[0175] The basic feature extraction module is used to extract basic image features from the balanced image data through a three-dimensional residual convolutional neural network. The basic image features include local features in the balanced image data that have case-discriminatory properties.

[0176] The key feature extraction module is used to extract key image features from the balanced image data based on the three-dimensional convolutional attention mechanism. The key image features include local information with transfer correlation in the balanced image data. The key image features are the feature maps output by modeling the importance of each channel based on the channel attention mechanism and weighting the spatial position of the feature maps through the spatial attention mechanism.

[0177] The model training module is used to train a prediction model based on the basic features and key features of the image. The prediction model is a neural network model with a weighted combination of binary cross-entropy loss and F1 loss as the loss function.

[0178] A probabilistic prediction model is used to output transition probabilities using the prediction model.

[0179] By applying the technical solutions of the above embodiments, this application provides a lymph node metastasis prediction system based on resampling and 3D attention convolution. After the data acquisition module acquires PET image data, the resampling module resamples the PET images in the PET image data to obtain balanced image data. Then, the basic feature extraction module extracts key image features from the balanced image data using a 3D residual convolutional neural network, and the key feature extraction module extracts key image features based on a 3D convolutional attention mechanism. The model training module trains a prediction model using the basic image features and key image features, enabling the probability prediction model to output the metastasis probability. The system can balance the distribution of positive and negative samples through resampling, mitigating model bias caused by the high imbalance between positive and negative samples in 3D PET image data. By employing a 3D attention convolutional network to deeply mine spatial features and fully extract the 3D spatial features of 3D PET, and then combining this with a weighted fusion loss function to optimize the model training process, the prediction accuracy of lymph node metastasis probability is improved.

[0180] It should be noted that other corresponding descriptions of the functional units involved in the transfer prediction system based on resampling and three-dimensional attention convolution provided in the embodiments of this application can be found in the corresponding descriptions in the transfer prediction method based on resampling and three-dimensional attention convolution provided in the above embodiments, and will not be repeated here.

[0181] This application also provides a computer device, specifically a personal computer, server, network device, etc. The computer device includes a bus, processor, memory, and communication interface, and may also include input / output interfaces and a display device. The processor of the computer device provides computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database of the computer device stores location information. The network interface of the computer device is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements the steps in the various method embodiments.

[0182] Those skilled in the art will understand that the structure of the computer device described above is only a partial structure related to the solution of this application, and does not constitute a limitation on the computer device to which the solution of this application is applied. A specific computer device may include more or fewer components, or combine certain components, or have different component arrangements.

[0183] In one embodiment, a computer-readable storage medium is also provided, which may be non-volatile or volatile, and a computer program is stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.

[0184] In one embodiment, a computer program product is also provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0185] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0186] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods.

[0187] Any references to memory, database, or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc.

[0188] Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can take many forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0189] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchain. The processors involved in the embodiments provided in this application may be, but are not limited to, general-purpose processors, graphics processors, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc.

[0190] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0191] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A transition prediction method based on resampling and 3D attention convolution, characterized in that, The method includes: Acquire positron emission tomography (PET) image data, wherein the PET image data includes multiple PET images; The PET images are resampled to obtain balanced image data, which includes positive samples and negative samples. The positive samples are transfer target images obtained by oversampling the PET images, and the negative samples are non-transfer target images obtained by undersampling the PET images. A three-dimensional residual convolutional neural network (3D Residual Convolutional Neural Network) is used to extract basic image features from the balanced image data. These basic image features include local features in the balanced image data that are case-discriminatory. The 3D Residual Convolutional Neural Network further includes a downsampling path and a feature fusion path. The extraction of basic image features from the balanced image data using the 3D Residual Convolutional Neural Network includes: extracting input features from the balanced image data using a feature extraction network; inputting the input features into the 3D Residual Convolutional Neural Network, which includes a first branch path and a second branch path; generating a first feature map through the first branch path; generating a second feature map through the second branch path; generating the basic image features based on the first and second feature maps, including: performing downsampling on the input features using the downsampling path to obtain a downsampling result; adding the second feature map and the downsampling result element-wise through the feature fusion path to obtain a first summation result; performing nonlinear activation on the first summation result using a nonlinear activation function to generate a third feature map; and outputting the first and third feature maps to obtain the basic image features. Based on a 3D convolutional attention mechanism, key image features are extracted from the balanced image data. These key features include local information with transfer correlation within the balanced image data. The extraction process includes: invoking a 3D convolutional attention module, which includes channel attention units and spatial attention units; inputting the third feature map into the 3D convolutional attention module; modeling the importance of each channel corresponding to the third feature map using the channel attention units to obtain modeling data; weighting the spatial position of the third feature map using the spatial attention unit to obtain a weighted result; fusing the modeling data and the weighted result to obtain fused key features; calculating a second summation result, which is the sum of the fused key features and the downsampling result element-wise; and applying a nonlinear activation function to the second summation result to generate the key image features. A prediction model is trained based on the basic features and key features of the image. The prediction model is a neural network model with a weighted combination of binary cross-entropy loss and F1 loss as the loss function. The predicted model is used to output the transition probability.

2. The method according to claim 1, characterized in that, Resampling is performed on the PET images to obtain balanced image data, including: By traversing the PET images in the PET image data, the original data structure information is obtained. The basic data structure information is used to characterize the ratio of the number of original positive samples to the number of original negative samples in the PET image data. Calculate the resampling factor based on the original data structure information; The PET image is resampled based on the resampling factor.

3. The method according to claim 2, characterized in that, Based on the resampling factor, resampling is performed on the PET image, including: The number of resampling times is calculated based on the resampling factor; The PET images are resampled according to the number of resampling operations to obtain generated sample data, which includes generated positive samples obtained by oversampling the original positive samples and / or generated negative samples obtained by undersampling the original negative samples. Obtain the number of original samples in the PET image data, wherein the number of original samples includes the number of original positive samples and the number of original negative samples; Calculate the number of generated samples, which is equal to the product of the original number of samples and the resampling factor; the number of generated samples includes the number of positive samples after oversampling the original positive samples, and / or the number of negative samples after undersampling the original negative samples; The positive-negative sample difference is calculated based on the number of positive samples and the number of negative samples, and the positive-negative sample difference is equal to the difference between the number of positive samples and the number of negative samples; If the positive and negative sample difference is greater than a preset deviation threshold, the number of resampling times is reset based on the positive and negative sample difference; If the positive and negative sample difference is less than or equal to a preset deviation threshold, the balanced image data is output based on the generated sample data and the PET image data.

4. The method according to claim 2, characterized in that, The method further includes: Extract the ratio of the number of original positive samples to the number of original negative samples from the original data structure information; If the quantity ratio is within a preset quantity ratio range, the PET image data is marked as the balanced image data; If the quantity ratio is not within the preset quantity ratio range, the PET image is resampled to obtain balanced image data.

5. The method according to claim 1, characterized in that, Training a prediction model based on the image's basic features and key features includes: Obtain the weighting parameters, which are used to control the ratio of binary cross-entropy loss to F1 loss; The loss function of the trained model is set according to the weighting parameters; The basic features and key features of the image are input into the trained model to obtain the training prediction results output by the trained model. Based on the loss function, the training loss is calculated according to the training labels of the PET images and the training prediction results; If the training loss is greater than the loss threshold, the model parameters of the trained model are adjusted according to the training loss. If the training loss is less than or equal to the loss threshold, the model parameters of the trained model are output to obtain the prediction model.

6. The method according to claim 1, characterized in that, The prediction model is used to output transition probabilities, including: Acquire PET images to be predicted; The basic features of the PET image to be predicted are extracted using a three-dimensional residual convolutional neural network. Based on a three-dimensional convolutional attention mechanism, key image features are extracted from the PET image to be predicted. The image's basic features and key features are input into the prediction model to obtain the transition probability output by the prediction model.

7. The method according to claim 1, characterized in that, The first feature map is generated by the first branch path performing a three-dimensional convolution operation on the input features, normalizing the convolution result, and then using a non-linear activation function. The second feature map is the feature map generated by normalizing the convolution result after the second branch path performs a three-dimensional convolution operation on the input features; The third feature map is a feature map that integrates multi-path information; The image key features are output as feature maps by modeling the importance of each channel based on the channel attention mechanism and then weighting the spatial position of the feature maps using the spatial attention mechanism.

8. A transition prediction system based on resampling and 3D attention convolution, characterized in that, The system includes: The data acquisition module is used to acquire positron emission tomography (PET) image data, which includes multiple PET images. A resampling module is used to perform resampling on the PET image to obtain balanced image data, the balanced image data including positive samples and negative samples, the positive samples being transfer target images obtained by oversampling the PET image; the negative samples being non-transfer target images obtained by undersampling the PET image; A basic feature extraction module is used to extract basic image features from the balanced image data using a three-dimensional residual convolutional neural network. These basic image features include local features in the balanced image data that are case-discriminatory. The three-dimensional residual convolutional neural network further includes a downsampling path and a feature fusion path. Extracting basic image features from the balanced image data using the three-dimensional residual convolutional neural network includes: extracting input features from the balanced image data using a feature extraction network; inputting the input features into the three-dimensional residual convolutional neural network, which includes a first branch path and a second branch path; generating a first feature map through the first branch path; generating a second feature map through the second branch path; generating the basic image features based on the first and second feature maps, including: performing downsampling on the input features using the downsampling path to obtain a downsampling result; adding the second feature map and the downsampling result element-wise through the feature fusion path to obtain a first summation result; performing nonlinear activation on the first summation result using a nonlinear activation function to generate a third feature map; and outputting the first and third feature maps to obtain the basic image features. A key feature extraction module is used to extract key image features from the balanced image data based on a three-dimensional convolutional attention mechanism. The key image features include local information with transfer correlation in the balanced image data. The extraction of key image features from the balanced image data based on the three-dimensional convolutional attention mechanism includes: calling a three-dimensional convolutional attention module, which includes channel attention units and spatial attention units; inputting the third feature map into the three-dimensional convolutional attention module; modeling the importance of each channel corresponding to the third feature map through the channel attention units to obtain modeling data; weighting the spatial position of the third feature map through the spatial attention unit to obtain a weighted result; fusing the modeling data and the weighted result to obtain fused key features; calculating a second summation result, which is the sum obtained by adding the fused key features and the downsampling result element-wise; and performing nonlinear activation on the second summation result using a nonlinear activation function to generate the key image features. The model training module is used to train a prediction model based on the basic features and key features of the image. The prediction model is a neural network model with a weighted combination of binary cross-entropy loss and F1 loss as the loss function. A probabilistic prediction model is used to output transition probabilities using the prediction model.

Images