Thermographic-based sample difficulty assessment and adaptive data augmentation method

Abstract

The application discloses a sample difficulty evaluation and adaptive data enhancement method based on a thermogram, and comprises the following steps: pre-processing the thermogram to obtain a normalized image, a breast segmentation result and a blood vessel segmentation result; calculating five-dimensional difficulty characteristic values of temperature gradient complexity, thermal distribution uniformity, blood vessel heat conduction characteristics, edge thermal change rate and left-right breast asymmetry; calculating adaptive initial weights based on information entropy and reliability coefficients, adopting a multi-level fusion strategy to obtain a comprehensive difficulty score and an attention weight distribution; calculating an up-sampling multiple based on the difficulty characteristic values, the comprehensive difficulty score and the attention weight distribution, and determining an enhancement strategy combination to perform adaptive data enhancement; and training a classification model adopting a cross-modal attention fusion mechanism by using the enhanced training sample set. The application relieves the sample imbalance problem through a difficulty-driven data enhancement strategy, and significantly improves the accuracy, robustness and generalization ability of thermogram benign and malignant classification.

Claims

1. A thermal image-based sample difficulty assessment and adaptive data augmentation method, characterized in that: Includes the following steps: Step S1: Preprocess the thermal image to obtain a normalized image, breast segmentation results, and blood vessel segmentation results; Step S2: Based on the normalized image, the breast segmentation result, and the blood vessel segmentation result, calculate the five-dimensional difficulty feature value, which includes temperature gradient complexity, heat distribution uniformity, blood vessel heat conduction characteristics, edge heat change rate, and left and right breast asymmetry. Step S3: Calculate adaptive initial weights based on information entropy and reliability coefficients, and use a multi-level fusion strategy to fuse the five-dimensional difficulty feature values ​​to obtain a comprehensive difficulty score and attention weight distribution; Step S4: Based on the five-dimensional difficulty feature values, the comprehensive difficulty score, and the attention weight distribution, calculate the upsampling factor and determine the combination of enhancement strategies, perform adaptive data augmentation, and obtain the enhanced training sample set; Step S5: Train the classification model using the enhanced training sample set to obtain a classification model for classifying benign and malignant thermal images; The method for calculating the five-dimensional difficulty feature value in step S2 includes: The temperature gradient complexity is obtained by calculating the first-order temperature gradients of the normalized image in the horizontal and vertical directions, synthesizing the gradient magnitude using the Euclidean norm, calculating the coefficient of variation of the gradient magnitude of the entire image and normalizing it to the [0,1] interval. The uniformity of heat distribution is obtained by dividing the normalized image into a regular grid, calculating the coefficient of variation of the average temperature in each grid region, and normalizing it to the [0,1] interval; The vascular thermal conduction characteristics are obtained by fusing three indicators: vascular distribution density, vascular orientation stability, and temperature difference between the vascular region and adjacent tissues, and normalizing them to the [0,1] interval. The edge thermal change rate is obtained by locating the pathological boundary region through edge detection, fusing the coefficient of variation of the edge pixel gradient magnitude and the standard deviation of the gradient direction and normalizing them to the [0,1] interval; The asymmetry of the left and right breasts is obtained by extracting the mean temperature, extreme range and standard deviation of the left and right sides respectively based on the breast segmentation results, calculating the normalized difference and merging it into the [0,1] interval; In step S3, the formula for calculating the adaptive initial weights is: ； in, Indicates the first One dimension, and The values ​​are all from 1 to 5, corresponding to the temperature gradient complexity, the heat distribution uniformity, the vascular heat conduction characteristics, the edge heat change rate, and the left and right breast asymmetry, respectively. For the first The information entropy of each dimension is calculated using the binary entropy formula: ； in, For the first Difficulty feature values ​​in each dimension; For the first The reliability coefficients for each dimension are calculated using an exponential decay function: ； in, For the first The standard deviation of each dimension feature value in historical data This is the attenuation coefficient.

2. The method according to claim 1, characterized in that: Step S1 includes: The Gaussian filtering algorithm is used to denoise the thermal image; The temperature values ​​are normalized to the [0,1] interval to obtain the normalized image; The left and right breast regions are automatically segmented based on the UNet segmentation network to obtain the breast segmentation results. The blood vessel segmentation result is obtained by using a blood vessel probability map output by the UNet segmentation network and thresholding processing.

3. The method according to claim 1, characterized in that: The reliability coefficient is calculated using a dynamic update mechanism based on historical data. After processing each batch of training samples, the five-dimensional difficulty feature values ​​of the training samples and their corresponding benign or malignant labels are added to the historical dataset. When the number of samples in the historical dataset is less than a preset threshold, the reliability coefficient adopts a preset default value; When the number of samples in the historical data set is greater than or equal to the preset threshold, the reliability coefficient is calculated based on the standard deviation of the feature values ​​of each dimension in the historical data set.

4. The method according to claim 1, characterized in that: In step S3, the multi-level fusion strategy includes: First-level self-attention mechanism fusion: using the five-dimensional difficulty feature value as the query vector, the element-wise product of the five-dimensional difficulty feature value and the adaptive initial weight as the key vector, and the five-dimensional difficulty feature value as the value vector, the attention weight distribution is calculated by scaling dot product attention; The second-level adaptive fusion mechanism calculates a preliminary difficulty estimate, which is a weighted sum of the five-dimensional difficulty feature values ​​and the adaptive initial weights. When the preliminary difficulty estimate is less than the first threshold, linear weighted fusion is used; When the initial difficulty estimate is between the first threshold and the second threshold, the adaptive initial weights and the attention weight distribution are weighted and fused according to a first ratio; When the initial difficulty estimate is greater than the second threshold, the adaptive initial weights and the attention weight distribution are weighted and fused according to the second ratio and a sigmoid nonlinear transformation is applied. Wherein, the proportion of the adaptive initial weight in the first ratio is greater than the proportion of the attention weight distribution, and the proportion of the attention weight distribution in the second ratio is greater than the proportion of the adaptive initial weight; The third level outputs the attention weight distribution and the overall difficulty score.

5. The method according to claim 1, characterized in that: In step S4, the calculation of the upsampling factor includes: Calculate the upsampling evaluation coefficient : ； in, , For the first Difficulty values ​​in each dimension, To integrate weights, The first in the attention weight distribution Weights of each dimension For the overall difficulty score, For class imbalance adjustment factor, This is the weighting coefficient for the overall difficulty score. Difficulty weighting coefficient and Greater than 0.5; Calculate the upsampling factor : ； in, Base multiple, Scaling factor This means limiting the value between the lower and upper limits. and This indicates the lower and upper limits of the upsampling factor.

6. The method according to claim 1, characterized in that: In step S4, the method for determining the combination of enhancement strategies includes: When the temperature gradient complexity exceeds a set threshold, the proportion of geometric transformations is increased, including rotation, flipping, and scaling. When the value of the heat distribution uniformity is greater than a set threshold, the proportion of color transformation is increased, and the color transformation includes brightness adjustment, contrast adjustment and hue adjustment; When the value of the vascular thermal conduction characteristic is greater than a set threshold, the ratio of noise processing and blurring processing is increased; When the value of the edge thermal change rate is greater than a set threshold, the edge enhancement ratio is increased; When the value of the left and right breast asymmetry is greater than a set threshold, the mirror transformation ratio is increased.

7. The method according to claim 1, characterized in that: In step S5, the classification model employs a cross-modal attention fusion mechanism, including: The feature map is extracted from the thermal image through a convolutional neural network, and the feature map and the five-dimensional difficulty feature value are projected onto a unified dimensional space. Execute bidirectional attention flow: the first direction uses the feature map projection as the query vector and the five-dimensional difficulty feature value projection as the key vector and value vector; the second direction uses the five-dimensional difficulty feature value projection as the query vector and the feature map projection as the key vector and value vector. The attention weights for the two directions are calculated by scaling the dot product attention, and then weighted and aggregated to obtain two attention outputs.

8. The method according to claim 7, characterized in that: The classification model also includes a gated fusion mechanism: The two attention outputs are concatenated; A gating value is calculated using the sigmoid activation function, and this gating value is used to dynamically adjust the contribution ratio of the feature map and the five-dimensional difficulty feature value. The gated and fused features are then used to output benign / malignant classification results through a multilayer perceptron.

Images