Diabetic foot image classification system and method based on small sample learning

Abstract

The embodiment of the invention provides a diabetic foot image classification system and method based on small sample learning. The diabetic foot image classification system based on small sample learning comprises a data enhancement module which is used for carrying out weighted fusion processing on a training image and a similar image on a diabetic foot image to generate an expanded data set; the feature extraction module is used for performing ResNet-101 backbone network feature extraction processing on the expanded data set generated by the data enhancement module to generate a feature tensor comprising global semantics and local details; the attention enhancement module is used for performing double-pool channel attention and enhancement space attention processing on the feature tensor generated by the feature extraction module to generate an attention enhancement feature map focusing on the lesion area; and the classification module is used for performing dynamic prototype aggregation processing on the attention enhancement feature map generated by the attention enhancement module so as to classify the diabetic foot image.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of image processing, and in particular to a diabetic foot image classification system and method based on small sample learning. Background Art

[0002] In the clinical diagnosis of diabetic foot, medical image classification is an important tool for assisting in lesion severity assessment. With the development of deep learning technology, image recognition-based classification models have shown promising application in diabetic foot diagnosis. However, practical applications face the significant challenge of scarce labeled data. Diabetic foot lesions are characterized by large individual differences and subtle early features. Acquiring a large number of high-quality labeled images requires significant clinical resources. Traditional deep learning models, relying on large amounts of labeled data, are prone to overfitting in small sample scenarios, resulting in a significant decrease in classification accuracy.

[0003] At present, the existing diabetic foot image classification schemes have the following major technical bottlenecks: First, traditional data augmentation methods (such as rotation and cropping) can only change the geometric properties of the image and are difficult to retain the semantic features of the lesion area, resulting in insufficient effectiveness of the expanded data; second, conventional feature extraction networks lack the ability to capture the multi-scale features of diabetic foot lesions, and are particularly difficult to take into account both the global lesion distribution and local subtle features (such as ulcer edge texture); third, the existing attention mechanism has limited ability to focus on the lesion area under complex backgrounds and is easily affected by noise such as skin texture and shooting environment; fourth, the classification model based on static prototypes cannot adapt to the dynamic changes of diabetic foot lesions and has weak generalization ability for new categories or early lesions.

[0004] These technical issues result in low classification accuracy for diabetic foot lesions in existing systems with small sample sizes, making it particularly difficult to meet the clinical needs of early lesion screening and disease stage segmentation. Improving the model's ability to represent and classify diabetic foot images with limited labeled data has become a pressing technical challenge in this field. Summary of the Invention

[0005] In view of this, an embodiment of the present invention provides a diabetic foot image classification system and method based on small sample learning to at least partially solve the above problems.

[0006] According to a first aspect of an embodiment of the present invention, a diabetic foot image classification system based on small sample learning is provided, comprising:

[0007] The data enhancement module is used to perform weighted fusion of training images and similar images on diabetic foot images to generate an expanded data set;

[0008] The feature extraction module is used to perform ResNet-101 backbone network feature extraction on the expanded dataset generated by the data enhancement module to generate a feature tensor containing global semantics and local details;

[0009] The attention enhancement module is used to perform dual-pool channel attention and enhanced spatial attention processing on the feature tensor generated by the feature extraction module to generate an attention-enhanced feature map focusing on the lesion area;

[0010] The classification module is used to perform dynamic prototype aggregation processing on the attention-enhanced feature map generated by the attention-enhancing module to classify diabetic foot images.

[0011] Optionally, the data enhancement module performs the following steps when performing weighted fusion processing of the diabetic foot image with the training image and the similar image to generate an expanded data set:

[0012] Training images and similar images are sampled from diabetic foot images, and weighted fusion is performed on the training images and similar images to generate deformed images and thereby generate an expanded data set, which includes: deformed images and diabetic foot images.

[0013] Optionally, the data enhancement module samples training images and similar images from the diabetic foot image, performs weighted fusion on the training images and similar images to generate a deformed image and generates an expanded data set accordingly, which includes: when the deformed image and the diabetic foot image are generated, performing the following steps:

[0014] Determine the base class images of diabetic foot images and construct a small sample learning paradigm based on the base class images;

[0015] Based on the small sample learning paradigm, diabetic foot images are sampled to obtain training images;

[0016] Perform neighbor retrieval in feature space based on training images to determine lesion features and generate similar images based on them;

[0017] The training image and similar images are linearly superimposed with weights to generate deformed images and thus generate an expanded dataset.

[0018] Optionally, when the data augmentation module performs weighted linear superposition on the training image and the similar image to generate the deformed image and generates the expanded data set accordingly, the following steps are performed:

[0019] Perform pixel-level fusion with weights on the training image and similar images to generate a preliminary fused image;

[0020] Performing lesion area mask weighting processing on the generated primary fusion image to generate a deformed image;

[0021] Lesion semantic annotation is performed on deformed images and diabetic foot images to generate an expanded dataset.

[0022] Optionally, the feature extraction module performs ResNet-101 backbone network feature extraction processing on the expanded data set generated by the data enhancement module to generate a feature tensor containing global semantics and local details, and performs the following steps:

[0023] Based on the ResNet-101 backbone network, the feature extraction of the expanded data set is performed with three-dimensional feature vectors. Where H / W is the feature map size and C is the number of channels;

[0024] The three-dimensional feature vector is decomposed into multiple local descriptors, each of which corresponds to the fine-grained features of the image block to generate a feature tensor that includes global semantics and local details.

[0025] Optionally, the feature extraction module decomposes the three-dimensional feature vector into multiple local descriptors, each local descriptor corresponding to a fine-grained feature of an image block, to generate a feature tensor including global semantics and local details, and performs the following steps:

[0026] Convolution operations are performed on the three-dimensional feature vector using convolution kernels with different receptive fields to generate a multi-scale feature map group;

[0027] The feature map at each scale is divided into grids of different sizes, and the features within each grid are subjected to maximum pooling or average pooling to generate spatial pyramid features;

[0028] The spatial pyramid features are divided into sliding windows of fixed size, and the features in each window are compressed into a vector of fixed dimension through a fully connected layer to generate multiple local descriptors;

[0029] Calculate the semantic self-attention weights between all local descriptors to generate semantic correlation features;

[0030] Perform global average pooling and maximum pooling on the semantic association features to obtain global semantic features;

[0031] The global semantic features are upsampled to generate a feature tensor that includes global semantics and local details.

[0032] Optionally, the attention enhancement module performs dual-pool channel attention and enhanced spatial attention processing on the feature tensor generated by the feature extraction module to generate an attention-enhanced feature map focused on the lesion area, performing the following steps:

[0033] Generate adaptive feature reweighting results based on the feature tensor generated by the feature extraction module;

[0034] Generate a channel attention map based on the adaptive feature reweighting results;

[0035] Generate an attention-enhanced feature map focusing on the lesion area based on the channel attention map.

[0036] Optionally, when the attention enhancement module generates an adaptive feature reweighting result based on the feature tensor generated by the feature extraction module, the following steps are performed:

[0037] The feature tensor generated by the feature extraction module is simultaneously subjected to cross-channel autocorrelation calculation and local feature entropy evaluation to obtain a channel autocorrelation matrix group and a channel entropy value vector;

[0038] The channel autocorrelation matrix group and the channel entropy value vector are concatenated and processed through attention gating to generate a channel weight vector;

[0039] The feature tensor is scaled using the channel weight vector and adaptive feature reweighting is obtained accordingly.

[0040] Optionally, when the attention enhancement module generates a channel attention map based on the adaptive feature reweighting result, the following steps are performed:

[0041] The adaptive feature reweighting results are processed through independent linear layers to obtain a dual-path output vector;

[0042] The dual-path output vectors are added element-by-element and normalized by the Sigmoid function to generate key feature channel weights, and irrelevant feature channels including feature channels corresponding to the background are suppressed to generate a channel attention map based on this.

[0043] Optionally, when the attention enhancement module generates an attention enhancement feature map of the focused lesion area based on the channel attention map, the channel attention map is mapped to the cross-channel interaction space and the local correlation metric space to calculate the local feature similarity matrix and evaluate the correlation of the lesion area based on it, locate the focus label of the diabetic foot lesion area to generate an attention enhancement feature map of the focused lesion area.

[0044] Optionally, when the classification module performs dynamic prototype aggregation processing on the attention enhancement feature map generated by the attention enhancement module to classify the diabetic foot image, the following steps are performed:

[0045] Metric support feature and query feature extraction are performed on the attention-enhanced feature map;

[0046] Generate enhanced perceptual masks based on the extracted metric support features and query features;

[0047] Generate dynamic category prototypes based on enhanced perceptual masks;

[0048] Dynamic category prototypes are fused to determine the classification of diabetic foot images.

[0049] Optionally, when the classification module measures support features and extracts query features from the attention-enhanced feature map, the following steps are performed:

[0050] Based on the constructed support feature extractor, the attention-enhanced feature map is measured to support feature extraction;

[0051] Based on the constructed query feature extractor, query features are extracted from the attention-enhanced feature map;

[0052] The extracted metric support features and query features are respectively embedded with the prior knowledge features of diabetic foot lesions to generate metric support features and query features guided by medical prior knowledge.

[0053] Optionally, when the classification module generates the enhanced perception mask based on the extracted metric support features and query features, the following steps are performed:

[0054] Reshape the metric support features and query features to calculate the association strength of each spatial position between the metric support features and the query features;

[0055] Based on the correlation strength, a neighborhood index matrix of the spatial position is constructed, and the neighborhood index matrix is ​​Gaussian smoothed to obtain a local smooth correlation matrix;

[0056] The local smoothed correlation matrix is ​​used to map the lesion region activations and generate an enhanced perceptual mask.

[0057] Optionally, when the classification module generates a dynamic category prototype based on the enhanced perceptual mask, the following steps are performed:

[0058] Perform spatial dimension mask expansion on the enhanced perception mask to generate initial category prototypes;

[0059] Performing principal component analysis and dimensionality reduction on the initial category prototype to construct the feature subspace of the dynamic category prototype;

[0060] The inter-class distance of the initial category prototype is maximized in the feature subspace and dynamically adapted according to the lesion stage to generate a dynamic category prototype.

[0061] Optionally, when the classification module fuses the dynamic category prototypes to determine the classification of the diabetic foot image, the following steps are performed:

[0062] Calculate the feature covariance matrix between dynamic category prototypes;

[0063] Calculate the similarity between the dynamic category prototype and the query feature based on the feature covariance matrix;

[0064] The dynamic category prototypes are fused based on similarity to generate weighted category representations;

[0065] Class-category decision boundaries were constructed based on statistically weighted class representation to determine the classification of diabetic foot images.

[0066] According to a second aspect of an embodiment of the present invention, a method for classifying diabetic foot images based on small sample learning is provided, comprising:

[0067] Perform weighted fusion of training images and similar images on diabetic foot images to generate an expanded dataset;

[0068] Perform ResNet-101 backbone network feature extraction on the expanded dataset to generate feature tensors that include global semantics and local details;

[0069] Perform dual-pool channel attention and enhanced spatial attention processing on the feature tensor to generate an attention-enhanced feature map focusing on the lesion area;

[0070] Dynamic prototype aggregation of attention-enhanced feature maps for classification of diabetic foot images.

[0071] The solution of the embodiment of the present invention has the following beneficial technical effects:

[0072] ① Solve the problems of scarcity and overfitting of labeled data and improve the effectiveness of small sample classification

[0073] The data augmentation module generates an augmented dataset by weightedly fusing training images with similar images. Unlike traditional geometric transformations (rotation and cropping), this method samples similar images based on feature similarity and linearly fuses them. This method preserves the semantic characteristics of diabetic foot lesions, effectively expands data distribution density in small sample sizes, and reduces the risk of model overfitting. For example, feature nearest neighbor retrieval ensures that the lesion features of similar images are highly consistent with those of the training images. The fused data increases the number of samples while maintaining the continuity of lesion semantics, enabling the model to learn stable lesion feature representations even with limited annotated data.

[0074] ② Strengthen the ability to capture multi-scale features, taking into account both global semantics and local details

[0075] The feature extraction module uses the ResNet-101 backbone network, whose deep convolutional structure extracts global semantic features (such as the overall distribution of lesions). Combined with the network's inherent hierarchical feature extraction mechanism, it can simultaneously capture both the global distribution and local subtle features of diabetic foot lesions. Compared to conventional feature extraction networks, this solution leverages the backbone network's inherent multi-scale representation capabilities to avoid the overhead of designing additional complex multi-scale modules. Furthermore, the fusion of global and local features in the feature tensor provides a more comprehensive feature foundation for the subsequent attention mechanism.

[0076] ③ Improve the focusing ability of the lesion area and suppress background noise interference

[0077] The attention enhancement module achieves precise focus on lesion areas through a hierarchical process of dual-pool channel attention and enhanced spatial attention. Channel attention generates channel weights through global average pooling and maximum pooling, suppressing irrelevant channels such as skin texture and shooting environment, and enhancing lesion-related channels such as ulcers and pigmentation. Spatial attention further locates the lesion area. The combination of the two can effectively filter complex background noise. For example, in diabetic foot images, this module can automatically reduce background interference such as socks and bed sheets, focusing attention on the foot lesion area and improving the discriminability of feature expression.

[0078] ④Adapt to the dynamic changes of lesions and enhance the generalization ability of new categories and early lesions

[0079] The classification module uses dynamic prototype aggregation to generate sample-updated category prototypes based on the attention-enhanced feature map, rather than static fixed prototypes. This mechanism dynamically adjusts the prototype representation based on the lesion characteristics of the input image. This adaptive updating of the prototype improves classification generalization, particularly for early, subtle lesions or newly emerging lesion types. For example, when encountering an early Wagner grade 1 ulcer, the dynamic prototype can optimize the category representation based on the sample characteristics, improving the model's recognition accuracy for similar cases and addressing the lack of generalization for new categories in traditional static prototype models. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments recorded in the embodiments of the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0081] Figure 1 This is a schematic diagram of a diabetic foot image classification system based on small sample learning in an embodiment of the present application.

[0082] Figure 2 A flowchart of a diabetic foot image classification method based on small sample learning is provided in this embodiment. DETAILED DESCRIPTION

[0083] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present invention, the technical solutions in the embodiments of the present invention will be clearly and detailedly described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. All other embodiments obtained by those skilled in the art based on the embodiments in the embodiments of the present invention should fall within the scope of protection of the embodiments of the present invention.

[0084] It should be understood that the terms "first," "second," and "third," etc. in the claims, specification, and drawings of this disclosure are used to distinguish different objects rather than to describe a specific order. The terms "include" and "comprising" used in the specification and claims of this disclosure indicate the presence of the described features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or combinations thereof.

[0085] It should also be understood that the terminology used in this disclosure is for the purpose of describing specific embodiments only and is not intended to limit the disclosure. As used in this disclosure and the claims, the singular forms "a," "an," and "the" are intended to include the plural forms unless the context clearly indicates otherwise. It should be further understood that the term "and / or" as used in this disclosure and the claims refers to any and all possible combinations of one or more of the associated listed items, including and including these combinations.

[0086] Figure 1 This is a schematic diagram of a diabetic foot image classification system based on small sample learning in an embodiment of the present application. Figure 1 As shown in FIG, a diabetic foot image classification system based on small sample learning includes:

[0087] The data enhancement module is used to perform weighted fusion of training images and similar images on diabetic foot images to generate an expanded data set;

[0088] The feature extraction module is used to perform ResNet-101 backbone network feature extraction on the expanded dataset generated by the data enhancement module to generate a feature tensor containing global semantics and local details;

[0089] The attention enhancement module is used to perform dual-pool channel attention and enhanced spatial attention processing on the feature tensor generated by the feature extraction module to generate an attention-enhanced feature map focusing on the lesion area;

[0090] The classification module is used to perform dynamic prototype aggregation processing on the attention-enhanced feature map generated by the attention-enhancing module to classify diabetic foot images.

[0091] Optionally, the data enhancement module performs the following steps when performing weighted fusion processing of the diabetic foot image with the training image and the similar image to generate an expanded data set:

[0092] Training images and similar images are sampled from diabetic foot images, and weighted fusion is performed on the training images and similar images to generate deformed images and thereby generate an expanded data set, which includes: deformed images and diabetic foot images.

[0093] Preferably, in a specific application scenario, the specific technical implementation when generating a data set is as follows:

[0094] 1. Similar image sampling mechanism (feature vector driven process)

[0095] For each image in the diabetic foot image library, the geometric features (perimeter, area, circularity), color features (mean / variance of ulcer hue in HSV space), and texture features (edge ​​complexity extracted by LBP operator) of the ulcer area are extracted to form a feature vector with a dimension of n = 12.

[0096] Map the feature vectors of all images into Euclidean space and construct a feature index tree (such as KD-Tree) to achieve fast neighbor retrieval.

[0097] For the training image I train , retrieve the feature vector through KD-Tree to satisfy ‖f(I train )-f(I similar )‖2<τ·Std(f) similar image I similar (τ=1.2 is the similarity threshold), and I similar The background features (such as shooting angle, foot occlusion) and I train The difference exceeds the threshold δ = 0.7, ensuring that the lesions are similar but the backgrounds are diverse.

[0098] 2. Weighted fusion deformation mechanism (feature vector guided fusion)

[0099] Using the feature vector f of the training image and similar images train ,f similar , calculate the feature difference vector Δf = f train -f similar , through TPS transformation parameter optimization to make ‖f(T(I similar ))-f train ‖2 is minimized to ensure that the features of similar images after geometric transformation are consistent with the training images.

[0100] Generate global fusion weights based on the modulus of the feature difference vector Δf At the same time, the local saliency map M is combined to generate pixel-by-pixel weights:

[0101] α(x,y)=M(x,y)·(0.7+0.3·α global )+(1-M(x,y))·0.2

[0102] Among them, the weight of the lesion area is dynamically adjusted with the feature similarity (the more similar the features are, the higher the weight of the training image is).

[0103] Calculate the deformed image after fusion I aug The eigenvector f aug , requiring to satisfy ‖f aug -f train ‖2<0.5·‖Δf‖2, otherwise reselect similar images or adjust the fusion parameters to ensure the feature consistency of the generated data.

[0104] In summary, in traditional methods, the randomly sampled similar images may have different ulcer types from the training images (such as fusing blister images with ulcer images), resulting in semantic confusion in the generated data. This scheme ensures that the sampled I similar with I train The same ulcer type (the difference in the dimension of "ulcer shape" in the feature vector is <0.2). Traditional fusion has no feature constraints, which makes the fused ulcer color feature deviate from the clinical true value; this scheme dynamically adjusts the fusion weight through Δf to make I aug The ulcer color characteristics (HSV hue dimension) and I train The deviation is <5%.

[0105] In addition, when there are only 30 labeled samples, about 40% of the 200 data points generated by traditional enhancement are invalid samples (the feature vector differs from any labeled sample by more than 2 standard deviations); this scheme controls the invalid sample rate below 5% through the feature vector verification mechanism, increases the amount of valid data by 3.2 times, and improves the classification accuracy from 60% to 85%.

[0106] Optionally, the data enhancement module samples training images and similar images from the diabetic foot image, performs weighted fusion on the training images and similar images to generate a deformed image and generates an expanded data set accordingly, which includes: when the deformed image and the diabetic foot image are generated, performing the following steps:

[0107] Determine the base class images of diabetic foot images and construct a small sample learning paradigm based on the base class images;

[0108] Based on the small sample learning paradigm, diabetic foot images are sampled to obtain training images;

[0109] Perform neighbor retrieval in feature space based on training images to determine lesion features and generate similar images based on them;

[0110] The training image and similar images are linearly superimposed with weights to generate deformed images and thus generate an expanded dataset.

[0111] In specific implementation, the above steps are exemplarily implemented as follows:

[0112] When screening base class images and constructing the small sample paradigm, images with complete lesions, typical features, and covering multiple types are selected from the diabetic foot image library as base classes to construct an "N-way K-shot" data framework that meets the clinical small sample diagnosis scenario (such as 5 types of lesions, 5 annotated images for each type) to ensure the clinical representativeness of the enhanced data.

[0113] In the feature space nearest neighbor retrieval mechanism, a vector containing 18-dimensional lesion features (6-dimensional geometric features + 6-dimensional color features + 6-dimensional texture features) is extracted for each image. Similar images with a feature distance less than 1.5 times the feature standard deviation from the training image are retrieved in the feature space through KD-Tree. At the same time, the background difference is required to exceed 0.6 to achieve sample screening with "consistent lesion features but different backgrounds".

[0114] In the dynamic weighted linear fusion strategy, the global fusion weight is determined by the feature distance between the training image and similar images: the smaller the feature distance, the higher the training image weight (the weight is inversely proportional to the distance). The local fusion weight is combined with the lesion saliency map: the weight of the lesion area is dynamically adjusted with the global weight (up to 0.8 times the global weight + 0.2), while the background area is fixed with a low weight of 0.3 to ensure lesion feature preservation and background noise suppression.

[0115] When the fused image needs to meet the feature distance constraint: the feature distance between the deformed image and the training image must be less than 0.6 times the feature distance between the training image and the similar image to ensure the feature consistency of the generated data.

[0116] Preferably, in a specific application scenario, the above steps are implemented as follows:

[0117] 1. Determination of base class images and construction of small sample paradigm

[0118] Base class screening criteria: Images that meet the following conditions are selected as base classes from the diabetic foot image library: lesion completeness: the ulcer area occlusion rate is <10%, verified by the mask area ratio generated by the semantic segmentation model; feature typicality: in line with the clinical Wagner grading standards (e.g., Wagner grade 2 ulcers must have dermal exposure features); data diversity: covering at least four major lesion types (ulcers, calluses, pigmentation, and vascular lesions).

[0119] Small sample paradigm construction: A 5-way 5-shot framework (5 types of lesions, 5 annotated images for each type) is used to simulate the small sample scenario during initial clinical diagnosis, ensuring that the subsequent enhanced data is consistent with the data distribution of the real diagnosis scenario.

[0120] 2. Neighbor retrieval mechanism based on feature space (high-dimensional feature vector driven process)

[0121] Feature vector construction (d = 18 dimensions) includes: Geometric features (6 dimensions): ulcer area A, perimeter P, circularity C = 4πA / P 2 , major axis length L, minor axis length W, eccentricity E; color features (6 dimensions): hue mean μ in HSV space of ulcer area h , saturation mean μ s , lightness mean μ v and standard deviation σ h ,σ s ,σ v Texture features (6 dimensions): ulcer edge roughness R, directionality D, contrast C and its standard deviation σ extracted by LBP operator R ,σ D ,σ C .

[0122] Feature space retrieval process:

[0123] For training image I train Extract feature vectors

[0124] Construct a KD-Tree index in the feature space and retrieve the train Images with ‖2<τ·Std(f) (τ=1.5, Std(f) is the standard deviation of each dimension of the feature vector);

[0125] The selected similar images must meet the background difference BgDiff>0.6 (calculated by the HSV color distribution difference of the background area) to ensure that the lesions are similar but the backgrounds are different.

[0126] 3. Deformed Image Generation with Weighted Linear Superposition

[0127] Global weight where f similar is the similar image feature vector;

[0128] The local weight α(x,y) is determined by the lesion saliency map M(x,y)∈[0,1] (generated by the segmentation model):

[0129] α(x,y)=M(x,y)·(0.8α global +0.2)+(1-M(x,y))·0.3

[0130] Among them: 0.8α global +0.2 is the weight of the lesion area (dynamically adjusted with feature similarity), and $0.3$ is a fixed low weight for the background area.

[0131] Linear superposition formula: I aug (x,y)=α(x,y)·I train (x,y)+(1-α(x,y))·I similar (x,y)

[0132] The generated deformed image must satisfy ‖f aug -f train ‖2<0.6·‖f train -f similar ‖2(f aug is the deformed image feature vector), otherwise reselect a similar image.

[0133] This program is implemented through ‖f train -f similar ‖2<1.5·Std(f) is strictly screened to ensure that the characteristics of similar lesions match. The traditional fusion weight is irrelevant to the characteristics, resulting in the drift of ulcer color characteristics after fusion; this scheme uses α global Dynamic adjustment is performed to control the hue shift to <5°, meeting clinical image consistency requirements.

[0134] In a 5-way, 5-shot scenario, traditional methods generate a low percentage of valid lesions of the same type in 100 data points due to lax feature matching. This solution, through feature vector retrieval, increases the proportion of valid data and significantly improves classification accuracy. When the feature vector dimension d = 18, the KD-Tree retrieval has a low false match rate.

[0135] After traditional fixed weight fusion, the geometric features (such as area) of the ulcer area have a high deviation rate; this scheme uses ‖f aug -f train ‖2<0.6·‖f train -f similar The ‖2 constraint significantly reduces the area deviation rate and improves the retention rate of edge roughness features. In the deformed images generated by this scheme, the pixel contrast of the spot bleeding feature is also improved, which optimizes the model's recognition rate of this feature.

[0136] Traditional methods use a background fusion weight of α = 0.5, resulting in a significant proportion of generated images containing significant background interference (such as sock texture). Our approach, with a fixed background weight of 0.3 and combined with saliency map filtering, significantly reduces the background interference rate and improves the model's feature extraction efficiency for lesion areas. The data generated by this approach significantly reduces the standard deviation of the model's accuracy under different backgrounds, demonstrating its strong background robustness.

[0137] Optionally, when the data augmentation module performs weighted linear superposition on the training image and the similar image to generate the deformed image and generates the expanded data set accordingly, the following steps are performed:

[0138] Perform pixel-level fusion with weights on the training image and similar images to generate a preliminary fused image;

[0139] Performing lesion area mask weighting processing on the generated primary fusion image to generate a deformed image;

[0140] Lesion semantic annotation is performed on deformed images and diabetic foot images to generate an expanded dataset.

[0141] In summary, pixel-wise weighted fusion involves linearly superimposing the training image and similar images pixel by pixel. Weight assignment follows the principle of prioritizing the preservation of training image features in lesion regions while introducing diversity from similar images into background regions. Specifically, fusion weights are dynamically adjusted based on the image significance of lesion regions: training image weights are increased in lesion regions like ulcers and pigmentation to preserve key diagnostic features; training image weights are decreased in areas like normal foot skin and background to introduce background variations from similar images, thereby expanding the data while avoiding distortion of lesion features.

[0142] During lesion mask weighting, a pre-generated lesion mask (e.g., the ulcer area outline obtained through semantic segmentation) is used to perform a secondary weighted optimization on the initial fused image. The core logic of mask weighting is to implement differentiated weight adjustments for the lesion core, edge, and background regions of the fused image based on the lesion boundary defined by the mask. The core region strengthens the training image features to preserve diagnostic information, the edge region balances the two features to avoid boundary blur, and the background region further suppresses non-lesion information. The result is a deformed image with clear features and a diverse background.

[0143] During the lesion semantic annotation mechanism, the generated deformed images and the original diabetic foot images were semantically annotated based on clinical diagnostic criteria. This annotation process strictly adheres to a three-dimensional annotation system for lesion type (ulcer, callus, etc.), severity (Wagner classification), and anatomical location (plantar, interdigital, etc.). This ensures that each sample in the expanded dataset has clear clinical semantic attributes, providing accurate labeling guidance for subsequent classification.

[0144] Optionally, the feature extraction module performs ResNet-101 backbone network feature extraction processing on the expanded data set generated by the data enhancement module to generate a feature tensor containing global semantics and local details, and performs the following steps:

[0145] Based on the ResNet-101 backbone network, the feature extraction of the expanded data set is performed with three-dimensional feature vectors. Where H / W is the feature map size and C is the number of channels;

[0146] The three-dimensional feature vector is decomposed into multiple local descriptors, each of which corresponds to the fine-grained features of the image block to generate a feature tensor that includes global semantics and local details.

[0147] In summary, the ResNet-101 backbone network feature extraction mechanism leverages the hierarchical feature extraction capabilities of the deep convolutional network to perform multi-scale feature extraction on the expanded diabetic foot images. This network, through its residual connection structure, alleviates the vanishing gradient problem of deep networks and is capable of extracting features from images at different levels of abstraction: shallow convolutional layers capture local details such as ulcer edges and color, while deep convolutional layers integrate global semantic information (such as the overall distribution of lesions and anatomical relationships). The final output is a three-dimensional feature vector H × W × C, where H × W corresponds to the spatial dimensions of the feature map and C is the number of channels, each representing a semantic or texture feature.

[0148] In addition, during feature vector decomposition and local descriptor generation, the three-dimensional feature vector is decomposed into multiple local descriptors according to the spatial dimension. Each descriptor corresponds to the fine-grained features of a specific image block in the original image. Specifically, the feature map is divided into multiple overlapping or non-overlapping sub-regions using a sliding window method. The features in each sub-region are aggregated (such as average pooling and maximum pooling) to generate local descriptors. These descriptors can not only preserve local subtle features such as ulcer edges and bleeding points, but also form a semantic representation of the global distribution of lesions by combining multiple descriptors, ultimately generating a feature tensor that contains both global semantics and local details.

[0149] Optionally, the feature extraction module decomposes the three-dimensional feature vector into multiple local descriptors, each local descriptor corresponding to a fine-grained feature of an image block, to generate a feature tensor including global semantics and local details, and performs the following steps:

[0150] Convolution operations are performed on the three-dimensional feature vector using convolution kernels with different receptive fields to generate a multi-scale feature map group;

[0151] The feature map at each scale is divided into grids of different sizes, and the features within each grid are subjected to maximum pooling or average pooling to generate spatial pyramid features;

[0152] The spatial pyramid features are divided into sliding windows of fixed size, and the features in each window are compressed into a vector of fixed dimension through a fully connected layer to generate multiple local descriptors;

[0153] Calculate the semantic self-attention weights between all local descriptors to generate semantic correlation features;

[0154] Perform global average pooling and maximum pooling on the semantic association features to obtain global semantic features;

[0155] The global semantic features are upsampled to generate a feature tensor that includes global semantics and local details.

[0156] In summary, the multi-scale feature map generation mechanism uses convolution kernels with different receptive fields to perform parallel convolution on the three-dimensional feature vector, capturing the multi-scale characteristics of diabetic foot lesions. Small receptive field convolution kernels (e.g., 3×3) focus on subtle local features such as ulcer edges and bleeding points, while large receptive field convolution kernels (e.g., 7×7) capture global semantics such as the overall distribution and anatomical location of the lesion. This creates a feature map group covering different spatial scales, ensuring the simultaneous representation of both small ulcers (diameter <3 mm) and large infected areas.

[0157] When constructing spatial pyramid features, feature maps at each scale can be divided into grids of varying densities (e.g., 1×1, 2×2, 4×4), and features within each grid are pooled. High-density grids (e.g., 4×4) retain fine-grained information such as the ulcer's edge texture, while low-density grids (e.g., 1×1) integrate global semantics, forming a pyramid feature that encompasses multi-scale spatial information. This mechanism enables features to describe both the local edge morphology of the ulcer and its overall positional relationship within the foot.

[0158] When generating local descriptors and strengthening semantic associations, sliding window feature segmentation is performed: spatial pyramid features are overlapped and segmented with fixed-size windows (such as 16×16 pixels). Each window corresponds to a local area of ​​the original image. The window features are compressed into a fixed-dimensional vector through a fully connected layer to generate local descriptors, ensuring that each descriptor focuses on the fine-grained features of a specific image block (such as the color and texture of a certain part of the ulcer).

[0159] When calculating the semantic self-attention weights, the semantic correlation between all local descriptors can be analyzed, and high weights can be given to descriptors that are highly correlated with ulcer lesions (such as descriptors containing bleeding point features), while the descriptor weights of normal skin areas are suppressed, the semantic association of lesion features is strengthened, and mutual interference of features of different lesion areas is avoided.

[0160] In the mechanism of fusing global semantics and local details, global pooling (average pooling + maximum pooling) is performed on the semantically associated features to extract global semantic features that can characterize the overall attributes of the lesion (such as ulcer type and severity). Then, the global features are aligned with the spatial position information of the local descriptor through upsampling, and finally a feature tensor containing both global semantics and local details is generated, realizing the dual feature expression of "overall diagnosis of lesions-verification of local details".

[0161] Therefore, for complex cases with combined micro-ulcers and large areas of pigmentation, traditional methods often submerge micro-ulcer features in global features due to the lack of multi-scale processing; this solution uses small receptive field convolution to extract the edges of micro-ulcers and large receptive field convolution to locate the pigmentation area, combined with spatial pyramid feature fusion, to significantly improve the classification accuracy of such cases. The low-density grid (such as 1×1) in the spatial pyramid feature of this application integrates the overall anatomical information of the foot, and combined with the position coordinates of the sliding window segmentation, it can accurately encode the features of lesions in different locations such as between the toes and the sole of the foot. This mechanism greatly improves the accuracy of the model in distinguishing the positions of "interdigital ulcers" and "plantar ulcers", and significantly reduces the classification errors caused by position misjudgment. Furthermore, when traditional local features are processed independently, the color features and edge features of the ulcer area may be misjudged as irrelevant information; this solution strengthens the semantic association between the two (such as the feature combination of "red ulcer edge") through self-attention weight calculation, which significantly improves the feature discriminability and significantly improves the accuracy of distinguishing "bleeding ulcers" from "non-bleeding ulcers". Finally, an upsampling operation spatially aligns global semantic features (such as "Wagner grade 2 ulcer") with local detail features (such as fibrin exudate at the base of the ulcer), ensuring consistency between the global diagnosis and the local details. For example, when the global semantic judgment is "moderate ulcer," the local descriptor can verify the presence of detailed features that meet this classification, significantly reducing the false positive rate.

[0162] Optionally, the attention enhancement module performs dual-pool channel attention and enhanced spatial attention processing on the feature tensor generated by the feature extraction module to generate an attention-enhanced feature map focused on the lesion area, performing the following steps:

[0163] Generate adaptive feature reweighting results based on the feature tensor generated by the feature extraction module;

[0164] Generate a channel attention map based on the adaptive feature reweighting results;

[0165] Generate an attention-enhanced feature map focusing on the lesion area based on the channel attention map.

[0166] In summary, this application introduces an adaptive feature reweighting mechanism. For the feature tensor output by the feature extraction module, differentiated weight adjustment is implemented by analyzing the correlation between the features of each channel and diabetic foot lesions. Specifically, global average pooling and maximum pooling are first performed on each channel of the feature tensor. The former captures the overall response intensity of the channel, and the latter highlights the local peak characteristics (such as the high-frequency response of the ulcer edge); then the two are combined to generate a channel importance metric, which increases the weight of lesion-related channels such as ulcer color and edge texture, and reduces the weight of irrelevant channels such as normal skin texture and background noise, thereby enhancing the expression of lesion features and suppressing background interference.

[0167] This application also introduces a channel attention map generation mechanism. Based on the adaptively reweighted features, a channel attention map is further constructed: by calculating the cross-channel correlation of each channel feature, the co-activation pattern of lesion features (such as the high-frequency co-activation of the color channel and edge channel in the ulcer area) is captured. This mechanism preferentially activates channel combinations that are highly correlated with diabetic foot lesions (such as ulcers and bleeding spots), forming a channel weight distribution that focuses on lesion features, ensuring that lesion-related features are retained in subsequent processing.

[0168] This application also introduces an attention enhancement mechanism that focuses on the lesion area. The channel attention map is weighted channel by channel with the original feature tensor to generate an attention-enhanced feature map. The high-weight channels in the channel attention map correspond to lesion features such as ulcers and bleeding. After weighting, the response intensity of these features is significantly improved; the low-weight channels correspond to irrelevant information such as normal skin and background, and their feature responses are suppressed. The final generated feature map can effectively filter out interference from non-lesion areas, making the feature expression of the lesion area more prominent, and providing more discriminative input for subsequent classification.

[0169] To this end, the classification of diabetic foot has the following technical benefits

[0170] 1. Traditional attention techniques have insufficient focus on lesion areas in diabetic foot images that include backgrounds such as socks and bed sheets, and can easily misjudge background texture as lesion features. This solution significantly improves the feature response strength of ulcer areas through dual-pool channel attention and enhanced spatial attention processing, while effectively suppressing background noise interference, making the expression of lesion features clearer against complex backgrounds.

[0171] 2. For early-stage minor ulcers such as Wagner grade 1, traditional methods often overwhelm normal skin features due to the lack of sensitivity of the attention mechanism to subtle features. This solution enhances the activation of subtle features such as the edges and color of minor ulcers through adaptive reweighting and channel attention maps, significantly improving the feature expression of early lesions and enhancing the detection capability of subtle lesions.

[0172] 3. For complex lesions with both ulcers and pigmentation, traditional technologies have difficulty effectively separating the features of different lesion types, leading to classification confusion. This solution uses channel attention maps to differentially activate channels related to different lesions. This can clearly distinguish the features of ulcer areas from those of pigmentation areas, reducing the mutual interference of complex lesion features and significantly improving feature separation capabilities.

[0173] 4. Traditional attention mechanisms lack targeted focus on lesions in different anatomical locations of the foot (such as between the toes and on the soles of the feet). This solution enhances spatial attention processing and combines the weight distribution of the channel attention map to dynamically adjust the attention focus based on the anatomical location characteristics of the lesion. This improves the accuracy of the model's focus on the features of lesions in different locations and reduces classification errors caused by location misjudgment.

[0174] Optionally, when the attention enhancement module generates an adaptive feature reweighting result based on the feature tensor generated by the feature extraction module, the following steps are performed:

[0175] The feature tensor generated by the feature extraction module is simultaneously subjected to cross-channel autocorrelation calculation and local feature entropy evaluation to obtain a channel autocorrelation matrix group and a channel entropy value vector;

[0176] The channel autocorrelation matrix group and the channel entropy value vector are concatenated and processed through attention gating to generate a channel weight vector;

[0177] The feature tensor is scaled using the channel weight vector and adaptive feature reweighting is obtained accordingly.

[0178] Specifically, in one application scenario, the above solution is implemented as follows:

[0179] This application introduces a cross-channel autocorrelation calculation and local feature entropy evaluation mechanism. For the feature tensor output by the feature extraction module, the autocorrelation matrix between the features of each channel is calculated to capture the co-activation relationship of the features of diabetic foot lesions (such as the correlation between the ulcer color channel and the edge texture channel). By analyzing the correlation between channels, the feature combination that is highly correlated with the lesion is identified, providing a basis for subsequent weight allocation. The local entropy value of the features of each channel is calculated to quantify the uncertainty of the features - the features of the lesion area (such as the ulcer edge) have a higher entropy value due to the complex texture, and the features of the normal skin area have a lower entropy value. The entropy value vector is used to distinguish between lesion-related channels and background channels, thereby enhancing the sensitivity to lesion features.

[0180] This application introduces a channel weight vector generation mechanism, which splices the cross-channel autocorrelation matrix group and the channel entropy value vector into a comprehensive feature vector, filters redundant information through an attention gating mechanism (such as a sigmoid activation function), and highlights the feature dimensions related to the lesion. This mechanism dynamically adjusts the importance weight of each channel based on the correlation between channels and the feature entropy value, suppresses background channels with low entropy and weak correlation, and activates lesion channels with high entropy and strong correlation. The feature vector after gating is mapped to a channel weight vector, in which each element corresponds to a channel of the feature tensor, and the weight reflects the channel's ability to characterize diabetic foot lesions.

[0181] This application also introduces an adaptive feature reweighting mechanism, which uses channel weight vectors to scale the feature tensor channel by channel. The feature response strength of high-weight channels (such as ulcer color and edge texture channels) is enhanced, while the features of low-weight channels (such as normal skin and background channels) are suppressed, ultimately generating an adaptive feature reweighting result. This result strengthens the coordinated expression of lesion features while reducing the interference of background noise, providing a more discriminative feature foundation for subsequent attention enhancement.

[0182] Therefore, it can be seen that the classification of diabetic foot has the following beneficial technical effects

[0183] 1. Traditional techniques for complex lesions with both ulcers and bleeding often lead to mutual suppression of different lesion features because they fail to exploit inter-channel correlation. This solution strengthens the synergistic weights of the ulcer color channel and the bleeding feature channel through cross-channel autocorrelation calculation, making the feature expression of complex lesions more complete and significantly reducing feature confusion.

[0184] 2. For subtle lesions such as Wagner grade 1 microulcers, traditional methods are insensitive to low-response features due to their channel weight generation mechanism, which can easily lead to features being overlooked. This solution uses local feature entropy evaluation to assign higher weights to subtle lesion features with high entropy values ​​(such as the texture of punctate hemorrhages), thereby enhancing the feature response strength of subtle lesions and significantly improving detection capabilities.

[0185] 3. In images with complex backgrounds such as bed sheet textures and sock patterns, traditional technologies have difficulty distinguishing between background channels and lesion channels. This solution uses the dual indicators of autocorrelation and entropy to accurately identify low-entropy background channels and suppress their weights, significantly reducing the interference of background noise on lesion features and improving feature purity.

[0186] 4. Traditional technologies use fixed weights or simple mapping, which cannot adapt to the differences in lesion characteristics across different cases. This solution dynamically generates channel weights through an attention gating mechanism, adaptively adjusting weight distribution for different types of diabetic foot lesions (such as ulcers and pigmentation), making the feature reweighting results more in line with the diagnostic needs of specific cases and improving the adaptability of feature expression.

[0187] Optionally, when the attention enhancement module generates a channel attention map based on the adaptive feature reweighting result, the following steps are performed:

[0188] The adaptive feature reweighting results are processed through independent linear layers to obtain a dual-path output vector;

[0189] The dual-path output vectors are added element-by-element and normalized by the Sigmoid function to generate key feature channel weights, and irrelevant feature channels including feature channels corresponding to the background are suppressed to generate a channel attention map based on this.

[0190] In summary, this application introduces a dual-path linear layer feature transformation mechanism, which performs feature transformation on the adaptive feature reweighting results through two independent linear layers to form a dual-path output vector. Each path uses a different weight matrix to map the input features: one path focuses on capturing the global semantic features of diabetic foot lesions (such as the overall distribution of ulcers), and the other path focuses on local detail features (such as the texture of the ulcer edge). This dual-path design can extract features from different dimensions to ensure comprehensive capture of lesion-related information.

[0191] In addition, this application also introduces a feature fusion and normalization mechanism, which adds the dual-path output vectors element by element, fusing global semantics with local detail features to form a comprehensive feature vector containing multi-dimensional lesion information. The comprehensive feature vector is normalized using the Sigmoid function, mapping the values ​​to the interval [0, 1] so that each element corresponds to a channel weight in the feature tensor—high values ​​indicate that the channel is highly correlated with the lesion, while low values ​​indicate that the channel is associated with irrelevant information such as background.

[0192] Finally, this application also introduces a channel attention map generation mechanism. This generates a channel attention map based on the normalized weight vector: high-weight channels (such as ulcer color and edge texture channels) are strongly activated, while low-weight channels (such as normal skin and background noise channels) are suppressed. This mechanism uses weight screening to retain key feature channels related to diabetic foot lesion diagnosis and filter out irrelevant interference channels, thereby generating a channel attention map that focuses on lesion features and provides precise weight guidance for subsequent feature enhancement.

[0193] To this end, the above solution brings the following beneficial technical effects in the diabetic foot classification scenario:

[0194] 1. Traditional technologies often overlook bleeding point features in complex lesions with both large-area ulcers and localized bleeding points because a single linear layer cannot account for both global and local features. This dual-path mechanism in our solution extracts both the global distribution of ulcers and the local details of bleeding points, making the feature representation of complex lesions more comprehensive and significantly reducing feature omissions.

[0195] 2. In images with complex backgrounds such as medical equipment and bed sheets, traditional technologies have difficulty accurately distinguishing between background and lesion channels. This solution, through dual-path fusion and normalization, can effectively identify low-weight background channels and suppress their responses, significantly reducing the interference of background noise on lesion features and improving feature purity.

[0196] 3. For subtle lesions such as Wagner grade 1 microulcers, traditional methods are insensitive to local details due to their weight generation mechanism, which can easily lead to insufficient feature response. However, the local detail path in the dual path of this solution is highly sensitive to features such as microulcer edges. By strengthening its response intensity through weight distribution, the feature expression of subtle lesions is significantly improved.

[0197] 4. Traditional technologies use a fixed weight generation method and are unable to adapt to the characteristic differences between different lesion types. This solution dynamically fuses global and local features through dual paths, adaptively adjusting weight distribution for different lesion types such as ulcers and pigmentation. This makes the channel attention map more closely match the characteristic distribution of specific lesions, improving the adaptability of feature screening.

[0198] Optionally, when the attention enhancement module generates an attention enhancement feature map of the focused lesion area based on the channel attention map, the channel attention map is mapped to the cross-channel interaction space and the local correlation metric space to calculate the local feature similarity matrix and evaluate the correlation of the lesion area based on it, locate the focus label of the diabetic foot lesion area to generate an attention enhancement feature map of the focused lesion area.

[0199] In summary, this application introduces a dual-channel space mapping mechanism, which maps the channel attention map to the cross-channel interaction space and the local correlation metric space respectively. The cross-channel interaction space analyzes the synergistic relationship between the features of each channel, captures the feature combination of diabetic foot lesions (such as the co-activation pattern of the ulcer color channel and the edge texture channel), and forms an inter-channel correlation matrix for identifying feature combinations that are highly correlated with lesions. Based on the local correlation metric space, the feature similarity of each local area in the feature tensor is evaluated, and by calculating the pixel-level feature distance matrix, the areas with high feature similarity in the lesion area (such as the texture consistency of the ulcer core area) are located, and the feature differences between the lesion and normal skin are distinguished.

[0200] In addition, when evaluating the correlation of lesion areas, based on the calculation of the local feature similarity matrix, in the local correlation metric space, the feature similarity between each pixel and its neighboring pixels is calculated with each pixel as the center to generate a similarity matrix. Pixels within the lesion area show high similarity due to high feature consistency (such as texture repetition at the edge of the ulcer), while the junction of normal skin and the lesion area shows low similarity due to large feature differences. When evaluating the correlation of lesions, the channel correlation matrix and the local similarity matrix of the cross-channel interaction space are combined to comprehensively evaluate the correlation between each area and diabetic foot lesions - areas with high channel correlation and high local feature similarity are judged as lesion core areas, and low correlation areas are judged as background or normal skin.

[0201] Finally, a focused label generation and feature enhancement mechanism was introduced. When locating the focused label of the lesion area, a binary focused label was generated based on the correlation evaluation results. The lesion core and edge areas were marked as high activation, and the background and normal skin were marked as low activation. This label accurately locates the spatial position of the diabetic foot lesion in the feature map and suppresses interference from irrelevant areas. When generating the attention-enhanced feature map, the focused label was element-wise multiplied with the original feature tensor. The lesion features in the high-activation area were retained and enhanced, while the background features in the low-activation area were suppressed. Finally, an attention-enhanced feature map for the focused lesion area was generated, highlighting the feature expression of key diagnostic areas such as ulcers and bleeding points.

[0202] Therefore, when applied to the classification of diabetic foot, beneficial technical effects were achieved:

[0203] 1. Traditional techniques often confuse different lesion regions for complex lesions with ulcers and pigmentation because they cannot simultaneously process channel correlation and spatial distribution. This solution uses dual-channel spatial mapping to separately locate the spatial positions of ulcers (high color channel activation) and pigmentation (high texture channel activation), significantly improving the regional separation of complex lesions and significantly reducing feature confusion.

[0204] 2. For the fuzzy boundaries between ulcers and normal skin (such as the early infiltration area of ​​Wagner grade 1 ulcers), traditional methods lack local feature similarity analysis, which easily leads to the neglect of boundary area features. This solution uses a local similarity matrix to enhance the feature differences of boundary pixels, thereby enhancing the feature response strength of the lesion boundary and significantly improving boundary clarity, providing more accurate feature support for lesion grading.

[0205] 3. For small-scale lesions such as interdigital ulcers with smaller diameters, traditional technologies are prone to missed detection due to the lack of sensitivity of the attention mechanism to local features. This solution calculates pixel-level similarity in the local correlation metric space, is sensitive to the consistency of features within small-scale lesions, and can effectively capture their local features, significantly improving the detection capability of small-scale lesions and significantly reducing missed detections.

[0206] 4. In images with complex backgrounds such as medical equipment and bed sheets, traditional techniques have difficulty distinguishing the characteristic correlation between the background and the lesion. This solution filters background-related channels in a cross-channel interaction space and combines it with a local correlation metric space to suppress the characteristic response of the background area. This significantly reduces the interference of background noise on lesion characteristics and improves the purity of the feature map.

[0207] Optionally, when the classification module performs dynamic prototype aggregation processing on the attention enhancement feature map generated by the attention enhancement module to classify the diabetic foot image, the following steps are performed:

[0208] Metric support feature and query feature extraction are performed on the attention-enhanced feature map;

[0209] Generate enhanced perceptual masks based on the extracted metric support features and query features;

[0210] Generate dynamic category prototypes based on enhanced perceptual masks;

[0211] Dynamic category prototypes are fused to determine the classification of diabetic foot images.

[0212] Optionally, when the classification module measures support features and extracts query features from the attention-enhanced feature map, the following steps are performed:

[0213] Based on the constructed support feature extractor, the attention-enhanced feature map is measured to support feature extraction;

[0214] Based on the constructed query feature extractor, query features are extracted from the attention-enhanced feature map;

[0215] The extracted metric support features and query features are respectively embedded with the prior knowledge features of diabetic foot lesions to generate metric support features and query features guided by medical prior knowledge.

[0216] To this end, this application introduces a feature extraction mechanism guided by medical priors, and through independently constructed support feature extractors and query feature extractors, support features (representing known lesion categories) and query features to be classified (current image features to be diagnosed) for category prototype construction are extracted from the attention-enhanced feature map. The extraction process introduces prior knowledge of diabetic foot lesions, such as the clinical Wagner grading standard, lesion anatomical location features (morphological differences of plantar / interdigital ulcers), etc., to perform guided filtering on the features at the semantic level, strengthen key diagnostic features such as ulcer edges and bleeding points, and suppress irrelevant information such as normal skin texture. The lesion prior knowledge is converted into a weight mask on the feature dimension. For example, high weights are given to channels that meet the Wagner grade 2 ulcer characteristics, and low weights are given to feature channels that are not related to diabetic foot (such as background color), so that the extracted support features and query features are more in line with clinical diagnosis needs.

[0217] In addition, this application also introduces an enhanced perceptual mask generation mechanism, which calculates the feature similarity matrix of the supporting features and query features guided by medical priors to generate an enhanced perceptual mask. This mask locates the areas in the query features that are highly correlated with known lesion categories by measuring the semantic relevance between the query features and each supporting feature - high similarity areas correspond to the core features of the lesions (such as the matching of ulcer color and grading features), and low similarity areas correspond to background or atypical features. The mask generation process combines prior knowledge of lesions, prioritizes the activation of feature associations that conform to clinical diagnostic logic, and suppresses abnormal feature matching.

[0218] In this application, a dynamic category prototype generation mechanism is introduced to use enhanced perception masks to perform weighted aggregation on supporting features to generate dynamic category prototypes. Unlike traditional static prototypes, this prototype is adaptively adjusted according to the lesion characteristics of the current query feature: if the query feature contains early ulcer features, the prototype will strengthen the feature expression of small ulcers; if it contains infection features, the weight of related features such as pus will be enhanced to make the prototype closer to the feature distribution of the current case to be classified. The generation of dynamic prototypes is constrained by the prior knowledge of diabetic foot lesions. For example, the prototype features must conform to clinical laws such as "the coexistence of color features and edge features in the ulcer area" to avoid generating prototypes that do not conform to medical logic.

[0219] In this application, a prototype fusion and classification mechanism is introduced to calculate the similarity between the dynamically generated prototypes of each category and the query features, and the final classification result is determined by weighted fusion. The fusion process combines the category association in the prior knowledge of the lesion (such as the feature overlap between Wagner grade 2 and grade 3 ulcers), assigns differential weights to similar category prototypes, and finally outputs a diabetic foot lesion classification that conforms to clinical diagnostic logic (such as ulcer type, grade, and whether there is infection).

[0220] To this end, the above technical solution has the following technical advantages when classifying diabetic foot:

[0221] 1. Traditional classification methods, due to their static prototypes, cannot capture the subtle features of early lesions and are prone to misclassifying Wagner grade 1 ulcers as normal skin. This solution uses medical priors to extract early lesion features (such as ulcer margins less than 0.5 mm) and combines this with adaptive dynamic prototype adjustment to significantly improve the accuracy of early lesion classification and reduce misclassifications.

[0222] 2. For complex cases with both ulcers and vascular lesions, traditional technologies often lead to classification confusion due to their inability to handle the medical associations between features. This solution significantly improves the classification clarity of complex lesions and reduces feature confusion by enhancing the feature association of "ulcer color-vascular texture" through perceptual masking and aggregating complex features through dynamic prototypes.

[0223] 3. Images collected by equipment from different hospitals vary in illumination and resolution. Traditional technologies, due to static prototypes, cannot adapt to feature shifts, resulting in poor classification stability. This solution embeds medical prior knowledge (such as the anatomical location of lesions and medical color standards) to filter out interference caused by equipment differences. Combined with the dynamic prototype's adaptive adjustment of feature shifts, the classification robustness of cross-center data is significantly enhanced, and the consistency of classification results is improved.

[0224] 4. The prototype construction of traditional classification methods lacks medical logic, making the results difficult to interpret. This solution introduces prior knowledge of lesions to ensure that the feature composition of the dynamic prototype meets clinical diagnostic standards (for example, Wagner grade 3 ulcers must include dermal exposure features). The classification process can be traced back to specific medical feature associations, significantly improving the clinical interpretability of the classification results and better aligning them with doctors' diagnostic habits.

[0225] Optionally, when the classification module generates the enhanced perception mask based on the extracted metric support features and query features, the following steps are performed:

[0226] Reshape the metric support features and query features to calculate the association strength of each spatial position between the metric support features and the query features;

[0227] Based on the correlation strength, a neighborhood index matrix of the spatial position is constructed, and the neighborhood index matrix is ​​Gaussian smoothed to obtain a local smooth correlation matrix;

[0228] The local smoothed correlation matrix is ​​used to map the lesion region activations and generate an enhanced perceptual mask.

[0229] In summary, this application introduces a feature association strength calculation mechanism to reshape the spatial dimension of the metric support features (representing known lesion categories) and query features (image features to be classified) extracted from the attention enhancement feature map, so that the association strength of the two can be calculated position by position in the feature space. Specifically, after converting the support features and the query features into vector form, the feature matching degree of each spatial position is calculated by cosine similarity or Euclidean distance-high association strength corresponds to the core area of ​​the lesion (such as the matching of ulcer color and known ulcer features), and low association strength corresponds to the background or non-lesion area. This process captures the spatial correspondence between the query feature and the supporting feature, providing a basis for locating the lesion area.

[0230] Furthermore, during neighborhood index matrix construction and smoothing, a neighborhood index matrix is ​​constructed for each pixel based on the strength of spatial correlation, representing the correlation between the pixel and the surrounding area. Pixels within the lesion region exhibit high correlation values ​​due to high feature consistency (e.g., similar texture at the ulcer edge). The boundary between the lesion and normal skin exhibits low correlation values ​​due to significant feature differences.

[0231] During Gaussian smoothing, this application applies a Gaussian kernel function to the neighborhood index matrix to reduce noise interference and enhance the continuity of regional features. This smoothing generates a local smoothed correlation matrix, which makes the boundaries of the lesion area clearer and avoids positioning deviations caused by minor feature fluctuations.

[0232] Furthermore, when generating activation maps and masks for lesion areas, this application generates activation maps for lesion areas through threshold segmentation or activation function mapping based on the local smooth correlation matrix. High-correlation areas (lesion core areas) are marked as high activation, and low-correlation areas (background or normal skin) are marked as low activation. The activation map is binarized to generate an enhanced perception mask, which accurately locates the lesion areas in the query features that are highly correlated with the supporting features, suppresses irrelevant background interference, and provides spatial weight guidance for dynamic prototype generation.

[0233] Therefore, the following technical advantages are available when classifying diabetic foot:

[0234] 1. Traditional classification methods, lacking spatial correlation analysis, are unable to distinguish the locational differences between interdigital ulcers and plantar ulcers, often leading to grading errors. This approach, through position-by-position correlation strength calculation, can precisely locate the anatomical location of the lesion on the foot, significantly improving the accuracy of positional differentiation and reducing classification errors caused by misjudgment of location.

[0235] 2. For the blurred boundaries between ulcers and normal skin (such as the infiltrated area of ​​Wagner grade 1 ulcers), traditional technologies lack neighborhood smoothing, making boundary positioning susceptible to pixel noise. This solution uses Gaussian smoothing to enhance the feature continuity of the boundary area, significantly improving the clarity of the ulcer edge and providing more accurate spatial feature support for lesion grading.

[0236] 3. In images with complex backgrounds such as bed sheet textures and sock patterns, traditional technologies have difficulty distinguishing the spatial correlation between the background and the lesion. This solution, through the combination of a neighborhood index matrix and Gaussian smoothing, can effectively suppress the correlation strength of the background area, making the lesion area stand out from the complex background. The interference of background noise is significantly reduced, and the accuracy of lesion segmentation is improved.

[0237] 4. Traditional activation mapping mechanisms (such as global threshold segmentation) are simple and cannot distinguish the characteristic differences between the core and edge areas of the ulcer. However, this scheme generates activation maps based on correlation strength, which can hierarchically activate different areas—high activation in the core area, moderate activation in the edge area, and low activation in the background area. This makes the hierarchical features of the lesion area more clearly expressed, provides richer spatial dimension information for classification, and improves classification accuracy.

[0238] Optionally, when the classification module generates a dynamic category prototype based on the enhanced perceptual mask, the following steps are performed:

[0239] Perform spatial dimension mask expansion on the enhanced perception mask to generate initial category prototypes;

[0240] Performing principal component analysis and dimensionality reduction on the initial category prototype to construct the feature subspace of the dynamic category prototype;

[0241] The inter-class distance of the initial category prototype is maximized in the feature subspace and dynamically adapted according to the lesion stage to generate a dynamic category prototype.

[0242] In summary, in this application, when the spatial dimension mask is expanded and the initial prototype is constructed, the spatial dimension of the enhanced perception mask is expanded, the two-dimensional spatial mask is mapped to the feature dimension, and the initial category prototype is generated. Specifically, based on the features of the high-activation area (lesion core area) in the mask, the spatial features and channel features of the lesion area are integrated by weighted aggregation - the features of the high-activation area (such as ulcer color, edge texture) are prioritized. The features of the low-activation area (background) are suppressed to form a preliminary category prototype to ensure that the prototype contains the most diagnostically valuable lesion features.

[0243] Furthermore, during principal component analysis (PCA) dimensionality reduction and feature subspace construction, principal component analysis (PCA) is performed on the initial class prototype to identify the main directions of variation in the features and construct a feature subspace for the dynamic class prototype. This subspace retains the feature dimensions most relevant to diabetic foot lesions (such as ulcer shape and bleeding point texture) while removing noisy or redundant dimensions (such as irrelevant background color), allowing the prototype features to focus more on key diagnostic information. The dimensionality reduction process retains the principal components that explain the majority of feature variation, ensuring that the prototype can still fully express the lesion characteristics in the low-dimensional space.

[0244] When maximizing inter-class distance and dynamically adapting to lesion stages, the prototype distribution is optimized in the feature subspace to maximize the distance between prototypes of different lesion categories (such as Wagner grade 1 and grade 2 ulcers), thereby enhancing the distinguishability between categories. By adjusting the feature weights of the prototypes, the characteristic differences between different lesion stages (such as ulcer depth and tissue damage range) are highlighted to avoid category confusion. Based on the lesion characteristics of the query feature, the prototype's feature subspace and inter-class distance are dynamically adjusted. For early lesions (such as Wagner grade 1), the prototype strengthens the expression of subtle features (such as punctate hemorrhages); for late lesions (such as Wagner grade 4), the prototype emphasizes large-area ulcers and infection features. This allows the prototype to adapt to the feature distribution of different lesion stages and improves classification flexibility.

[0245] Therefore, the following technical advantages are available when classifying diabetic foot:

[0246] 1. Traditional classification methods often misclassify grade 1 and grade 2 ulcers because static prototypes cannot capture the characteristic differences between Wagner grades. This solution maximizes inter-class distances and dynamically adapts to enhance the characteristic differences between lesions at different stages (such as ulcer depth and edge morphology), significantly improving the accuracy of lesion stage differentiation and reducing classification errors.

[0247] 2. When there are few labeled samples, the static prototypes of traditional technologies are susceptible to sample bias and have poor classification stability. This solution uses PCA dimensionality reduction to retain key lesion features and dynamically optimizes the prototype distribution, allowing the prototype to accurately express lesion categories in small sample scenarios. This significantly enhances generalization capabilities and improves the reliability of classification results.

[0248] 3. For complex cases of combined ulcers and neuropathy, traditional technical prototypes are unable to separate the characteristics of different lesions. This solution prioritizes the aggregation of core features of each lesion through spatial masking and optimizes the distribution of prototypes of different categories in the feature subspace, so that the characteristics of the complex lesions can be accurately expressed, feature confusion is significantly reduced, and classification clarity is significantly improved.

[0249] 4. Traditional prototype construction does not filter out noise features, which are easily affected by factors such as shooting angle and equipment differences. This solution uses PCA dimensionality reduction to eliminate noise dimensions unrelated to the lesion, combined with dynamic adaptation to suppress feature offsets caused by equipment differences. This significantly improves the purity of prototype features, significantly enhances anti-interference capabilities, and improves the classification stability of cross-center data.

[0250] Optionally, when the classification module fuses the dynamic category prototypes to determine the classification of the diabetic foot image, the following steps are performed:

[0251] Calculate the feature covariance matrix between dynamic category prototypes;

[0252] Calculate the similarity between the dynamic category prototype and the query feature based on the feature covariance matrix;

[0253] The dynamic category prototypes are fused based on similarity to generate weighted category representations;

[0254] Class-category decision boundaries were constructed based on statistically weighted class representation to determine the classification of diabetic foot images.

[0255] In summary, this application introduces a feature covariance matrix calculation mechanism to perform feature space analysis on dynamic category prototypes and calculate the feature covariance matrix between each prototype. This matrix captures the statistical dependency between features of different lesion categories (such as Wagner classification) - high covariance indicates that similar features are shared between categories (such as ulcer edge texture), and low covariance indicates that the category features are significantly different (such as the tissue damage pattern of deep ulcers and superficial ulcers). Through covariance analysis, the feature correlation between categories is quantified, providing a statistical basis for subsequent similarity calculations.

[0256] This application calculates the similarity between each dynamic category prototype and the query feature based on the feature covariance matrix when calculating similarity and assigning dynamic weights. Unlike traditional Euclidean distance or cosine similarity, this calculation takes into account the feature correlation between categories - the feature space is whitened through the covariance matrix, so that the similarity metric focuses more on the unique features between categories rather than shared features. For example, when distinguishing between Wagner grade 2 and grade 3 ulcers, the similarity calculation will strengthen the weight of specific features such as "whether there is bone exposure", weaken the influence of common features such as "redness and swelling of the ulcer edge", and improve the ability to distinguish categories.

[0257] In the present application, when fusing dynamic prototypes and weighted category representations, the dynamic category prototypes are weightedly fused according to the similarity calculation results to generate a comprehensive weighted category representation. Prototypes with high similarity (such as Wagner 2-level prototypes that match the query features) have a dominant weight in the fusion, and the weights of prototypes with low similarity (such as Wagner 4-level prototypes) are suppressed. This process dynamically integrates the information of multiple prototypes so that the final representation can retain the main category features of the query features and include supplementary information of adjacent categories (such as the features of the transition from early lesions to late lesions), forming a more comprehensive expression of lesion features.

[0258] When constructing and classifying decision boundaries, this application constructs a statistical decision boundary based on weighted category representation, and determines the classification threshold by analyzing the probability density function of the feature distribution. The construction of the decision boundary takes into account the feature covariance and similarity distribution between categories, so that the boundary is closer to the natural separation between actual lesion categories - for example, in the characteristic transition area between Wagner grade 2 and grade 3 ulcers, the decision boundary will be flexibly divided according to the gradual change law of clinical characteristics to avoid classification errors caused by hard thresholds. Ultimately, the query feature is assigned to the most matching diabetic foot lesion category based on its position in the decision boundary.

[0259] To this end, the following technical benefits are available when classifying diabetic foot

[0260] 1. Traditional classification methods, due to their fixed decision boundaries, have difficulty distinguishing the transition between Wagner grade 2 and grade 3 ulcers, often leading to grade jumps. This approach, through the construction of a flexible decision boundary, can accurately classify adjacent lesion stages based on the gradual change of features, significantly improving the accuracy of stage differentiation and reducing classification errors.

[0261] 2. For complex cases with both ulcers and infections, traditional technologies cannot integrate the characteristics of different lesions due to independent processing of prototypes. This solution can effectively combine ulcer morphology and infection characteristics (such as the extent of redness and swelling, and the amount of exudate) through dynamic prototype fusion, making the characteristic expression of complex lesions more comprehensive and significantly improving the completeness of classification.

[0262] 3. When labeled samples are limited, traditional static prototypes are prone to overfitting. This solution, based on statistical modeling of feature covariance, can explore the essential connections between categories from a small number of samples, enabling the classifier to maintain high stability in small sample scenarios, significantly enhancing generalization capabilities and reducing classification fluctuations caused by insufficient samples.

[0263] 4. Traditional classification decision boundaries are out of line with clinical diagnostic standards. This solution uses statistical probability density functions to construct decision boundaries that conform to the progressive nature of lesions, making classification results more closely aligned with clinical reality. For example, in the process of early ulcers progressing to deep ulcers, the classification results can reflect the progressive nature of the lesions, significantly improving consistency with doctors' diagnostic logic and enhancing diagnostic reference value.

[0264] In order to facilitate understanding of the above content of this application, the following examples are given for specific explanation:

[0265] Take the classification process of a Wagner grade 2 diabetic foot ulcer as an example:

[0266] 1. Data Enhancement Module: Building High-Quality Training Datasets

[0267] Base class image screening and small sample paradigm construction

[0268] Typical Wagner grade 2 ulcer images were selected from the image library as the base class (features: exposed dermis, redness and swelling at the ulcer edge, and no bone exposure). A 5-way 5-shot small-sample learning framework (five types of lesions, five annotated images per type) was constructed to simulate the few-shot scenario during initial clinical diagnosis.

[0269] Similar image sampling and weighted fusion

[0270] For a training image (ulcer on the middle part of the plantar, area about 10mm 2 ), extract its geometric features (area A = 10, perimeter P = 12), color features (HSV hue of ulcer area H = 15, S = 0.8, V = 0.6), and texture features (edge ​​roughness R = 0.7) to form an 18-dimensional feature vector F = [10, 12, ... 0.7].

[0271] Three similar images were retrieved in the feature space through KD-Tree, and the Euclidean distances between their feature vectors and F were 0.5, 0.6, and 0.7 respectively (the distance was less than the threshold of 1.5×feature standard deviation).

[0272] Lesion saliency map (mask example): The binary mask M generated by the segmentation model has the same size as the image, with the pixel value of the ulcer area being 1 and the background being 0. For example, the mask of a 5×5 area is: 0 0 1 0 0 0 1 1 1 0 1 1 1 1 1 0 1 1 1 0 0 0 1 0 0

[0278] Dynamic weighted fusion: For each pixel (x, y), if M(x, y) = 1 (ulcer area), the fusion weight α = 0.8; otherwise α = 0.3, generating a deformed image I_{aug}(x, y) = α·I_train(x, y) + (1-α)·I_similar(x, y).

[0279] 2. Feature Extraction Module: Multi-Scale Feature Fusion

[0280] ResNet-101 backbone network feature extraction

[0281] The shallow layer captures the pixel-level texture of the ulcer edge, and the deep layer integrates the overall morphology of the ulcer.

[0282] After the input image is processed by ResNet-101, the output is a three-dimensional feature vector H×W×C=32×32×2048, where the local area of ​​a channel feature map is as follows: [[0.2 0.5 0.7] [0.3 0.8 0.6] [0.1 0.7 0.9]]

[0286] Spatial pyramid features and local descriptor generation

[0287] Example of a multi-scale feature map: In the feature map generated by a 3×3 convolution kernel, the feature value of the punctate bleeding at the edge of the ulcer captured in a certain 3×3 region is [0.9, 0.8, 0.7]. In the feature map generated by a 7×7 convolution kernel, the feature value of the integrated ulcer position in the corresponding region is [0.6, 0.7, 0.5].

[0288] Spatial pyramid features (4×4 grid example): The feature map is divided into 4×4 grids, and maximum pooling is performed within each grid. The pooling result of a certain grid is 0.8 (indicating that the edge features in this area are significant).

[0289] Local descriptor generation: The feature map is segmented using a 16×16 sliding window. The features within a certain window are compressed into a 128-dimensional vector D = [0.3, 0.5, ... 0.7] using a fully connected layer, corresponding to the color and texture features of a certain part of the ulcer edge.

[0290] Example of semantic self-attention weight matrix: Calculate the attention matrix A between descriptors, where A[i,j] = 0.9, indicating that the i-th descriptor (red edge) is highly correlated with the j-th descriptor (rough texture), strengthening their association.

[0291] 3. Attention Enhancement Module: Focusing on the Characteristics of Lesion Area

[0292] Adaptive feature reweighting

[0293] Cross-channel autocorrelation matrix example: Calculate the autocorrelation matrix R of the color channel C1 and the edge channel C2:

[0294] R=[[1.0 0.8] [0.8 1.0]]

[0296] Indicates that the two channel features are highly correlated.

[0297] Example of a local entropy vector: The entropy calculation result of a 5×5 area is E = [0.9, 0.8, 0.7, 0.6, 0.5]. The entropy value of the ulcer area is high (0.9), and the entropy value of normal skin is low (0.5).

[0298] Example of a channel weight vector: After concatenating R and E and performing gating, a weight vector W = [0.9, 0.8, 0.3, 0.2] is generated. The first two dimensions are the weights of the lesion channel, and the last two dimensions are the weights of the background channel.

[0299] Channel attention map generation

[0300] Example of a dual-path output vector: global path output G = [0.7, 0.6, 0.5], local path output L = [0.9, 0.8, 0.7], which are added and normalized by Sigmoid to obtain the channel attention map A = [0.8, 0.7, 0.6], with high weights activating lesion-related channels.

[0301] Feature enhancement of focused lesion areas

[0302] Local feature similarity matrix example: Calculate the pixel similarity matrix S in a 5×5 area: [[1.0 0.9 0.8 0.7 0.6] [0.9 1.0 0.9 0.8 0.7] [0.8 0.9 1.0 0.9 0.8] [0.7 0.8 0.9 1.0 0.9] [0.6 0.7 0.8 0.9 1.0]]

[0308] The pixel similarity in the ulcer core area is high (the value near the diagonal is large).

[0309] Focus label (mask example): After Gaussian smoothing, a threshold of 0.8 is set to generate a binary mask L, with the ulcer core area as 1 and the background as 0: [[0 0 1 0 0] [0 1 1 1 0] [1 1 1 1 1] [0 1 1 1 0] [0 0 1 0 0]]

[0315] 4. Classification Module: Dynamic Prototype Aggregation and Classification Decision

[0316] Feature extraction guided by medical priors

[0317] Example of supporting features: Given the supporting features S of Wagner grade 2 ulcer, S = [0.8 (exposed dermis), 0.7 (red and swollen edge), 0.2 (normal skin)], the weights after embedding prior are [0.9, 0.8, 0.1].

[0318] Enhanced perceptual mask generation

[0319] Example of correlation strength matrix: Calculate the correlation strength matrix C between the query feature and the supporting features. The value of a 3×3 area is: [[0.9 0.8 0.7] [0.8 0.9 0.8] [0.7 0.8 0.9]]

[0323] The association strength was high in the ulcer area.

[0324] Neighborhood index matrix example: Construct a 3×3 neighborhood matrix N, and the associated index between the center pixel and the neighborhood is: [[0.8 0.9 0.8] [0.9 1.0 0.9] [0.8 0.9 0.8]]

[0328] After Gaussian smoothing, the activation map is generated and converted into an enhanced perceptual mask.

[0329] A threshold (e.g., 0.8) is set for the smoothed correlation matrix, and regions above the threshold are marked as 1 (lesion activation) and regions below the threshold are marked as 0 (background suppression), generating an enhanced perception mask L. For example, the mask shape of a 5×5 region is: [[0 0 1 0 0] [0 1 1 1 0] [1 1 1 1 1] [0 1 1 1 0] [0 0 1 0 0]]

[0335] Dynamic category prototype generation

[0336] Initial prototype example: Based on mask expansion, a 10-dimensional initial prototype P = [0.7, 0.8, 0.6, 0.5, 0.9, 0.8, 0.7, 0.6, 0.5, 0.4] is generated. After PCA dimensionality reduction, the first three principal components are retained to form a 3D feature subspace.

[0337] Prototype fusion and classification decision

[0338] Example of feature covariance matrix: Covariance matrix Σ for Wagner 2-level and 3-level prototypes:

[0339] Σ=[[1.0 0.5] [0.5 1.0]]

[0341] Indicates the correlation between two types of features.

[0342] Similarity calculation example: The similarity between the query feature Q and the Wagner 2nd level prototype is 0.9, and the similarity with the 3rd level prototype is 0.4. After weighted fusion, the Wagner 2nd level weight accounts for 70%. The final classification result is:

[0343] Wagner grade 2 diabetic foot ulcer with exposed dermis and marginal redness and swelling, without exposed bone or deep infection.

[0344] Decision boundary visualization: In the 3D feature subspace, the peak of the probability density function of Wagner level 2 is located at the coordinates (0.7, 0.8, 0.6). The coordinates of the query feature Q (0.68, 0.79, 0.58) fall within 1.5 standard deviations of this peak, so it is determined to be Wagner level 2.

[0345] like Figure 2 As shown, the embodiment of the present application provides a diabetic foot image classification method based on small sample learning, including:

[0346] Perform weighted fusion of training images and similar images on diabetic foot images to generate an expanded dataset;

[0347] Perform ResNet-101 backbone network feature extraction on the expanded dataset to generate feature tensors that include global semantics and local details;

[0348] Perform dual-pool channel attention and enhanced spatial attention processing on the feature tensor to generate an attention-enhanced feature map focusing on the lesion area;

[0349] Dynamic prototype aggregation of attention-enhanced feature maps for classification of diabetic foot images.

[0350] above Figure 2 For an exemplary introduction to each step in Figure 1 Description of Examples.

[0351] The above embodiments are only used to illustrate the embodiments of the present invention, and are not intended to limit the embodiments of the present invention. Ordinary technicians in the relevant technical field can make various changes and modifications without departing from the spirit and scope of the embodiments of the present invention. Therefore, all equivalent technical solutions also fall within the scope of the embodiments of the present invention, and the scope of patent protection of the embodiments of the present invention should be defined by the claims. The systems, devices, modules or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions.

Claims

1. A diabetic foot image classification system based on small sample learning, comprising: The data enhancement module is used to perform weighted fusion of training images and similar images on diabetic foot images to generate an expanded data set; The feature extraction module is used to perform ResNet-101 backbone network feature extraction on the expanded dataset generated by the data enhancement module to generate a feature tensor containing global semantics and local details; The attention enhancement module is used to perform dual-pool channel attention and enhanced spatial attention processing on the feature tensor generated by the feature extraction module to generate an attention-enhanced feature map focusing on the lesion area; The classification module is used to perform dynamic prototype aggregation processing on the attention-enhanced feature map generated by the attention-enhancing module to classify diabetic foot images.

2. The diabetic foot image classification system based on small sample learning according to claim 1, characterized in that: The data enhancement module performs the following steps when processing the diabetic foot image by weighted fusion of the training image and the similar image to generate an expanded data set: Training images and similar images are sampled from diabetic foot images, and weighted fusion is performed on the training images and similar images to generate deformed images and thereby generate an expanded data set, which includes: deformed images and diabetic foot images.

3. The diabetic foot image classification system based on small sample learning according to claim 2, characterized in that: The data enhancement module samples training images and similar images from diabetic foot images, performs weighted fusion on the training images and similar images to generate deformed images and generates an expanded data set based on the deformed images and diabetic foot images, and performs the following steps: Determine the base class images of diabetic foot images and construct a small sample learning paradigm based on the base class images; Based on the small sample learning paradigm, diabetic foot images are sampled to obtain training images; Perform neighbor retrieval in feature space based on training images to determine lesion features and generate similar images based on them; The training image and similar images are linearly superimposed with weights to generate deformed images and thus generate an expanded dataset.

4. The diabetic foot image classification system based on small sample learning according to claim 1, characterized in that: When the data augmentation module performs weighted linear superposition on the training image and the similar image to generate the deformed image and generates the expanded data set based on the deformed image, the following steps are performed: Perform pixel-level fusion with weights on the training image and similar images to generate a preliminary fused image; Performing lesion area mask weighting processing on the generated primary fusion image to generate a deformed image; Lesion semantic annotation is performed on deformed images and diabetic foot images to generate an expanded dataset.

5. The diabetic foot image classification system based on small sample learning according to claim 1, characterized in that: The feature extraction module performs ResNet-101 backbone network feature extraction processing on the expanded data set generated by the data enhancement module to generate a feature tensor containing global semantics and local details, performing the following steps: Based on the ResNet-101 backbone network, the feature extraction of the expanded data set is performed with three-dimensional feature vectors. Where H / W is the feature map size and C is the number of channels; The three-dimensional feature vector is decomposed into multiple local descriptors, each of which corresponds to the fine-grained features of the image block to generate a feature tensor that includes global semantics and local details.

6. The diabetic foot image classification system based on small sample learning according to claim 5, characterized in that: The feature extraction module decomposes the three-dimensional feature vector into multiple local descriptors, each of which corresponds to a fine-grained feature of an image block, to generate a feature tensor that includes global semantics and local details. The following steps are performed: Convolution operations are performed on the three-dimensional feature vector using convolution kernels with different receptive fields to generate a multi-scale feature map group; The feature map at each scale is divided into grids of different sizes, and the features within each grid are subjected to maximum pooling or average pooling to generate spatial pyramid features; The spatial pyramid features are divided into sliding windows of fixed size, and the features in each window are compressed into a vector of fixed dimension through a fully connected layer to generate multiple local descriptors; Calculate the semantic self-attention weights between all local descriptors to generate semantic correlation features; Perform global average pooling and maximum pooling on the semantic association features to obtain global semantic features; The global semantic features are upsampled to generate a feature tensor that includes global semantics and local details.

7. The diabetic foot image classification system based on small sample learning according to claim 1, characterized in that: The attention enhancement module performs dual-pool channel attention and enhanced spatial attention processing on the feature tensor generated by the feature extraction module to generate an attention-enhanced feature map focusing on the lesion area, performing the following steps: Generate adaptive feature reweighting results based on the feature tensor generated by the feature extraction module; Generate a channel attention map based on the adaptive feature reweighting results; Generate an attention-enhanced feature map focusing on the lesion area based on the channel attention map.

8. The diabetic foot image classification system based on small sample learning according to claim 7, characterized in that: When the attention enhancement module generates an adaptive feature reweighting result based on the feature tensor generated by the feature extraction module, the following steps are performed: The feature tensor generated by the feature extraction module is simultaneously subjected to cross-channel autocorrelation calculation and local feature entropy evaluation to obtain a channel autocorrelation matrix group and a channel entropy value vector; The channel autocorrelation matrix group and the channel entropy value vector are concatenated and processed through attention gating to generate a channel weight vector; The feature tensor is scaled using the channel weight vector and adaptive feature reweighting is obtained accordingly.

9. The diabetic foot image classification system based on small sample learning according to claim 7, characterized in that: When the attention enhancement module generates a channel attention map based on the adaptive feature reweighting result, the following steps are performed: The adaptive feature reweighting results are processed through independent linear layers to obtain a dual-path output vector; The dual-path output vectors are added element-by-element and normalized by the Sigmoid function to generate key feature channel weights, and irrelevant feature channels including feature channels corresponding to the background are suppressed to generate a channel attention map based on this.

10. The diabetic foot image classification system based on small sample learning according to claim 7, characterized in that: When the attention enhancement module generates an attention enhancement feature map focused on the lesion area based on the channel attention map, the channel attention map is mapped to the cross-channel interaction space and the local correlation metric space to calculate the local feature similarity matrix and evaluate the correlation of the lesion area based on it, locate the focus label of the diabetic foot lesion area to generate an attention enhancement feature map focused on the lesion area.

Images