Skin disease image classification method and device based on multi-dimensional aggregation mechanism, equipment, storage medium and computer program product

Abstract

The invention relates to the technical field of artificial intelligence, in particular to a skin disease image classification method and device based on a multi-dimensional aggregation mechanism, equipment, a storage medium and a computer program product. The method comprises the following steps: acquiring to-be-classified skin image data, and performing feature extraction on the skin image data to obtain initial features; spatial gating processing is applied to the initial features to obtain spatial gating features; inputting the spatial gating features into a plurality of depth separable convolution paths with different expansion rates, extracting multi-scale features, and fusing the multi-scale features to obtain multi-order aggregation features; applying channel gating processing to the multi-order aggregation feature to obtain a channel gating feature; single-head attention processing is applied to partial channels of the channel gating features, and fusion features are obtained; and performing linear mapping processing on the fusion feature to obtain a classification vector, and inputting the classification vector into a preset classification model to obtain a target classification result, thereby improving the classification accuracy of the skin disease image.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a method, apparatus, device, storage medium, and computer program product for classifying skin disease images based on a multidimensional aggregation mechanism. Background Technology

[0002] With the development of deep learning technology, intelligent analysis of medical images has become an important research direction in the field of artificial intelligence. Among these, the automatic classification of skin disease images, as a key step in assisted diagnosis, has received widespread attention. Traditional skin disease image classification methods mainly rely on manual feature extraction and shallow classifiers. These methods depend on human experience, have poor robustness, and have limited ability to represent complex lesion textures and blurred boundaries, resulting in poor classification accuracy for skin disease images. Therefore, how to improve the classification accuracy of skin disease images has become an urgent technical problem to be solved. Summary of the Invention

[0003] The main objective of this application is to provide a method, apparatus, device, storage medium, and computer program product for classifying skin disease images based on a multidimensional aggregation mechanism, aiming to solve the technical problem of how to improve the classification accuracy of skin disease images.

[0004] To achieve the above objectives, this application provides a skin disease image classification method based on a multidimensional aggregation mechanism, the method comprising the following steps: Acquire skin image data to be classified, and extract features from the skin image data to obtain initial features; Spatial gating is applied to the initial features to obtain spatially gated features; The spatial gated features are input into multiple depthwise separable convolutional paths with different dilation rates to extract multi-scale features and fuse them to obtain multi-level aggregated features. Channel gating is applied to the multi-level aggregated features to obtain channel-gated features; Single-head attention processing is applied to a portion of the channels of the channel-gated feature to obtain the fused feature; The fused features are linearly mapped to obtain a classification vector, which is then input into a preset classification model to obtain the target classification result.

[0005] In one embodiment, the step of applying single-head attention processing to a portion of the channels of the channel-gated features to obtain fused features includes: The channel gating features are used to divide the channels and determine the target channel set; Perform a linear mapping on the target channel set to generate a query vector, a key vector, and a value vector; The attention weight is determined based on the correlation between the query vector and the key vector; The attention weights are weighted together with the value vector to obtain a weighted output, and the weighted output is fused with the residual path to generate the fused feature.

[0006] In one embodiment, the step of inputting the spatially gated features into multiple depthwise separable convolutional paths with different dilation rates, extracting multi-scale features, and fusing them to obtain multi-level aggregated features includes: The spatial gated features are input into multiple parallel depthwise separable convolutional paths, wherein the kernel size of each depthwise separable convolutional path is the same, but the dilation rate is different. Spatial alignment and dimensionality unification are performed on the output features of the separable convolutional paths at each depth. The multi-level aggregated features are obtained by fusing the results of spatial alignment and dimensional unification.

[0007] In one embodiment, the step of applying spatial gating processing to the initial features to obtain spatially gated features includes: The initial features are subjected to global average pooling to generate the corresponding spatial global description vector; Based on the spatial global description vector, a spatial weight map is generated by combining a preset convolution operation with a scaling parameter. The spatial weight map and the initial feature are multiplied element-wise to obtain the spatial gating feature.

[0008] In one embodiment, the step of applying channel gating processing to the multi-level aggregated features to obtain channel-gated features includes: The multi-level aggregated features are subjected to global average pooling along the channel dimension to generate a channel global description vector; The channel global description vector is input into a convolution operation module that matches the number of channels of the multi-level aggregated feature to obtain the channel weight vector; The channel weight vector and the multi-level aggregated feature are multiplied one channel at a time along the channel dimension to obtain the channel-gated feature.

[0009] In one embodiment, the step of performing linear mapping processing on the fused features to obtain a classification vector, and inputting the classification vector into a preset classification model to obtain a target classification result includes: The fused features are flattened to obtain one-dimensional features; The one-dimensional feature is input into a fully connected layer and linearly mapped to generate the classification vector. The classification vector is normalized to obtain a normalized vector; The normalized vector is input into a preset classification model to obtain the target classification result.

[0010] Furthermore, to achieve the above objectives, this application also proposes a skin disease image classification device based on a multidimensional aggregation mechanism, the skin disease image classification device based on a multidimensional aggregation mechanism comprising: The initial feature module is used to acquire skin image data to be classified and to extract features from the skin image data to obtain initial features; The spatial gating module is used to apply spatial gating processing to the initial features to obtain spatially gating features; The multi-level aggregation module is used to input the spatial gated features into multiple depthwise separable convolutional paths with different dilation rates, extract multi-scale features and fuse them to obtain multi-level aggregated features; The channel gating module is used to apply channel gating processing to the multi-level aggregated features to obtain channel-gated features; The fusion module is used to apply single-head attention processing to a portion of the channels of the channel-gated feature to obtain the fused feature; The target module is used to perform linear mapping processing on the fused features to obtain a classification vector, and input the classification vector into a preset classification model to obtain the target classification result.

[0011] Furthermore, to achieve the above objectives, this application also proposes a skin disease image classification device based on a multidimensional aggregation mechanism. The device includes: a memory, a processor, and a skin disease image classification program based on a multidimensional aggregation mechanism stored in the memory and executable on the processor. The skin disease image classification program based on a multidimensional aggregation mechanism is configured to implement the steps of the skin disease image classification method based on a multidimensional aggregation mechanism as described above.

[0012] Furthermore, to achieve the above objectives, this application also proposes a storage medium storing a skin disease image classification program based on a multidimensional aggregation mechanism. When the skin disease image classification program based on the multidimensional aggregation mechanism is executed by a processor, it implements the steps of the skin disease image classification method based on the multidimensional aggregation mechanism described above.

[0013] In addition, to achieve the above objectives, this application also proposes a computer program product comprising a computer program that, when executed by a processor, implements the steps of the skin disease image classification method based on the multidimensional aggregation mechanism described above.

[0014] This application acquires skin image data to be classified and extracts features from the skin image data to obtain initial features. Spatial gating is applied to the initial features to obtain spatially gated features. These spatially gated features are then input into multiple depthwise separable convolutional paths with different dilation rates to extract multi-scale features, which are then fused to obtain multi-level aggregated features. Channel gating is applied to these multi-level aggregated features to obtain channel-gated features. Single-head attention is applied to some channels of the channel-gated features to obtain fused features. Linear mapping is then applied to the fused features to obtain a classification vector, which is then input into a preset classification model to obtain the target classification result. This application sequentially uses spatial gating to select key regions of the initial features, multi-dilation rate convolutional paths to extract and fuse multi-scale features, channel gating to enhance classification-related channel features, and single-head attention to highlight global information in key channels. This approach suppresses irrelevant background and redundant channel interference while simultaneously considering the expression and fusion of detailed and overall features, thereby improving the classification accuracy of skin disease images. Attached Figure Description

[0015] Figure 1 This is a flowchart illustrating the first embodiment of the skin disease image classification method based on a multidimensional aggregation mechanism of this application; Figure 2 This is a schematic diagram of a sub-process in the second embodiment of the skin disease image classification method based on multidimensional aggregation mechanism of this application; Figure 3 This is a schematic diagram of a sub-process in the third embodiment of the skin disease image classification method based on multidimensional aggregation mechanism of this application; Figure 4 This is a schematic diagram of the module structure of the skin disease image classification device based on the multidimensional aggregation mechanism in an embodiment of this application; Figure 5 This is a schematic diagram of the device structure of the hardware operating environment involved in the skin disease image classification method based on the multidimensional aggregation mechanism in the embodiments of this application.

[0016] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0017] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of this application.

[0018] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0019] It should be noted that with the development of deep learning technology, intelligent analysis of medical images has become an important research direction in the field of artificial intelligence. Among them, the automatic classification of skin disease images, as a key link in auxiliary diagnosis, has received widespread attention. Traditional skin disease image classification methods mainly rely on manual feature extraction and shallow classifiers. These methods depend on human experience, have poor robustness, and have limited ability to represent features such as complex lesion textures and blurred boundaries, resulting in poor classification accuracy of skin disease images. Therefore, how to improve the classification accuracy of skin disease images has become an urgent technical problem to be solved.

[0020] The main solution of this application is as follows: acquire skin image data to be classified, and extract features from the skin image data to obtain initial features; apply spatial gating to the initial features to obtain spatially gated features; input the spatially gated features into multiple depthwise separable convolutional paths with different dilation rates to extract multi-scale features and fuse them to obtain multi-level aggregated features; apply channel gating to the multi-level aggregated features to obtain channel-gated features; apply single-head attention to some channels of the channel-gated features to obtain fused features; perform linear mapping on the fused features to obtain a classification vector, and input the classification vector into a preset classification model to obtain the target classification result.

[0021] This application sequentially employs spatial gating to screen key regions of initial features, multi-scale features to be extracted and fused through multi-dilation convolution paths, channel gating to enhance classification-related channel features, and single-head attention to highlight global information of key channels. While suppressing interference from irrelevant backgrounds and redundant channels, it also takes into account the expression and fusion of details and overall features, thereby improving the classification accuracy of skin disease images.

[0022] It should be noted that the executing entity of the method in this embodiment can be a computing service device with data processing, network communication, and program execution functions, or it can be the aforementioned skin disease image classification device based on the multidimensional aggregation mechanism with the same or similar functions. This embodiment and the following embodiments will be described using a skin disease image classification device based on the multidimensional aggregation mechanism as an example.

[0023] Based on this, a first embodiment of the skin disease image classification method based on a multidimensional aggregation mechanism is proposed in this application. Please refer to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the skin disease image classification method based on a multidimensional aggregation mechanism of this application.

[0024] In this embodiment, the skin disease image classification method based on a multidimensional aggregation mechanism includes the following steps: S1: Obtain skin image data to be classified, and extract features from the skin image data to obtain initial features; It should be noted that skin image data refers to images of skin lesions used for classification tasks. Feature extraction refers to processing the input image using a deep learning model to extract high-dimensional feature representations that represent the local and global attributes of the image. Initial features refer to the first-stage feature representations obtained after processing by the feature extraction module (such as convolutional layers).

[0025] Specifically, the first step is to acquire images of skin lesions using an image acquisition module or from a medical image database, and then preprocess them to a standard format that meets the model's input requirements. This typically includes operations such as unifying resolution, normalizing pixel values, and adjusting channel order. The purpose of this step is to ensure that the image is compatible with the input interface of the subsequent deep neural network and to eliminate interference from unstructured noise.

[0026] Furthermore, the preprocessed image is input into the front-end feature extraction layer of the neural network model and processed using the basic structure of a convolutional neural network (CNN). Through multi-layer convolution operations and non-linear activation functions, the model extracts representative feature maps from the original image. These feature maps contain low-level image semantic information such as edges, local textures, and color distribution, forming the input basis for subsequent attention mechanisms and feature fusion modules, i.e., "initial features".

[0027] This step involves standardizing and preprocessing the skin image and extracting initial features. This allows the model to effectively filter out irrelevant noise regions while preserving the structural information of the original lesion area in the image. This improves the stability and expressive power of the feature representation, providing a reliable input foundation for subsequent attention mechanisms, multi-scale feature extraction, and classification models. It also helps to improve the accuracy and robustness of the overall classification task.

[0028] S2: Apply spatial gating to the initial features to obtain spatially gated features; It's important to note that spatial gating refers to generating weights for each location along the spatial dimension (H×W) and adjusting the strength of the feature map position by position accordingly. Spatial-gated features refer to the feature tensor output after spatial gating. Global average pooling (GAP) is an operation that aggregates the features along a certain dimension to obtain global statistical information; here, it's used to extract an overall description of the spatial distribution. The spatial weight map refers to a weight matrix with the same size as the feature space. The scaling parameter is a coefficient used to adjust the density and contrast of the spatial weight distribution. Position-by-position multiplication refers to multiplying the spatial weight map and the feature at corresponding positions to achieve position-level suppression and enhancement.

[0029] Specifically, the initial features are first subjected to global statistics to capture the overall distribution information of their spatial responses. This overall distribution is then refined into a spatial weight map of the same size as the features through lightweight convolution / non-linear mapping. A scaling parameter is introduced during this process to adjust the sparsity and contrast of the weight map: sparsity is increased when more focus is needed on a few salient regions; sparsity is decreased when smoother coverage of potential lesion boundaries is needed. To avoid the impact of extreme values ​​on subsequent propagation, normalized / truncated activation can be used at this stage to limit the weights to a reasonable range, facilitating stable training and inference.

[0030] Furthermore, the obtained spatial weight map is multiplied positionally with the initial features to selectively enhance or suppress each spatial location: locations related to lesions and with stronger responses are assigned higher weights, while irrelevant or noisy locations are assigned lower weights. This positional gating operates on all channels (broadcast and shared in the spatial dimension), thereby performing fine-grained recalibration of the spatial distribution without altering the semantic structure of the channels, and the output is the spatially gated feature.

[0031] Spatial gating first learns interpretable weights for each spatial location in a lightweight manner, and then applies them to the initial features one location at a time. This shifts the network's focus from "large and comprehensive" to "selective spatial regions." This weakens background interference that is irrelevant to classification, while retaining necessary transitional information around lesions and highlighting truly critical areas. This makes it easier for subsequent multi-scale extraction and channel selection to work on "clean and focused" features, thereby improving the overall discriminativeness of features and the final classification accuracy.

[0032] S3: Input the spatial gated features into multiple depthwise separable convolutional paths with different dilation rates, extract multi-scale features and fuse them to obtain multi-level aggregated features; It should be noted that spatially gated features refer to feature maps obtained after spatial gating processing. The dilation rate is the interval between sampling points of adjacent convolution kernels in a convolution operation. Depthwise separable convolution refers to decomposing standard convolution into depthwise convolution (channel-wise convolution) and pointwise convolution (1×1 convolution) to reduce computational cost and parameter count while maintaining feature extraction capabilities. Multi-level aggregated features are comprehensive features obtained by fusing features extracted from multiple sets of convolutions with different dilation rates in either the channel or spatial dimensions.

[0033] Specifically, spatially gated features are simultaneously input into multiple parallel depthwise separable convolutional branches, each with a different kernel dilation rate, thus creating different receptive field settings: branches with smaller dilation rates better preserve local details of the lesion, such as edge textures; while branches with larger dilation rates capture global structural and semantic information of the lesion and surrounding tissues. The introduction of depthwise separable convolution ensures that the overall computational cost and parameter count remain low during multi-branch parallel computation, making it suitable for tasks with real-time requirements.

[0034] Furthermore, the output features of each convolutional branch with different dilation rates are uniformly processed (e.g., size alignment, channel number matching), and then a feature fusion strategy (e.g., channel concatenation or weighted summation) is used to merge them into a comprehensive feature map. This fusion result simultaneously contains feature information at different scales and levels, which can express both fine-grained local lesion features and preserve the overall morphology and contextual information of the lesions, thus forming a multi-level aggregated feature for further optimization in subsequent channel selection and attention mechanisms.

[0035] By inputting spatially gated features into multiple depthwise separable convolutional branches with different dilation rates, it is possible to simultaneously acquire lesion details and global structural features at different scales in a single computation. These multi-scale features are then fused into multi-level aggregated features, enabling the feature representation to simultaneously possess low-order, mid-order, and high-order information. This enhances the model's adaptability to skin lesions of different shapes and sizes, providing a more comprehensive and discriminative feature foundation for subsequent channel optimization and classification.

[0036] S4: Apply channel gating processing to the multi-level aggregated features to obtain channel-gated features; It's important to note that channel gating refers to the process of dynamically assigning weights to channels in the feature map based on their importance, enhancing channels containing critical information, and suppressing irrelevant or interfering channels. The channel global description vector is generated by pooling the feature map spatially into the overall statistical value for each channel using methods such as global average pooling. The channel weight vector is obtained by mapping the channel global description vector to each channel. Channel-wise multiplication (channel weighting) refers to multiplying the channel weight vector by the corresponding channels of the feature map, thereby adjusting the response intensity of each channel.

[0037] Specifically, global average pooling is performed on the multi-level aggregated features in the spatial dimension, compressing the two-dimensional features of each channel into a single value to form a global description vector for each channel, which is used to quantify the contribution of each channel to the overall feature representation. Next, this description vector is input into a convolutional or fully connected mapping module matching the number of channels to generate channel weight vectors; non-linear activation functions can be introduced during this process to enhance the discriminative power of the weights.

[0038] Furthermore, the generated channel weight vectors are multiplied one by one with the multi-level aggregated features along the channel dimension to obtain a weighted feature map. Channels with higher weights have enhanced feature responses, while channels with lower weights have suppressed responses. As a result, the output feature map focuses more on key information related to the classification task at the channel level, and this result is the channel-gated feature.

[0039] By assigning dynamic weights to multi-level aggregated features along the channel dimension, we can selectively enhance feature channels that are highly relevant to the classification task and suppress noisy or redundant channels. This makes the feature representation more concentrated and discriminative at the channel level, thereby providing more refined and effective feature inputs for subsequent attention processing and classification models, and improving the classification accuracy of the final skin disease images.

[0040] S5: Apply single-head attention processing to a portion of the channels of the channel-gated feature to obtain the fused feature; It should be noted that channel-gated features refer to the feature maps obtained after channel-gated processing. Single-head attention processing is a form of attention mechanism that calculates attention weights using a single set of queries (Q), keys (K), and values ​​(V), models the dependencies between feature elements, and captures global information. Fusion features are the output features after single-head attention processing.

[0041] Specifically, a subset of channels is selected from the channel-gated features based on a pre-defined channel selection strategy (such as weight ranking or task relevance labeling) to reduce computational load and focus on the most important feature information. The features of these channels are used as input to a single-head attention module, and corresponding query vectors (Q), key vectors (K), and value vectors (V) are generated through linear mapping or convolution operations, respectively.

[0042] Furthermore, in the single-head attention module, correlation weights between feature locations are calculated based on Q and K, and these weights are used to perform a weighted summation on V to obtain the global contextual feature representation for each location. The attention output is then fused with the original input channel features (e.g., by addition or concatenation) to form a fused feature that combines local features with global dependencies, providing a more comprehensive feature representation for subsequent classification stages.

[0043] By applying single-head attention to only a subset of high-value channels in the channel-gated features, the computational complexity is reduced while effectively capturing the global dependencies and contextual information of these key channels. By fusing global dependency features with local detail features, both discriminative details are preserved and global consistency is improved, thereby significantly enhancing the expressive power of the features and providing stronger support for the accuracy of the final classification results.

[0044] S6: Perform linear mapping on the fused features to obtain a classification vector, and input the classification vector into a preset classification model to obtain the target classification result.

[0045] It should be noted that linear mapping refers to mapping high-dimensional features into fixed-dimensional vector representations through fully connected layers or affine transformations. The classification vector refers to the intermediate representation of the image's class probability. The pre-built classification model refers to a pre-constructed classification sub-network, including fully connected layers, normalization processes, and a softmax function, used to output the final classification prediction result. The target classification result refers to the final determined image class label, which is the output of the entire classification process.

[0046] Specifically, the fused features are first flattened, converting them from a three-dimensional structure (height × width × channels) into a one-dimensional vector representation. This vector is then fed into a linear mapping layer (such as a fully connected layer) to compress the feature dimension and generate a classification vector with a uniform structure, whose dimension matches the number of task categories. Subsequently, the classification vector is input into a pre-defined classification model for final discrimination. The classification model includes a Softmax or other normalized activation function to transform the classification vector into a set of probability distributions reflecting the confidence level of each category. Finally, the category corresponding to the highest probability is used as the prediction result for the current input skin image, i.e., the target classification result.

[0047] By performing linear mapping on the fused features and inputting them into a pre-defined classification model, high-dimensional semantic information can be effectively extracted and utilized after structural integration to complete a precise mapping from the feature space to the category space, enabling the model to output clear classification results. This step realizes the transformation of feature information into discrimination results, ensuring the closed loop of the overall classification process, thereby improving the accuracy and practicality of intelligent recognition of skin disease images.

[0048] This embodiment acquires skin image data to be classified and extracts features from the skin image data to obtain initial features. Spatial gating is applied to the initial features to obtain spatially gated features. These spatially gated features are then input into multiple depthwise separable convolutional paths with different dilation rates to extract multi-scale features and fuse them to obtain multi-level aggregated features. Channel gating is applied to these multi-level aggregated features to obtain channel-gated features. Single-head attention is applied to some channels of the channel-gated features to obtain fused features. Linear mapping is performed on the fused features to obtain a classification vector, which is then input into a preset classification model to obtain the target classification result. This embodiment sequentially uses spatial gating to select key regions of the initial features, multi-dilation rate convolutional paths to extract and fuse multi-scale features, channel gating to enhance classification-related channel features, and single-head attention to highlight global information in key channels. This approach suppresses irrelevant background and redundant channel interference while balancing the expression and fusion of detailed and overall features, thereby improving the classification accuracy of skin disease images.

[0049] Based on the first embodiment described above, a second embodiment of the skin disease image classification method based on a multidimensional aggregation mechanism is proposed in this application. Please refer to... Figure 2 , Figure 2 This is a schematic diagram of a sub-process in the second embodiment of the skin disease image classification method based on the multidimensional aggregation mechanism of this application.

[0050] like Figure 2 As shown, in this embodiment, step S5 includes: S51: Divide the channel gating features into channels to determine the target channel set; S52: Perform a linear mapping on the target channel set to generate a query vector, a key vector, and a value vector; S53: Determine the attention weight based on the correlation between the query vector and the key vector; S54: The attention weights are weighted with the value vector to obtain a weighted output, and the weighted output is fused with the residual path to generate the fused feature.

[0051] It should be noted that channel partitioning refers to grouping or filtering the channel dimensions of the feature map according to a preset ratio or strategy. The target channel set is a subset of channels selected from the initial feature map. Query vector, key vector, and value vector are the three types of vectors in the attention mechanism, used to model the current focus, match reference content, and generate the actual weighted output, respectively. Attention weights are weight coefficients obtained by calculating the similarity relationship between the query and the key, used to control the degree of information retention in the value vector. Residual path fusion refers to directly adding or concatenating the weighted attention output with the original features to preserve the original feature information and enhance the model training stability.

[0052] Specifically, in the channel-gated features, a subset of channels is selected according to a preset channel sampling ratio or strategy to form a target channel set for attention calculation. Subsequently, this target channel set is linearly mapped to generate query vectors, key vectors, and value vectors, preparing for subsequent attention weight calculations. Attention weights are obtained by calculating the correlation between the query vector and the key vector, measuring the contribution of each channel to the target features. These weights are then applied to the value vectors to weight the features of each channel, resulting in a weighted output feature. The weighted output is then fused with the residual path of the original channel-gated features to achieve a balance between information preservation and feature enhancement, ultimately generating a fused feature.

[0053] By performing single-head attention processing on a subset of the channel-gated features, the model can dynamically highlight key channel information relevant to the classification task while maintaining a lightweight structure. Residual fusion preserves the original features, preventing feature loss. This step effectively improves the focus and discriminative power of features along the channel dimension, providing higher-quality input for subsequent operations, thereby contributing to improved accuracy and robustness in skin disease image classification.

[0054] In this embodiment, step S2 includes: S21: Perform global average pooling on the initial features to generate the corresponding spatial global description vector; S22: Based on the spatial global description vector, a spatial weight map is generated by combining a preset convolution operation with a scaling parameter; S23: Perform element-wise multiplication of the spatial weight map and the initial feature to obtain the spatial gating feature.

[0055] It's important to note that global average pooling refers to averaging the spatial dimensions (height and width) of the feature map to obtain the average response value of each channel globally. The spatial global description vector is a vector obtained from global average pooling, reflecting the overall spatial distribution characteristics of the initial features. Convolution operation here refers to the convolution operation used to generate the weight map. The scaling parameter is a coefficient used to adjust the sparsity and magnitude of the weight map distribution. The spatial weight map is a matrix with the same size as the spatial dimensions of the input features. Element-wise positional multiplication refers to multiplying each position of the feature map by its corresponding weight value, achieving positional feature enhancement or suppression.

[0056] Specifically, the initial features are subjected to global average pooling in the spatial dimension to obtain the average response value of each channel in the global scope, forming a spatial global description vector. This vector reflects the spatial distribution trend of the overall features. Subsequently, this vector is input into a preset convolution operation module to extract position-related weight information, and the distribution of weight values ​​is adjusted in combination with scaling parameters so that the weight distribution is concentrated in the salient region while retaining necessary background information, thereby generating a spatial weight map.

[0057] Furthermore, the generated spatial weight map is multiplied element-wise with the initial features. That is, at each spatial location, the feature value is multiplied by the corresponding weight coefficient, thereby enhancing task-relevant salient regions and weakening irrelevant or interfering regions. The resulting weighted features are spatially gated features, providing more focused feature input for subsequent multi-scale convolutional extraction.

[0058] By extracting the spatial global distribution information of the initial features through global average pooling, and combining convolution operations with scaling parameters to generate a position-sensitive spatial weight map, which is then applied to the initial features position by position, the spatial dimension can effectively highlight the significant regions related to the task and suppress the interference of irrelevant background. This enables more efficient and focused information utilization in subsequent feature extraction processes, thereby improving the final classification accuracy.

[0059] This embodiment acquires skin image data to be classified and extracts features from the skin image data to obtain initial features. Spatial gating is applied to the initial features to obtain spatially gated features. These spatially gated features are then input into multiple depthwise separable convolutional paths with different dilation rates to extract multi-scale features and fuse them to obtain multi-level aggregated features. Channel gating is applied to these multi-level aggregated features to obtain channel-gated features. Single-head attention is applied to some channels of the channel-gated features to obtain fused features. Linear mapping is performed on the fused features to obtain a classification vector, which is then input into a preset classification model to obtain the target classification result. This embodiment sequentially uses spatial gating to select key regions of the initial features, multi-dilation rate convolutional paths to extract and fuse multi-scale features, channel gating to enhance classification-related channel features, and single-head attention to highlight global information in key channels. This approach suppresses irrelevant background and redundant channel interference while balancing the expression and fusion of detailed and overall features, thereby improving the classification accuracy of skin disease images.

[0060] Based on the second embodiment described above, a third embodiment of the skin disease image classification method based on a multidimensional aggregation mechanism is proposed in this application. Please refer to... Figure 3 , Figure 3 This is a schematic diagram of a sub-process in the third embodiment of the skin disease image classification method based on the multidimensional aggregation mechanism of this application.

[0061] In this embodiment, step S3 includes: S31: The spatial gated features are input into multiple parallel depthwise separable convolutional paths, wherein the kernel size of each depthwise separable convolutional path is the same, but the dilation rate is different. S32: Perform spatial alignment and dimensionality unification processing on the output features of the separable convolutional paths at each depth; S33: The spatial alignment and dimensional unification processing results are fused to obtain the multi-level aggregated features.

[0062] Specifically, spatially gated features are input in parallel into multiple depthwise separable convolutional paths. Each path has the same kernel size but different dilation rates. Paths with low dilation rates can capture local texture and edge information, while paths with high dilation rates can perceive a larger range of overall structure and blurred boundary features. Through depthwise separable convolution, the model extracts multi-scale features while reducing the number of parameters and computational complexity.

[0063] Furthermore, the feature maps output by multiple convolutional paths may have inconsistencies in the number of channels or spatial dimensions. Therefore, they are first spatially aligned and dimensionally unified, for example, by adjusting the number of channels and the feature map size through interpolation or convolution. After alignment, the features of each path are concatenated or weighted and summed along the channel dimension to generate multi-level aggregated features that integrate information from different scales, which are then used for subsequent processing by the attention and gating modules.

[0064] By inputting spatially gated features into multiple depthwise separable convolutional paths with different dilation rates and fusing these features, the model can simultaneously perceive local and global features while maintaining a lightweight structure, capturing detailed textures, blurred boundaries, and overall structure of skin images. The multi-level aggregated feature information, after spatial alignment and dimensionality unification, is complete and rich in hierarchy, providing high-quality input for subsequent attention processing and classification, thereby improving the accuracy and robustness of skin disease image classification.

[0065] Based on the second embodiment described above, in this embodiment, step S4 includes: S41: Perform global average pooling on the multi-level aggregated features along the channel dimension to generate a channel global description vector; S42: Input the channel global description vector into the convolution operation module that matches the number of channels of the multi-level aggregated feature to obtain the channel weight vector; S43: Perform a channel-by-channel multiplication operation between the channel weight vector and the multi-order aggregated feature in the channel dimension to obtain the channel-gated feature.

[0066] It should be noted that the channel dimension refers to the arrangement of feature maps along the channel direction, with each channel typically corresponding to a type of feature response pattern. The channel global description vector is a vector obtained by global average pooling. The convolution module is used here to map and transform the channel global description vector to generate a channel weight vector with the same number of input channels. Channel-wise multiplication refers to multiplying the channel weight vector with multi-level aggregated features one-to-one along the channel dimension to achieve channel-level feature enhancement or suppression.

[0067] Specifically, global average pooling is performed on the multi-level aggregated features in the spatial dimension to obtain the average response value of each channel globally, forming a channel global description vector. This vector reflects the contribution of each channel to the overall feature representation. Subsequently, the channel global description vector is input into a convolution operation module that matches the number of channels in the multi-level aggregated features. This module transforms and assigns weights to the channel descriptions, thereby generating channel weight vectors.

[0068] Furthermore, the channel weight vectors are multiplied one-to-one with the multi-level aggregated features along the channel dimension. Channels with higher weights are enhanced, while channels with lower weights are suppressed, resulting in an output that is more focused on task-related features. After weighting, the output feature map becomes the channel-gated feature, providing optimized input for subsequent attention mechanism processing.

[0069] By performing global average pooling to extract the channel-wide description of multi-level aggregated features, and using convolution operations to generate channel weight vectors, and then weighting the features channel by channel, we can selectively strengthen channel features that are highly relevant to the classification task, suppress irrelevant or redundant channel information, and make the features more concentrated and discriminative at the channel level, thereby improving the accuracy of subsequent attention processing and classification models.

[0070] Based on the second embodiment described above, in this embodiment, step S6 includes: S61: Flatten the fused features to obtain one-dimensional features; S62: Input the one-dimensional feature into the fully connected layer and perform linear mapping to generate the classification vector; S63: Normalize the classification vector to obtain a normalized vector; S64: Input the normalized vector into the preset classification model to obtain the target classification result.

[0071] It should be noted that flattening refers to the operation of converting a multidimensional feature tensor (such as height × width × channels) into a one-dimensional vector. Normalization refers to the numerical standardization operation performed on the classification vector.

[0072] Specifically, firstly, the fused features are flattened from a three-dimensional structure into a one-dimensional vector to fit the fully connected structure. This one-dimensional feature vector retains semantic information in both the spatial and channel dimensions, serving as the input basis for the final classification. Next, this one-dimensional feature is input to the fully connected layer for linear mapping, generating a fixed-length classification vector. This vector dimension is typically equal to the number of categories in the target task, representing the response level of each category. Subsequently, the classification vector is normalized, commonly by using the Softmax function to convert each element in the vector into a probability value between [0,1], satisfying the probability distribution characteristics. Finally, the normalized vector is input into a pre-defined classification model, and by outputting the category label corresponding to the highest probability, the final classification of the skin image is completed, generating the target classification result.

[0073] By progressively processing fused features through flattening, linear mapping, normalization, and classification prediction, the model can effectively transform high-dimensional semantic features into category prediction outputs, achieving an accurate mapping from the feature space to the category space. Simultaneously, normalization enhances predictive interpretability, giving the final classification result a clear probability indication, thereby improving the classification accuracy and output stability of skin disease images.

[0074] This embodiment acquires skin image data to be classified and extracts features from the skin image data to obtain initial features. Spatial gating is applied to the initial features to obtain spatially gated features. These spatially gated features are then input into multiple depthwise separable convolutional paths with different dilation rates to extract multi-scale features and fuse them to obtain multi-level aggregated features. Channel gating is applied to these multi-level aggregated features to obtain channel-gated features. Single-head attention is applied to some channels of the channel-gated features to obtain fused features. Linear mapping is performed on the fused features to obtain a classification vector, which is then input into a preset classification model to obtain the target classification result. This embodiment sequentially uses spatial gating to select key regions of the initial features, multi-dilation rate convolutional paths to extract and fuse multi-scale features, channel gating to enhance classification-related channel features, and single-head attention to highlight global information in key channels. This approach suppresses irrelevant background and redundant channel interference while balancing the expression and fusion of detailed and overall features, thereby improving the classification accuracy of skin disease images.

[0075] In one embodiment, a skin disease category prediction method based on multi-gated aggregation single-head attention is proposed. Its core is to effectively extract global information from medical images while reducing the model parameter size through channel-selective single-head attention. Simultaneously, a multi-gated aggregation mechanism is introduced to enhance the discriminative power of feature semantic expression by comprehensively modeling the intermediate-order interactions between local texture, boundary information, and global structure. This improves the model's ability to resolve blurred lesions and complex textures, as well as its ability to discriminate fine-grained categories.

[0076] Specifically, the framework mainly includes three key modules: (1) Single-head selective attention module. It selectively applies single-head attention to a subset of feature channels to obtain global features, while effectively reducing computational redundancy in the channel dimension. (2) Multi-level aggregation module. It utilizes various dilated depthwise separable convolutions to capture low-order, mid-order, and high-order interactive features in parallel. (3) Multi-dimensional gating module. It introduces channel aggregation to enhance the collaborative capabilities of features between channels, while introducing gating mechanisms in the spatial and channel dimensions to emphasize important features and suppress invalid background information.

[0077] The core component of the single-head selective attention module is the single-head attention module (OHA). It performs attention calculations by selecting only a subset of feature channels, thereby reducing computational complexity while extracting effective global information. Its formula is as follows:

[0078]

[0079]

[0080] , in, , , as well as All are projected weights. Representation matrix sum matrix Dimensions This indicates a splicing operation. This represents the number of channels that require attention operations in single-head attention.

[0081] To effectively model the interactive features of blurred boundaries and multi-scale textures, a multi-level aggregation module is introduced to improve the model's ability to handle similar but different class samples. Input features are processed by depthwise separable convolutions with different scales and dilation levels.

[0082]

[0083]

[0084] in, This represents the number of channels in the input feature. Indicates size is Depth-separable convolution with a dilation factor of 2. This represents the activation function, which in this embodiment is set to the SiLU function:

[0085] Therefore, by using separable convolutions of different sizes, it is possible to effectively model the interactive features of blurred boundaries and multi-scale textures.

[0086] To emphasize important features and suppress irrelevant background information, a multi-dimensional gating module is used:

[0087] in, This represents the scaling parameter, which is set to 0.3 in this embodiment; This indicates global average pooling.

[0088] In addition, this embodiment introduces a channel gating mechanism to extract important image features:

[0089]

[0090]

[0091] in, express Convolution, which is used for matching and and output features The number of channels for subsequent features; This represents element-wise matrix multiplication.

[0092] This application also provides a skin disease image classification device based on a multidimensional aggregation mechanism. Please refer to... Figure 4 , Figure 4 This is a schematic diagram of the module structure of a skin disease image classification device based on a multidimensional aggregation mechanism according to an embodiment of this application. The skin disease image classification device based on a multidimensional aggregation mechanism includes: The initial feature module 401 is used to acquire skin image data to be classified and to extract features from the skin image data to obtain initial features; The spatial gating module 402 is used to apply spatial gating processing to the initial feature to obtain a spatial gating feature; The multi-level aggregation module 403 is used to input the spatial gated features into multiple depthwise separable convolutional paths with different dilation rates, extract multi-scale features and fuse them to obtain multi-level aggregated features; Channel gating module 404 is used to apply channel gating processing to the multi-level aggregated features to obtain channel-gated features; The fusion module 405 is used to apply single-head attention processing to a portion of the channel gating features to obtain fused features; The target module 406 is used to perform linear mapping processing on the fused features to obtain a classification vector, and input the classification vector into a preset classification model to obtain a target classification result.

[0093] The skin disease image classification device based on a multidimensional aggregation mechanism provided in this application adopts the skin disease image classification method based on a multidimensional aggregation mechanism in the above embodiments, and can solve the technical problem of how to improve the classification accuracy of skin disease images. Compared with the prior art, the beneficial effects of the skin disease image classification device based on a multidimensional aggregation mechanism provided in this application are the same as the beneficial effects of the skin disease image classification method based on a multidimensional aggregation mechanism provided in the above embodiments, and other technical features in the skin disease image classification device based on a multidimensional aggregation mechanism are the same as the features disclosed in the methods of the above embodiments, and will not be repeated here.

[0094] This application provides a skin disease image classification device based on a multidimensional aggregation mechanism. The skin disease image classification device based on a multidimensional aggregation mechanism includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the skin disease image classification method based on the multidimensional aggregation mechanism in the above embodiments.

[0095] The following is for reference. Figure 5 , Figure 5 This is a schematic diagram of the hardware operating environment of the skin disease image classification method based on the multidimensional aggregation mechanism in the embodiments of this application. It shows a schematic diagram of the structure of the skin disease image classification device based on the multidimensional aggregation mechanism suitable for implementing the embodiments of this application. Figure 5 The skin disease image classification device based on the multidimensional aggregation mechanism shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0096] like Figure 5As shown, a skin disease image classification device based on a multidimensional aggregation mechanism may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for the operation of the skin disease image classification device based on the multidimensional aggregation mechanism. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to I / O interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. Communication device 1009 allows the skin disease image classification device based on a multidimensional aggregation mechanism to wirelessly or wiredly communicate with other devices to exchange data. Although the figure shows a skin disease image classification device based on a multidimensional aggregation mechanism with various systems, it should be understood that it is not required to implement or possess all the systems shown. More or fewer systems can be implemented alternatively.

[0097] In particular, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. When the computer program is executed by the processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0098] The skin disease image classification device based on a multidimensional aggregation mechanism provided in this application, employing the skin disease image classification method based on a multidimensional aggregation mechanism in the above embodiments, can solve the technical problem of how to improve the classification accuracy of skin disease images. Compared with the prior art, the beneficial effects of the skin disease image classification device based on a multidimensional aggregation mechanism provided in this application are the same as those of the skin disease image classification method based on a multidimensional aggregation mechanism provided in the above embodiments, and other technical features in this skin disease image classification device based on a multidimensional aggregation mechanism are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0099] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0100] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0101] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the skin disease image classification method based on the multidimensional aggregation mechanism in the above embodiments.

[0102] The aforementioned computer-readable storage medium carries one or more programs. When these programs are executed by a skin disease image classification device based on a multidimensional aggregation mechanism, the device performs the following actions: acquires skin image data to be classified and extracts features from the skin image data to obtain initial features; applies spatial gating to the initial features to obtain spatially gated features; inputs the spatially gated features into multiple depthwise separable convolutional paths with different dilation rates to extract multi-scale features and fuse them to obtain multi-level aggregated features; applies channel gating to the multi-level aggregated features to obtain channel-gated features; applies single-head attention to some channels of the channel-gated features to obtain fused features; performs linear mapping on the fused features to obtain a classification vector, and inputs the classification vector into a preset classification model to obtain the target classification result. Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof. These programming languages ​​include object-oriented programming languages—such as Java, Smalltalk, and C++—and conventional procedural programming languages—such as the "C" language or similar programming languages. The program code can execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0103] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0104] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0105] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described skin disease image classification method based on a multidimensional aggregation mechanism, thereby solving the technical problem of how to improve the classification accuracy of skin disease images. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the skin disease image classification method based on a multidimensional aggregation mechanism provided in the above embodiments, and will not be repeated here.

[0106] This application provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the skin disease image classification method based on the multidimensional aggregation mechanism described above.

[0107] The computer program product provided in this application can solve the technical problem of how to improve the classification accuracy of skin disease images. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the skin disease image classification method based on multidimensional aggregation mechanism provided in the above embodiments, and will not be repeated here.

[0108] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent scope of this application.

Claims

1. A skin disease image classification method based on a multi-dimensional aggregation mechanism, characterized in that, The method comprises: obtaining skin image data to be classified, and performing feature extraction on the skin image data to obtain initial features; applying spatial gating processing to the initial features to obtain spatial gating features; inputting the spatial gating features into multiple deep separable convolution paths with different expansion rates respectively, extracting multi-scale features and performing fusion to obtain multi-order aggregation features; applying channel gating processing to the multi-order aggregation features to obtain channel gating features; applying single-head attention processing to part of the channels of the channel gating features to obtain fusion features; performing linear mapping processing on the fusion features to obtain a classification vector, and inputting the classification vector into a preset classification model to obtain a target classification result.

2. The method of claim 1, wherein, The step of applying single-head attention processing to part of the channels of the channel gating features to obtain fusion features comprises: performing channel division on the channel gating features to determine a target channel set; performing linear mapping on the target channel set to generate a query vector, a key vector and a value vector; determining an attention weight according to the correlation between the query vector and the key vector; performing weighting processing on the attention weight and the value vector to obtain a weighted output, and fusing the weighted output with a residual path to generate the fusion features.

3. The method of claim 1, wherein, The step of inputting the spatial gating features into multiple deep separable convolution paths with different expansion rates respectively, extracting multi-scale features and performing fusion to obtain multi-order aggregation features comprises: inputting the spatial gating features into multiple parallel deep separable convolution paths respectively, wherein the convolution kernel sizes of each deep separable convolution path are the same, and the expansion rates are different from each other; performing spatial alignment and dimension unification processing on the output features of each deep separable convolution path; performing feature fusion on the processing results of spatial alignment and dimension unification to obtain the multi-order aggregation features.

4. The method of claim 1, wherein, The step of applying spatial gating processing to the initial features to obtain spatial gating features comprises: performing global average pooling processing on the initial features to generate a corresponding spatial global description vector; generating a spatial weight map based on the spatial global description vector by combining a preset convolution operation with a scale parameter; performing element-level position-by-position multiplication operation on the spatial weight map and the initial features to obtain the spatial gating features.

5. The method of claim 1, wherein, The step of applying channel gating processing to the multi-order aggregation features to obtain channel gating features comprises: performing global average pooling processing on the multi-order aggregation features along the channel dimension to generate a channel global description vector; inputting the channel global description vector into a convolution operation module matched with the number of channels of the multi-order aggregation features to obtain a channel weight vector; performing channel-by-channel multiplication operation on the channel weight vector and the multi-order aggregation features along the channel dimension to obtain the channel gating features.

6. The method of claim 1, wherein, The step of performing linear mapping processing on the fusion features to obtain a classification vector, and inputting the classification vector into a preset classification model to obtain a target classification result comprises: performing flattening processing on the fusion features to obtain a one-dimensional feature; The one-dimensional feature is input into a full connection layer and linear mapping processing is performed to generate the classification vector; The classification vector is normalized to obtain a normalized vector; The normalized vector is input into a preset classification model to obtain the target classification result.

7. A skin disease image classification apparatus based on a multi-dimensional aggregation mechanism, characterized by, The device comprises: An initial feature module configured to obtain skin image data to be classified and perform feature extraction on the skin image data to obtain initial features; A spatial gating module configured to perform spatial gating processing on the initial features to obtain spatial gating features; A multi-order aggregation module configured to input the spatial gating features into a plurality of depth separable convolution paths with different dilation rates, extract multi-scale features and perform fusion to obtain multi-order aggregation features; A channel gating module configured to perform channel gating processing on the multi-order aggregation features to obtain channel gating features; A fusion module configured to perform single-head attention processing on part of the channels of the channel gating features to obtain fusion features; A target module configured to perform linear mapping processing on the fusion features to obtain a classification vector, and input the classification vector into a preset classification model to obtain a target classification result.

8. A computer device, comprising: The device comprises a memory, a processor, and a skin disease image classification program based on a multi-dimensional aggregation mechanism stored on the memory and executable on the processor, which is configured to implement the steps of the skin disease image classification method based on the multi-dimensional aggregation mechanism according to any one of claims 1 to 6.

9. A storage medium, characterized by The storage medium stores a skin disease image classification program based on a multi-dimensional aggregation mechanism, which implements the steps of the skin disease image classification method based on the multi-dimensional aggregation mechanism according to any one of claims 1 to 6 when executed by a processor.

10. A computer program product, characterised in that, The computer program product comprises a computer program, which implements the steps of the skin disease image classification method based on the multi-dimensional aggregation mechanism according to any one of claims 1 to 6 when executed by a processor.

Images