Skin mirror image processing method and system based on multi-branch feature aggregation

Abstract

The application discloses a dermoscope image processing method and system based on multi-branch feature aggregation, the method comprises the following steps: a multi-branch feature extraction step, a high-frequency information fusion step, and a symmetric decoding reconstruction step; by designing a multi-branch collaborative hybrid attention network, using mechanisms such as multi-scale context extraction, global channel context attention and multi-semantics collaborative attention, the segmentation accuracy is greatly improved; meanwhile, an asymmetric context aggregation network is proposed, and technologies such as axial deep convolution and edge operator are adopted, so that the ideal balance between the calculation efficiency and the segmentation performance is realized while the model complexity is greatly reduced; finally, an auxiliary diagnosis platform deployed on an edge device is designed and implemented, and a complete technical scheme and practice paradigm are provided for the intelligent clinical application of the dermoscope image.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, specifically relating to a dermoscopic image processing method and system based on multi-branch feature aggregation. Background Technology

[0002] Dermoscopy is an important non-invasive method for early screening of skin lesions, and the accuracy of its lesion image segmentation directly determines the reliability of subsequent feature extraction and quantitative analysis. In clinical practice, the accurate definition of lesion regions is not only related to the determination of regional grading, but also directly affects downstream tasks such as surgical boundary planning and efficacy evaluation. However, achieving high-precision segmentation of skin lesions still faces multiple challenges: First, skin lesions often exhibit highly irregular shapes and significant scale differences, requiring models to have robust multi-scale feature capture capabilities; second, lesions and backgrounds are often accompanied by insufficient contrast, blurred boundaries, and artifacts such as hair and bubbles, causing existing deep learning models to easily lose key geometric and topological information during downsampling, making it difficult to achieve fine-grained boundary structure restoration. In addition, mainstream deep learning models generally suffer from complex structures and redundant parameters, and the high computational cost severely restricts the efficient deployment of models in resource-constrained edge medical devices. Therefore, how to construct a high-precision image processing model that can handle lesions with diverse shapes and complex structures while meeting the requirements of lightweight deployment is a key obstacle to promoting the clinical application of intelligent assisted analysis systems. Summary of the Invention

[0003] The purpose of this invention is to overcome the shortcomings of the prior art and provide a dermoscopic image processing method and system based on multi-branch feature aggregation.

[0004] A dermoscopic image processing method based on multi-branch feature aggregation includes, The multi-branch feature extraction step involves inputting the image to be processed into the encoder and the Laplacian pyramid branch, respectively. The encoder extracts multi-scale semantic features through multiple cascaded processing stages, and the Laplacian pyramid branch extracts Laplacian features at the corresponding scale. The high-frequency information fusion step utilizes a high-frequency context-aware module to interactively fuse the semantic features output from each stage with the corresponding scale Laplacian features, in order to capture boundary detail information and enhance feature representation. The symmetric decoding and reconstruction step involves upsampling the fused features step by step through a decoder that is symmetrical to the encoder structure, and fusing them with the corresponding encoded features during each upsampling process, ultimately outputting a pixel-level segmentation mask of the target region in the image to be processed.

[0005] As one implementation, the coding network branch includes five cascaded processing stages, wherein shallow spatial features are extracted through a 3×3 convolutional layer in the first stage, feature processing is performed using a hierarchical parallel perceptual fusion module in the second to third stages, and deep semantic features are extracted using a multi-branch semantic collaborative attention module in the fourth to fifth stages. The hierarchical parallel perception fusion module synchronously models local details and long-range dependencies through parallel multi-branch deep convolution and global channel context weights. In the decoding network branch, a 1×1 convolutional layer corresponding to the first stage is used to perform channel integration and dimensionality compression on the features fused by skip connections.

[0006] As one implementation method, the processing steps of the hierarchical parallel perception fusion module include: The feature splitting process involves processing the input features using the first normalized convolutional layer and dividing the processed features into multiple feature sub-streams along the channel dimension using a channel splitting operator. The multi-dimensional parallel perception step inputs each feature substream after division into corresponding parallel processing branches, and extracts local spatial features and long-range structural features simultaneously through depth convolution operators and asymmetric strip depth convolution operators of different dimensions. The feature aggregation and calibration steps involve concatenating the features output from each branch and using the channel prior convolutional attention mechanism module to calibrate the importance of the channel and spatial dimensions of the concatenated multi-scale features. The residual fusion output step involves summing the calibrated features element-wise with the initial input features and outputting the fused features using a residual connection method.

[0007] As one implementation method, the processing procedure of the channel prior convolutional attention mechanism module includes: Global average pooling and global max pooling are performed on the spliced ​​multi-scale features to generate corresponding channel description information; Based on the channel description information, channel attention weights are generated, and the channel attention weights are used to weight the concatenated multi-scale features to obtain intermediate prior features. Spatial augmentation is performed on the intermediate prior feature input multi-branch deep convolutional structure to extract spatial context information under different receptive fields; The features output by the multi-branch deep convolutional structure are fused to generate a spatial attention map; The intermediate prior features are weighted based on the spatial attention map to output enhanced features.

[0008] As one implementation method, the execution logic of the multi-branch semantic collaborative attention module includes: The multi-scale spatial semantic capture step divides the input features into multiple sub-features along the channel dimension, and processes them in parallel using dilated convolution operators with different dilation rates to extract complementary multi-scale contextual information. The feature integration and dimensional decoupling steps involve concatenating the processed sub-features by channels and performing group normalization and pointwise convolution operations in sequence to compress channel redundancy. Subsequently, the resulting features are decomposed into single dimensions along the height and width dimensions to generate two one-dimensional sequence features representing the spatial structure. The fine-grained subspace modeling step divides each of the one-dimensional sequence features into multiple independent sub-features, and applies depth convolution operators with different kernel sizes to each independent sub-feature for processing, so as to adapt to spatial structures of different scales and suppress semantic interference. The collaborative attention calculation and output steps involve converging and reconstructing the processed sub-features, calculating the self-attention weight mapping through matrix multiplication operators and normalized exponential functions, and finally applying the weight mapping to the initial input features to output enhanced multi-semantic collaborative features.

[0009] As one implementation method, the specific steps of the collaborative attention calculation include: Embedded representation generation: Average pooling and group normalization are performed on the preprocessed features, and query vector, key vector and value vector are generated respectively using 1×1 depthwise convolution operator; Weight graph calculation: The query vector and the key vector are reconstructed into a preset dimension form, and the similarity weight graph between them is calculated based on the matrix multiplication operator. Normalization and regularization processing are then performed on the weight graph in sequence. Adaptive recalibration: The processed weight map is multiplied with the value vector to obtain attention-enhanced features; after remapping the attention-enhanced features back to the original spatial dimension, adaptive recalibration of the features is achieved using an average pooling layer and a sigmoid activation function. Residual fusion: The recalibrated features are fused with the preprocessed features through residual connections to enhance the feature discrimination capability guided by high-level semantics.

[0010] As one implementation method, the execution logic of the high-frequency context-aware module includes: The initial feature fusion step involves concatenating the input features with the pre-extracted Laplacian geometric features through channels, and then integrating spatial information using a 3×3 convolutional layer to obtain an intermediate feature map. The multi-path global pooling step inputs the features to be perceived into two different global average pooling operators in parallel to capture the spatial context descriptors of the features from different dimensions. The shared weight generation step involves inputting the two sets of spatial context descriptors into two twice-cascaded shared weight one-dimensional convolution operators to perform local cross-channel interaction, generating preliminary deep attention features. Then, two learnable scalar adjustment factors are used to weight the two sets of preliminary deep attention features, and the weighted features are subjected to element-wise addition, batch normalization, and activation function processing to generate the final channel attention map. The adaptive feature enhancement step involves performing element-wise multiplication between the channel attention map and the intermediate feature map to output enhanced high-frequency context-aware features.

[0011] A dermoscopic image processing system based on multi-branch feature aggregation includes a model development platform and an edge inference device with communication connections; The model development platform includes: The model building unit is used to build a multi-branch feature aggregation neural network model including an encoding branch, a Laplacian pyramid branch, and a symmetric decoding branch. The model achieves interactive fusion of semantic features and high-frequency features through a high-frequency context awareness mechanism at each processing stage. The format conversion engine is used to obtain the initial model weight file of the neural network model trained under the first deep learning framework, export it as an intermediate model in the Open Neural Network Exchange Format, and perform topology verification and operator validity checks on the intermediate model. The heterogeneous compilation engine, with a built-in Ascend tensor compiler, is used to receive the verified intermediate model and perform operator fusion and computation graph optimization processing on the multi-branch feature structure to compile the intermediate model into an offline model adapted to the target heterogeneous hardware instruction set. The edge inference device is an edge computing node equipped with a neural network processing unit (NPU). The edge inference device is used for: Obtain the offline model issued by the model development platform; The offline model is loaded into the computing architecture corresponding to the neural network processing unit through the heterogeneous computing programming interface integrated within the device, so as to drive the hardware to perform parallel inference computing and output pixel-level segmentation results based on the multi-branch feature fusion results.

[0012] As one implementation, the neural network model includes an encoding branch, a Laplacian pyramid branch, and a symmetric decoding branch, wherein: The encoding branch includes multiple cascaded processing stages for extracting multi-scale semantic features step by step; The Laplacian pyramid branch is used to perform multi-scale decomposition of the input image and extract high-frequency Laplacian features corresponding to each processing stage. High-frequency context-aware units are set up in each processing stage to interactively fuse the semantic features and Laplacian features of the corresponding stage. The symmetric decoding branch is used to upsample the fused features step by step, and to fuse them with the corresponding encoding stage features through skip connections during each upsampling process, so as to output pixel-level segmentation results.

[0013] As one implementation, the model development platform is used to compile the ONNX intermediate model into an OM offline model that can run under the CANN architecture using the Ascend Tensor Compiler (ATC). The edge inference device is an edge computing terminal with a built-in Ascend AI processing unit, and loads the OM offline model through the Ascend Computing Language (ACL) interface to perform lesion segmentation inference on the local neural network processing unit.

[0014] The advantages and beneficial effects of this invention are as follows: This invention addresses the limitation in segmentation accuracy caused by the complex features of skin lesions, such as irregular shapes, significant scale differences, low contrast, and blurred boundaries. First, a hierarchical parallel perceptual fusion module employs a multi-branch structure and deep convolution decomposition design to efficiently capture multi-scale contextual information, and introduces a CPCA attention mechanism to enhance the modeling ability of distant pixel dependencies. Second, a multi-branch semantic collaborative attention module utilizes multi-scale dilated convolution and dimensionality decoupling mechanisms to capture multi-view feature representations, and introduces a multi-head self-attention mechanism to achieve dynamic feature fusion, thereby effectively reducing semantic interference caused by lesion blurring and low contrast. Furthermore, a high-frequency channel attention module fuses high-frequency features extracted from the Laplacian pyramid in skip connections, and significantly enhances the model's ability to capture lesion edge details through channel attention and adaptive gating mechanisms. Experiments on multiple public datasets demonstrate that this model outperforms existing state-of-the-art methods on various evaluation metrics without pre-training, showcasing its significant application potential in complex medical image segmentation tasks.

[0015] By designing a multi-branch collaborative hybrid attention network, the segmentation accuracy is significantly improved by utilizing mechanisms such as multi-scale context extraction, global channel context attention, and multi-semantic collaborative attention. At the same time, an asymmetric context aggregation network is proposed, which adopts techniques such as axial depth convolution and edge operators to achieve an ideal balance between computational efficiency and segmentation performance while significantly reducing model complexity. Finally, by designing and implementing an auxiliary diagnostic platform deployed on edge devices, a complete technical solution and practical paradigm are provided for the intelligent clinical application of dermoscopy images. Attached Figure Description

[0016] Figure 1 It is a multi-branch collaborative hybrid attention network.

[0017] Figure 2 It is a hierarchical parallel perception fusion module.

[0018] Figure 3 Channel prior convolutional attention.

[0019] Figure 4 This is a multi-branch semantic collaborative attention module.

[0020] Figure 5 This is a high-frequency channel attention module.

[0021] Figure 6 This is a comparison of different models in terms of parameter count and computational complexity.

[0022] Figure 7 A comparison of the mIoU index for different models on four datasets.

[0023] Figure 8 A comparison of DSC metrics for different models across four datasets.

[0024] Figure 9 A segmented visualization comparison chart.

[0025] Figure 10 Visualize the feature heatmap.

[0026] For those skilled in the art, other related figures can be obtained from the above figures without any creative effort. Detailed Implementation

[0027] To enable those skilled in the art to better understand the present invention, the technical solution of the present invention will be further described below in conjunction with the accompanying drawings and specific embodiments.

[0028] This invention addresses the complex challenges of skin lesion segmentation, particularly for lesions with irregular shapes, blurred boundaries, hair occlusion, and low contrast with the background, achieving higher segmentation accuracy. This invention proposes a dermoscopic image processing method based on multi-branch feature aggregation, including... The multi-branch feature extraction step involves inputting the image to be processed into the encoder and the Laplacian pyramid branch, respectively. The encoder extracts multi-scale semantic features through multiple cascaded processing stages, and the Laplacian pyramid branch extracts Laplacian features at the corresponding scale. The high-frequency information fusion step utilizes a high-frequency context-aware module to interactively fuse the semantic features output from each stage with the corresponding scale Laplacian features, in order to capture boundary detail information and enhance feature representation. The symmetric decoding and reconstruction step involves upsampling the fused features through the decoder of the symmetric decoder of the encoder structure at each level, and fusing them with the corresponding encoded features during each upsampling process, ultimately outputting a pixel-level segmentation mask of the target region in the image to be processed.

[0029] The network implementing the dermoscopy image processing method of this invention adopts a symmetrical U-shaped encoder-decoder structure. The encoder contains five stages, with the number of channels gradually increasing from an initial 32 to 512. The initial stage of the encoder uses 3×3 standard convolutions for shallow feature extraction, while the corresponding stage in the decoder uses 1×1 convolutions for feature integration. Stages 2 and 3 embed a hierarchical parallel perceptual fusion module, which simultaneously models local details and global context through parallel multi-branch deep convolutions and global channel context weights, strengthening multi-scale long-range dependencies. Simultaneously, to promote the synergistic utilization of multi-semantic feature information and enhance feature interaction to improve segmentation accuracy, a multi-branch semantic collaborative attention module is used in stages 4 and 5. Unlike the standard skip connections in U-Net, this invention introduces a high-frequency channel attention module for feature fusion. This high-frequency channel attention module captures multi-scale features and high-frequency Laplacian edge features, and then, through an adaptive gating mechanism and channel-by-channel calibration with encoder features, compensates for the loss of boundary information caused by downsampling, significantly reducing the semantic difference between the encoder and decoder, thereby improving the accuracy of lesion contour reconstruction.

[0030] This invention relates to a Multi-Branch Collaborative Hybrid Attention Network (MCHANet). Based on a decoder-encoder architecture, this network utilizes three core modules to comprehensively enhance the model's feature representation and segmentation capabilities across three dimensions: multi-scale context modeling, multi-semantic information collaboration, and high-frequency boundary detail enhancement. Specifically, the hierarchical parallel perceptual fusion module efficiently extracts multi-scale context through a parallel multi-branch structure and deep convolution decomposition, and strengthens the modeling of long-range dependencies between pixels by combining global channel context attention. To address low contrast and semantic ambiguity, the multi-branch semantic collaborative attention module captures multi-view features using multi-scale dilated convolutions and promotes dynamic feature fusion through dimensional decoupling and multi-head self-attention mechanisms to suppress semantic interference. Finally, to overcome boundary information loss caused by downsampling, the high-frequency channel attention module is applied to skip connections, fusing high-frequency edge features extracted by the Laplacian operator with encoder features. Through an adaptive gating mechanism, this significantly enhances the decoder's ability to recover lesion contour details.

[0031] The scale variability and morphological irregularities of segmented targets in skin lesion images pose a significant challenge to the feature extraction capabilities of networks, necessitating robust multi-scale feature capture capabilities in the model. However, conventional methods of deepening or widening the backbone network not only lead to a linear increase in parameters and introduce redundant computation but also weaken the model's generalization ability. Based on this, this invention proposes a Hierarchical Parallel Perception Fusion Module (HPPF), such as... Figure 2 As shown, this module first uses small kernel convolution to extract local detail information, and then uses a set of parallel depthwise convolutions to capture multi-scale contextual information. To further establish long-distance dependencies between pixels and enhance feature representation capabilities, the CPCA module applies a dual channel and spatial attention mechanism to the fused features of the multi-branch output. Finally, the enhanced features output by the CPCA module are fused with the initial input features of the HPPF module to achieve multi-dimensional complementary enhancement of the original features, thereby effectively improving the segmentation accuracy of irregular lesions.

[0032] Input feature map First, a normalized convolutional layer, such as a 1×1 convolution, is used to adjust the channel dimensions. Then, the system is divided into four parallel branches along each channel dimension. One branch utilizes a depthwise convolution operator to extract local spatial features; for example, a 3×3 convolution is used to extract local details, avoiding the use of the computationally inefficient large square kernel found in Inception. The other two branches utilize asymmetric strip-shaped depthwise convolution operators to extract long-range structural features in the horizontal and vertical directions, i.e., large convolutional kernels. Decomposed into and The convolutional layer efficiently captures multi-scale contextual information, and the fourth branch preserves the original feature information through identity mapping. Then comes the feature aggregation and calibration step, where the features output from each branch are concatenated by channel, and the channel prior convolutional attention mechanism module is used to calibrate the importance of the channel and spatial dimensions of the concatenated multi-scale features. Finally, the residual fusion output step takes the calibrated features through a second normalized convolutional layer (e.g., a 1×1 convolution), and then sums them element-wise with the initial input features, outputting the fused features via residual connections.

[0033] Among them, such as Figure 3 As shown, a Channel Prior Convolutional Attention (CPCA) module is set in the HPPF structure to enhance the channel representation capability and spatial context modeling capability of multi-branch fusion features. The Channel Prior Convolutional Attention (CPCA) module includes: The channel prior extraction step involves performing global average pooling and global max pooling on the concatenated multi-scale features to obtain the corresponding channel description information; based on the channel description information, channel attention weights are generated and applied to the input features to obtain intermediate prior features; The multi-scale spatial augmentation step involves spatially augmenting the intermediate prior features using a multi-branch deep convolutional structure. The multi-branch deep convolutional structure includes multiple convolutional branches with different kernel sizes, used to extract spatial context information under different receptive fields. The attention mapping and output steps involve fusing the features output by each convolutional branch to generate a spatial attention map; then applying the spatial attention map to the intermediate prior features to output enhanced features.

[0034] Preferably, the enhanced features are residually fused with the initial input features of the HPPF module to improve the responsiveness to key regions while preserving the original feature information.

[0035] That is, the CPCA module processes the input feature map Two separate channel context descriptors are generated using average pooling and max pooling operations. These descriptors are then fed into a shared multilayer perceptron. To reduce parameter overhead, the hidden activation size is set to... , where r is the reduction ratio. After applying the shared network to each descriptor, element-wise summation is used to merge the output feature vectors. The feature maps are then passed through 5×5 depthwise convolutions, and then simultaneously computed in parallel through multiple depthwise strip convolutions.

[0036] In short, CPCA attention is calculated as follows: In the formula, These are input features. Indicates average pooling. This indicates max pooling. This represents a multilayer perceptron. For channel attention weights, These are the input features after channel splicing or preprocessing. The feature map after channel attention weighting. The feature map is after multi-branch spatial enhancement. This represents the sigmoid activation function. This represents depthwise convolution. , This represents the i-th branch in the graph. It is a residual connection. This is the final output of the CPCA submodule.

[0037] Compared to mechanisms like CBAM, which suffer from a uniform distribution of spatial attention weights across all channels due to channel compression calculations, CPCA generates a dynamic attention map that better reflects the actual feature distribution, significantly improving the model's ability to model complex spatial relationships. Ablation experiments (Table 7) validate the effectiveness of the CPCA module's attention mechanism. The CPCA module outperforms Squeeze-and-Excitation, SE, and the ConvolutionalBlock Attention Module and CBAM in all four key metrics: mIoU, DSC, Acc, and Spe. It only slightly lags behind SE in Sen, demonstrating the most balanced and outstanding overall performance. This indicates that it can more effectively improve feature representation and task performance within the HPPF module. Finally, the enhanced features output by the CPCA module are fused with the initial input features to achieve complementary enhancement of the original features across multiple dimensions, thereby improving the segmentation accuracy of irregular lesions.

[0038] The specific formula is as follows: In the formula, These are input features. For height-width feature branches, For wide feature branches, For high feature branches, It is an identity branch. , , For the corresponding convolutional features, With input equal, This indicates segmentation along the channel dimension. This represents the number of channels in the convolution branch. Set a ratio for the final output characteristics of the HPPF module. =0.125, passed To determine the number of branch channels. This represents depthwise convolution. This indicates that multiple vectors are concatenated along the channel dimension. This refers to the CPCA attention mechanism.

[0039] Table 1. The impact of different attention mechanisms on HPPF modules.

[0040]

[0041] Compared to natural image segmentation, skin lesion segmentation often faces challenges such as blurred lesion boundaries or low regional contrast. Comprehensive data information helps the model understand the lesion structure and its relationship with the background, thereby achieving more accurate lesion localization; local information helps capture fine corner features of the lesion area, improving prediction accuracy. To effectively integrate these two types of information and improve segmentation performance, this invention proposes a Multi-banch Semantic Collaborative Attention Module (MSCA), such as... Figure 4 As shown, this module further captures more accurate and comprehensive information through multi-scale perspectives, dimensional decoupling, and multi-semantic collaboration strategies to enhance the model's segmentation capabilities. The following sections will describe the specific operation of this module in detail.

[0042] To effectively capture features from the previous level The multi-scale spatial semantic information is first captured through a multi-scale spatial semantic capture step, which divides the feature along the channel dimension into multiple sub-features, such as four sub-features. Each sub-feature is processed by a dilated convolution with a kernel size of 3, with dilation rates set to {1, 2, 5, 7}, to extract complementary multi-scale contextual information under different receptive fields. Then, a feature integration and dimensional decoupling step is performed, concatenating the four outputs along the channel dimension and applying Group Normalization (GN). GN is more advantageous in understanding the semantic differences between sub-features. To further integrate multi-view features and compress channel redundancy, a pointwise convolution operation with a kernel size of 1 is applied to the normalized features. Finally, the resulting features are... Perform single-dimensional decomposition along both the height and width dimensions: apply global average pooling to the height dimension to generate a one-dimensional sequence. Similarly, a one-dimensional sequence is generated for the width dimension. To enhance the ability to model single-dimensional spatial structures with fine granularity, a fine-grained subspace modeling step is performed. and Each feature is divided into K independent sub-features (K=4), and the number of channels in each sub-feature is C / K, denoted as... and .

[0043] To suppress semantic interference between sub-features and avoid dilution of attention distribution, depthwise convolutions with different kernel sizes are applied to each sub-feature, with a kernel size set of {3, 5, 7, 9} to adapt to spatial structures at different scales. Subsequently, a collaborative attention calculation and output step is performed, aggregating the sub-features and unifying their scales again through group normalization. The normalized features are mapped using a sigmoid activation function to generate a spatial attention map, which adaptively enhances salient regions and suppresses irrelevant background. Then, a residual connection mechanism is used to... By integrating attention-weighted features, cross-layer information transfer and gradient flow are enhanced. This process can be formally described by the following mathematical expression.

[0044]

[0045]

[0046]

[0047] In the formula, These are input features. For block operations, Represents dilated convolutions at different dilation rates. . This indicates splicing along the channel dimension. Represents a normal convolution operation. These are the four sub-features after block division. Features after convolution This represents the GELU activation function. This is the feature map after multi-scale fusion. , These are average pooling features in the height and width directions, respectively. , Represents the i-th sub-feature. , These are the attention features after convolution, where . Let represent the depthwise convolution applied to the i-th sub-feature, where the kernel size is . . It is a group normalization, This represents the sigmoid activation function. , Attention weights are given in the height and width directions, respectively. This is the final output feature.

[0048] To efficiently utilize multi-semantic space information and enhance the synergistic effect between features, the collaborative attention computation described in this invention optimizes performance by calculating the similarity between different tokens. Specifically, the embedding representation is first generated, i.e., the features output from the aforementioned stages are processed... Average pooling is performed, followed by single-group (group=1) normalization to standardize the feature distribution. Then, 1×1 depthwise convolutions are used to generate embedded representations of the query vector (Query, Q), key vector (Key, K), and value vector (Value, V), respectively. Weight graph computation is then performed, first reconstructing the feature tensor into... The format is adapted to accommodate subsequent calculations. The formal representation indicates that the current feature has been reconstructed into a three-dimensional real tensor, with dimensions corresponding to the batch size (B), number of channels (C), and flattened spatial sequence length (N). Here, N represents the spatial sequence length, which is the product of the feature map height and width (N=H×W). Then, based on matrix multiplication and scaled dot product attention mechanisms, an attention weight map between Q and K is calculated. Adaptive recalibration is then performed; this weight map is normalized using Softmax and regularized using Dropout, and then multiplied with V to obtain the enhanced feature representation. This feature is subsequently remapped back to the original spatial dimensions. To further enhance feature discrimination capabilities, an average pooling layer is introduced to capture global semantic information, and an adaptive feature recalibration is achieved through a sigmoid activation function. Finally, residual connections are used to convert the final output features... By integrating with attention-enhanced features, local spatial details are effectively preserved, while the feature discrimination capability guided by high-level semantics is strengthened.

[0049]

[0050] In the formula, For global pooling normalization features, Indicates average pooling. Indicates group normalization, This represents a 1×1 depthwise convolution. Indicates normalization, This represents the sigmoid activation function. This is a self-attention enhancement feature. Scaling factor The number of feature dimensions. Multiply the transposes of Q and K. It is a value vector used for weighted fusion.

[0051] In skin lesion segmentation, boundary information loss often occurs due to blurred boundaries and missing details, and traditional skip connections are insufficient to effectively compensate for high-frequency details, thus affecting the accuracy of lesion contour reconstruction. To address this, this invention proposes a High Frequency Channel Attention Module (HFCA), whose core lies in the spatial hierarchical features of the co-encoder and the high-frequency edge information extracted by the Laplacian pyramid. The Laplacian operator, as a parameter-free second-order differential operator, can effectively capture high-frequency structural changes in images, especially in areas with low contrast between lesions and normal skin, where such high-frequency features play a crucial role in region differentiation. To enhance the semantic discriminative power of high-frequency features and guide the network to focus on key structural regions, the HFCA introduces a channel attention mechanism. By modeling inter-channel dependencies, it adaptively strengthens feature channels related to lesion boundaries while suppressing irrelevant background responses. Furthermore, the module structure embeds residual connections to promote stable gradient propagation in deep networks, mitigating gradient decay caused by downsampling. Compared to the direct concatenation or addition operations in traditional skip connections, HFCA employs a multi-stage downsampling adaptation and feature recalibration strategy to achieve efficient aggregation of cross-scale spatial context, significantly enhancing local edge representation while preserving global semantics. The ultimate goal of this module is to systematically compensate for the high-frequency details lost due to spatial compression in the encoder-decoder architecture, thereby comprehensively improving the segmentation accuracy of lesion contours.

[0052] The execution logic of the high-frequency context awareness module includes: a preliminary feature fusion step, which concatenates the input features with pre-extracted Laplacian geometric features. Specifically, the HFCA module receives two types of input at the i-th layer: the output features from the i-th level encoder, which are defined as... , i-level high-frequency edge features extracted with Laplacian pyramid The Laplacian pyramid is constructed by cascading a series of DoG operators, which encapsulates and represents key low-level details of an image at multiple scales.

[0053] In the formula, I is the input image. This indicates the number of pyramid levels; the default value is 4. It is a convolution operator with a Gaussian filter. This represents a 2× downsampling operation. Each level of the Laplace pyramid... By starting from the current level Subtract the smaller level obtained from the Gaussian pyramid upsampled version ( (The image of the i-th level of the Laplace pyramid). The high-frequency characteristics at a given location are provided to the i-th HFCA module.

[0054] To achieve multi-scale feature fusion, the initial feature fusion step first processes high-frequency features. Perform i-fold downsampling to match The spatial resolution is then determined, followed by concatenation along the channel dimension, and a 3×3 convolution is used for feature fusion and channel dimensionality reduction to finally generate an intermediate feature map. To effectively aggregate spatial context information, a multi-path global pooling step is then performed, which affects the intermediate feature maps. Average pooling and max pooling operations are applied in parallel to generate two complementary spatial distribution information. and Subsequently, to efficiently capture local cross-channel interaction information, a shared weight generation step is performed, employing two concatenated one-dimensional convolutions (kernel size 5) on each of the two branches. Compared to fully connected layers, this design not only significantly reduces the number of model parameters and computational complexity but also effectively models a wider range of inter-channel dependencies by expanding the receptive field, thereby obtaining deep attention features. and .

[0055] To adaptively balance the importance of features extracted from these two branches, two learnable scalar parameters α and β are introduced and optimized during training via backpropagation. The BceDice loss function used in this invention is a weighted sum of the binary cross-entropy (BCE) loss and the Dice loss, dynamically adjusting... and The contribution weights are then assigned. Finally, the weighted features are summed element-wise, and batch normalization and a sigmoid activation function are applied to generate the final channel attention map. This attention map is then compared with the intermediate features of the input. Element-wise multiplication is performed to dynamically recalibrate the feature channels, enhance key information related to high-frequency edge details, and output enhanced high-frequency context-aware features.

[0056]

[0057]

[0058]

[0059] In the formula, It is the input image of encoder layer i. It is the input image of layer i of the Laplacian pyramid. This indicates splicing along the channel dimension. This indicates an i× downsampling operation. Indicates average pooling. This indicates max pooling. This represents a one-dimensional convolution with a kernel size of 5×5. and This represents two learnable scalar parameters. It is batch normalization. It is the sigmoid activation function. This is the final output of the HFCA submodule.

[0060] Experimental Results and Analysis 1. Experimental setup All experiments were performed on a single NVIDIA RTX 3090 GPU. During data preprocessing, all images were normalized and resized to a uniform 256×256 resolution. Data augmentation strategies, including vertical flipping, horizontal flipping, and random rotation, were also employed.

[0061] During the model training phase, the BceDice loss function is used to optimize the network parameters, where the BCE loss weights are... =0.5, Dice loss weight =0.5, this combination balances pixel classification stability and lesion region overlap optimization, adapting to the class imbalance and boundary blurring problems in dermoscopic image segmentation. The optimizer chosen is AdamW, with an initial learning rate set to 1×10⁻⁵. -3 To achieve dynamic adjustment of the learning rate and promote stable convergence of the model in the later stages of training, a cosine annealing learning rate scheduler was configured, with a maximum of 50 iterations and a minimum learning rate decay to 1×10⁻⁶. -5 The entire training process consists of 300 cycles, with a batch size of 8.

[0062] 2. Comparative Experiment To comprehensively evaluate the effectiveness of the proposed MCHANet model, image segmentation comparison tests were conducted on four publicly available dermoscopy datasets. MCHANet was quantitatively compared with seven segmentation models, including UNet, Att-Unet, UNet++, UNet3+, PDF-UNet, HSH-UNet, and ConDSeg.

[0063] Table 2 shows the comparative experimental results on the ISIC2018 dataset. (Bold indicates best, underline indicates second best)

[0064] Table 3 shows the comparative experimental results on the ISIC2017 dataset. (Bold indicates best, underline indicates second best)

[0065] Tables 2 and 3 present the experimental results of MCHANet on the ISIC2017 and ISIC2018 datasets. Quantitative experimental data show that the model achieves significant advantages in key comprehensive metrics for measuring segmentation quality: on the ISIC2018 dataset, mIoU reaches 81.24%, DSC reaches 89.65%, and ACC reaches 95.00%; on the ISIC2017 dataset, these metrics are 80.43%, 89.15%, and 96.49%, respectively. Figure 6 As shown, the distribution of parameters and computational complexity clearly reveals that MCHANet is located in the "double low" region in the lower left of the coordinate axis. This significant performance gain is mainly attributed to the effectiveness of the multi-branch collaborative architecture: the hierarchical parallel perceptual fusion module effectively captures the global dependencies of lesions through global channel context, while the multi-branch semantic collaborative attention module specifically addresses the semantic ambiguity problem in low-contrast regions. The synergistic effect of both endows the model with powerful feature representation capabilities, enabling it to maintain extremely high segmentation accuracy even when facing lesions with significant scale differences.

[0066] While MCHANet performs best on key metrics such as mIoU and DSC, it exhibits specificity related to design trade-offs on some individual metrics. Specifically, MCHANet's Sen scores on the two datasets are 88.86% and 86.22%, slightly lower than ConDSeg's 92.31% and 87.74%. However, analysis of the Spe metric reveals that ConDSeg's Spe score on ISIC2018 is only 95.26%, significantly lower than MCHANet's 96.98%. This difference suggests that ConDSeg's higher sensitivity is partly due to its oversegmentation of lesion boundaries, introducing excessive background noise. In contrast, MCHANet, by introducing a high-frequency channel attention module to specifically enhance high-frequency boundary features, can more accurately remove artifacts such as hair and restore fine boundaries. This design ensures that the model achieves optimal mIoU performance while maintaining high specificity, better meeting the stringent requirements of precise lesion boundary localization in clinical surgical planning. Furthermore, although HSH-UNet and PDF-UNet achieved a slight lead of 0.05% to 0.65% in the Spe metric thanks to their complex feature interaction mechanisms, MCHANet achieved comparable background suppression capabilities with only about 2.6M parameters and surpassed them in overall accuracy. Experimental results demonstrate that MCHANet does not rely on accumulating computational resources to improve a single metric, but rather effectively solves the problem of traditional models struggling to balance segmentation accuracy and boundary integrity while maintaining a large parameter scale advantage through efficient feature decoupling and synergy mechanisms, showcasing extremely high clinical application value.

[0067] Table 4 presents the comparative experimental results on the ISIC2016 dataset. (Bold indicates best, underline indicates second best)

[0068] Table 5 shows the comparative experimental results on the PH2 dataset. (Bold indicates best, underline indicates second best)

[0069] To further verify the model's generalization ability and robustness in small-sample scenarios, this section conducted extended experiments on the smaller ISIC2016 and PH2 datasets, with results detailed in Tables 4 and 5, respectively. The experiments show that MCHANet maintains its excellent performance even under data constraints, effectively overcoming the overfitting problem easily caused by small-sample training. On the ISIC2016 dataset, MCHANet again achieved the best results in the three key metrics of mIoU, DSC, and Acc, reaching 87.11%, 93.11%, and 95.09%, respectively. Compared to the parameter-intensive ConDSeg, MCHANet not only surpasses it in segmentation accuracy but also outperforms ConDSeg's 96.01% in the Spe metric with a score of 96.36%, demonstrating that this architecture has superior discriminative stability when dealing with the problem of extreme imbalance between lesion and background classes.

[0070] This performance advantage is even more pronounced on the PH2 dataset, where MCHANet ranks first in both mIoU (91.28%) and DSC (95.44%). To more intuitively demonstrate the model's overall performance stability, Figure 7 and Figure 8 The trends of mIoU and DSC for each model across the four datasets are summarized separately. Figure 7 As can be observed in the bar chart, the data bars representing MCHANet consistently maintain the highest level, regardless of whether it's the large ISIC2018 dataset or the small PH2 dataset. This visualization strongly confirms the robustness of MCHANet. Although ConDSeg leads MCHANet in the Sen metric with a value of 96.37% (MCHANet 94.79%), combined with the PH2 dataset's high mIoU benchmark analysis (90.92%), it's clear that ConDSeg's high Sen is mainly due to its broader judgment mechanism for ambiguous areas. While this covers more lesion pixels, it inevitably misclassifies surrounding healthy skin as lesion areas, causing its Spe to drop to 95.97%. Conversely, MCHANet, through the precise capture of boundary details by its HFCA module, successfully captures lesions accurately while minimizing false positive predictions. Experimental results across four datasets confirm that MCHANet, requiring only 2.675M parameters and 2.979G of computational overhead, consistently maintains leading mIoU and DSC metrics across scenarios with varying data scales and complexities. This demonstrates that the proposed multi-branch collaborative architecture achieves a dual breakthrough in generalization performance and segmentation accuracy through the complementary fusion of deep semantics and high-frequency details.

[0071] Figure 9This visually demonstrates the qualitative segmentation results of MCHANet on the test set compared to seven other models: UNet, Att-Unet, UNet++, UNet3+, PDF-UNet, HSH-UNet, and ConDSeg. Each row in the figure represents a typical dermoscopic image case, and each column represents a different segmentation model. Image is the original input image, and Groundtruth is the expert-annotated ground truth mask.

[0072] Analysis of the visualization results reveals varying performance across different scenarios of varying complexity. In the simpler cases shown in rows 1 and 2, the lesions primarily exhibit regular elliptical shapes with high contrast, allowing most models to achieve relatively accurate segmentation. However, UNet and Att-UNet still show coarse edge processing. Row 3 displays lesion regions with extremely irregular shapes, where UNet3+ and HSH-UNet struggle to capture complex geometric transformations, resulting in noticeable jagged edges or loss of detail in the segmentation results. Rows 5 and 6 present the most challenging scenarios, with severe hair occlusion and lesions exhibiting extremely similar colors to the background. In these scenarios, ConDSeg tends to misclassify surrounding blurred skin tissue as lesions, demonstrating significant oversegmentation. Att-UNet and HSH-UNet are susceptible to high-frequency noise from hair, leading to fragmented segmentation regions. In contrast, MCHANet demonstrates superior robustness. Thanks to the high-frequency channel attention module's accurate extraction of boundary information and the multi-branch collaborative architecture's deep decoupling of semantic features, MCHANet can effectively filter out hair artifacts and accurately locate lesion contours under low contrast. Its segmentation results not only have smooth edges but also complete internal filling, maintaining the highest consistency with Groundtruth, intuitively verifying the model's superior performance in complex clinical environments.

[0073] In summary, the above results clearly demonstrate that the MCHANet model has achieved state-of-the-art segmentation performance, primarily due to the synergistic effect of the HPPF, MSCA, and HFCA modules. The HPPF module, by integrating multi-branch initial decomposition depth operations with the CPCA attention mechanism, achieves dual capture of local details and long-range contextual information, enhancing its ability to perceive multi-scale features. The MSCA module employs a multi-scale perspective and dimensionality decoupling strategy, effectively extracting multi-semantic features and enhancing the synergistic effect between features through convolutions with different dilation rates and multi-head self-attention mechanisms. This is particularly effective in handling cases of blurred lesion features and low regional contrast. The HFCA module, by fusing high-frequency features extracted from the Laplacian pyramid and encoder spatial hierarchical features, and introducing a channel attention mechanism, significantly improves the model's ability to recognize lesion edges and details. This multi-branch collaborative architecture enables MCHANet to effectively address challenges such as irregular boundaries, hair occlusion, and low contrast in skin lesion segmentation, thus achieving excellent performance on multiple evaluation metrics.

[0074] 3 Ablation Experiment 3.1 Ablation of a single module To verify the independent effectiveness of each component in MCHANT, this section conducts a single-module ablation study on the ISIC2017 dataset. The experiments use the basic five-stage U-shaped network (Baseline) as a benchmark. This baseline model features a symmetrical encoder and decoder design, with each stage containing only one ordinary convolution with a kernel size of 3. The specific settings are as follows: the HPPF module replaces the standard convolutional operations in stages 2 and 3 to enhance multi-scale perception; the MSCA module is applied in stages 4 and 5 to achieve semantic collaboration and dimensionality reduction of features; and the HFCA module is embedded at skip connections to introduce high-frequency edge information. Table 6 shows that with the individual embedding of each component, all evaluation metrics show varying degrees of improvement compared to the benchmark model. Specifically, the introduction of the HPPF module increases mIoU from 77.41% to 78.79% and DSC by 1.20%. This is mainly attributed to the parallel branching structure employed by HPPF, which efficiently captures multi-scale contextual information through decomposed large-kernel convolutions. Combined with the CPCA dynamic spatial attention mechanism, it effectively solves the problem of feature extraction for lesions with varying scales and irregular shapes. The MSCA module significantly improved mIoU to 78.78% while reducing the number of model parameters from 3.136M to 0.948M, and also reduced computational cost by 41%. This result demonstrates that MSCA, by decomposing 2D convolution into 1D sequences and depthwise convolutions, and combining this with a multi-head self-attention mechanism, ensures efficient representation of semantic features. Notably, the HFCA module performs exceptionally well, achieving an mIoU of 78.97%. This result confirms that HFCA utilizes high-frequency edge features extracted from the Laplacian pyramid and effectively compensates for lost boundary details during downsampling through a channel attention mechanism, thereby enhancing the model's ability to segment blurred areas at the edges of skin lesions.

[0075] Table 6 shows a comparison of the ablation performance of individual modules.

[0076] 3.2 Ablation of the Hybrid Module Ablation experiments of the hybrid modules were conducted on the ISIC2017 dataset. To verify the effectiveness of the hybridization of the three modules proposed in this invention, comparative experiments were performed on different combinations of the proposed modules, and the results are shown in Table 7. Here, "BL+HPPF+MSCA" represents the baseline embedding of the HPPF module and the MSCA module, "BL+HPPF+HFCA" represents the baseline embedding of the HPPF module and the HFCA module, "BL+MSCA+HFCA" represents the baseline embedding of the MSCA module and the HFCA module, and "MCHANet" is the proposed complete network architecture.

[0077] Table 7 shows the comparison of ablation performance of the hybrid modules.

[0078] As shown in Table 7, the quantitative results reveal a steady increase in the model's segmentation performance as the modules are integrated. Specifically, the "BL+HPPF+MSCA" combination achieved an mIoU of 80.00% and a DSC of 88.88%. This significant gain is attributed to the deep interaction established in the feature space between the multi-scale perception capability provided by HPPF and the semantic collaboration mechanism of MSCA, which effectively suppresses redundant information while strengthening feature representation. Meanwhile, the "BL+HPPF+HFCA" combination achieved an mIoU of 80.05% and a DSC of 88.92%, strongly demonstrating the complementary advantages between global contextual information and high-frequency boundary features. This allows the model to achieve high-precision fitting of irregular boundaries while ensuring the integrity of the lesion's semantic structure. Furthermore, the mIoU of the "BL+MSCA+HFCA" combination also improved to 79.48%, further revealing the effective coupling between the semantic features optimized by MSCA and the detailed textures extracted by HFCA, thereby further refining the segmentation boundaries based on the construction of efficient feature representations. Finally, the key metrics of the complete MCHANet architecture, mIoU and DSC, reached 80.43% and 89.15%, respectively, representing improvements of 3.02% and 1.89% compared to the baseline. These gradual improvements reflect the effectiveness of the proposed modules in the final segmentation performance, while multi-branch feature fusion enhances the network's feature capture capability. Furthermore, these modules can be well combined, ensuring that adding a module does not lead to a decrease in final accuracy.

[0079] 3.3 Visualization of Feature Heatmaps Figure 10 The feature visualization heatmaps of different stages of the ablation experiment are shown. Rows A, B, and C correspond to the feature visualization results of the model with only HPPF integration, the model with HPPF and MSCA integration, and the complete MCHANet model, respectively.

[0080] Observing row A, when only the HPPF module is integrated, although the model can locate the main lesion, confirming the effectiveness of multi-scale context extraction, the central activation region shows a state of distraction and is accompanied by a small amount of background noise interference when faced with hair occlusion or low-contrast boundaries in columns b and c. This indicates that the single spatial attention mechanism is still insufficient in focusing on the core region features. With the introduction of the MSCA module, the heatmap response in row B is significantly optimized. Thanks to the module's suppression of feature redundancy and semantic synergy, the model significantly weakens the irrelevant activation of the surrounding healthy skin region and strengthens the focus on the interior of the lesion; especially in column c, most of the hair interference is effectively filtered out, and the attention is highly focused on the core semantic region. Row C further demonstrates the optimal boundary fit of the complete MCHANT. The highlighted area not only completely covers the lesion, but also has a clear and distinguishable edge contour, which is highly consistent with the original image morphology. The visualization results confirm that the HFCA module, with the high-frequency information introduced by the Laplacian pyramid, effectively compensates for the loss of detail during the downsampling process, enabling the model to accurately capture the subtle edges of irregular lesions while maintaining semantic consistency, intuitively demonstrating the comprehensive improvement of the quantitative indicators in Tables 6 and 7.

[0081] This invention proposes a high-performance skin lesion image segmentation network, MCHANT, which solves the challenge of segmenting complex lesions through a multi-module collaborative design. The network introduces an HPPF module, utilizing a multi-branch structure, depthwise convolution decomposition, and CPCA attention mechanism to effectively model long-range pixel dependencies while extracting multi-scale context. To address feature redundancy, the MSCA module achieves dynamic fusion of multi-view features and suppression of semantic interference through multi-scale dilated convolution, dimensional decoupling, and multi-head self-attention mechanisms. Furthermore, in conjunction with the HFCA module for fusing encoder features and high-frequency information from the Laplacian pyramid, the model significantly enhances its ability to capture high-frequency details such as lesion edge contours through an adaptive gating mechanism. Experimental validation on four public datasets demonstrates that this method achieves significant performance improvements without pre-training, proving its effectiveness as an efficient and robust segmentation scheme.

[0082] In one embodiment, the present invention also provides a dermoscopic image processing system based on multi-branch feature aggregation, the system being deployed on an edge computing device equipped with an Eternal NPU and operating based on the CANN computing architecture.

[0083] The system includes a model building unit and a model execution unit, wherein: The model building unit is used to export the trained neural network model from the PyTorch framework into an ONNX intermediate model, and to perform operator adaptation, computation graph optimization and operator fusion processing on the ONNX model through the ATC tensor compiler to generate an OM offline model suitable for the target hardware platform. The model execution unit is used to load the OM offline model based on the AscendCL programming interface and perform inference tasks on the Yiteng NPU to perform pixel-level segmentation processing on the input dermoscopic image.

[0084] During model execution, the neural network model includes an encoding branch, a Laplacian pyramid branch, and a decoding branch, wherein: The encoding branch is used to extract multi-scale semantic features through multiple cascaded processing stages; The Laplacian pyramid branch is used to extract high-frequency Laplacian features at the corresponding scale; Between each processing stage, semantic features and Laplacian features are interactively fused through a high-frequency context-aware mechanism; The decoding branch is used to upsample the fused features step by step, and combined with skip connections to restore the spatial resolution, outputting the segmentation mask of the target region.

[0085] Through the above system architecture, efficient migration and deployment of the model from the training framework to the edge hardware platform are realized. Combined with the multi-branch feature fusion mechanism, the accuracy and efficiency of dermoscopic image segmentation are improved.

[0086] In one implementation, based on model format standardization, the Ascend Tensor Compiler (ATC) is used to compile the ONNX model into an offline OM model that can run on the CANN architecture. During compilation, the ATC performs operator adaptation, operator fusion, and memory access optimization on the model computation graph to realize the transformation of the model from a framework-level description to a target heterogeneous hardware execution model. After compilation, the generated OM model is deployed to the edge computing terminal Orange Pi AI Pro. The edge computing terminal has a built-in Ascend AI processing unit and loads and invokes the OM model through the AscendComputing Language (ACL) interface.

[0087] In one implementation, during system operation, the user uploads a dermoscopic image through a front-end interactive interface. The service layer receives the dermoscopic image and sends the image data to the edge computing terminal. The edge computing terminal calls the loaded OM model, completes lesion region segmentation inference on the local neural network processing unit (NPU), and returns the segmentation result to the front-end interactive interface for visualization.

[0088] Preferably, the segmentation results are displayed in the form of a mask image, a contour image, or an overlay of the original image and the segmentation mask, so as to facilitate the location and observation of the target lesion area.

[0089] Through the above model compilation and edge deployment methods, the segmentation model is fully migrated from the training environment to the edge device operating environment. This enables the system to maintain segmentation accuracy while making full use of the parallel computing capabilities of heterogeneous acceleration hardware, improving inference efficiency and interactive response speed, and reducing processing latency. This is beneficial for the edge deployment and clinical auxiliary application of the dermoscopic image segmentation model.

[0090] In a preferred embodiment, the neural network model in the system adopts a multi-branch collaborative structure, including an encoding branch, a Laplace pyramid branch, and a symmetric decoding branch.

[0091] Specifically, the coding branch consists of multiple cascaded processing stages used to extract features layer by layer from the input dermoscopy image, forming multi-scale semantic features with different receptive fields at different levels to characterize the overall structural information of the lesion region.

[0092] Meanwhile, the Laplacian pyramid branch performs multi-scale decomposition on the input image, extracts high-frequency Laplacian features at each scale, which are used to highlight boundary information and fine-grained texture features in the image, and form a scale correspondence with each processing stage in the coding branch.

[0093] In each processing stage, a high-frequency context-aware unit is set up to interactively fuse semantic features with high-frequency Laplacian features at the corresponding scale. By introducing high-frequency information to enhance boundary representation, and using semantic context to constrain noise interference, a fused feature with both structural and detailed information is obtained.

[0094] In the decoding stage, a decoding structure symmetrical to the encoding branch is adopted to upsample the fused features step by step, and the features of the corresponding encoding stage are introduced into the current layer for fusion through skip connections, so as to gradually restore the spatial resolution and retain shallow detail information, and finally generate a pixel-level segmentation mask of the target area.

[0095] By introducing Laplacian pyramid branches and high-frequency context-aware units into the system, co-modeling of semantic features and high-frequency detail information was achieved. On the one hand, the Laplacian feature is used to enhance the boundary information and detail representation of the lesion area, effectively improving the accuracy of the segmentation contour; On the other hand, semantic features provide contextual constraints, suppressing the interference of high-frequency noise on the segmentation results and improving the stability and robustness of the model. Meanwhile, by using a symmetric decoding structure and a skip connection mechanism, shallow feature information is preserved during the process of restoring spatial resolution, thereby further improving the integrity and precision of the segmentation results.

[0096] The present invention has been described above by way of example. It should be noted that any simple modifications, alterations or other equivalent substitutions that can be made by those skilled in the art without creative effort without departing from the core of the present invention fall within the protection scope of the present invention.

Claims

1. A dermoscopic image processing method based on multi-branch feature aggregation, characterized in that, include, The multi-branch feature extraction step involves inputting the image to be processed into the encoder and the Laplacian pyramid branch, respectively. The encoder extracts multi-scale semantic features through multiple cascaded processing stages, and the Laplacian pyramid branch extracts Laplacian features at the corresponding scale. The high-frequency information fusion step utilizes a high-frequency context-aware module to interactively fuse the semantic features output from each stage with the corresponding scale Laplacian features, in order to capture boundary detail information and enhance feature representation. The symmetric decoding and reconstruction step involves upsampling the fused features step by step through a decoder that is symmetrical to the encoder structure, and fusing them with the corresponding encoded features during each upsampling process, ultimately outputting a pixel-level segmentation mask of the target region in the image to be processed.

2. The dermoscopic image processing method based on multi-branch feature aggregation according to claim 1, characterized in that, The encoding network branch includes five cascaded processing stages. In the first stage, shallow spatial features are extracted through a 3×3 convolutional layer. In the second and third stages, feature processing is performed using a hierarchical parallel perceptual fusion module. In the fourth and fifth stages, deep semantic features are extracted using a multi-branch semantic collaborative attention module. The hierarchical parallel perception fusion module synchronously models local details and long-range dependencies through parallel multi-branch deep convolution and global channel context weights. In the decoding network branch, a 1×1 convolutional layer corresponding to the first stage is used to perform channel integration and dimensionality compression on the features fused by skip connections.

3. The dermoscopic image processing method based on multi-branch feature aggregation according to claim 1, characterized in that, The processing steps of the hierarchical parallel perception fusion module include: The feature splitting process involves processing the input features using the first normalized convolutional layer and dividing the processed features into multiple feature sub-streams along the channel dimension using a channel splitting operator. The multi-dimensional parallel perception step inputs each feature substream after division into corresponding parallel processing branches, and extracts local spatial features and long-range structural features simultaneously through depth convolution operators and asymmetric strip depth convolution operators of different dimensions. The feature aggregation and calibration steps involve concatenating the features output from each branch and using the channel prior convolutional attention mechanism module to calibrate the importance of the channel and spatial dimensions of the concatenated multi-scale features. The residual fusion output step involves summing the calibrated features element-wise with the initial input features and outputting the fused features using a residual connection method.

4. The dermoscopic image processing method based on multi-branch feature aggregation according to claim 3, characterized in that, The processing steps of the channel prior convolutional attention mechanism module include: Global average pooling and global max pooling are performed on the spliced ​​multi-scale features to generate corresponding channel description information; Based on the channel description information, channel attention weights are generated, and the channel attention weights are used to weight the concatenated multi-scale features to obtain intermediate prior features. Spatial augmentation is performed on the intermediate prior feature input multi-branch deep convolutional structure to extract spatial context information under different receptive fields; The features output by the multi-branch deep convolutional structure are fused to generate a spatial attention map; The intermediate prior features are weighted based on the spatial attention map to output enhanced features.

5. The dermoscopic image processing method based on multi-branch feature aggregation according to claim 1, characterized in that, The execution logic of the multi-branch semantic collaborative attention module includes: The multi-scale spatial semantic capture step divides the input features into multiple sub-features along the channel dimension, and processes them in parallel using dilated convolution operators with different dilation rates to extract complementary multi-scale contextual information. The feature integration and dimensional decoupling steps involve concatenating the processed sub-features by channels and performing group normalization and pointwise convolution operations in sequence to compress channel redundancy. Subsequently, the resulting features are decomposed into single dimensions along the height and width dimensions to generate two one-dimensional sequence features representing the spatial structure. The fine-grained subspace modeling step divides each of the one-dimensional sequence features into multiple independent sub-features, and applies depth convolution operators with different kernel sizes to each independent sub-feature for processing, so as to adapt to spatial structures of different scales and suppress semantic interference. The collaborative attention calculation and output steps involve converging and reconstructing the processed sub-features, calculating the self-attention weight mapping through matrix multiplication operators and normalized exponential functions, and finally applying the weight mapping to the initial input features to output enhanced multi-semantic collaborative features.

6. The dermoscopic image processing method based on multi-branch feature aggregation according to claim 5, characterized in that, The specific steps for calculating collaborative attention include: Embedded representation generation: Average pooling and group normalization are performed on the preprocessed features, and query vector, key vector and value vector are generated respectively using 1×1 depthwise convolution operator; Weight graph calculation: The query vector and the key vector are reconstructed into a preset dimension form, and the similarity weight graph between them is calculated based on the matrix multiplication operator. Normalization and regularization processing are then performed on the weight graph in sequence. Adaptive recalibration: The processed weight map is multiplied with the value vector to obtain attention-enhanced features; after remapping the attention-enhanced features back to the original spatial dimension, adaptive recalibration of the features is achieved using an average pooling layer and a sigmoid activation function. Residual fusion: The recalibrated features are fused with the preprocessed features through residual connections to enhance the feature discrimination capability guided by high-level semantics.

7. The dermoscopic image processing method based on multi-branch feature aggregation according to claim 3, characterized in that, The execution logic of the high-frequency context awareness module includes: The initial feature fusion step involves concatenating the input features with the pre-extracted Laplacian geometric features through channels, and then integrating spatial information using a 3×3 convolutional layer to obtain an intermediate feature map. The multi-path global pooling step inputs the features to be perceived into two different global average pooling operators in parallel to capture the spatial context descriptors of the features from different dimensions. The shared weight generation step involves inputting the two sets of spatial context descriptors into two twice-cascaded shared weight one-dimensional convolution operators to perform local cross-channel interaction, generating preliminary deep attention features. Then, two learnable scalar adjustment factors are used to weight the two sets of preliminary deep attention features, and the weighted features are subjected to element-wise addition, batch normalization, and activation function processing to generate the final channel attention map. The adaptive feature enhancement step involves performing element-wise multiplication between the channel attention map and the intermediate feature map to output enhanced high-frequency context-aware features.

8. A dermoscopic image processing system based on multi-branch feature aggregation, characterized in that, This includes a model development platform and edge inference devices with communication connectivity; The model development platform includes: The model building unit is used to build a multi-branch feature aggregation neural network model including an encoding branch, a Laplacian pyramid branch, and a symmetric decoding branch. The model achieves interactive fusion of semantic features and high-frequency features through a high-frequency context awareness mechanism at each processing stage. The format conversion engine is used to obtain the initial model weight file of the neural network model trained under the first deep learning framework, export it as an intermediate model in the Open Neural Network Exchange Format, and perform topology verification and operator validity checks on the intermediate model. The heterogeneous compilation engine, with a built-in Ascend tensor compiler, is used to receive the verified intermediate model and perform operator fusion and computation graph optimization processing on the multi-branch feature structure to compile the intermediate model into an offline model adapted to the target heterogeneous hardware instruction set. The edge inference device is an edge computing node equipped with a neural network processing unit (NPU). The edge inference device is used for: Obtain the offline model issued by the model development platform; The offline model is loaded into the computing architecture corresponding to the neural network processing unit through the heterogeneous computing programming interface integrated within the device, so as to drive the hardware to perform parallel inference computing and output pixel-level segmentation results based on the multi-branch feature fusion results.

9. The dermoscopic image processing system based on multi-branch feature aggregation according to claim 8, characterized in that, The neural network model includes an encoding branch, a Laplacian pyramid branch, and a symmetric decoding branch, wherein: The encoding branch includes multiple cascaded processing stages for extracting multi-scale semantic features step by step; The Laplacian pyramid branch is used to perform multi-scale decomposition of the input image and extract high-frequency Laplacian features corresponding to each processing stage. High-frequency context-aware units are set up in each processing stage to interactively fuse the semantic features and Laplacian features of the corresponding stage. The symmetric decoding branch is used to upsample the fused features step by step, and to fuse them with the corresponding encoding stage features through skip connections during each upsampling process, so as to output pixel-level segmentation results.

10. The dermoscopic image processing system based on multi-branch feature aggregation according to claim 9, characterized in that, The model development platform is used to compile the ONNX intermediate model into an OM offline model that can run under the CANN architecture using the Ascend Tensor Compiler (ATC). The edge inference device is an edge computing terminal with a built-in Ascend AI processing unit, and loads the OM offline model through the Ascend Computing Language (ACL) interface to perform lesion segmentation inference on the local neural network processing unit.

Images