A dual-path skin lesion image segmentation method based on lightweight multi-scale fusion

Abstract

The present invention proposes a dual-path skin lesion image segmentation method based on lightweight multi-scale fusion. The method achieves efficient and accurate image segmentation in a resource-constrained environment through an innovative network architecture and a multi-scale fusion module. The method includes the acquisition and preprocessing of a data set, the construction of a lightweight network, iterative training and verification, model selection based on multiple evaluation indicators, and the application of the best model to obtain the segmentation results of the lesion area. The method of the present invention not only maintains high segmentation accuracy, but also reduces computing and storage requirements. It is suitable for mobile medical care and remote diagnosis and has important clinical application value.

Description

Technical Field

[0001] The present invention relates to the field of image processing and analysis, and in particular to a dual-path skin lesion image segmentation method based on lightweight multi-scale fusion. Background Art

[0002] With the rapid development of mobile medical and remote diagnosis technology, the segmentation of skin lesion images has become an important research field. The segmentation of skin lesion images is a key step in the early diagnosis of skin diseases such as skin cancer. It can help doctors quickly locate the lesion area and provide important information for subsequent diagnosis and treatment.

[0003] Traditional skin lesion segmentation methods mainly rely on manual feature extraction and classic image processing techniques such as edge detection, region growing, etc. However, these methods often require a lot of prior knowledge, have poor adaptability to the diversity and complexity of lesions, and have limited segmentation accuracy.

[0004] In recent years, with the rise of deep learning technology, automatic segmentation methods based on convolutional neural networks have made significant progress. In particular, U-Net and its variants have been widely used in the field of medical image segmentation. U-Net can effectively capture the contextual information in the image and maintain the spatial resolution of the image through its symmetrical encoder-decoder structure and jump connection, thus achieving good results in the task of skin lesion segmentation.

[0005] However, U-Net and its derivative models are still large in terms of computing resources and number of parameters, which limits their deployment on resource-constrained mobile devices and edge computing platforms.

[0006] In addition, these models still face certain challenges when dealing with lesion and normal skin areas with high similarity, especially in distinguishing subtle lesion boundaries.

[0007] As an emerging model, the Transformer architecture has shown strong ability in capturing long-distance dependencies with its self-attention mechanism. However, the standard Transformer model faces the problems of low computational efficiency and difficulty in adapting to local features in image segmentation tasks.

[0008] Directly applying Transformer to the skin lesion segmentation task also faces the difficulty of deployment on mobile devices due to its high demand for computing resources.

[0009] In summary, the development of skin lesion image segmentation technology can not only improve the work efficiency and diagnostic accuracy of clinicians, but is also expected to be more widely used in the future, providing more powerful support for the early diagnosis and treatment of skin diseases.

[0010] In view of the above problems, the present invention proposes a dual-path skin lesion image segmentation method based on lightweight multi-scale fusion, which can not only maintain high segmentation accuracy but also significantly reduce the computing and storage requirements.

[0011] By designing a novel dual-path network architecture and multi-scale fusion module, the present invention can achieve efficient and accurate skin lesion segmentation in resource-constrained environments, providing strong technical support for mobile medical care and remote diagnosis.

[0012] In order to achieve this goal, the present invention mainly comprises the following steps:

[0013] Step 1: Obtain a skin lesion image dataset, preprocess the skin lesion image dataset, and divide it into a training set and a validation set;

[0014] Step 2: Construct a dual-path skin lesion image segmentation network based on lightweight multi-scale fusion. The lightweight architecture innovation of this network is mainly reflected in two key aspects: first, the dual-path design for skin lesion image processing, and second, the innovative implementation of the multi-scale fusion module in the low-resolution path;

[0015] Step 3: Use the training set and validation set to perform multiple rounds of iterative training and validation on the lightweight multi-scale fusion-based dual-path skin lesion image segmentation network to monitor whether it is overfitting and provide feedback for further network optimization;

[0016] Step 4: Select the best network model based on the scores of each model on multiple evaluation indicators, and save the model parameters of the best network model;

[0017] Step 5: Use the best network model saved in step 4 to segment the input skin lesion image and obtain the segmentation result of the lesion area.

[0018] Furthermore, in step 1, the skin lesion images in the data set are adjusted to an image size of 256 × 256. In order to ensure that the network model proposed in the present invention has sufficient data for learning during the training process, and the validation set is used to adjust the network model parameters and prevent overfitting, the present invention divides the data set into a training set and a validation set in a ratio of 7:3.

[0019] As a further improvement of the present invention, in step 2, on the one hand, a dual-path design is performed, including a high-resolution path and a low-resolution path, and on the other hand, a multi-scale fusion module is designed. The specific description is:

[0020] Step 2.1: For the building blocks of the dual path, the present invention innovatively designs a three-stage feature extraction block as the basis for building the dual path. Each three-stage feature extraction block consists of a series of efficient convolution operations, including a 1×1 convolution for dimensionality reduction, a 3×3 depth convolution for feature extraction, and another 1×1 convolution for dimensionality increase;

[0021] Furthermore, the design of the building blocks of the dual paths in step 2.1 is specifically described as follows:

[0022] Step 2.1.1: Use a 1×1 convolutional layer to reduce the dimensionality of the input feature map to reduce the number of parameters and computational load of subsequent layers while ensuring that the depth information of the feature map is effectively preserved;

[0023] Step 2.1.2: Apply 3×3 depthwise convolution to process the features of each channel independently, focusing on extracting the deep features of each channel without mixing them in the spatial dimension, so as to reduce the amount of computation while maintaining the richness and expressiveness of the features;

[0024] Step 2.1.3: After the 3×3 depthwise convolution, batch normalization is used to normalize the data, and then the ReLU activation function is used to introduce nonlinearity. This combination accelerates the training process and promotes rapid convergence of the model.

[0025] Step 2.1.4: Finally, another 1×1 convolutional layer is used to perform feature dimensionality upscaling to restore the dimension of the feature map to the required output size to ensure that the feature information can be effectively used in subsequent network layers;

[0026] Step 2.1.5: Provide a flexible configuration option that allows the ReLU activation function to be selectively removed from the output of the three-stage feature extraction block under specific task requirements to adapt to different application scenarios.

[0027] Step 2.2: In the low-resolution path, the input image undergoes a series of downsampling operations with scaling factors of 1 / 4, 1 / 8, 1 / 16, and 1 / 32, gradually reducing the image size, while image features are extracted and processed by a carefully designed three-stage feature extraction block.

[0028] At the end of this path, a multi-scale fusion module is used to adaptively integrate features of different scales to enhance the ability of feature representation;

[0029] Furthermore, the design of the dual-path multi-scale fusion module in step 2.2 is specifically described as follows:

[0030] Step 2.2.1: Use a 1×1 convolutional layer to perform channel compression on the input feature map. The formula is as follows:

[0031] ,

[0032] Conv1( ) represents a 1×1 convolution operation, X is the input feature map, K is the convolution kernel, and C 输入 is the number of input channels, C 输出 is the number of channels after compression.

[0033] Step 2.2.2: Apply dilated convolution with different dilation rates to the compressed feature map for multi-scale feature extraction. For the i-th scale, the feature extraction is expressed as:

[0034] ,

[0035] F i represents the feature map extracted by dilated convolution at the i-th scale, R represents the compressed feature map, and d i represents the dilation rate of the atrous convolution at the i-th scale (d = 1, 2, 4, 8), so i = 1, 2, 3, 4;

[0036] Step 2.2.3: Use global average pooling GAP (R) combined with Conv1 and SiLU activation function to calculate the attention weight A:

[0037] ,

[0038] The SiLU activation function is defined as:

[0039] ,

[0040] Among them, z represents the input value of the activation function, σ(z) represents the Sigmoid function, and the SiLU function transforms the input value z nonlinearly through a Sigmoid function, and then multiplies the result of this transformation by the original input value z to generate the final output. σ(z) compresses any real number input to between 0 and 1, so that the output range of the SiLU function is between 0 and the input value z itself;

[0041] Step 2.2.4: Multiply the feature maps extracted at different scales by the corresponding attention weights element by element to obtain the weighted feature maps, and then merge them in the channel dimension. The specific calculation formula is as follows:

[0042] , ,

[0043] Among them, F i is the feature map extracted at different scales, M' represents the attention weight, F i ' is the obtained weighted feature map, Concat means the connection operation in the channel dimension;

[0044] Step 2.2.5: Use 1×1 convolution to compress the merged feature map F' to generate the final output feature map. The calculation formula is as follows:

[0045] ,

[0046] Among them, Out is the final output feature map.

[0047] Step 2.3: The high-resolution path starts from the initial feature extraction stage and maintains the image size after downsampling by a scaling factor of 1 / 8 to achieve finer feature extraction. The third three-stage feature extraction block and the fourth three-stage feature extraction block use bilinear interpolation and downsampling techniques to integrate the features from the two paths;

[0048] Step 2.4: An auxiliary segmentation head is introduced after the third three-stage feature extraction block stage of the high-resolution path to provide additional supervision signals to accelerate the convergence of the network model proposed in the present invention and enhance the network model's ability to capture key features.

[0049] Furthermore, for step 2.4, the auxiliary segmentation head provides outputs in the third three-stage feature extraction block of the network, which are used to calculate the auxiliary loss, providing the model with additional gradient information in the early stages of training, thereby helping to accelerate the convergence of the model. The specific description is:

[0050] Step 2.4.1: After the third three-stage feature extraction block of the network, an auxiliary segmentation head is integrated to provide additional prediction and supervision signals during training;

[0051] Step 2.4.2: Design an auxiliary loss function to calculate the prediction loss of the auxiliary segmentation head;

[0052] Step 2.4.3: Introduce weight parameters α and β to adjust the contribution ratio between the auxiliary loss function and the main loss function. The specific implementation formula is as follows:

[0053] ,

[0054] Among them, L total Represents the final loss function, L ohem represents the main loss function, L dice is the loss term generated by the auxiliary segmentation head;

[0055] Specific L ohem and L dice The function implementation formula is as follows:

[0056] ,

[0057] ,

[0058] Where N is the number of samples; p target is the predicted probability of the target class by the network model of the present invention; target Represents the true label, where the value 1 represents the skin lesion area and 0 represents the skin non-lesion area.

[0059] Step 2.4.4: Use the auxiliary loss function and the main loss function designed in step 2.4.3 to jointly guide the network parameter update to improve the model's convergence speed and segmentation performance.

[0060] Furthermore, in step 3, the iterative training and verification process adopts the following strategy:

[0061] On the one hand, the Adam optimizer is used to optimize the network weights, and the weight decay rate is set to 0.0001 to regularize the model and prevent overfitting;

[0062] On the one hand, a polynomial decay strategy is used to manage the learning rate. The initial learning rate is set to 0.01 and gradually decreases according to the decay exponent 0.9 to help the model converge smoothly during the training process;

[0063] On the other hand, the training process iterates 20,000 times, and each batch is set to 8 samples to ensure that the model can effectively learn from the data in each iteration.

[0064] Furthermore, in step 4, the evaluation indicators include mean intersection over union (mIoU), Dice similarity coefficient (DSC), accuracy (Acc) and sensitivity (Sen), which comprehensively evaluate the segmentation ability of the model. The network model with the best performance is selected based on the scores of these evaluation indicators, and its parameters are saved for subsequent use. The specific calculation formula is as follows:

[0065] ,

[0066] Among them, TP represents the correctly identified lesion area, FP represents the incorrectly identified normal skin area, TN represents the correctly unlabeled area of ​​normal skin, and FN represents the missed lesion area;

[0067] Furthermore, in step 5, the saved optimal network model is applied to the new skin lesion image, and the accurate segmentation result of the lesion area is obtained through forward propagation, which provides important information for clinical diagnosis.

[0068] As a further improvement of the present invention, the execution of steps 4 and 5 pays special attention to the balance between efficiency and performance, and particularly considers the computational complexity and number of parameters of the model. The quantification of computational complexity is achieved by measuring the number of floating point operations (GFLOPs), which is directly related to the running efficiency of the model on the hardware.

[0069] The technical solution provided by the present invention has the following three main beneficial effects:

[0070] The dual-path skin lesion image segmentation method based on lightweight multi-scale fusion proposed in the present invention can effectively process skin lesion images, and achieves high-precision segmentation of the lesion area through an innovative dual-path network architecture and multi-scale fusion module.

[0071] The method of the present invention combines an auxiliary segmentation head and an auxiliary loss function, and significantly reduces the computational complexity and storage requirements of the model through optimized network design, so that the model can run efficiently even in a resource-constrained environment, and is suitable for mobile medical and remote diagnosis applications.

[0072] The present invention ensures high segmentation accuracy of the model through multi-dimensional evaluation indicators (mIoU, DSC, Acc, Sen), and achieves lightweight design by optimizing computational complexity (GFLOPs). This design enables the model to run quickly on resource-constrained devices while maintaining efficient segmentation performance, making it suitable for mobile medical and remote diagnosis scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] The drawings in the specification, which constitute a part of the present application, are used to provide further understanding of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute improper limitations on the present application.

[0074] Figure 1 This is a flowchart of the dual-path skin lesion image segmentation method based on lightweight multi-scale fusion in this application.

[0075] Figure 2 This is the network architecture diagram of the dual-path skin lesion image segmentation method based on lightweight multi-scale fusion in this application.

[0076] Figure 3 This is the specific design diagram of the three-stage feature extraction block of this application.

[0077] Figure 4 Schematic diagram of the internal structure of the multi-scale feature fusion module designed for this application.

[0078] Figure 5 This is a diagram showing the skin lesion segmentation effect according to an embodiment of the present application. DETAILED DESCRIPTION

[0079] The present application is further described in detail below in conjunction with embodiments and with reference to the accompanying drawings.

[0080] One embodiment of the present application is a dual-path skin lesion image segmentation method based on lightweight multi-scale fusion. The overall process is as follows: Figure 1 As shown, the following steps are included:

[0081] Step 1: Obtain a skin lesion image dataset, preprocess the skin lesion image dataset, and divide it into a training set and a validation set;

[0082] Step 2: Construct a dual-path skin lesion image segmentation network based on lightweight multi-scale fusion. The lightweight architecture innovation of this network is mainly reflected in two key aspects: first, the dual-path design for skin lesion image processing, and second, the innovative implementation of the multi-scale fusion module in the low-resolution path;

[0083] Step 3: Use the training set and validation set to perform multiple rounds of iterative training and validation on the lightweight multi-scale fusion-based dual-path skin lesion image segmentation network to monitor whether it is overfitting and provide feedback for further network optimization;

[0084] Step 4: Select the best network model based on the scores of each model on multiple evaluation indicators, and save the model parameters of the best network model;

[0085] Step 5: Use the best network model saved in step 4 to segment the input skin lesion image and obtain the segmentation result of the lesion area.

[0086] Furthermore, in step 1, the skin lesion images in the data set are adjusted to an image size of 256 × 256. In order to ensure that the network model proposed in the present invention has sufficient data for learning during the training process, and the validation set can be used to adjust the network model parameters and prevent overfitting, the present invention divides the data set into a training set and a validation set in a ratio of 7:3.

[0087] As a further improvement of the present invention, in step 2, the network architecture of the dual-path skin lesion image segmentation method based on lightweight multi-scale fusion is as follows: Figure 2 As shown in FIG. 1 , on the one hand, there is a dual-path design, including a high-resolution path and a low-resolution path, and on the other hand, there is a multi-scale fusion module design. The specific description is:

[0088] Step 2.1: For the building blocks of the dual path, the present invention innovatively designs a three-stage feature extraction block as the basis for building the dual path. Each three-stage feature extraction block consists of a series of efficient convolution operations, including a 1×1 convolution for dimensionality reduction, a 3×3 depth convolution for feature extraction, and another 1×1 convolution for dimensionality increase;

[0089] Furthermore, for the dual path building block design in step 2.1, Figure 3 As shown, it can be specifically described as:

[0090] Step 2.1.1: Use a 1×1 convolutional layer to reduce the dimensionality of the input feature map to reduce the number of parameters and computational load of subsequent layers while ensuring that the depth information of the feature map is effectively preserved;

[0091] Step 2.1.2: Apply 3×3 depthwise convolution to process the features of each channel independently, focusing on extracting the deep features of each channel without mixing them in the spatial dimension, so as to reduce the amount of computation while maintaining the richness and expressiveness of the features;

[0092] Step 2.1.3: After the 3×3 depthwise convolution, batch normalization is used to normalize the data, and then the ReLU activation function is used to introduce nonlinearity. This combination accelerates the training process and promotes rapid convergence of the model.

[0093] Step 2.1.4: Finally, another 1×1 convolutional layer is used to perform feature dimensionality upscaling to restore the dimension of the feature map to the required output size to ensure that the feature information can be effectively used in subsequent network layers;

[0094] Step 2.1.5: Provide a flexible configuration option that allows the ReLU activation function to be selectively removed from the output of the three-stage feature extraction block under specific task requirements to adapt to different application scenarios.

[0095] Step 2.2: In the low-resolution path, the input image undergoes a series of downsampling operations with scaling factors of 1 / 4, 1 / 8, 1 / 16, and 1 / 32, gradually reducing the image size, while image features are extracted and processed through a carefully designed three-stage feature extraction block. At the end of this path, a multi-scale fusion module is used to adaptively integrate features of different scales to enhance the ability of feature representation;

[0096] Furthermore, for the design of the dual-path multi-scale fusion module in step 2.2, Figure 4 As shown, it can be specifically described as:

[0097] Step 2.2.1: Use a 1×1 convolutional layer to perform channel compression on the input feature map. The formula is as follows:

[0098] ,

[0099] Conv1( ) represents a 1×1 convolution operation, X is the input feature map, K is the convolution kernel, and C 输入 is the number of input channels, C 输出 is the number of channels after compression.

[0100] Step 2.2.2: Apply dilated convolution with different dilation rates to the compressed feature map for multi-scale feature extraction. For the i-th scale, the feature extraction is expressed as:

[0101] ,

[0102] F i represents the feature map extracted by dilated convolution at the i-th scale, R represents the compressed feature map, and d i represents the dilation rate of the dilated convolution at the i-th scale (d = 1, 2, 4, 8), so i = 1, 2, 3, 4;

[0103] Step 2.2.3: Use global average pooling GAP (R) combined with Conv1 and SiLU activation function to calculate the attention weight A:

[0104] ,

[0105] The SiLU activation function is defined as:

[0106] ,

[0107] Among them, z represents the input value of the activation function, σ(z) represents the Sigmoid function, and the SiLU function transforms the input value z nonlinearly through a Sigmoid function, and then multiplies the result of this transformation by the original input value z to generate the final output. σ(z) compresses any real number input to between 0 and 1, so that the output range of the SiLU function is between 0 and the input value z itself;

[0108] Step 2.3.4: Multiply the feature maps extracted at different scales by the corresponding attention weights element by element to obtain the weighted feature maps, and then merge them in the channel dimension. The specific calculation formula is as follows:

[0109] ,

[0110] Among them, F i is the feature map extracted at different scales, M' represents the attention weight, F i ' is the obtained weighted feature map, Concat means the connection operation in the channel dimension;

[0111] Step 2.3.5: Use 1×1 convolution to compress the merged feature map F' to generate the final output feature map. The calculation formula is as follows:

[0112] ,

[0113] Among them, Out is the final output feature map.

[0114] Step 2.3: The high-resolution path starts from the initial feature extraction stage and maintains the image size after downsampling by a scaling factor of 1 / 8 to achieve finer feature extraction. The third three-stage feature extraction block and the fourth three-stage feature extraction block use bilinear interpolation and downsampling techniques to integrate the features from the two paths;

[0115] Step 2.4: An auxiliary segmentation head is introduced after the third three-stage feature extraction block stage of the high-resolution path to provide additional supervision signals to accelerate the convergence of the network model proposed in the present invention and enhance the network model's ability to capture key features.

[0116] Furthermore, for step 2.4, the auxiliary segmentation head provides outputs in the third three-stage feature extraction block of the network, which are used to calculate the auxiliary loss, providing the model with additional gradient information in the early stages of training, thereby helping to accelerate the convergence of the model. It can be described as:

[0117] Step 2.4.1: After the third three-stage feature extraction block of the network, an auxiliary segmentation head is integrated to provide additional prediction and supervision signals during training;

[0118] Step 2.4.2: Design an auxiliary loss function to calculate the prediction loss of the auxiliary segmentation head;

[0119] Step 2.4.3: Introduce weight parameters α and β to adjust the contribution ratio between the auxiliary loss function and the main loss function. The specific implementation formula is as follows:

[0120] ,

[0121] Among them, L total Represents the final loss function, L ohem represents the main loss function, L dice is the loss term generated by the auxiliary segmentation head.

[0122] Specific L ohem and L dice The function implementation formula is as follows:

[0123] ,

[0124] ,

[0125] Where N is the number of samples; p target is the predicted probability of the target class by the network model of the present invention; target Represents the true label, where the value 1 represents the skin lesion area and 0 represents the skin non-lesion area.

[0126] Step 2.4.4: Use the auxiliary loss function and the main loss function designed in step 2.4.3 to jointly guide the network parameter update to improve the model's convergence speed and segmentation performance.

[0127] Furthermore, in step 3, the iterative training and verification process adopts the following strategy:

[0128] On the one hand, the Adam optimizer is used to optimize the network weights, which accelerates convergence and improves training efficiency by adaptively adjusting the learning rate.

[0129] The weight decay rate is set to 0.0001 to regularize the model and effectively prevent overfitting.

[0130] On the one hand, a dynamic adjustment strategy of the learning rate is implemented. The initial learning rate is set to 0.01, and the learning rate is gradually reduced according to the polynomial decay strategy with a decay exponent of 0.9 to ensure that the model converges quickly in the early stage of training and is carefully adjusted in the later stage.

[0131] On the one hand, the training process is set to 20,000 iterations, with each batch containing 8 samples, to achieve a balance between computational efficiency and model generalization ability, ensuring that the model can stably learn from the data;

[0132] On the other hand, during the training process, the performance of the model on the validation set is continuously monitored to assess the risk of overfitting, and an early stopping strategy is adopted when necessary to avoid the model from overfitting the training data.

[0133] At the same time, a model checkpoint mechanism is implemented to periodically save the best performing model state during training so that the model with the best performance can be restored at the end of training.

[0134] Furthermore, step 4 involves a comprehensive evaluation of the network model performance to determine the optimal model and save its parameters, which is specifically described as follows:

[0135] On the one hand, the evaluation indicators include:

[0136] Mean Intersection over Union (mIoU): measures the average value of the overlap between the skin lesion segmentation area predicted by the model and the actual labeled area, reflecting the segmentation accuracy of the model.

[0137] Dice Similarity Coefficient (DSC): The segmentation similarity of the model is evaluated by calculating the ratio of twice the intersection of the predicted skin lesion segmentation area and the actual annotated area to the union of the two areas.

[0138] Accuracy (Acc): reflects the model's ability to correctly predict (including skin lesion areas and normal skin areas).

[0139] Sensitivity (Sen): Also known as recall, it measures the model’s ability to identify skin lesion areas.

[0140] The present invention comprehensively compares the performance of different models through the scores of these evaluation indicators and selects the network model with the best performance in all evaluation indicators.

[0141] The selected model is considered to be the most suitable for the purpose of the present invention and its parameters are then saved for use in subsequent skin lesion segmentation tasks.

[0142] The specific calculation formula is as follows:

[0143] ,

[0144] Among them, TP represents the correctly identified lesion area, FP represents the incorrectly identified normal skin area, TN represents the correct unlabeled area of ​​normal skin, and FN represents the missed lesion area.

[0145] Furthermore, in step 5, the present invention involves segmenting the new skin lesion image using the optimal network model selected in step 4 to obtain an accurate segmentation result of the lesion area. The specific description is:

[0146] Step 5.1: Load the optimal network model parameters evaluated and saved in step 4 into the model architecture to ensure that the model can perform the segmentation task in the best state;

[0147] Step 5.2: Preprocess the new skin lesion image, including image resizing, normalization, etc., to ensure that the image data meets the requirements of the model input;

[0148] Step 5.3: Input the preprocessed image into the model, perform the forward propagation process, and use the deep structure of the model to extract and analyze features to predict the lesion area;

[0149] Step 5.4: Based on the output of the model, the continuous prediction results are converted into binary segmentation masks through appropriate thresholding or classification techniques to accurately define the lesion area;

[0150] Step 5.5: Verify the generated segmentation results to ensure that they meet the accuracy requirements for clinical applications. If necessary, fine-tune the model or post-processing steps based on the verification results;

[0151] Step 5.6: Provide clinical diagnosis information. Provide the verified accurate segmentation results to clinicians to assist in accurate diagnosis and treatment planning, thereby improving clinical work efficiency.

[0152] Step 5.7: Continuously monitor the performance of the model in actual applications and establish a feedback mechanism to collect feedback from clinicians to further optimize model performance.

[0153] As a further improvement of the present invention, the execution of step 4 and step 5 pays special attention to the balance between efficiency and performance. Furthermore, in step 5, the best model selected in step 4 is used to accurately segment the image, and the result is as follows: Figure 5 As shown, these results show the effectiveness and accuracy of the model in practical applications. In order to highlight the lightweight advantage of the present invention, the present invention also specifically considers the computational complexity and number of parameters of the model. The quantification of computational complexity is achieved by measuring the number of floating-point operations (GFLOPs), which is directly related to the operating efficiency of the model on hardware. In this way, the present invention not only pursues segmentation accuracy, but also focuses on the practicality and deployment feasibility of the model in resource-constrained environments.

[0154] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. The present invention proposes a dual-path skin lesion image segmentation method based on lightweight multi-scale fusion, comprising the following steps: Step 1: Obtain a skin lesion image dataset, preprocess the skin lesion image dataset, and divide it into a training set and a validation set; Step 2: Construct a lightweight image segmentation network for skin lesions based on dual-path multi-scale fusion. The lightweight architecture of this network is reflected in two aspects: first, the dual-path design for skin lesion image processing, and second, the implementation of the multi-scale fusion module in the low-resolution path: Step 2.1: For the building blocks of the dual path, the present invention designs a three-stage feature extraction block as the basis for building the dual path, each of which is composed of efficient convolution operations, including a 1×1 convolution for dimensionality reduction, a 3×3 depth convolution for feature extraction, and another 1×1 convolution for dimensionality increase; Furthermore, the design of the building blocks of the dual paths in step 2.1 is specifically described as follows: Step 2.1.1: Use a 1×1 convolutional layer to reduce the dimensionality of the input feature map to reduce the number of parameters and computational load of subsequent layers while ensuring that the depth information of the feature map is effectively preserved; Step 2.1.2: Apply 3×3 depthwise convolution to process the features of each channel independently, focusing on extracting the deep features of each channel without mixing them in the spatial dimension, so as to reduce the amount of computation while maintaining the richness and expressiveness of the features; Step 2.1.3: After the 3×3 depthwise convolution, batch normalization is used to normalize the data, and then the ReLU activation function is used to introduce nonlinearity. This combination accelerates the training process and promotes rapid convergence of the model. Step 2.1.4: Finally, another 1×1 convolutional layer is used to perform feature dimensionality upscaling to restore the dimension of the feature map to the required output size to ensure that the feature information can be effectively used in subsequent network layers; Step 2.1.5: Provide a configuration option to allow the ReLU activation function to be selectively removed from the output of the three-stage feature extraction block to adapt to different application scenarios; Step 2.2: In the low-resolution path, the input image undergoes a downsampling operation with scaling factors of 1 / 4, 1 / 8, 1 / 16, and 1 / 32, gradually reducing the image size. At the same time, the image features are extracted and processed through a three-stage feature extraction block. At the end, a multi-scale fusion module is used to adaptively integrate features of different scales to enhance the ability of feature representation. Furthermore, the design of the dual-path multi-scale fusion module in step 2.2 is specifically described as follows: Step 2.2.1: Use a 1×1 convolutional layer to perform channel compression on the input feature map. The formula is as follows: Conv1() represents a 1×1 convolution operation, X is the input feature map, K is the convolution kernel, and C 输入 is the number of input channels, C 输出 is the number of channels after compression; Step 2.2.2: Apply dilated convolution with different dilation rates to the compressed feature map for multi-scale feature extraction. For the i-th scale, the feature extraction is expressed as: F i =R*d i , F i represents the feature map extracted by dilated convolution at the i-th scale, R represents the compressed feature map, and d i represents the dilation rate of the atrous convolution at the i-th scale (d=1, 2, 4, 8), so i=1, 2, 3, 4; Step 2.2.3: Use global average pooling GAP(R) combined with Conv1 and SiLU activation function to calculate the attention weight A: A=GAP(R), The SiLU activation function is defined as: Among them, z represents the input value of the activation function, σ(z) represents the Sigmoid function, and the SiLU function performs a nonlinear transformation on the input value z through a Sigmoid function, and then multiplies the result of this transformation by the original input value z to generate the final output. σ(z) compresses any real number input to between 0 and 1, so that the output range of the SiLU function is between 0 and the input value z itself; Step 2.2.4: Multiply the feature maps extracted at different scales by the corresponding attention weights element by element to obtain the weighted feature maps, and then merge them in the channel dimension. The specific calculation formula is as follows: F′=Concat(F′1, F′2, F′3, F′4), Among them, F i is the feature map extracted at different scales, M' represents the attention weight, F i ' is the obtained weighted feature map, Concat means the connection operation in the channel dimension; Step 2.2.5: Use 1×1 convolution to compress the merged feature map F' to generate the final output feature map. The calculation formula is as follows: Out = Conv1(F′), Among them, Out is the final output feature map; Step 2.3: The high-resolution path starts from the initial feature extraction stage and keeps the image size after downsampling by a scaling factor of 1 / 8 to achieve finer feature extraction; Step 2.4: introducing an auxiliary segmentation head after the second three-stage feature extraction block stage of the high-resolution path to provide additional supervision signals to accelerate the convergence of the model proposed by the present invention and enhance the model's ability to capture key features; Step 3: Use the training set and validation set to iteratively train and validate the dual-path lightweight image segmentation network based on multi-scale fusion, monitor whether it is overfitting, and provide feedback for further network optimization; Step 4: Select the best network model according to the scores of each model, and save the model parameters of the best network model; Step 5: Use the best network model saved in step 4 to segment the input skin lesion image and obtain the segmentation result of the lesion area.

Images