Melanoma segmentation method fusing convolutional neural network and Transform

Abstract

The invention provides a melanoma segmentation method fusing a convolutional neural network and Transform. The method comprises the following specific steps: step 1, obtaining a melanoma training set; 2, expanding the data volume of the training set by adopting a data enhancement technology, and further enhancing the general applicability or generalization ability of the constructed model; and step 3, by innovatively introducing an SViT-C encoder, the excellent local feature extraction capability of a convolutional neural network (CNN), the excellent global dependence modeling advantage of Transform and the spatial pyramid pooling characteristic of PSP-Net are fused. The encoder ingeniously utilizes local features to guide global features, so that higher-level semantic association is accurately captured; 4, implementing a model training process to optimize and determine a training weight; and 5, implementing detailed performance evaluation and robustness test on the independent test set.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision and medical image processing, and in particular to a melanoma segmentation method integrating a convolutional neural network and a Transformer. Background Art

[0002] Skin cancer is a common disease worldwide, with a wide variety of types, including melanoma, MerKel cell carcinoma, carcinoma in situ, and seborrheic keratosis. Among these types, malignant melanoma is particularly serious due to its high invasiveness and lethality, unfortunately claiming approximately 55,500 precious lives each year, highlighting the importance of in-depth research, early screening, and effective treatment. It is estimated that in the United States alone, there will be 97,610 new cases of melanoma by 2023. Once melanoma proliferates and spreads, it will threaten life. In the early stages of the disease, timely intervention, such as simple surgical procedures, can often achieve significant results in the treatment of melanoma. In this context, computer-aided diagnosis (CAD) systems have shown their inestimable unique value. The system can provide medical professionals with more accurate and efficient medical image interpretation, accurately depicting the morphological characteristics, size range, and spatial location of the lesion area, and thus provide strong support for the formulation of treatment plans and the implementation of surgical operations.

[0003] In the past decade, the rapid progress in the field of deep learning has greatly promoted the diversified development of convolutional neural network (CNN) technology, giving rise to a series of innovative application strategies and methods, and showing excellent performance in medical image processing, especially in the segmentation of medical images. However, there are also limitations. Due to the difficulty in establishing long-range dependencies between pixels, the accuracy of the segmentation results is directly affected, making it difficult to achieve the ideal diagnostic standards in clinical practice.

[0004] Although a variety of algorithms have been successfully applied to the accurate segmentation of medical images such as brain tissue, lungs and blood vessels, there is still a lack of effective targeted technology to achieve efficient segmentation in the specific field of malignant melanoma detection. In view of this, developing an efficient segmentation method specifically for melanoma to assist in clinical accurate diagnosis has become an important issue that needs to be solved urgently.

[0005] The information disclosed in this background technology section is only intended to increase the understanding of the overall background of the invention and should not be regarded as an admission or suggestion in any form that the information constitutes the prior art already known to a person skilled in the art. Summary of the invention

[0006] The purpose of the present invention is to provide a melanoma segmentation method that integrates convolutional neural network and Transformer, so as to fill the gap in melanoma segmentation in clinical diagnosis.

[0007] To achieve the above object, the present invention discloses a melanoma segmentation method integrating a convolutional neural network and a Transformer, comprising:

[0008] Step 1: Obtain a public dataset of melanoma;

[0009] Step 2: Perform data enhancement on the training set of the public dataset;

[0010] Step 3: Integrate the local feature extraction capability of convolutional neural network (CNN), the global dependency modeling advantage of Transformer, and the spatial pyramid pooling characteristics of PSP-Net, and innovatively introduce the SViT-C encoder, which guides global features through local features to capture higher-level semantic associations. The local features are fused with the global features by designing the proposed DBF Moudle, and the multi-branch deep cross-feature attention structure Multi-Branch Deep Cross-FeatureAttention (MBCFA) is used as the decoder to improve the traditional jump connection, making good use of the multi-scale information generated by the encoder, processing the changes of lesion areas of different sizes and shapes, achieving good segmentation performance, reducing parameters, and speeding up network training;

[0011] Step 4: Train the model and obtain the corresponding weights;

[0012] Step 5: Test the segmentation effect of the trained model on the test set.

[0013] Preferably, in the above technical solution, step 1 obtains a large-scale dermoscopic image dataset - ISIC dataset, which contains 2000 training images, 150 verification images and 600 test images, and all images have corresponding ground truths. The data structure is carefully designed and divided into training subsets, verification subsets and test subsets according to a scientific ratio of 7:2:1 to ensure the comprehensiveness and accuracy of model training, performance verification and final evaluation.

[0014] Preferably, in the above technical solution, step 2 performs data enhancement twice on the training set of the melanoma dataset, including seven data enhancement methods of random rotation, random translation, brightness adjustment, noise injection, shearing, random cropping and flipping.

[0015] Preferably, in the above technical solution, step 3 of constructing the SVMB-Net model includes:

[0016] A hybrid efficient hierarchical feature extraction structure Super ViT-CNN (SViT-C) based on CNN and transformer is designed, and a Global Feature Recover module is proposed to filter redundant semantic relationships to obtain key global features. The workflow of the local feature annotation module of this structure is as follows:

[0017] L i =BN(f 3×3 (ReLU(BN(f 3×3 ))))+f 1×1 (L i-1 ) (1)

[0018] Among them, Li-1 is the input of the local feature extraction module of the previous layer, f3×3 represents the conventional convolution operation with a kernel size of 3×3, and f1×1 represents the convolution kernel transformation of 1x1.

[0019] In this framework, BatchNorm (BN for short) plays the role of a normalization layer, f3×3 corresponds to a 3x3 convolution kernel operation, and f1×1 represents a 1x1 convolution kernel transformation. The ReLU function is introduced as a nonlinear activation mechanism, and Li-1 is the input of the local feature extraction module of the previous layer.

[0020] In order to achieve continuous global feature extraction, we use the Window-based Self-Attention Module (W-MSA) and its variant, the Shifted Window Partitioned Self-Attention Module (SW-MSA). These two modules work in tandem to efficiently represent and enhance the ability to capture global features:

[0021]

[0022] Among them, zl-1 represents the feature map output by the double-branch fusion module in the previous stage, ti' and Ti' represent the output feature matrices of the feature extraction blocks based on W-MSA and SW-MSA respectively, and LN is the LayerNorm operation.

[0023] The calculation of the global feature recovery module can be expressed as:

[0024]

[0025] where Q, K, and V represent query, key, and value respectively, which are linear transformations of the input matrix. dk is the dimension of the key vector.

[0026] We designed a dual-branch fusion module DBF Module with good polarization attention, which integrates the local feature extraction capability of CNN and the global feature capture advantage of ViT. The polarization filtering of local and global fusion significantly improves the performance of the model for pathological image segmentation. DBF Module is based on convolutional block attention module (CBAM) and position attention mechanism (PAM).

[0027] The processing flow of the CBAM module is shown in (3):

[0028]

[0029] Where represents the element-by-element multiplication of the matrix, f3×3 and f7×7 represent the conventional convolution operations with kernel sizes of 3×3 and 7×7, respectively.

[0030] The fusion process of the dual-branch fusion module in the encoder is as follows:

[0031]

[0032] Where Fo is the output of the dual-branch fusion module, Ti is the global feature extraction module, Li is the local feature extraction module, and f3×3 represents a conventional convolution operation with a kernel size of 3×3.

[0033] In the formula, the output of the dual-branch fusion module is defined as Fo, which fuses the information provided by the global feature extraction module (whose output is Ti) and the local feature extraction module (whose output is Li) in the previous layer. This process is achieved through a 1×1 convolution kernel (denoted as f1×1). In addition, Layer Normalization (LN) is introduced for inter-layer normalization, and SiLU is used as an activation function to enhance nonlinear expression capabilities.

[0034] We proposed a multi-branch deep cross-feature attention structure called Multi-Branch Deep Cross-FeatureAttention (MBCFA) as the decoder core, which effectively integrates the multi-scale feature information generated by the encoder to flexibly cope with the diverse changes in the size and shape of the lesion area, thereby achieving excellent segmentation results. This design not only reduces the model parameters, but also significantly accelerates the network training process.

[0035] The encoder architecture consists of five layers, each corresponding to a different feature scale, labeled S1 to S5. The feature sizes of these layers are H / 2*H / 2, H / 4*H / 4, H / 8*H / 8, H / 16*H / 16, H / 32*H / 32, depending on the height H and width W of the input image. Among them, H represents the height of the input image and W represents the width of the input image. The feature maps S1, S2, S3, S4, and S5 are then linearly upsampled to the same size as the input image to obtain S1', S2', S3', S4', and S5'. Next, S2', S3', S4', and S5' are concatenated in the channel dimension and the result is input into a 1×1 convolution to reduce the channel dimension and enhance the information exchange between channels. Make sure the output has the same number of channels as S2.

[0036] Afterwards, the output of the 1×1 convolution is fed into the MBCFA module to further explore the deep interactions between cross-scale features. In order to make full use of the rich boundary information contained in S1', we concatenate the output of MBCFA with S1' along the channel dimension. Finally, this combined feature is processed by another pair of 1×1 convolutional layers to optimize the feature representation and provide strong support for subsequent tasks. The calculation formula is as follows (5):

[0037]

[0038] Among them, Fup is upper linear interpolation, MSCFA is multi-branch deep cross feature attention, and Sx is the different feature scales corresponding to each layer.

[0039] The cross attention formula is expressed as follows (6):

[0040]

[0041] Where C is the multi-branch deep cross feature attention input, LN is the LayerNorm operation, represents three 1×k convolutions or k×1 convolutions in parallel. CFA(·) is cross feature attention. The query, key, and value of the upper branch are Q1, K1, and V1, respectively, and the query, key, and value of the lower branch are Q2, K2, and V2, respectively.

[0042] In the optimized technical process, step 4 focuses on the implementation of the training phase. This step specifically conducts an in-depth model training process for the training set processed by data augmentation technology. This process aims to refine and construct the core training weights of the proposed method. These weights are the cornerstone of model performance and accuracy, and enhance the model's adaptability and generalization ability in the face of diverse input data.

[0043] Preferably, in the above technical solution, the strategy adopted is to feed the test set data into the optimized model, and then execute the prediction process to accurately generate the test results. In addition, binary prediction images that have been carefully segmented are also provided. These images intuitively and clearly demonstrate the model's efficient recognition and accurate segmentation capabilities for test samples, greatly improving the visualization and practical value of the results.

[0044] Preferably, in the above technical solution, the step of testing the effect of the training model on the test set in step 5 includes inputting the test set into the trained model to obtain the test result of the model, and setting the confidence and IoU to 0.25 and 0.45, and the model outputs a picture with a target prediction box and confidence.

[0045] Preferably, in the above technical solution, step 5, implementing a detailed performance evaluation and robustness test on an independent test set, includes inputting the test set into the trained model to obtain the test result of the model, and setting the confidence and IoU to 0.25 and 0.45, and the model outputs the picture with the target prediction box and confidence. The advantages of the present invention over modern technology are as follows:

[0046] (1) This model achieves a dual expansion of the number and diversity of training samples by integrating advanced data augmentation strategies. Data augmentation technology effectively simulates various data variations that may be encountered in practical applications, enabling the model to maintain stable performance under complex and changing input conditions, thereby ensuring the high accuracy and reliability of melanoma segmentation tasks under complex environmental influences.

[0047] (2) The invention discloses a melanoma segmentation method that integrates convolutional neural network and Transformer, which integrates the design features of convolutional neural network (CNN), Transformer and PSP-Net, and introduces an encoder SViT-C that guides global features to obtain high-level semantic relationships through local features. The local features are fused with the global features through the proposed DBF Moudle, and the multi-branch deep cross-feature attention structure Multi-Branch Deep Cross-FeatureAttention (MBCFA) is used as the decoder to improve the traditional jump connection, making good use of the multi-scale information generated by the encoder, processing the changes of lesion areas of different sizes and shapes, achieving good segmentation performance, reducing parameters, and accelerating network training. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 It is an overall flow chart of a melanoma segmentation method integrating a convolutional neural network and a Transformer according to the present invention;

[0049] Figure 2 This is the improved SVIT-Net model framework diagram;

[0050] Figure 3 It is the structure diagram of the Multi-Branch Deep Cross-FeatureAttention (MBCFA) decoder model;

[0051] Figure 4 This is the module structure diagram of the encoder DBF Moudle that incorporates the convolutional neural network (CNN) and Transformer;

[0052] Figure 5 This is the clean image detection effect diagram of the improved method model. DETAILED DESCRIPTION

[0053] Here, we elaborate on the specific implementation scheme of the present invention with the help of the accompanying drawings. The implementation details presented are only exemplary, and the actual protection scope of the present invention is not limited to these specific implementation methods.

[0054] Unless explicitly stated otherwise, throughout the specification and claims, the term “comprise” or variations such as “include” or “comprising”, etc., will be understood to include the stated elements or components but not to exclude other elements or components.

[0055] like Figures 1 to 5 As shown, according to a specific embodiment of the present invention, a melanoma segmentation method integrating a convolutional neural network and a Transformer comprises the following steps:

[0056] Step 1: Obtain a public dataset of melanoma

[0057] A melanoma segmentation method that integrates convolutional neural networks and Transformer, which is characterized by all-round optimization from data preparation to model evaluation. First, we adopted the large-scale ISIC dermoscopic image dataset, which exhaustively includes 2000 training images, 150 validation images, and 600 test images. Each image is accompanied by precise ground truth annotations to ensure the quality and accuracy of the data. In the division of the dataset, we strictly followed the golden ratio principle of 7:2:1. The training set contains the main learning samples, which are used to train the SVMB-Net model to grasp the characteristics of melanoma images; the validation set is used to continuously adjust and optimize the model parameters during the training process to avoid overfitting; and the test set is completely independent of the training and validation process, and is used to finally evaluate the segmentation performance and generalization ability of the model;

[0058] Step 2: Perform data enhancement on the training set of the public dataset

[0059] (1) The step of performing data enhancement on the training set of the public data set includes using data enhancement technology to enhance the training set of the data set twice using seven enhancement methods including random rotation, random translation, brightness adjustment, noise injection, shearing, random cropping and flipping. The data set can be expanded, effectively expanding the original data set to include more diversified samples, enhancing the robustness of the model, and thus improving the generalization ability of the model;

[0060] Step 3: Integrate the design features of convolutional neural network (CNN), Transformer and PSP-Net, and introduce an encoder SViT-C that uses local features to guide global features to obtain high-level semantic relationships.

[0061] (1) We designed a hybrid efficient hierarchical feature extraction structure Super ViT-CNN (SViT-C) based on CNN and transformer, and proposed a Global Feature Recover module to filter redundant semantic relationships to obtain key global features. The workflow of the local feature annotation module of this structure is as follows:

[0062] L i =BN(f 3×3 (ReLU(BN(f 3×3 ))))+f 1×1 (L i-1 ) (1)

[0063] Where Li is the output of the local feature extraction module, and f1×1 represents the 1x1 convolution kernel transformation.

[0064] In this framework, BatchNorm (BN for short) plays the role of a normalization layer, f3×3 corresponds to a 3x3 convolution kernel operation, while f1×1 represents a 1x1 convolution kernel transformation, and the ReLU function is introduced as a nonlinear activation mechanism.

[0065] In order to achieve continuous global feature extraction, we use the Window-based Self-Attention Module (W-MSA) and its variant, the Shifted Window Partitioned Self-Attention Module (SW-MSA). These two modules work in tandem to efficiently represent and enhance the ability to capture global features:

[0066]

[0067] Among them, zl-1 represents the feature map output by the double-branch fusion module in the previous stage, ti' and Ti' represent the output feature matrices of the feature extraction blocks based on W-MSA and SW-MSA respectively, and LN is the LayerNorm operation.

[0068] The calculation of the global feature recovery module can be expressed as:

[0069]

[0070] where Q, K, and V represent query, key, and value respectively, which are linear transformations of the input matrix. dk is the dimension of the key vector.

[0071] (2) We designed a dual-branch fusion module DBF Module with good polarization attention, which integrates the local feature extraction capability of CNN and the global feature capture advantage of ViT. The polarization filtering of local and global fusion significantly improves the performance of the model for pathological image segmentation. DBF Module is based on convolutional block attention module (CBAM) and position attention mechanism (PAM).

[0072] The processing flow of the CBAM module is shown in (3):

[0073]

[0074] Where represents the element-by-element multiplication of the matrix, f3×3 and f7×7 represent the conventional convolution operations with kernel sizes of 3×3 and 7×7, respectively.

[0075] The fusion process of the dual-branch fusion module in the encoder is as follows:

[0076]

[0077] Where Fo is the output of the dual-branch fusion module, Ti and Li are the outputs of the global feature extraction module and the local feature extraction module of the previous layer, f1×1 is a 1×1 convolution, LN is a LayerNorm operation, and SiLU is an activation function.

[0078] (3) A multi-branch deep cross-feature attention structure Multi-Branch Deep Cross-FeatureAttention (MBCFA) is proposed as a decoder, which makes good use of the multi-scale information generated by the encoder to handle the changes in lesion areas of different sizes and shapes, achieve good segmentation performance, reduce parameters, and speed up network training.

[0079] The encoder architecture consists of five layers, each corresponding to a different feature scale, labeled S1 to S5. The feature maps S1, S2, S3, S4, and S5 are then linearly upsampled to the same size as the input image to obtain S1', S2', S3', S4', and S5'. Next, S2', S3', S4', and S5' are concatenated in the channel dimension, and the result is input into a 1×1 convolution to reduce the channel dimension and enhance information exchange between channels. The output has the same number of channels as S2. The 1x1 convolution output is then fed into the multi-branch deep cross feature attention. In order to utilize the rich boundary information of S1', the output of the multi-branch deep cross feature attention and S1' are concatenated along the channel dimension. The result is then passed to a pair of 1×1 convolutions. The calculation formula is as follows (5):

[0080]

[0081] Among them, Fup is upper linear interpolation and MSCFA is multi-branch deep cross feature attention.

[0082] The cross attention formula is expressed as follows (6):

[0083]

[0084] Where C is the multi-branch deep cross feature attention input, LN is the LayerNorm operation, represents three 1×k convolutions or k×1 convolutions in parallel. CFA(·) is cross feature attention. The query, key, and value of the upper branch are Q1, K1, and V1, respectively, and the query, key, and value of the lower branch are Q2, K2, and V2, respectively.

[0085] Step 4: Train the model to obtain the training weights of the proposed method.

[0086] Step 5: Perform a detailed performance evaluation and robustness test on an independent test set, and output the segmented binary prediction image.

[0087] The above description of specific exemplary embodiments of the present invention is intended to provide illustration and examples, and is not intended to limit the scope of the present invention to the precise form described. Obviously, based on the above teachings, many modifications and variations are feasible. The original intention of selecting and detailing specific exemplary embodiments is to clarify the core principles of the present invention and how they are applied in actual scenarios, thereby enabling those skilled in the art to flexibly implement and effectively utilize the present invention, explore a variety of implementation schemes, and implement various adaptive adjustments and improvements. Therefore, the true scope of the present invention should be defined by the attached claims and their equivalent expressions.

Claims

1. A melanoma segmentation method integrating convolutional neural network and Transformer, characterized in that: include: Step 1: Obtain a public dataset of melanoma; Step 2: Use data augmentation technology to expand the amount of data in the training set, thereby enhancing the general applicability or generalization ability of the constructed model and ensuring that its performance on unseen data is more robust; Step 3: Integrate the local feature extraction capability of convolutional neural network (CNN), the global dependency modeling advantage of Transformer, and the spatial pyramid pooling characteristics of PSP-Net, and innovatively introduce the SViT-C encoder. The encoder guides global features through local features to capture higher-level semantic associations. The local features are fused with global features through the proposed DBF Moudle, and the multi-branch deep cross-feature attention structure Multi-Branch Deep Cross-FeatureAttention (MBCFA) is used as the decoder to improve the traditional jump connection. The multi-scale information generated by the encoder is used to process the changes in lesion areas of different sizes and shapes, achieve good segmentation performance, reduce parameters, and speed up network training. Step 4: Implement the model training process to optimize and determine the training weights; Step 5: Conduct a detailed performance evaluation and robustness test on an independent test set.

2. The melanoma segmentation method integrating convolutional neural network and Transformer according to claim 1, characterized in that: Step 1: Get the ISIC dataset, a large collection of dermoscopic images, which is carefully organized and contains 2,000 images for model training, 150 images for parameter validation, and 600 images dedicated to testing. This dataset strictly follows the 7:2:1 ratio principle and is divided into training subsets, validation subsets, and test subsets to ensure the comprehensiveness and fairness of the evaluation. All images in the dataset are equipped with accurate Ground Truth annotations. This way of organizing data not only ensures the scientific nature of the method, but also meets the requirements of academic papers for data rigor and standardization.

3. The melanoma segmentation method integrating convolutional neural network and Transformer according to claim 1, characterized in that: Step 2 performs two data enhancements on the training set of the melanoma SU dataset, including seven data enhancement methods: random rotation, random translation, dynamic adjustment of brightness, clever injection of noise elements, local shearing of image content, cropping operations at random positions, and image flipping transformation.

4. The melanoma segmentation method integrating convolutional neural network and Transformer according to claim 1, characterized in that: Step 3: Building the SVMB-Net model includes: 1). A hybrid efficient hierarchical feature extraction structure SuperViT-CNN (SViT-C) based on CNN and transformer is designed, and the GlobalFeatureRecover module is proposed to filter redundant semantic relationships to obtain key global features. The workflow of the local feature annotation module of this structure is as follows: L i =BN(f 3×3 (ReLU(BN(f 3×3 ))))+f 1×1 (L i-1 ) (1) Where Li-1 is the input of the local feature extraction module of the previous layer, f3×3 represents the conventional convolution operation with a kernel size of 3×3, and f1×1 represents the convolution kernel transformation of 1x1. In this framework, BatchNorm (BN for short) plays the role of a normalization layer, f3×3 corresponds to a 3x3 convolution kernel operation, and f1×1 represents a 1x1 convolution kernel transformation. The ReLU function is introduced as a nonlinear activation mechanism, and Li-1 is the input of the local feature extraction module of the previous layer. In order to achieve continuous global feature extraction, we use the Window-based Self-Attention Module (W-MSA) and its variant, the Shifted Window Partitioned Self-Attention Module (SW-MSA). These two modules work in tandem to efficiently represent and enhance the ability to capture global features: Among them, zl-1 represents the feature map output by the double-branch fusion module in the previous stage, ti' and Ti' represent the output feature matrices of the feature extraction blocks based on W-MSA and SW-MSA respectively, and LN is the LayerNorm operation. The calculation of the global feature recovery module can be expressed as: where Q, K, and V represent query, key, and value respectively, which are linear transformations of the input matrix. dk is the dimension of the key vector. We designed a dual-branch fusion module DBF Module with good polarization attention, which integrates the local feature extraction capability of CNN and the global feature capture advantage of ViT. The polarization filtering of local and global fusion significantly improves the performance of the model for pathological image segmentation. DBF Module is based on convolutional block attention module (CBAM) and position attention mechanism (PAM). The processing flow of the CBAM module is shown in (3): where represents the element-by-element matrix multiplication, and f3×3 and f7×7 represent regular convolution operations with kernel sizes of 3×3 and 7×7, respectively. The fusion process of the dual-branch fusion module in the encoder is as follows: Among them, Fo is the output of the dual-branch fusion module, Ti is the global feature extraction module, Li is the local feature extraction module, and f3×3 represents the conventional convolution operation with a kernel size of 3×3. In the formula, the output of the dual-branch fusion module is defined as Fo, which fuses the information provided by the global feature extraction module (whose output is Ti) and the local feature extraction module (whose output is Li) in the previous layer. This process is achieved through a 1×1 convolution kernel (denoted as f1×1). In addition, Layer Normalization (LN) is introduced for inter-layer normalization, and SiLU is used as an activation function to enhance nonlinear expression capabilities. We proposed a multi-branch deep cross-feature attention structure called Multi-Branch Deep Cross-FeatureAttention (MBCFA) as the decoder core, which effectively integrates the multi-scale feature information generated by the encoder to flexibly cope with the diverse changes in the size and shape of the lesion area, thereby achieving excellent segmentation results. This design not only reduces the model parameters, but also significantly accelerates the network training process. The encoder architecture consists of five layers, each corresponding to a different feature scale, labeled S1 to S5. The feature sizes of these layers are H / 2*H / 2, H / 4*H / 4, H / 8*H / 8, H / 16*H / 16, H / 32*H / 32, depending on the height H and width W of the input image. Among them, H represents the height of the input image and W represents the width of the input image. The feature maps S1, S2, S3, S4, and S5 are then linearly upsampled to the same size as the input image to obtain S1', S2', S3', S4', and S5'. Next, S2', S3', S4', and S5' are concatenated in the channel dimension and the result is input into a 1×1 convolution to reduce the channel dimension and enhance the information exchange between channels. Make sure the output has the same number of channels as S2. Afterwards, the output of the 1×1 convolution is fed into the MBCFA module to further explore the deep interactions between cross-scale features. In order to make full use of the rich boundary information contained in S1', we concatenate the output of MBCFA with S1' along the channel dimension. Finally, this combined feature is processed by another pair of 1×1 convolutional layers to optimize the feature representation and provide strong support for subsequent tasks. The calculation formula is as follows (5): Among them, Fup is upper linear interpolation, MSCFA is multi-branch deep cross feature attention, and Sx is the different feature scales corresponding to each layer. The cross attention formula is expressed as follows (6): Where C is the multi-branch deep cross feature attention input, LN is the LayerNorm operation, represents three 1×k convolutions or k×1 convolutions in parallel. CFA(·) is cross feature attention. The query, key, and value of the upper branch are Q1, K1, and V1, respectively, and the query, key, and value of the lower branch are Q2, K2, and V2, respectively.

5. The melanoma segmentation method integrating convolutional neural network and Transformer according to claim 1, characterized in that: Step 4 trains the training set after data enhancement to obtain the training weights of the proposed method.

6. The carefully prepared test set is used as input to the trained model, and the model will then output the test results for each test image. These test results are presented in the form of binary prediction images, which clearly show the model's segmentation effect on the melanoma area.

Images