Skin cancer detection method based on double-branch network structure

Abstract

The invention discloses a skin cancer detection method based on a double-branch network structure, and the method comprises the following steps: S1, designing a skin cancer detection model of a double-branch network structure suitable for analyzing a skin cancer image, the skin cancer detection model of the double-branch network structure comprises a feature extraction module, a multi-source feature fusion module and a detection prediction module. S2, training the designed skin cancer detection model of the double-branch network to obtain a trained skin cancer detection model of the double-branch network; and S3, analyzing the skin cancer picture by using the trained skin cancer detection model of the double-branch network to generate a prediction picture. The features on different branches can be fully utilized, the accuracy of skin cancer detection is improved, and the practicability is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of skin cancer detection, and particularly to a skin cancer detection method based on a dual-branch network structure. Background Art

[0002] Skin cancer is one of the most common cancers globally. Early detection of skin cancer is crucial for improving the survival rate and treatment effect of patients. Traditional skin cancer detection methods mainly rely on dermatologists using a dermoscope for examination. However, this method not only takes a long time but also has a high cost.

[0003] With the progress of technology, the rapid development of deep learning technology has brought unprecedented breakthroughs to skin cancer detection. Deep learning algorithms can automatically identify and classify skin lesions by analyzing a large number of skin images. This method not only improves the accuracy of detection but also significantly shortens the diagnosis time and reduces medical costs.

[0004] In the prior art, Chinese Patent No. CN112263217B discloses "a method for detecting lesion regions in non-melanoma skin cancer pathological images based on an improved convolutional neural network". This method improves the YOLOv3 network model and uses pre-training to detect non-melanoma skin cancer images, solving the problem of slow training speed of the convolutional neural network. However, the accuracy of this method is not high, and the detection method needs to be improved.

[0005] Therefore, there is an urgent need to design a skin cancer detection method based on a dual-branch network structure to solve the problems existing in the above prior art. Summary of the Invention

[0006] Aiming at the above defects or improvement requirements of the prior art, the purpose of the present invention is to provide a skin cancer detection method based on a dual-branch network structure. This method simultaneously extracts local features and global features and effectively combines them through a multi-source feature fusion module, significantly improving the detection accuracy. Local features focus on capturing details and lesion information, while global features enhance the understanding of the overall structure. By fusing these two types of features, the method not only improves the accuracy and robustness of skin cancer images but also enhances its reliability and practicality in actual applications.

[0007] To achieve the above object, according to one aspect of the present invention, there is provided a skin cancer detection method based on a dual-branch network structure, the method comprising the following steps:

[0008] S1: Design a skin cancer detection model with a dual-branch network structure suitable for skin cancer images; the skin cancer detection model with a dual-branch network structure includes a feature extraction module, a multi-source feature fusion module, and a detection and prediction module, and comprises the following steps;

[0009] S11: Feed the skin cancer image into Feature Extraction Module 1 to extract local feature information;

[0010] S12: At the same time, feed the skin cancer image into Feature Extraction Module 2 to extract global feature information;

[0011] S13: Feed the local feature information obtained by Feature Extraction Module 1, the global feature information obtained by Feature Extraction Module 2, and the original image information into the multi-source feature fusion module to obtain fused feature information;

[0012] S14: Feed the fused feature information after passing through the multi-source feature fusion module into the detection and prediction module for detection, and finally obtain the final prediction result;

[0013] S2: Train the designed skin cancer detection model with a double-branch network structure to obtain a trained skin cancer detection model with a double-branch network structure;

[0014] S3: Use the trained skin cancer detection model with a double-branch network structure to analyze the skin cancer image and generate a prediction image.

[0015] Further, Feature Extraction Module 1 in step S11 includes 1 convolutional layer and 2 extraction units:

[0016] S111: Input the picture into a convolutional layer with a kernel size of 3×3, and perform activation function and normalization operations;

[0017] S112: Enter the first extraction unit, which contains five convolutional layers with kernel sizes of 1×1, 3×3, 5×5, 1×1, and 1×1 respectively, and the first extraction unit repeats five times;

[0018] S113: Enter the second extraction unit, which contains two convolutional layers with kernel sizes of 3×3 and 1×1 respectively, and one activation function. The second extraction unit repeats four times to obtain local feature information.

[0019] Further, Feature Extraction Module 2 in step S12 includes:

[0020] S121: Input the picture into convolutional layers with kernel sizes of 1×1, 3×3, and 5×5 respectively, and add and fuse the feature information passing through the three convolutional layers to obtain multi-scale feature information;

[0021] S122: Perform global average pooling on the multi-scale feature information, a convolutional layer with a kernel size of 1×1, and global max pooling, a convolutional layer with a kernel size of 1×1, and send the fused feature information to the Sigmoid function;

[0022] S123: Process the multi-scale feature information and the feature information after passing through the Sigmoid function to obtain global feature information;

[0023] Further, the multi-source feature fusion module in step S13 includes three fusion units:

[0024] S131: Send the local feature information obtained by the feature extraction module 1 to the first fusion unit, send the original image to the second fusion unit, and send the global feature information obtained by the feature extraction module 2 to the third fusion unit;

[0025] S132: In the first fusion unit, the local feature information passes through a 1×1 convolutional layer twice, a 3×3 convolutional layer, is normalized, then passes through a 5×5 convolutional layer, is normalized, passes through a 1×1 convolutional layer, is normalized, and an activation function operation is performed;

[0026] S133: In the second fusion unit, the original image passes through a 1×1 convolutional layer twice, is normalized, an activation function operation is performed, then passes through a 3×3 convolutional layer, is normalized, a 5×5 convolutional layer, is normalized, and finally passes through a 1×1 convolutional layer, is normalized, and an activation function operation is performed;

[0027] S134: In the third fusion unit, the global feature information passes through a 1×1 convolutional layer, a 3×3 convolutional layer, is normalized, then passes through a 5×5 convolutional layer, is normalized, passes through a 3×3 convolutional layer, is normalized, and finally passes through a 1×1 convolutional layer, is normalized, and an activation function operation is performed;

[0028] S135: Add the features passed through the first fusion unit and the second fusion unit and pass through a 1×1 convolutional layer;

[0029] S136: Add the features passed through the second fusion unit and the third fusion unit and pass through a 1×1 convolutional layer;

[0030] S137: Add the two features passed through the 1×1 convolutional layer to obtain the fused feature information.

[0031] Further, the detection and prediction module in step S14 specifically includes a 3×3 convolutional layer and a 1×1 convolutional layer:

[0032] S141: Input the fused feature information after passing through the multi-source feature fusion module into the 3×3 convolutional layer;

[0033] S142: Then directly output the final prediction result through a 1×1 convolutional layer;

[0034] As an embodiment of the present application, the loss function SkinLoss used in step S2 is used to train the skin cancer detection model with the designed dual-branch network structure. The loss function SkinLoss includes the loss function Loss of feature extraction module 1 fe1 , the loss function Loss of feature extraction module 2 fe2 , the loss of the multi-source feature fusion module fusion , and the loss of the detection and prediction module pre ; The calculation formula of the loss function SkinLoss is as follows:

[0035] SkinLoss = αLoss fe1 + βLoss fe2 + γLoss fusion + κLoss pre

[0036] Where α, β, γ, and κ are hyperparameters for weighing each loss function.

[0037] Furthermore, the calculation formula of the loss function Loss of feature extraction module 1 fe1 is as follows:

[0038]

[0039] Where C represents the number of categories, y i represents the true label, represents the probability that a sample belongs to a certain category, λ 1 and λ 2 represent weights, N represents the number of samples, represents the feature of the i-th sample output by feature extraction module 1, represents the true feature of the i-th sample, L represents the total number of convolutional layers in the network, and W l represents the convolutional kernel weight of the l-th layer.

[0040] The calculation formula of the loss function Loss of feature extraction module 2 fe2 is as follows:

[0041]

[0042] Where λ 3 , λ 4 , λ 5 , λ 6 represent weight coefficients, N represents the number of samples, represents the feature information obtained by the i-th sample through a 1×1 convolutional kernel, It represents the feature information obtained by the i-th sample through a 3×3 convolutional kernel. It represents the feature information obtained by the i-th sample through a 5×5 convolutional kernel. It represents the multi-scale feature information of the i-th sample. It represents the original feature of the i-th sample, GAP represents global average pooling, GMP represents global max pooling, and y i It represents the true label of the i-th sample. It represents the predicted output of the i-th sample after passing through the activation function, and Var represents the variance of the feature map.

[0043] The loss function Loss of the multi-source feature fusion module fusion The calculation formula is as follows:

[0044]

[0045] Among them, α1, β1, γ1, δ1, λ 7 , λ 8 , λ 9 are all hyperparameters, F fk represents the output feature of the multi-source feature fusion module, y represents the true label, and N represents the number of samples. represents the i-th element of the fused feature. represents the i-th element of the local feature. represents the i-th element of the global feature. represents the i-th element of the original image.

[0046] The loss function Loss of the detection and prediction module pre The calculation formula is as follows:

[0047]

[0048] Among them, N represents the number of samples, c represents the number of categories, and y i，c represents the true label of the i-th sample. represents the predicted value of the i-th sample on category c, and W conv3x3 represents the weight of the 3×3 convolutional layer, and W conv1x1 represents the weight of the 1×1 convolutional layer, and λ represents the regularization coefficient.

[0049] The beneficial effects of the present invention are:

[0050] (1) The present invention designs a skin cancer detection model with a dual-branch network structure. The model includes a feature extraction module 1, a feature extraction module 2, a multi-source feature fusion module, and a detection and prediction module. The model is trained through a loss function, and finally, the trained model is used to analyze skin cancer images to generate corresponding prediction results. While maintaining high accuracy, the present invention provides more possibilities for actual application scenarios and has broad application value.

[0051] (2) The feature extraction module 1 used in the present invention extracts local feature information of skin cancer images through a multi-layer convolution structure. First, the image undergoes preliminary feature extraction through a 3×3 convolution layer, and then through extraction units combined with multiple different convolution kernels (1×1, 3×3, 5×5) to capture local detail features at different scales. This design enables the model to finely extract information such as the edges and textures of the lesion area and adapt to skin lesions of different sizes and shapes.

[0052] (3) The feature extraction module 2 used in the present invention combines multi-scale convolution operations with a global pooling mechanism to fully capture the global structural information of skin cancer images. Through the combination of 1×1, 3×3, and 5×5 different convolution kernels, multi-level and multi-scale feature information is extracted, and then global average pooling and max pooling are used to focus on the global view and capture macroscopic features in the image, such as the morphology of the lesion, color gradient, and texture changes. This design can enhance the model's understanding of the global structure, especially when facing complex lesion morphologies, improve the expression ability of the global context, and enable the model to better distinguish different stages and types of skin cancer.

[0053] (4) The multi-source feature fusion module used in the present invention comprehensively processes local features, global features, and original image information through three independent fusion units. Each fusion unit uses multi-layer convolution and normalization operations to effectively fuse feature information from different sources. Through this multi-source fusion, the model can combine information at different levels, improve the representation ability of skin cancer images, and thus improve the accuracy of classification. Description of the Drawings

[0054] Figure 1 It is a flowchart of the technical solution of a skin cancer detection method based on a dual-branch network structure provided in an embodiment of the present invention;

[0055] Figure 2 It is a schematic diagram of the feature extraction module 1 of a skin cancer detection method based on a dual-branch network structure provided in an embodiment of the present invention;

[0056] Figure 3 It is a schematic diagram of the feature extraction module 2 of a skin cancer detection method based on a dual-branch network structure provided in an embodiment of the present invention;

[0057] Figure 4 This is a schematic diagram of the multi-source feature fusion module of a skin cancer detection method based on a dual-branch network structure provided in an embodiment of the present invention.

[0058] Figure 5 This is a schematic diagram of the detection and prediction module of a skin cancer detection method based on a dual-branch network structure provided in an embodiment of the present invention. Detailed implementation manners

[0059] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without making creative efforts belong to the scope of protection of the present invention.

[0060] It should be noted that all directional indications (such as up, down, left, right, front, back...) in the embodiments of the present invention are only used to explain the relative positional relationship and movement conditions between components in a specific posture (as shown in the drawings). If the specific posture changes, the directional indications will also change accordingly.

[0061] In the present invention, unless otherwise clearly defined and limited, terms such as "connection" and "fixation" should be understood in a broad sense. For example, "fixation" can be a fixed connection, a detachable connection, or integrated; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium, and it can be the communication inside two components or the interaction relationship between two components, unless otherwise clearly limited. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific situations.

[0062] In addition, if there is a description involving "first", "second", etc. in the embodiments of the present invention, the descriptions of "first", "second", etc. are only for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include at least one of such features. In addition, the meaning of "and / or" appearing throughout the text includes three parallel solutions. Taking "A and / or B" as an example, it includes solution A, solution B, or a solution that satisfies both A and B at the same time. In addition, the technical solutions between the various embodiments can be combined with each other, but it must be based on the fact that those of ordinary skill in the art can implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0063] Refer toFigures 1 to 5 , in the first aspect of the present invention, a skin cancer detection method based on a dual-branch network structure is provided, and the method includes the following steps:

[0064] S1: Design a skin cancer detection model with a dual-branch network structure suitable for skin cancer images; the skin cancer detection model with a dual-branch network structure includes a feature extraction module, a multi-source feature fusion module, and a detection and prediction module, and includes the following steps;

[0065] S11: Send the skin cancer image into the feature extraction module 1 to extract local feature information;

[0066] S12: At the same time, send the skin cancer image into the feature extraction module 2 to extract global feature information;

[0067] S13: Send the local feature information obtained by the feature extraction module 1, the global feature information obtained by the feature extraction module 2, and the original image information into the multi-source feature fusion module to obtain fused feature information;

[0068] S14: Send the fused feature information after passing through the multi-source feature fusion module into the detection and prediction module for detection, and finally obtain the final prediction result;

[0069] S2: Train the designed skin cancer detection model with a dual-branch network structure to obtain a trained skin cancer detection model with a dual-branch network structure;

[0070] S3: Use the trained skin cancer detection model with a dual-branch network structure to analyze the skin cancer image and generate a prediction image;

[0071] Specifically, the present invention designs a skin cancer detection model with a dual-branch network structure. The model includes a feature extraction module 1, a feature extraction module 2, a multi-source feature fusion module, and a detection and prediction module. The model is trained through a loss function, and finally the trained model is used to analyze the skin cancer image to generate corresponding prediction results. While maintaining high accuracy, the present invention provides more possibilities for actual application scenarios and has broad application value.

[0072] As an embodiment of the present application, the feature extraction module 1 in step 11 includes 1 convolutional layer and 2 extraction units:

[0073] S111: Input the picture into a convolutional layer with a convolutional kernel size of 3×3, and perform activation function and normalization operations;

[0074] S112: Enter the first extraction unit, and the first extraction unit contains five convolutional layers with convolutional kernel sizes of 1×1, 3×3, 5×5, 1×1, and 1×1 respectively. The first extraction unit is repeated five times.

[0075] S113: Enter the second extraction unit, which contains two convolutional layers with convolutional kernel sizes of 3×3 and 1×1 respectively, and one activation function. The second extraction unit is repeated four times to obtain local feature information.

[0076] Specifically, the feature extraction module 1 extracts local feature information of skin cancer images through a multi-layer convolutional structure. This design enables the model to finely extract information such as the edges and textures of the lesion area and adapt to skin lesions of different sizes and shapes.

[0077] As an embodiment of the present application, the feature extraction module 2 in the step S12 includes:

[0078] S121: Input the pictures into convolutional layers with convolutional kernel sizes of 1×1, 3×3, and 5×5 respectively, and add and fuse the feature information after passing through the three convolutional layers to obtain multi-scale feature information;

[0079] S122: Perform global average pooling on the multi-scale feature information respectively, use a convolutional layer with a convolutional kernel size of 1×1 and global max pooling, use a convolutional layer with a convolutional kernel size of 1×1, and send the fused feature information to the Sigmoid function;

[0080] S123: Process the multi-scale feature information and the feature information after passing through the Sigmoid function to obtain global feature information;

[0081] Specifically, the feature extraction module 2 used in the present invention combines multi-scale convolutional operations with a global pooling mechanism to fully capture the global structural information of skin cancer images. This design can enhance the model's understanding of the global structure, especially when facing complex lesion morphologies, improve the expression ability of the global context, and enable the model to better distinguish different stages and types of skin cancer.

[0082] As an embodiment of the present application, the multi-source feature fusion module in the step S13 includes three fusion units:

[0083] S131: Send the local feature information obtained after passing through the feature extraction module 1 to the first fusion unit, send the original image to the second fusion unit, and send the global feature information obtained after passing through the feature extraction module 2 to the third fusion unit;

[0084] S132: In the first fusion unit, the local feature information passes through a convolutional layer with a size of 1×1 twice, a convolutional layer with a size of 3×3, is normalized, then passes through a convolutional layer with a size of 5×5, is normalized, passes through a convolutional layer with a size of 1×1, is normalized, and undergoes an activation function operation;

[0085] S133: In the second fusion unit, the original image passes through a 1×1 convolutional layer twice, followed by normalization and activation function operations, then through a 3×3 convolutional layer, followed by normalization, a 5×5 convolutional layer, followed by normalization, and finally through a 1×1 convolutional layer, followed by normalization and activation function operations;

[0086] S134: In the third fusion unit, the global feature information passes through a 1×1 convolutional layer, a 3×3 convolutional layer, followed by normalization, then through a 5×5 convolutional layer, followed by normalization, through a 3×3 convolutional layer, followed by normalization, and finally through a 1×1 convolutional layer, followed by normalization and activation function operations;

[0087] S135: Add the features that have passed through the first and second fusion units and pass them through a 1×1 convolutional layer;

[0088] S136: Add the features that have passed through the second and third fusion units and pass them through a 1×1 convolutional layer;

[0089] S137: Add the two features that have passed through the 1×1 convolutional layer to obtain the fused feature information.

[0090] Specifically, the multi-source feature fusion module used in the present invention comprehensively processes local features, global features, and original image information through three independent fusion units. Each fusion unit adopts multi-layer convolution and normalization operations to effectively fuse feature information from different sources. Through this multi-source fusion, the model can combine information at different levels, enhance the representation ability of skin cancer images, and thus improve the accuracy of classification.

[0091] As an embodiment of the present application, the detection and prediction module in step S14 specifically includes a 3×3 convolutional layer and a 1×1 convolutional layer:

[0092] S141: Input the fused feature information after passing through the multi-source feature fusion module into the 3×3 convolutional layer;

[0093] S142: Then directly output the final prediction result through the 1×1 convolutional layer;

[0094] Specifically, the detection and prediction module receives the information after multi-source feature fusion. First, it extracts deep features through the 3×3 convolutional layer, and then performs classification prediction through the 1×1 convolutional layer. The 3×3 convolutional layer further strengthens the detailed features of the image, and the 1×1 convolutional layer reduces redundant calculations and simplifies the structure of the output layer, enabling the model to efficiently make the final classification decision.

[0095] As an embodiment of the present application, the loss function SkinLoss used in step S2 is used to train the skin cancer detection model with the designed dual-branch network structure. The loss function SkinLoss includes the loss function Loss of feature extraction module 1 fe1 、the loss function Loss of feature extraction module 2 fe2 、the Loss of multi-source feature fusion module fusion 、the Loss of detection prediction module pre ; The calculation formula of the loss function SkinLoss is as follows:

[0096] SkinLoss = αLoss fe1 + βLoss fe2 + γLoss fusion + κLoss pre

[0097] Where α, β, γ, and κ are hyperparameters that balance each loss function.

[0098] As an embodiment of the present application, the loss function Loss of feature extraction module 1 fe1 The calculation formula is as follows:

[0099]

[0100] Where C represents the number of categories, y i represents the true label, represents the probability that a sample belongs to a certain category, λ 1 and λ 3 represent weights, N represents the number of samples, represents the feature of the i-th sample output by feature extraction module 1, represents the true feature of the i-th sample, L represents the total number of convolutional layers in the network, and W l represents the convolutional kernel weight of the l-th layer.

[0101] Specifically, the loss function of feature extraction module 1 ensures that the feature extraction process can not only effectively capture key local information but also be highly consistent with the actual label by comprehensively optimizing multiple objectives. At the same time, by introducing a regularization mechanism to control the complexity of the model and prevent overfitting. Through this multi-dimensional optimization, the model can retain important features while enhancing the generalization ability to unknown data during the training process, thereby improving the overall detection performance.

[0102] The loss function Loss of feature extraction module 2 fe2 The calculation formula is as follows:

[0103]

[0104] Among them, λ 4 and λ 5 and λ 6 and λ 7 represent weight coefficients, N represents the number of samples, represents the feature information obtained by the i-th sample through a 1×1 convolutional kernel, represents the feature information obtained by the i-th sample through a 3×3 convolutional kernel, represents the feature information obtained by the i-th sample through a 5×5 convolutional kernel, represents the multi-scale feature information of the i-th sample, represents the original feature of the i-th sample, GAP represents global average pooling, GMP represents global max pooling, y i represents the true label of the i-th sample, represents the predicted output of the i-th sample after passing through the activation function, and Var represents the variance of the feature map.

[0105] Specifically, the feature extraction module 2 can effectively integrate information of different scales and levels, strengthen the synergistic effect of local and global features, improve the model's ability to capture details and overall structures, avoid information loss, and thus enhance the sensitivity to minor differences. In addition, it optimizes the distribution consistency between feature maps, ensures the balance of global and local features at the spatial and semantic levels, and further improves the stability and generalization ability of the model.

[0106] The loss function Loss of the multi-source feature fusion module fusion is calculated as follows:

[0107]

[0108] Among them, α1, β1, γ1, δ1, λ 8 and λ 9 and λ 10 are all hyperparameters, F fk represents the output feature of the multi-source feature fusion module, y represents the true label, N represents the number of samples, represents the i-th element of the fused feature, represents the i-th element of the local feature, represents the i-th element of the global feature, represents the i-th element of the original image.

[0109] Specifically, the loss function of the multi-source feature fusion module constructs a multi-source information collaborative optimization mechanism by fusing local, global, and original image features, aiming to maximize information flow and minimize information loss. It not only emphasizes the capture of details in the feature extraction process but also ensures efficient feature fusion in the global context, thereby enhancing the model's performance and robustness in complex tasks.

[0110] The loss function Loss of the detection and prediction module pre The calculation formula is as follows:

[0111]

[0112] where N represents the number of samples, c represents the number of categories, y i，c represents the true label of the i-th sample, represents the predicted value of the i-th sample on category c, W conv3x3 represents the weight of the 3×3 convolutional layer, W conv1x1 represents the weight of the 1×1 convolutional layer, and λ represents the regularization coefficient.

[0113] Specifically, the loss function of the detection and prediction module optimizes the detection accuracy and improves the generalization ability of the model. Such a design can ensure that the network performs detection quickly and efficiently.

[0114] The use of this comprehensive loss function design in the present invention helps the network better learn the feature representation suitable for the skin cancer image detection task, improving the model's performance and generalization ability.

[0115] The above description is only some preferred embodiments of the present disclosure and an explanation of the applied technical principles. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by the specific combination of the above technical features, but also covers other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above inventive concept. For example, the technical solutions formed by mutually replacing the above features with the technical features (but not limited to) disclosed in the embodiments of the present disclosure that have similar functions.

Claims

1. A skin cancer detection method based on a double-branch network structure, characterized in that: The method comprises the following steps: S1: Design a skin cancer detection model with a dual-branch network structure suitable for skin cancer images; the skin cancer detection model with a dual-branch network structure includes a feature extraction module, a multi-source feature fusion module and a detection prediction module, including the following steps; S11: sending the skin cancer image to the feature extraction module 1 for extracting local feature information; S12: At the same time, the skin cancer image is also sent to the feature extraction module 2 for extracting global feature information; S13: Send the local feature information obtained by the feature extraction module 1, the global feature information obtained by the feature extraction module 2, and the original image information to the multi-source feature fusion module to obtain fused feature information; S14: sending the fused feature information after the multi-source feature fusion module to the detection prediction module for detection, and finally obtaining the final prediction result; S2: training the designed skin cancer detection model with a dual-branch network structure to obtain a trained skin cancer detection model with a dual-branch network structure; S3: Use the trained dual-branch network structure skin cancer detection model to parse skin cancer images and generate prediction images.

2. A skin cancer detection method based on a dual-branch network structure according to claim 1, characterized in that: The feature extraction module 1 in step S11 includes 1 convolution layer and 2 extraction units: S111: Input the image into a convolution layer with a convolution kernel size of 3×3, and then go through an activation function and normalization operation; S112: Entering a first extraction unit, the first extraction unit includes five convolutional layers, the convolution kernel sizes are 1×1, 3×3, 5×5, 1×1, 1×1 respectively, and the first extraction unit is repeated five times; S113: Entering a second extraction unit, the second extraction unit includes two convolutional layers, the convolution kernel sizes are 3×3 and 1×1 respectively, and an activation function. The second extraction unit is repeated four times to obtain local feature information.

3. The skin cancer detection method based on a double-branch network structure according to claim 1, characterized in that: The feature extraction module 2 in step S12 includes: S121: Input the image into convolution layers with convolution kernel sizes of 1×1, 3×3, and 5×5 respectively, and add and fuse the feature information of the three convolution layers to obtain multi-scale feature information; S122: Perform global average pooling on the multi-scale feature information, a convolution layer with a convolution kernel size of 1×1, and global maximum pooling on the convolution layer with a convolution kernel size of 1×1, and send the fused feature information to the Sigmoid function; S123: Process the multi-scale feature information and the feature information after the Sigmoid function to obtain global feature information.

4. The skin cancer detection method based on a dual-branch network structure according to claim 1, characterized in that: The multi-source feature fusion module in step S13 includes three fusion units: S131: Send the local feature information obtained by feature extraction module 1 to the first fusion unit, send the original image to the second fusion unit, and send the global feature information obtained by feature extraction module 2 to the third fusion unit; S132: In the first fusion unit, the local feature information passes through a convolution layer of size 1×1 twice, a convolution layer of size 3×3, is normalized, then passes through a convolution layer of size 5×5, is normalized, passes through a convolution layer of size 1×1, is normalized, and an activation function operation is performed; S133: In the second fusion unit, the original image passes through a convolution layer of size 1×1 twice, is normalized, and an activation function is operated, then passes through a convolution layer of size 3×3, is normalized, and a convolution layer of size 5×5, is normalized, and finally passes through a convolution layer of size 1×1, is normalized, and an activation function is operated; S134: In the third fusion unit, the global feature information passes through a convolution layer of size 1×1, a convolution layer of size 3×3, is normalized, then passes through a convolution layer of size 5×5, is normalized, passes through a convolution layer of size 3×3, is normalized, and finally passes through a convolution layer of size 1×1, is normalized, and an activation function operation is performed; S135: Add the features of the first fusion unit and the second fusion unit and pass them through a convolution layer of size 1×1; S136: Add the features of the second fusion unit and the third fusion unit and pass them through a convolution layer of size 1×1; S137: Add the features of the two convolutional layers with a size of 1×1 to obtain fused feature information.

5. The skin cancer detection method based on a dual-branch network structure as claimed in claim 1, characterized in that: The detection prediction module in step S14 specifically includes a 3×3 convolution layer and a 1×1 convolution layer, and the specific steps include: S141: Input the fused feature information after the multi-source feature fusion module into the 3×3 convolution layer; S142: The final prediction result is directly output through a 1×1 convolutional layer.

6. The skin cancer detection method based on a double-branch network structure according to claim 1, characterized in that: The loss function SkinLoss used in step S2 trains the designed skin cancer detection model with a dual-branch network structure; the loss function SkinLoss includes the loss function Loss of feature extraction module 1 fe1 , Feature extraction module 2 loss function Loss fe2 , Multi-source feature fusion module Loss fusion , Detection prediction module Loss pre ; The calculation formula of the loss function SkinLoss is as follows: SkinLoss=αLoss fe1 +βLoss fe2 +γLoss fusion +κLoss pre Among them, α, β, γ, and κ are hyperparameters that weigh each loss function.

7. The skin cancer detection method based on a dual-branch network structure according to claim 6, characterized in that: The feature extraction module 1 loss function Loss fe1 The calculation formula is as follows: Where C represents the number of categories, y i represents the true label, represents the probability that a sample belongs to a certain category, λ1 and λ2 represent weights, and N represents the number of samples. represents the feature of the i-th sample output by feature extraction module 1, represents the true feature of the i-th sample, L represents the total number of convolutional layers in the network, and W l Represents the convolution kernel weight of the lth layer.

8. The skin cancer detection method based on a double-branch network structure according to claim 6, characterized in that: The feature extraction module 2 loss function Loss fe2 The calculation formula is as follows: Among them, λ3, λ4, λ5, λ6 represent weight coefficients, N represents the number of samples, It represents the feature information obtained by the i-th sample through the convolution kernel of size 1×1. It represents the feature information obtained by the i-th sample through the convolution kernel of size 3×3. It represents the feature information obtained by the i-th sample through the convolution kernel of size 5×5. Represents the multi-scale feature information of the i-th sample, represents the original features of the i-th sample, GAP represents global average pooling, GMP represents global maximum pooling, and y i represents the true label of the i-th sample, It represents the predicted output of the i-th sample after the activation function, and Var represents the variance of the feature map.

9. The skin cancer detection method based on a double-branch network structure according to claim 6, characterized in that: The multi-source feature fusion module loss function Loss fusion The calculation formula is as follows: Among them, α1, β1, γ1, δ1, λ7, λ8, and λ9 are all hyperparameters. fs represents the output features of the multi-source feature fusion module, y represents the true label, N represents the number of samples, represents the i-th element of the fusion feature, Represents the i-th element of the local feature, represents the i-th element of the global feature, Represents the i-th element of the original image; The detection prediction module loss function Loss pre The calculation formula is as follows: Among them, N represents the number of samples, c represents the number of categories, and y i，c represents the true label of the i-th sample, represents the predicted value of the i-th sample in category c, W conv3x3 represents the weight of the 3×3 convolutional layer, W con1x1 represents the weight of the 1×1 convolutional layer, and λ represents the regularization coefficient.

10. A skin cancer detection method based on a double-branch network structure, characterized in that: include: Feature extraction module 1: used to extract local feature information of skin cancer images; Feature extraction module 2: used to extract global feature information of skin cancer images; Multi-source feature fusion module: used to fuse local feature information, global feature information and original image information to obtain fused feature information; Detection prediction module: used to receive the fusion feature information of the multi-source feature fusion module for detection, and finally obtain the final prediction result.

Images