Skin lesion image segmentation method and device, storage medium and computer device

Abstract

The application provides a skin damage image segmentation method and device, a storage medium and a computer device. The method comprises the following steps: inputting a skin damage image sample into a backbone network layer to obtain image feature data; inputting the image feature data into a feature convolution processing module to obtain image convolution feature data; inputting the image feature data into a plurality of feature convolution attention processing modules to obtain image convolution attention feature data; inputting the image convolution feature data and the image convolution attention feature data into a feature fusion layer to obtain fusion feature data; inputting the fusion feature data into a classifier to obtain a skin damage segmentation sample image; training an initial network model according to the skin damage segmentation sample image to obtain a skin damage image segmentation network; and inputting a to-be-processed skin damage image into the skin damage image segmentation network to obtain an accurate target skin damage segmentation image output by the skin damage image segmentation network.

Description

Technical Field

[0001] This application relates to the technical field of skin lesion image segmentation, specifically to a skin lesion image segmentation method, apparatus, storage medium, and computer device. Background Technology

[0002] Skin is the body's first line of defense and largest organ, playing a vital role in various physiological functions. Skin damage reflects the health and condition of the skin. However, manually searching for skin damage in batches of images is extremely labor-intensive and time-consuming, and its accuracy in identifying minute skin lesions is very low. Therefore, existing technologies suffer from the technical limitation of failing to accurately acquire images of skin damage. Summary of the Invention

[0003] The purpose of this application is to overcome the shortcomings and deficiencies in the prior art and provide a method, apparatus, storage medium and computer device for skin damage image segmentation, which can accurately acquire skin damage images.

[0004] The first aspect of this application provides a method for skin lesion image segmentation, including:

[0005] An initial segmentation network model is obtained; the initial segmentation network model includes a backbone network layer, a feature convolution processing module, several feature convolution attention processing modules, a feature fusion layer, and a classifier; wherein, the output of the backbone network layer is connected to the input of the feature convolution processing module and the input of several feature convolution attention processing modules, the output of the feature convolution processing module and the input of several feature convolution attention processing modules are respectively connected to the input of the feature fusion layer, the output of the feature fusion layer is connected to the input of the classifier, and the output of the classifier is the output of the initial segmentation network model;

[0006] Skin injury image samples are input into the backbone network layer for feature extraction processing to obtain image feature data;

[0007] The image feature data is input into the feature convolution processing module for convolution processing to obtain image convolution feature data;

[0008] The image feature data is input into the plurality of feature convolutional attention processing modules for convolutional fusion and channel attention processing to obtain image convolutional attention feature data;

[0009] The image convolutional feature data and the image convolutional attention feature data are input into the feature fusion layer for fusion processing to obtain fused feature data;

[0010] The fused feature data is input into the classifier to obtain skin lesion segmentation sample images;

[0011] The initial network model is trained based on skin lesion segmentation sample images to obtain a skin lesion image segmentation network;

[0012] The skin damage image to be processed is input into the skin damage image segmentation network to obtain the target skin damage segmentation image output by the skin damage image segmentation network.

[0013] A second aspect of this application provides a skin damage image segmentation apparatus, comprising:

[0014] A model acquisition module is used to acquire an initial segmentation network model. The initial segmentation network model includes a backbone network layer, a feature convolution processing module, several feature convolution attention processing modules, a feature fusion layer, and a classifier. The output of the backbone network layer is connected to the input of the feature convolution processing module and the input of each of the feature convolution attention processing modules. The output of the feature convolution processing module and the input of each of the feature convolution attention processing modules are connected to the input of the feature fusion layer. The output of the feature fusion layer is connected to the input of the classifier. The output of the classifier is the output of the initial segmentation network model.

[0015] The image feature data acquisition module is used to input skin damage image samples into the backbone network layer for feature extraction processing to obtain image feature data.

[0016] The image convolution feature data acquisition module is used to input the image feature data into the feature convolution processing module for convolution processing to obtain image convolution feature data;

[0017] The image convolutional attention feature data acquisition module is used to input the image feature data into the plurality of feature convolutional attention processing modules for convolutional fusion and channel attention processing to obtain image convolutional attention feature data;

[0018] The feature data acquisition module is used to input the image convolutional feature data and the image convolutional attention feature data into the feature fusion layer for fusion processing to obtain fused feature data;

[0019] A skin lesion segmentation sample image acquisition module is used to input the fused feature data into the classifier to obtain skin lesion segmentation sample images;

[0020] The training module is used to train the initial network model based on skin lesion segmentation sample images to obtain a skin lesion image segmentation network.

[0021] The target skin lesion segmentation image acquisition module is used to input the skin lesion image to be processed into the skin lesion image segmentation network to obtain the target skin lesion segmentation image output by the skin lesion image segmentation network.

[0022] A third aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the skin damage image segmentation method described above.

[0023] A fourth aspect of this application provides a computer device including a storage device, a processor, and a computer program stored in the storage device and executable by the processor, wherein the processor executes the computer program to implement the steps of the skin damage image segmentation method described above.

[0024] Compared to related technologies, this application extracts features from skin injury image samples through a backbone network layer to obtain image feature data. Then, a feature convolution processing module performs convolution processing on the image feature data to obtain image convolutional feature data. Next, several feature convolutional attention processing modules perform convolutional fusion and channel attention processing on the image feature data to obtain image convolutional attention feature data. Then, a feature fusion layer fuses the image convolutional feature data and the image convolutional attention feature data to obtain fused feature data. The fused feature data is then input into a classifier to obtain skin injury segmentation sample images. Finally, the initial network model is trained based on the skin injury segmentation sample images. The resulting skin injury image segmentation network can compensate for the loss of spatial information in the skin image through convolution and channel attention, thereby improving the accuracy of the acquired target skin injury segmentation images.

[0025] To provide a clearer understanding of this application, the specific embodiments of this application will be described below in conjunction with the accompanying drawings. Attached Figure Description

[0026] Figure 1 This is a flowchart of a skin lesion image segmentation method according to an embodiment of this application.

[0027] Figure 2 This is a schematic diagram of the initial segmentation network model for a skin lesion image segmentation method according to an embodiment of this application.

[0028] Figure 3 This is a schematic diagram of the feature convolution processing module of a skin damage image segmentation method according to an embodiment of this application.

[0029] Figure 4 This is a schematic diagram of the feature convolutional attention processing module of a skin damage image segmentation method according to an embodiment of this application.

[0030] Figure 5 This is a schematic diagram of the dual-convolution feature fusion module of a skin damage image segmentation method according to an embodiment of this application.

[0031] Figure 6 This is a schematic diagram of the module connections of a skin damage image segmentation device according to an embodiment of this application.

[0032] 100. Skin lesion image segmentation device; 101. Model acquisition module; 102. Image feature data acquisition module; 103. Image convolution feature data acquisition module; 104. Image convolution attention feature data acquisition module; 105. Fusion feature data acquisition module; 106. Skin lesion segmentation sample image acquisition module; 107. Training module; 108. Target skin lesion segmentation image acquisition module. Detailed Implementation

[0033] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0034] It should be understood that the described embodiments are merely some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of the embodiments of this application.

[0035] In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. In the description of this application, it should be understood that the terms "first," "second," "third," etc., are used only to distinguish similar objects and are not necessarily used to describe a specific order or sequence, nor should they be construed as indicating or implying relative importance. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances. The singular forms "a," "the," and "the" used in this application and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise. The word "if" as used herein can be interpreted as "when," "when," or "in response to determination."

[0036] Furthermore, in the description of this application, unless otherwise stated, "multiple" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0037] Please see Figure 1 This is a flowchart of a skin lesion image segmentation method according to an embodiment of this application, including:

[0038] S1: Obtain the initial segmentation network model; the initial segmentation network model includes a backbone network layer, a feature convolution processing module, several feature convolution attention processing modules, a feature fusion layer, and a classifier; wherein, the output of the backbone network layer is connected to the input of the feature convolution processing module and the input of several feature convolution attention processing modules, the output of the feature convolution processing module and the input of several feature convolution attention processing modules are respectively connected to the input of the feature fusion layer, the output of the feature fusion layer is connected to the input of the classifier, and the output of the classifier is the output of the initial segmentation network model.

[0039] S2: Input the skin damage image sample into the backbone network layer for feature extraction processing to obtain image feature data.

[0040] The skin lesion image samples can be standardized in resolution using random materialization techniques, and then the hair in the skin lesion image samples is denoised to obtain skin lesion image samples with uniform resolution and free from hair drying. Optionally, data augmentation operations can be performed on the images to increase the number of images used as skin lesion image samples. Data augmentation operations include, but are not limited to, rotation, Gaussian noise, Gaussian blur, contrast adjustment, perspective transformation, gamma transformation, and mirroring. These data augmentation operations can increase the amount of skin lesion image samples and improve the robustness of the trained model.

[0041] In order to improve the accuracy of the model, the backbone network layer used in this embodiment may include multiple feature acquisition modules.

[0042] S3: Input the image feature data into the feature convolution processing module for convolution processing to obtain image convolution feature data.

[0043] The feature convolution processing module is used to perform convolution processing on the image feature data output by the first feature acquisition module of the backbone network layer to obtain the image convolution feature data.

[0044] S4: Input the image feature data into the plurality of feature convolutional attention processing modules for convolutional fusion and channel attention processing to obtain image convolutional attention feature data.

[0045] Specifically, there are two feature convolutional attention processing modules. These modules are used to perform convolutional fusion and channel attention processing on the image feature data output by modules other than the first feature acquisition module. The "non-first feature acquisition module" refers to any feature acquisition module other than the first one.

[0046] S5: Input the image convolutional feature data and the image convolutional attention feature data into the feature fusion layer for fusion processing to obtain fused feature data.

[0047] The feature fusion layer includes multiple feature fusion modules. Each feature fusion module is used to fuse the image convolutional feature data and the image convolutional attention feature data. During the fusion process, the fused feature data output by the lower-level feature fusion module is also transmitted to the upper-level feature fusion module for fusion processing.

[0048] S6: Input the fused feature data into the classifier to obtain skin lesion segmentation sample images.

[0049] The fused feature data includes the fused feature data output by each feature fusion module. Optionally, image feature data output by a feature acquisition module can be directly input into the classifier to obtain the corresponding skin lesion segmentation sample image. In this embodiment, the classifier can be a linear classifier, such as Linear Classification.

[0050] S7: Train the initial network model based on the skin damage segmentation sample images to obtain the skin damage image segmentation network.

[0051] Specifically, training the initial network model with all skin lesion segmentation sample images output by the classifier can effectively improve the accuracy of the skin lesion image segmentation network output.

[0052] S8: Input the skin damage image to be processed into the skin damage image segmentation network to obtain the target skin damage segmentation image output by the skin damage image segmentation network.

[0053] In this feature fusion layer, the fused feature data output by the first feature fusion module includes the fused feature data output by the lower-level feature fusion modules, indicating that the fused feature data output by the first feature fusion module has a larger and more accurate information content. Therefore, the skin lesion segmentation image output by the classifier corresponding to the first feature fusion module is used as the target skin lesion segmentation image output by the skin lesion image segmentation network.

[0054] Compared to related technologies, this application extracts features from skin injury image samples through a backbone network layer to obtain image feature data. Then, a feature convolution processing module performs convolution processing on the image feature data to obtain image convolutional feature data. Next, several feature convolutional attention processing modules perform convolutional fusion and channel attention processing on the image feature data to obtain image convolutional attention feature data. Then, a feature fusion layer fuses the image convolutional feature data and the image convolutional attention feature data to obtain fused feature data. The fused feature data is then input into a classifier to obtain skin injury segmentation sample images. Finally, the initial network model is trained based on the skin injury segmentation sample images. The resulting skin injury image segmentation network can compensate for the loss of spatial information in the skin image through convolution and channel attention, thereby improving the accuracy of the acquired target skin injury segmentation images.

[0055] Please see Figure 2 In one feasible embodiment, the backbone network layer includes a first shallow feature acquisition module, a second shallow feature acquisition module, a first deep feature acquisition module, and a second deep feature acquisition module connected sequentially. The input of the first shallow feature acquisition module is the input of the backbone network; the input of the second shallow feature acquisition module is connected to the output of the first shallow feature acquisition module; the input of the first deep feature acquisition module is connected to the output of the second shallow feature acquisition module; and the input of the second deep feature acquisition module is connected to the output of the first deep feature acquisition module. That is, the first shallow feature acquisition module is the first feature acquisition module, the second shallow feature acquisition module is a subordinate feature acquisition module of the first shallow feature acquisition module, the first deep feature acquisition module is a subordinate feature acquisition module of the second shallow feature acquisition module, and the second deep feature acquisition module is a subordinate feature acquisition module of the first deep feature acquisition module.

[0056] The output of the first shallow feature acquisition module is connected to the input of the feature convolution processing module, and the outputs of the second shallow feature acquisition module and the first deep feature acquisition module are respectively connected to the inputs of the corresponding feature convolution attention processing modules.

[0057] The feature fusion layer includes a dual convolutional feature fusion module, a first feature fusion module, and a second feature fusion module.

[0058] The input of the dual convolutional feature fusion module is connected to the output of the feature convolutional attention processing module corresponding to the first deep feature acquisition module, the input of the first feature fusion module is connected to the output of the feature convolutional attention processing module corresponding to the second shallow feature acquisition module, and the output of the second feature fusion module is connected to the output of the feature convolutional processing module.

[0059] The output of the dual-convolution feature fusion module is connected to the input of the first feature fusion module, and the output of the first feature fusion module is connected to the input of the second feature fusion module. That is, the dual-convolution feature fusion module is a subordinate feature fusion module of the first feature fusion module, and the first feature fusion module is a subordinate feature fusion module of the second feature fusion module.

[0060] The image feature data includes a first image feature, a second image feature, a third image feature, and a fourth image feature; the image convolutional attention feature data includes a first image convolutional attention feature, a second image convolutional attention feature, and a third image convolutional attention feature; the fusion feature data includes a dual convolutional fusion feature, a first fusion feature, and a second fusion feature.

[0061] S2: The step of inputting skin injury image samples into the backbone network layer for feature extraction processing to obtain image feature data includes:

[0062] S21: Input the skin damage image sample into the first shallow feature acquisition module for feature extraction processing to obtain the first image features.

[0063] S22: Input the first image features into the second shallow feature acquisition module for feature extraction processing to obtain the second image features.

[0064] S23: Input the second image features into the first deep feature acquisition module for feature extraction processing to obtain the third image features.

[0065] S24: Input the third image feature into the second deep feature acquisition module for feature extraction processing to obtain the fourth image feature.

[0066] Step S3: The step of inputting the image feature data into the feature convolution processing module for convolution processing to obtain image convolution feature data includes: inputting the first image feature into the feature convolution processing module for convolution processing to obtain the image convolution feature data. A schematic diagram of the feature convolution processing module is shown below. Figure 3 As shown, the feature convolution processing module uses a ResPath of length 3.

[0067] Step S4: The step of inputting the image feature data into the plurality of feature convolutional attention processing modules for convolutional fusion and channel attention processing to obtain image convolutional attention feature data includes:

[0068] S41: Input the second image features into the feature convolutional attention processing module corresponding to the second shallow feature acquisition module to perform convolutional fusion and channel attention weighting processing to obtain the first image convolutional attention features.

[0069] S42: Input the third image features into the feature convolutional attention processing module corresponding to the first deep feature acquisition module for convolutional fusion and channel attention weighting processing to obtain the second image convolutional attention features.

[0070] S43: Input the fourth image features into the feature convolutional attention processing module corresponding to the second deep feature acquisition module to perform convolutional fusion and channel attention processing to obtain the third image convolutional attention features.

[0071] Step S5: The step of inputting the image convolutional feature data and the image convolutional attention feature data into the feature fusion layer for fusion processing to obtain fused feature data includes:

[0072] S51: Input the second image convolutional attention features into the dual convolutional feature fusion module to perform dual convolutional fusion processing to obtain the dual convolutional fused features.

[0073] S52: Input the dual convolutional fusion feature and the first image convolutional attention feature into the first feature fusion module for fusion processing to obtain the first fusion feature.

[0074] S53: Input the first fusion feature and the image convolution feature data into the second feature fusion module for fusion processing to obtain the second fusion feature.

[0075] Step S6: The step of inputting the fused feature data into the classifier to obtain a skin lesion segmentation sample image includes:

[0076] The second fusion feature, the first fusion feature, the dual convolutional fusion feature, and the third image convolutional attention feature are respectively input into the classifier to obtain multiple skin damage segmentation sample images.

[0077] S7: The step of training the initial network model based on skin damage segmentation sample images to obtain a skin damage image segmentation network includes: correcting the model parameters of the initial network model based on multiple skin damage segmentation sample images and the skin damage regions marked in the skin damage image samples to obtain the skin damage image segmentation network.

[0078] In this embodiment, image features of different depths can be acquired through a first shallow feature acquisition module, a second shallow feature acquisition module, a first deep feature acquisition module, and a second deep feature acquisition module. Then, image features of different depths are processed by a feature convolution processing module and several feature convolution attention processing modules respectively. Finally, the image features are fused by the dual convolution feature fusion module, the first feature fusion module, and the second feature fusion module of the feature fusion layer. This can improve the accuracy of the acquired image features and better train the initial network model.

[0079] During the training of the skin damage image segmentation network, metrics such as Dice Coefficient (DICE), Intersection over Union (IoU), Hausdorff Distance (HD), and Average Surface Distance (ASSD) can be used to comprehensively evaluate the network, quantifying its performance. Furthermore, optimization experiments can be conducted, including ablation experiments, to verify the effectiveness of the feature convolutional attention processing module and the dual convolutional feature fusion module. Pruning can be used to optimize the network, making it lighter and improving its speed. Finally, the effectiveness of the improvements can be verified on a test set.

[0080] In one feasible embodiment, the feature convolution attention processing module includes a first convolution compensation submodule, a first feature extraction submodule, a first fusion submodule, a channel attention processing module, and a second feature extraction submodule.

[0081] The outputs of the first convolutional compensation submodule and the first feature extraction submodule are respectively connected to the input of the first fusion submodule. The output of the first fusion submodule is connected to the input of the channel attention processing module. The output of the channel attention processing module is connected to the input of the second feature extraction submodule. The output of the second feature extraction submodule is the output of the feature convolutional attention processing module.

[0082] Please see Figure 4 The feature convolutional attention processing module is named CB-Transformer. In CB-Transformer, the first convolutional compensation submodule can be composed of a 3x3 convolution (3×3Conv) shorted with a 1x1 convolution (1×1Conv). The first feature extraction submodule and the second feature extraction submodule both use Transformer. The first fusion submodule is Add, and the channel attention processing module uses Channel Attention.

[0083] Step S4: The step of inputting the image feature data into the plurality of feature convolutional attention processing modules for convolutional fusion and channel attention processing to obtain image convolutional attention feature data includes:

[0084] S401: Input the image feature data into the first convolution compensation submodule for convolution processing to obtain the feature data output by the first convolution compensation submodule.

[0085] S402: Input the image feature data into the first feature extraction submodule for feature extraction processing to obtain the feature data output by the first feature extraction submodule.

[0086] S403: Input the feature data output by the first convolution compensation submodule and the feature data output by the first feature extraction submodule into the first fusion submodule for fusion processing to obtain the feature data output by the first fusion submodule.

[0087] S404: Input the feature data output by the first fusion submodule into the channel attention processing module for channel attention weighting processing to obtain the feature data output by the channel attention processing module.

[0088] S405: Input the feature data output by the channel attention processing module into the second feature extraction submodule for feature extraction processing to obtain the image convolutional attention feature data.

[0089] In this embodiment, the feature convolution attention processing module can use the first convolution compensation submodule and the channel attention processing module to make up for the shortcomings of the Transformer, and at the same time process the fused features so that the model can better accept the processing.

[0090] In one feasible embodiment, the dual convolutional feature fusion module includes a first convolutional branch, a second convolutional branch, a second convolutional compensation submodule, and a second fusion submodule.

[0091] Please see Figure 5 The dual convolutional feature fusion module is named DCA-Block. The first convolutional branch includes a first convolutional submodule, a second convolutional submodule, a first feature fusion submodule, a first channel attention processing submodule, and a second feature fusion submodule. The second convolutional branch includes a third convolutional submodule, a fourth convolutional submodule, a third feature fusion submodule, a second channel attention processing submodule, and a fourth feature fusion submodule.

[0092] In this design, the first, second, third, and fourth convolutional submodules all use 3×3 Conv; the first, second, third, and fourth feature fusion submodules all use Concatenation; the first and second channel attention processing submodules both use Channel Attention; the second convolutional compensation submodule uses 1×1 Conv; and the second fusion submodule is called Add. The connection relationships between the first convolutional branch, the second convolutional branch, the second convolutional compensation submodule, and the second fusion submodule are as follows: Figure 5 As shown.

[0093] S51: The step of inputting the second image convolutional attention features into the dual convolutional feature fusion module for dual convolutional fusion processing to obtain the dual convolutional fused features includes:

[0094] S511: Input the convolutional attention features of the second image into the first convolutional branch for convolutional processing and fusion processing to obtain the feature data output by the first convolutional branch.

[0095] Step S511 includes:

[0096] S5111: Input the second image convolutional attention features into the first convolutional submodule for convolution processing to obtain the feature data output by the first convolutional submodule.

[0097] S5112: Input the feature data output by the first convolution submodule into the second convolution submodule for convolution processing to obtain the feature data output by the second convolution submodule.

[0098] S5113: Input the feature data output by the first convolutional submodule and the feature data output by the second convolutional submodule into the first feature fusion submodule for fusion processing to obtain the feature data output by the first feature fusion submodule.

[0099] The feature data output by the first feature fusion submodule is obtained using the following formula:

[0100] Output B-CNN =0.32*Conv 3x3 (x)+0.68*Conv 5x5 (x)

[0101] Among them, Output B-CNN For the feature data output by the first feature fusion submodule, Conv 3x3 (x) represents the feature data output by the first convolutional submodule, Conv 5x5(x) represents the feature data output by the second convolutional submodule.

[0102] One of them is a 5×5 convolution Conv 5x5 (x) is essentially equivalent to two consecutive 3×3 convolutions Conv 3x3 (x), and the computational cost of using two 3×3 convolutions is less than that of using a single 5×5 convolution. Therefore, Conv 5x5 (x) can be obtained using the following formula:

[0103] Conv 5x5 (x)=Conv 3x3 (Conv 3x3 (x))

[0104] Among them, Conv 3x3 (x) represents the feature data output by the first convolutional submodule.

[0105] S5114: Input the feature data output by the first feature fusion submodule into the first channel attention processing submodule for channel attention weighting processing to obtain the feature data output by the first channel attention processing submodule.

[0106] S5115: Input the feature data output by the second convolution submodule and the feature data output by the first channel attention processing submodule into the second feature fusion submodule for fusion processing to obtain the feature data output by the first convolution branch.

[0107] The feature data output by the first convolution branch is obtained using the following formula:

[0108] Output B =0.5 * Output B-CNN +0.5*CA(Output B-CNN )

[0109] Among them, Output B The feature data output by the first convolutional branch, Output B-CNN The feature data output by the first feature fusion submodule, CA (Output) B-CNN ) represents the feature data output by the first channel attention processing submodule.

[0110] S512: Input the second image convolutional attention features into the second convolutional branch for convolutional processing and fusion processing to obtain the feature data output by the second convolutional branch.

[0111] Step S512 includes:

[0112] S5121: Input the second image convolutional attention features into the third convolutional submodule for convolution processing to obtain the feature data output by the third convolutional submodule.

[0113] S5122: Input the feature data output by the third convolution submodule into the fourth convolution submodule for convolution processing to obtain the feature data output by the fourth convolution submodule.

[0114] S5123: Input the feature data output by the third convolution submodule and the feature data output by the fourth convolution submodule into the third feature fusion submodule for fusion processing to obtain the feature data output by the third feature fusion submodule.

[0115] S5124: Input the feature data output by the third feature fusion submodule into the second channel attention processing submodule for channel attention weighting processing to obtain the feature data output by the second channel attention processing submodule.

[0116] S5125: Input the feature data output by the third convolution submodule, the feature data output by the fourth convolution submodule, and the feature data output by the second channel attention processing submodule into the fourth feature fusion submodule for fusion processing to obtain the feature data output by the second convolution branch.

[0117] S513: Input the second image convolutional attention features into the second convolutional compensation submodule for convolutional processing to obtain the feature data output by the second convolutional compensation submodule.

[0118] S514: Input the feature data output by the first convolution branch, the feature data output by the second convolution branch, and the feature data output by the second convolution compensation submodule into the second fusion submodule for fusion processing to obtain the dual convolution fusion feature.

[0119] The dual-convolution fusion feature is obtained using the following formula:

[0120] Output DCA =2*Output B +Conv 1x1 (x)

[0121] Among them, Output DCA For the dual-convolution fused feature, 2*Output B For the feature data output by the first convolutional branch and the feature data output by the second convolutional branch, Conv 1x1 (x) represents the feature data output by the second convolution compensation submodule.

[0122] In this embodiment, the dual-convolution feature fusion module, through a first convolution branch, a second convolution branch, a second convolution compensation submodule, and a second fusion submodule, achieves the effect of enhancing feature information acquisition by utilizing dual enhanced convolution branches, which is beneficial to improving the accuracy of feature information acquisition.

[0123] In a feasible embodiment, step S8: inputting the skin damage image to be processed into the skin damage image segmentation network to obtain the target skin damage segmentation image output by the skin damage image segmentation network includes:

[0124] When the skin damage image to be processed is input into the skin damage image segmentation network, the predicted skin damage segmentation image output by the classifier in the skin damage image segmentation network corresponding to the second fusion feature is determined as the target skin damage segmentation image.

[0125] In this embodiment, since the predicted skin damage segmentation image corresponding to the second fusion feature simultaneously fuses image convolutional feature data and the first fusion feature, and the first fusion feature further fuses dual convolutional fusion feature and the first image convolutional attention feature, the second fusion feature is the fusion feature with the most and most comprehensive feature information. That is, the accuracy of the predicted skin damage segmentation image corresponding to the second fusion feature is the highest. Therefore, by determining the predicted skin damage segmentation image as the target skin damage segmentation image, the skin damage segmentation image with the highest accuracy can be obtained.

[0126] Please see Figure 6 The second embodiment of this application provides a skin damage image segmentation device 100, comprising:

[0127] The model acquisition module 101 is used to acquire an initial segmentation network model. The initial segmentation network model includes a backbone network layer, a feature convolution processing module, several feature convolution attention processing modules, a feature fusion layer, and a classifier. The output of the backbone network layer is connected to the input of the feature convolution processing module and the input of the several feature convolution attention processing modules. The output of the feature convolution processing module and the input of the several feature convolution attention processing modules are respectively connected to the input of the feature fusion layer. The output of the feature fusion layer is connected to the input of the classifier. The output of the classifier is the output of the initial segmentation network model.

[0128] The image feature data acquisition module 102 is used to input skin damage image samples into the backbone network layer for feature extraction processing to obtain image feature data.

[0129] The image convolution feature data acquisition module 103 is used to input the image feature data into the feature convolution processing module for convolution processing to obtain image convolution feature data.

[0130] The image convolutional attention feature data acquisition module 104 is used to input the image feature data into the plurality of feature convolutional attention processing modules to perform convolutional fusion and channel attention processing to obtain image convolutional attention feature data;

[0131] The feature data acquisition module 105 is used to input the image convolutional feature data and the image convolutional attention feature data into the feature fusion layer for fusion processing to obtain fused feature data;

[0132] The skin lesion segmentation sample image acquisition module 106 is used to input the fused feature data into the classifier to obtain a skin lesion segmentation sample image;

[0133] Training module 107 is used to train the initial network model based on skin damage segmentation sample images to obtain a skin damage image segmentation network;

[0134] The target skin damage segmentation image acquisition module 108 is used to input the skin damage image to be processed into the skin damage image segmentation network to obtain the target skin damage segmentation image output by the skin damage image segmentation network.

[0135] It should be noted that the skin damage image segmentation device 100 provided in the second embodiment of this application is only illustrated by the above-described division of functional modules when performing the skin damage image segmentation method. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the skin damage image segmentation device 100 provided in the second embodiment of this application and the skin damage image segmentation method of the first embodiment of this application belong to the same concept, and its implementation process is detailed in the method embodiment, which will not be repeated here.

[0136] A third aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the skin damage image segmentation method described above.

[0137] A fourth aspect of this application provides a computer device including a storage device, a processor, and a computer program stored in the storage device and executable by the processor, wherein the processor executes the computer program to implement the steps of the skin damage image segmentation method described above.

[0138] The device embodiments described above are merely illustrative. The components described as separate parts may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this application according to actual needs. Those skilled in the art can understand and implement this without any inventive effort.

[0139] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0140] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 The computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function selected in one or more boxes.

[0141] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function selected in one or more boxes.

[0142] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0143] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0144] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0145] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0146] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A method for segmenting skin lesion images, characterized in that, include: An initial segmentation network model is obtained; the initial segmentation network model includes a backbone network layer, a feature convolution processing module, several feature convolution attention processing modules, a feature fusion layer, and a classifier; wherein, the output of the backbone network layer is connected to the input of the feature convolution processing module and the input of several feature convolution attention processing modules, the output of the feature convolution processing module and the input of several feature convolution attention processing modules are respectively connected to the input of the feature fusion layer, the output of the feature fusion layer is connected to the input of the classifier, and the output of the classifier is the output of the initial segmentation network model; the feature convolution attention processing module includes a first convolution compensation submodule, a first feature extraction submodule, a first fusion submodule, a channel attention processing module, and a second feature extraction submodule; the backbone network layer includes a first shallow feature acquisition module, a second shallow feature acquisition module, a first deep feature acquisition module, and a second deep feature acquisition module connected in sequence; the feature fusion layer includes a dual convolution feature fusion module, a first feature fusion module, and a second feature fusion module; Skin injury image samples are input into the backbone network layer for feature extraction processing to obtain image feature data; the image feature data includes a first image feature, a second image feature, a third image feature, and a fourth image feature; including: The skin damage image sample is input into the first shallow feature acquisition module for feature extraction processing to obtain the first image features; The first image features are input into the second shallow feature acquisition module for feature extraction processing to obtain the second image features; The second image features are input into the first deep feature acquisition module for feature extraction processing to obtain the third image features; The third image feature is input into the second deep feature acquisition module for feature extraction processing to obtain the fourth image feature; The image feature data is input into the feature convolution processing module for convolution processing to obtain image convolution feature data; including: inputting the first image feature into the feature convolution processing module for convolution processing to obtain the image convolution feature data; The image feature data is input into the plurality of feature convolutional attention processing modules for convolutional fusion and channel attention processing to obtain image convolutional attention feature data; the image convolutional attention feature data includes a first image convolutional attention feature, a second image convolutional attention feature, and a third image convolutional attention feature; including: The second image features are input into the feature convolutional attention processing module of the corresponding second shallow feature acquisition module for convolutional fusion and channel attention weighting to obtain the first image convolutional attention features. The third image features are input into the feature convolutional attention processing module corresponding to the first deep feature acquisition module for convolutional fusion and channel attention weighting to obtain the second image convolutional attention features. The fourth image feature is input into the feature convolutional attention processing module corresponding to the second deep feature acquisition module for convolutional fusion and channel attention processing to obtain the third image convolutional attention feature. The image convolutional feature data and the image convolutional attention feature data are input into the feature fusion layer for fusion processing to obtain fused feature data; the fused feature data includes dual convolutional fusion features, a first fusion feature, and a second fusion feature; including: The second image convolutional attention features are input into the dual convolutional feature fusion module to perform dual convolutional fusion processing to obtain the dual convolutional fused features; The dual-convolutional fusion feature and the first image convolutional attention feature are input into the first feature fusion module for fusion processing to obtain the first fused feature; The first fusion feature and the image convolutional feature data are input into the second feature fusion module for fusion processing to obtain the second fusion feature; The fused feature data is input into the classifier to obtain skin lesion segmentation sample images; including: The second fusion feature, the first fusion feature, the dual convolutional fusion feature, and the third image convolutional attention feature are respectively input into the classifier to obtain multiple skin damage segmentation sample images; The initial network model is trained based on skin lesion segmentation sample images to obtain a skin lesion image segmentation network; including: correcting the model parameters of the initial network model based on multiple skin lesion segmentation sample images and the skin lesion regions marked in the skin lesion image samples to obtain the skin lesion image segmentation network; The skin damage image to be processed is input into the skin damage image segmentation network to obtain the target skin damage segmentation image output by the skin damage image segmentation network.

2. The skin lesion image segmentation method according to claim 1, characterized in that, The step of inputting the image feature data into the plurality of feature convolutional attention processing modules for convolutional fusion and channel attention processing to obtain image convolutional attention feature data includes: The image feature data is input into the first convolution compensation submodule for convolution processing to obtain the feature data output by the first convolution compensation submodule. The image feature data is input into the first feature extraction submodule for feature extraction processing to obtain the feature data output by the first feature extraction submodule. The feature data output by the first convolution compensation submodule and the feature data output by the first feature extraction submodule are input into the first fusion submodule for fusion processing to obtain the feature data output by the first fusion submodule. The feature data output by the first fusion submodule is input into the channel attention processing module for channel attention weighting processing to obtain the feature data output by the channel attention processing module. The feature data output by the channel attention processing module is input into the second feature extraction submodule for feature extraction processing to obtain the image convolutional attention feature data.

3. The skin lesion image segmentation method according to claim 1, characterized in that, The step of inputting the skin lesion image to be processed into the skin lesion image segmentation network to obtain the target skin lesion segmentation image output by the skin lesion image segmentation network includes: When the skin damage image to be processed is input into the skin damage image segmentation network, the predicted skin damage segmentation image output by the classifier in the skin damage image segmentation network corresponding to the second fusion feature is determined as the target skin damage segmentation image.

4. The skin lesion image segmentation method according to claim 1, characterized in that, The dual-convolution feature fusion module includes a first convolution branch, a second convolution branch, a second convolution compensation submodule, and a second fusion submodule; The step of inputting the second image convolutional attention features into the dual convolutional feature fusion module for dual convolutional fusion processing to obtain the dual convolutional fused features includes: The second image convolutional attention features are input into the first convolutional branch for convolution and fusion processing to obtain the feature data output by the first convolutional branch; The second image convolutional attention features are input into the second convolutional branch for convolution and fusion processing to obtain the feature data output by the second convolutional branch; The second image convolutional attention features are input into the second convolutional compensation submodule for convolutional processing to obtain the feature data output by the second convolutional compensation submodule. The feature data output from the first convolutional branch, the feature data output from the second convolutional branch, and the feature data output from the second convolutional compensation submodule are input into the second fusion submodule for fusion processing to obtain the dual convolutional fused feature.

5. The skin lesion image segmentation method according to claim 4, characterized in that, The first convolutional branch includes a first convolutional submodule, a second convolutional submodule, a first feature fusion submodule, a first channel attention processing submodule, and a second feature fusion submodule; The step of inputting the second image convolutional attention features into the first convolutional branch for convolutional processing and fusion processing to obtain the feature data output by the first convolutional branch includes: The second image convolutional attention features are input into the first convolutional submodule for convolution processing to obtain the feature data output by the first convolutional submodule; The feature data output by the first convolutional submodule is input into the second convolutional submodule for convolution processing to obtain the feature data output by the second convolutional submodule; The feature data output by the first convolutional submodule and the feature data output by the second convolutional submodule are input into the first feature fusion submodule for fusion processing to obtain the feature data output by the first feature fusion submodule. The feature data output by the first feature fusion submodule is input into the first channel attention processing submodule for channel attention weighting processing to obtain the feature data output by the first channel attention processing submodule. The feature data output by the second convolution submodule and the feature data output by the first channel attention processing submodule are input into the second feature fusion submodule for fusion processing to obtain the feature data output by the first convolution branch.

6. The skin lesion image segmentation method according to claim 4, characterized in that, The second convolutional branch includes a third convolutional submodule, a fourth convolutional submodule, a third feature fusion submodule, a second channel attention processing submodule, and a fourth feature fusion submodule; The step of inputting the second image convolutional attention features into the second convolutional branch for convolutional processing and fusion processing to obtain the feature data output by the second convolutional branch includes: The second image convolutional attention features are input into the third convolutional submodule for convolution processing to obtain the feature data output by the third convolutional submodule; The feature data output by the third convolution submodule is input into the fourth convolution submodule for convolution processing to obtain the feature data output by the fourth convolution submodule; The feature data output by the third convolution submodule and the feature data output by the fourth convolution submodule are input into the third feature fusion submodule for fusion processing to obtain the feature data output by the third feature fusion submodule. The feature data output by the third feature fusion submodule is input into the second channel attention processing submodule for channel attention weighting processing to obtain the feature data output by the second channel attention processing submodule. The feature data output by the third convolution submodule, the feature data output by the fourth convolution submodule, and the feature data output by the second channel attention processing submodule are input into the fourth feature fusion submodule for fusion processing to obtain the feature data output by the second convolution branch.

7. A skin lesion image segmentation device, characterized in that, include: A model acquisition module is used to acquire an initial segmentation network model. The initial segmentation network model includes a backbone network layer, a feature convolution processing module, several feature convolution attention processing modules, a feature fusion layer, and a classifier. The output of the backbone network layer is connected to the inputs of the feature convolution processing modules and the inputs of the several feature convolution attention processing modules. The outputs of the feature convolution processing modules and the several feature convolution attention processing modules are connected to the inputs of the feature fusion layer. The output of the feature fusion layer is connected to the input of the classifier, and the output of the classifier is the output of the initial segmentation network model. The feature convolution attention processing module includes a first convolution compensation submodule, a first feature extraction submodule, a first fusion submodule, a channel attention processing module, and a second feature extraction submodule. The backbone network layer includes a first shallow feature acquisition module, a second shallow feature acquisition module, a first deep feature acquisition module, and a second deep feature acquisition module connected sequentially. The feature fusion layer includes a dual convolution feature fusion module, a first feature fusion module, and a second feature fusion module. The image feature data acquisition module is used to input skin injury image samples into the backbone network layer for feature extraction processing to obtain image feature data; the image feature data includes a first image feature, a second image feature, a third image feature, and a fourth image feature; including: The skin damage image sample is input into the first shallow feature acquisition module for feature extraction processing to obtain the first image features; The first image features are input into the second shallow feature acquisition module for feature extraction processing to obtain the second image features; The second image features are input into the first deep feature acquisition module for feature extraction processing to obtain the third image features; The third image feature is input into the second deep feature acquisition module for feature extraction processing to obtain the fourth image feature; An image convolutional feature data acquisition module is used to input the image feature data into the feature convolution processing module for convolution processing to obtain image convolutional feature data; including: inputting the first image feature into the feature convolution processing module for convolution processing to obtain the image convolutional feature data; An image convolutional attention feature data acquisition module is used to input the image feature data into the plurality of feature convolutional attention processing modules for convolutional fusion and channel attention processing to obtain image convolutional attention feature data; the image convolutional attention feature data includes a first image convolutional attention feature, a second image convolutional attention feature, and a third image convolutional attention feature; including: The second image features are input into the feature convolutional attention processing module of the corresponding second shallow feature acquisition module for convolutional fusion and channel attention weighting to obtain the first image convolutional attention features. The third image features are input into the feature convolutional attention processing module corresponding to the first deep feature acquisition module for convolutional fusion and channel attention weighting to obtain the second image convolutional attention features. The fourth image feature is input into the feature convolutional attention processing module corresponding to the second deep feature acquisition module for convolutional fusion and channel attention processing to obtain the third image convolutional attention feature. A fusion feature data acquisition module is used to input the image convolutional feature data and the image convolutional attention feature data into the feature fusion layer for fusion processing to obtain fusion feature data; the fusion feature data includes dual convolutional fusion features, a first fusion feature, and a second fusion feature; including: The second image convolutional attention features are input into the dual convolutional feature fusion module to perform dual convolutional fusion processing to obtain the dual convolutional fused features; The dual-convolutional fusion feature and the first image convolutional attention feature are input into the first feature fusion module for fusion processing to obtain the first fused feature; The first fusion feature and the image convolutional feature data are input into the second feature fusion module for fusion processing to obtain the second fusion feature; A skin lesion segmentation sample image acquisition module, used to input the fused feature data into the classifier to obtain a skin lesion segmentation sample image; including: The second fusion feature, the first fusion feature, the dual convolutional fusion feature, and the third image convolutional attention feature are respectively input into the classifier to obtain multiple skin damage segmentation sample images; The training module is used to train the initial network model based on skin damage segmentation sample images to obtain a skin damage image segmentation network; it includes: correcting the model parameters of the initial network model based on multiple skin damage segmentation sample images and the skin damage regions marked in the skin damage image samples to obtain the skin damage image segmentation network; The target skin lesion segmentation image acquisition module is used to input the skin lesion image to be processed into the skin lesion image segmentation network to obtain the target skin lesion segmentation image output by the skin lesion image segmentation network.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, it implements the steps of the skin lesion image segmentation method as described in any one of claims 1 to 6.

9. A computer device, characterized in that: The device includes a storage device, a processor, and a computer program stored in the storage device and executable by the processor, wherein the processor executes the computer program to implement the steps of the skin lesion image segmentation method as described in any one of claims 1 to 6.

Images