A psoriasis image intelligent recognition method and device based on ensemble learning

Abstract

The application discloses a kind of based on ensemble learning's psoriasis image intelligent identification method and device;Among them, the method includes: obtaining image data to be identified;The image data to be identified is input into the feature extraction network in ensemble learning model and is extracted feature extraction processing, and obtains extraction image feature;Based on extraction image feature in the image segmentation network in the ensemble learning model, the image to be identified is carried out image segmentation processing, and obtains segmentation image data;Based on the segmentation image data and the image data to be identified are spliced processing, and based on splicing image, psoriasis image classification identification is carried out.In the embodiment of the application, accurate classification identification of psoriasis image is realized, and better auxiliary doctor is diagnosed with psoriasis.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image recognition, and in particular to a psoriasis image intelligent recognition method and device based on ensemble learning. BACKGROUND

[0002] Psoriasis, commonly known as psoriasis, is a chronic inflammatory skin disease with a long course and a tendency to relapse. Some cases are almost lifelong. However, the earlier the disease is detected, the better it is for treatment. However, patients generally do not pay much attention to some diseases that are not serious on their own skin, and it is a tedious and time-consuming task for professional doctors to identify and diagnose skin diseases. Based on the principle of "early detection, early treatment, and early recovery", it is now urgently needed to detect diseases earlier and use deep learning technology to assist doctors in their work in order to better diagnose and reduce the time cost of doctors.

[0003] The prior art pre-trains a pre-training model on a skin image dataset using deep learning technology. The pre-training model can segment and identify skin lesions in skin images. The steps can be roughly divided into the following four steps: image data preprocessing, feature extraction, image segmentation, and image recognition.

[0004] The prior art uses gradient descent method to iteratively update the gradient. Until the loss no longer decreases, the update is stopped. The final pre-training result is compared with the label data, and mIoU is calculated by comparison. This method only uses the label as an evaluation index, but not as auxiliary data to help train the pre-training model, which does not fully utilize the role of the label. SUMMARY

[0005] The present application aims to overcome the shortcomings of the prior art, and provides a psoriasis image intelligent recognition method and device based on ensemble learning, which realizes accurate classification and recognition of psoriasis images and better assists doctors in diagnosing psoriasis.

[0006] To solve the above technical problems, the present application provides a psoriasis image intelligent recognition method based on ensemble learning, which comprises:

[0007] Obtaining image data to be recognized;

[0008] Inputting the image data to be recognized into a feature extraction network in an ensemble learning model for feature extraction processing to obtain extracted image features;

[0009] Based on the extracted image features, performing image segmentation processing on the image data to be recognized in an image segmentation network in the ensemble learning model to obtain segmented image data;

[0010] Based on the segmented image data and the image data to be identified, stitching processing is performed, and based on the stitched image, psoriasis image classification recognition is performed.

[0011] Optionally, the integrated learning model integrates a feature extraction network and an image segmentation network.

[0012] The feature extraction network is composed of two encoder module structures, and the image segmentation network is composed of two decoder module structures.

[0013] Optionally, the training process of the integrated learning model is as follows:

[0014] Obtain training sample data required for training, and perform image preprocessing on the training sample data based on the ostu method to obtain preprocessed training sample data.

[0015] The preprocessed training sample data is sequentially subjected to image enhancement processing of rotation, scaling and cropping to obtain enhanced training sample data.

[0016] The enhanced training sample data is input into the integrated learning model for training processing until the training converges or the number of training times is reached.

[0017] Optionally, the image preprocessing of the training sample data based on the ostu method to obtain the preprocessed training sample data comprises:

[0018] The training sample data is subjected to grayscale processing to obtain grayscale training sample data.

[0019] The grayscale training sample data is subjected to morphological grayscale closing operation based on a structural element to obtain the operated training sample data.

[0020] The operated training sample data is subjected to binary segmentation processing based on interactive threshold segmentation to obtain a binary segmentation result.

[0021] The binary segmentation result is subjected to noise reduction processing based on a labeled connected domain to obtain a noise-reduced binary segmentation result.

[0022] The noise-reduced binary segmentation result is used as a mask to reconstruct the hair region in the training sample data to obtain the preprocessed training sample data.

[0023] Optionally, the feature extraction processing of the image data to be identified in the feature extraction network of the integrated learning model comprises:

[0024] Input the image data to be identified into a feature extraction network in an ensemble learning model, and respectively perform feature extraction in two encoder modules in the feature extraction network to obtain encoder1 extracted features and encoder2 extracted features;

[0025] Perform splicing processing on the encoder1 extracted features and the encoder2 extracted features to form extracted image features.

[0026] The two encoder modules are based on Resnet50 to extract features, and use the network structure before the full connection layer of Resnet50 to perform feature extraction processing.

[0027] Optionally, the image segmentation network in the ensemble learning model based on the extracted image features performs image segmentation processing on the image data to be identified to obtain segmented image data, including:

[0028] Input the extracted image features into the image segmentation network in the ensemble learning model, and respectively perform image segmentation processing in two decoder modules in the image segmentation network to form a first segmentation result and a second segmentation result.

[0029] Perform weighted processing on the first segmentation result and the second segmentation result to obtain segmented image data.

[0030] Optionally, the weighted processing on the first segmentation result and the second segmentation result is as follows:

[0031] Seg=W1*seg1+W2*seg2.

[0032] Wherein, W1 represents the weight parameter of decoder1 in the two decoder modules; seg1 represents the first segmentation result obtained by performing image segmentation processing on decoder1 in the two decoder modules; W2 represents the weight parameter of decoder2 in the two decoder modules; seg2 represents the second segmentation result obtained by performing image segmentation processing on decoder2 in the two decoder modules; and Seg represents the segmented image data.

[0033] Optionally, the image segmentation processing in the two decoder modules in the image segmentation network to form the first segmentation result and the second segmentation result includes:

[0034] Perform upsampling processing in decoder1 in the image segmentation network to obtain the first segmentation result.

[0035] The features of the decoder1 are cascaded in the decoder2 in the image segmentation network for image segmentation processing to obtain a second segmentation result.

[0036] The decoder1 is based on the original unet and uses a CBAM mechanism.

[0037] Optionally, the segmentation image data and the to-be-identified image data are spliced, and psoriasis image classification recognition is performed based on the spliced image.

[0038] The segmentation image data and the original image of the to-be-identified image data are spliced to obtain a spliced image.

[0039] The spliced image is normalized to [0, 1] based on a sigmoid function, and psoriasis image classification recognition is performed based on the normalized result.

[0040] In addition, the embodiment of the present application also provides a psoriasis image intelligent recognition device based on ensemble learning, which comprises:

[0041] An obtaining module is configured to obtain to-be-identified image data.

[0042] A feature extraction module is configured to input the to-be-identified image data into a feature extraction network in an ensemble learning model for feature extraction processing to obtain extracted image features.

[0043] An image segmentation module is configured to perform image segmentation processing on the to-be-identified image based on the extracted image features in an image segmentation network in the ensemble learning model to obtain segmentation image data.

[0044] A classification recognition module is configured to splice the segmentation image data and the to-be-identified image data, and perform psoriasis image classification recognition based on the spliced image.

[0045] In the specific implementation process of the present application, accurate classification and recognition of psoriasis images are realized, and the value of labels is fully utilized to assist in the training and adjustment of network parameters, and doctors are better assisted in diagnosing psoriasis. BRIEF DESCRIPTION OF DRAWINGS

[0046] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without creative labor.

[0047] Figure 1 is a flowchart of the intelligent identification method of psoriasis images based on ensemble learning in the embodiment of the present application;

[0048] Figure 2 is a structural composition diagram of the intelligent identification device of psoriasis images based on ensemble learning in the embodiment of the present application;

[0049] Figure 3 is a structural composition diagram of the ensemble learning model in the embodiment of the present application. DETAILED DESCRIPTION

[0050] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the protection scope of the present application.

[0051] Embodiment one

[0052] Please refer to Figure 1 , Figure 1 is a flowchart of the intelligent identification method of psoriasis images based on ensemble learning in the embodiment of the present application.

[0053] As shown in Figure 1 , an intelligent identification method of psoriasis images based on ensemble learning comprises the following steps:

[0054] S11: obtaining image data to be identified;

[0055] In the specific implementation process of the present application, the images collected by the image collection device or received from other intelligent terminals are obtained, and then the obtained images are processed to obtain the image data to be identified.

[0056] Specifically, the corresponding processing is processing by using the ostu method, that is, including the following steps: gray value processing: the skin melanocytoma color image is subjected to gray value processing, and the color image is changed into a gray scale format; wave trough detector: a structural element is used to perform morphological gray scale closing operation on a given gray scale image, first inflation and then corrosion, fill small cavities in the object, connect adjacent objects, and then subtract the original image from the image obtained by the gray scale closing operation to obtain a hair extraction image with darker background color and brighter hair region; threshold segmentation: the image after wave trough detection can basically extract the hair region, interactive threshold segmentation is used to perform binary segmentation on the hair extraction image to prepare for making a hair mask for region growing; label connected domain, and remove weak and small noise: the region growing method is used to extract a connected domain, and the hair region is labeled, the size of each connected domain is counted, a threshold is set, small connected domains are shielded, and small noise points in the background are removed to as little as possible damage the information of the original image; mask, and restore the original skin information: the binary image after removing the noise is used as a mask to restore and reconstruct the hair region, thereby obtaining the image data to be recognized.

[0057] S12: inputting the image data to be recognized into a feature extraction network in the integrated learning model for feature extraction processing to obtain extracted image features;

[0058] In the specific implementation process of the present application, the integrated learning model integrates a feature extraction network and an image segmentation network; the feature extraction network is composed of two encoder module structures; and the image segmentation network is composed of two decoder module structures.

[0059] Further, the training process of the integrated learning model is as follows: obtaining training sample data required for training, performing image preprocessing on the training sample data based on the ostu method to obtain preprocessed training sample data; performing image enhancement processing of rotation, scaling and cropping on the preprocessed training sample data in sequence to obtain enhanced training sample data; inputting the enhanced training sample data into the integrated learning model for training processing until the training converges or the number of training times reaches.

[0060] Further, the image preprocessing of the training sample data based on the ostu method to obtain preprocessed training sample data comprises: performing gray scale processing on the training sample data to obtain gray scale training sample data; performing morphological gray scale closing operation on the gray scale training sample data based on a structural element to obtain the operated training sample data; performing binary segmentation processing on the operated training sample data based on interactive threshold segmentation to obtain a binary segmentation result; performing noise reduction processing on the binary segmentation result based on labeled connected domain to obtain a noise-reduced binary segmentation result; and performing reconstruction processing on the hair region in the training sample data by taking the noise-reduced binary segmentation result as a mask to obtain the preprocessed training sample data.

[0061] Further, the feature extraction network in the integrated learning model is used for feature extraction processing of the to-be-identified image data, and extracted image features are obtained, which includes: inputting the to-be-identified image data into the feature extraction network in the integrated learning model, and respectively performing feature extraction on the two encoder modules in the feature extraction network to obtain encoder1 extracted features and encoder2 extracted features; and performing splicing processing on the encoder1 extracted features and the encoder2 extracted features to form the extracted image features; wherein the two encoder modules are based on Resnet50 to extract features, and the network structure before the full connection layer of Resnet50 is used for feature extraction processing.

[0062] Specifically, as shown in the integrated learning model, Figure 3 the feature extraction network (encoder1 and encoder2) and the image segmentation network (Decoder1 and Decoder2); that is, the feature extraction network is composed of two encoder module structures; the image segmentation network is composed of two decoder module structures; in Figure 3 CBAM (Convolutional Block Attention Module): convolutional block attention module.

[0063] When training the integrated learning model, training sample data required for training is needed, and then the ostu method is used for image preprocessing of the training sample data to obtain preprocessed training sample data; then the preprocessed training sample data is sequentially subjected to image enhancement processing of rotation, scaling and cropping to obtain enhanced training sample data; finally, the enhanced training sample data is input into the integrated learning model for training processing until the training converges or the number of training times reaches the limit.

[0064] In the image preprocessing of the training sample data by using the Ostu method, the main steps include gray scale, wave trough detector, threshold segmentation, label connected domain, remove weak and small noise and mask, and restore the original skin information. That is, the color image of the skin melanocytoma is processed by gray value, and the color image is converted into a gray scale format. The given gray scale image is subjected to morphological gray scale closing operation using a structure element, first expanded and then corroded, the small cavities in the object are filled, the adjacent objects are connected, and then the original image is subtracted from the image obtained by the gray scale closing operation to obtain a hair extraction image with a dark background color and a bright hair region. The image after wave trough detection can basically extract the hair region. The hair extraction image is subjected to binary segmentation using interactive threshold segmentation, which prepares for hair mask making for region growing. The region growing method is used to extract connected domains and label the hair region. The size of each connected domain is counted, a threshold is set, small connected domains are shielded, and small noise points in the background are removed to minimize the damage to the original image information. The binary image after noise removal is used as a mask to restore and reconstruct the hair region.

[0065] Specifically, feature extraction is performed in the image segmentation network (encoder1 and encoder2) in the ensemble learning model. The structure of encoder1 and encoder2 is based on the structure of Resnet50 network, and is the network structure before the full connection layer of Resnet50. In feature extraction, a high-dimensional feature matrix can be obtained. Encoder2 cascades the features from encoder1 (cascading means splicing the features of the corresponding layer of encoder1 to the corresponding layer of encoder2). Resnet50 network contains 49 convolution layers and 1 full connection layer. The Resnet50 network structure can be divided into seven parts. The first part does not contain residual blocks, and mainly performs convolution, regularization, activation function and maximum pooling calculation on the input. The second, third, fourth and fifth parts contain residual blocks. In the Resnet50 network structure, the residual block has three convolution layers, and the network has a total of 1+3x(3+4+6+3)=49 convolution layers, plus the last full connection layer, a total of 50 layers.

[0066] Finally, the features extracted by encoder1 and encoder2 are spliced to form the extracted image features.

[0067] S13: performing image segmentation processing on the to-be-identified image in the image segmentation network of the ensemble learning model based on the extracted image features, to obtain segmented image data;

[0068] In the implementation of the present application, the image segmentation processing of the to-be-recognized image in the image segmentation network in the ensemble learning model based on the extracted image features includes: inputting the extracted image features into the image segmentation network in the ensemble learning model, performing image segmentation processing in two decoder modules in the image segmentation network respectively to form a first segmentation result and a second segmentation result; and performing weighted processing on the first segmentation result and the second segmentation result to obtain segmentation image data.

[0069] Further, the weighted processing on the first segmentation result and the second segmentation result is as follows:

[0070] Seg=W1*seg1+W2*seg2;

[0071] wherein W1 represents a weight parameter of decoder1 in the two decoder modules; seg1 represents the first segmentation result obtained by performing image segmentation processing on decoder1 in the two decoder modules; W2 represents a weight parameter of decoder2 in the two decoder modules; seg2 represents the second segmentation result obtained by performing image segmentation processing on decoder2 in the two decoder modules; and Seg represents the segmentation image data.

[0072] Further, the image segmentation processing in the two decoder modules in the image segmentation network respectively to form the first segmentation result and the second segmentation result includes: performing upsampling processing on decoder1 in the image segmentation network to obtain the first segmentation result; and performing image segmentation processing on decoder2 in the image segmentation network which is cascaded with the features of decoder1 to obtain the second segmentation result; wherein decoder1 is based on the original unet and uses the CBAM mechanism.

[0073] Specifically, decoder1 is used to upsample the encoder features (for the convenience of description, the splicing features of encoder1 and encoder2 are referred to as encoder features) extracted by encoder1 and encoder2 into segmentation result 1, that is, the high-dimensional features extracted before are restored to the same size as the original image through the method of bilinear interpolation. Figure 1The size of the image is the same, but the original RGB three channels become a gray image (single channel, and the label is also single channel, which is done to facilitate the evaluation of the segmentation result). Among them, decoder1 is based on the original unet using the CBAM mechanism, and decoder2 is the feature of the cascaded decoder1, and thus the segmentation result 2 is obtained. The process of obtaining the segmentation result 1 can be regarded as the result of the basic classifier 1, and similarly, the segmentation result 2 can be regarded as the result of the basic classifier 2. The mIoU of the data set in the two classifiers is calculated respectively, and the weighted average method is used to integrate the final result according to the calculation result. The final segmentation result is obtained. That is:

[0074] Seg=W1*seg1+W2*seg2;

[0075] Wherein, W1 represents the weight parameter of decoder1 in the two decoder modules; seg1 represents the first segmentation result obtained by decoder1 in the two decoder modules for image segmentation processing; W2 represents the weight parameter of decoder2 in the two decoder modules; seg2 represents the second segmentation result obtained by decoder2 in the two decoder modules for image segmentation processing; and Seg represents the segmented image data.

[0076] In the training stage, the mIoU results of the data set under decoder1 and decoder2 need to be calculated, and in the test or verification stage, these data do not need to be calculated.

[0077] S14: based on the segmented image data and the to-be-identified image data for splicing processing, and based on the spliced image for psoriasis image classification and identification.

[0078] In the specific implementation process of the present application, the splicing processing of the segmented image data and the to-be-identified image data is performed, and the spliced image is used for psoriasis image classification and identification, which includes: splicing the original image of the segmented image data and the to-be-identified image data to obtain a spliced image; based on the sigmoid function, the spliced image is normalized to [0, 1], and based on the normalized result, the psoriasis image classification and identification is performed.

[0079] Specifically, based on the fact that there are thousands of skin diseases, it is difficult for a system / method to recognize all skin disease types. Therefore, a simple method is adopted here for the identification of psoriasis, and we divide the thousands of skin diseases into three categories: one is psoriasis, and the other two are normal skin and non-psoriasis image data. Similarly, if you want to be able to identify a specified set of skin diseases, you can select the skin disease types you need to identify, and add image data of other skin diseases and normal skin image data.

[0080] The segmented image data (gray image) and the original image (color image with RGB three channels) are sequentially spliced, and the purpose of splicing here is to extract only the features of the lesion corresponding area when performing classification task. To improve the accuracy of identification. The classification network here can be any network with classification function, such as Resnet, VGG, etc. Here it is also realized by deep learning. The difference between here and segmentation is that the extracted features will be mapped to a feature vector through a fully connected layer. Then through the sigmoid function, it is normalized to [0, 1]. Finally, the class corresponding to the maximum value in the vector is taken as the final prediction result.

[0081] In the specific implementation process of the present application, accurate classification and identification of psoriasis images are realized, and the value of the label is more fully utilized. The label is used to assist in the training and adjustment of network parameters, and better assists doctors in diagnosing psoriasis.

[0082] Embodiment two

[0083] Please refer to Figure 2 , Figure 2 is a structure composition schematic diagram of the psoriasis image intelligent identification device based on ensemble learning in the embodiment of the present application.

[0084] As Figure 1 shown, a psoriasis image intelligent identification device based on ensemble learning, the device comprises:

[0085] The obtaining module 21 is used for obtaining image data to be identified;

[0086] In the specific implementation process of the present application, the image acquisition device or the image collected by other intelligent terminals is used; then the obtained image needs to be processed accordingly, so as to obtain the image data to be identified.

[0087] Specifically, the corresponding processing is processing by using the ostu method, that is, including the following steps: gray value processing: the skin melanocytoma color image is subjected to gray value processing, and the color image is changed into a gray scale format; trough detector: a structural element is used to perform morphological gray scale closing operation on a given gray scale image, and then the original image and the image obtained by the gray scale closing operation are subtracted, so that a hair extraction image with a darker background color and a brighter hair region is obtained; threshold segmentation: the image after the trough detection can basically extract the hair region, and interactive threshold segmentation is used to perform binary segmentation on the hair extraction image, so as to prepare a hair mask for region growing; label connected domain and remove weak small noise: the region growing method is used to extract a connected domain, and the hair region is labeled, the size of each connected domain is counted, a threshold is set, small connected domains are shielded, and small noise points in the background are removed, so as to as little as possible damage the information of the original image; mask and restore original skin information: the binary image after the noise removal is used as a mask, the hair region is restored and reconstructed, and thus the to-be-recognized image data is obtained.

[0088] The feature extraction module 22 is configured to input the to-be-recognized image data into a feature extraction network in the integrated learning model to perform feature extraction processing, and obtain extracted image features.

[0089] In the specific implementation process of the present application, the integrated learning model integrates a feature extraction network and an image segmentation network; the feature extraction network is composed of two encoder module structures; and the image segmentation network is composed of two decoder module structures.

[0090] Further, the training process of the integrated learning model is as follows: training sample data required for training is obtained, the training sample data is subjected to image preprocessing based on the ostu method, and preprocessed training sample data is obtained; the preprocessed training sample data is subjected to image enhancement processing of rotation, scaling and cropping in sequence, and enhanced training sample data is obtained; and the enhanced training sample data is input into the integrated learning model for training processing until the training converges or the number of training reaches a certain number.

[0091] Further, the ostu method is used to pre-process the training sample data to obtain pre-processed training sample data, including: performing grayscale processing on the training sample data to obtain grayscale training sample data; performing morphological grayscale closing operation on the grayscale training sample data based on a structure element to obtain operation training sample data; performing binary segmentation processing on the operation training sample data based on interactive threshold segmentation to obtain a binary segmentation result; performing noise reduction processing on the binary segmentation result based on a labeled connected domain to obtain a noise reduction binary segmentation result; and reconstructing a hair region in the training sample data by taking the noise reduction binary segmentation result as a mask to obtain the pre-processed training sample data.

[0092] Further, the feature extraction network in the integrated learning model is used to extract features from the to-be-identified image data to obtain extracted image features, including: inputting the to-be-identified image data into the feature extraction network in the integrated learning model, and performing feature extraction by two encoder modules in the feature extraction network to obtain encoder1 extracted features and encoder2 extracted features; and performing splicing processing on the encoder1 extracted features and the encoder2 extracted features to form the extracted image features; wherein the two encoder modules are based on Resnet50 to extract features, and the network structure before the full connection layer of Resnet50 is used for feature extraction processing.

[0093] Specifically, as shown in the integrated learning model, Figure 3 the feature extraction network (encoder1 and encoder2) and the image segmentation network (Decoder1 and Decoder2); that is, the feature extraction network is composed of two encoder module structures; the image segmentation network is composed of two decoder module structures; in Figure 3 CBAM (Convolutional Block Attention Module): convolutional block attention module.

[0094] When training the integrated learning model, training sample data required for training is needed, and then the ostu method is used to pre-process the training sample data to obtain pre-processed training sample data; then the pre-processed training sample data is sequentially subjected to image enhancement processing of rotation, scaling and cropping to obtain enhanced training sample data; finally, the enhanced training sample data is input into the integrated learning model for training processing until the training converges or the number of training times reaches a preset number.

[0095] In the image preprocessing of the training sample data by using the Ostu method, the main steps include gray scale, wave trough detector, threshold segmentation, label connected domain, eliminate weak and small noise and mask, and restore the original skin information. That is, the color image of the skin melanocytoma is processed by gray value, and the color image is converted into gray scale format. The given gray scale image is processed by morphological gray scale closing operation using a structure element, which is first expanded and then eroded, fills small cavities in the object, connects adjacent objects, and then subtracts the original image from the image obtained by the gray scale closing operation to obtain a hair extraction image with darker background color and brighter hair region. The image after wave trough detection can basically extract the hair region. The hair extraction image is processed by interactive threshold segmentation to prepare for hair mask making by region growing. The region growing method is used to extract connected domains and label the hair region. The size of each connected region is counted, and a threshold is set to shield small connected regions and remove small noise points in the background to minimize the damage to the original image information. The binary image after noise removal is used as a mask to restore and reconstruct the hair region.

[0096] Specifically, feature extraction is performed in the image segmentation network (encoder1 and encoder2) in the integrated learning model. The structure of encoder1 and encoder2 is based on the structure of Resnet50 network, and is the network structure before the full connection layer of Resnet50. In feature extraction, a high-dimensional feature matrix can be obtained. Encoder2 cascades the features from encoder1 (cascading means splicing the features of the corresponding layer of encoder1 to the corresponding layer of encoder2). Resnet50 network contains 49 convolution layers and 1 full connection layer. The Resnet50 network structure can be divided into seven parts. The first part does not contain residual blocks, and mainly performs convolution, regularization, activation function and maximum pooling calculation on the input. The second, third, fourth and fifth parts contain residual blocks. In the Resnet50 network structure, the residual block has three convolution layers, and the network has a total of 1+3x(3+4+6+3)=49 convolution layers, plus the last full connection layer, a total of 50 layers.

[0097] Finally, the features extracted by encoder1 and encoder2 are spliced to form the extracted image features.

[0098] The image segmentation module 23 is used for image segmentation processing of the to-be-identified image in the image segmentation network of the integrated learning model based on the extracted image features, to obtain segmented image data.

[0099] In the implementation of the present application, the image segmentation processing of the to-be-recognized image in the image segmentation network in the ensemble learning model based on the extracted image features includes: inputting the extracted image features into the image segmentation network in the ensemble learning model, performing image segmentation processing in two decoder modules in the image segmentation network respectively to form a first segmentation result and a second segmentation result; and performing weighted processing on the first segmentation result and the second segmentation result to obtain segmentation image data.

[0100] Further, the weighted processing on the first segmentation result and the second segmentation result is as follows:

[0101] Seg=W1*seg1+W2*seg2;

[0102] Wherein, W1 represents the weight parameter of decoder1 in the two decoder modules; seg1 represents the first segmentation result obtained by performing image segmentation processing on decoder1 in the two decoder modules; W2 represents the weight parameter of decoder2 in the two decoder modules; seg2 represents the second segmentation result obtained by performing image segmentation processing on decoder2 in the two decoder modules; and Seg represents the segmentation image data.

[0103] Further, the image segmentation processing in the two decoder modules in the image segmentation network respectively to form the first segmentation result and the second segmentation result includes: performing upsampling processing on decoder1 in the image segmentation network to obtain the first segmentation result; and performing image segmentation processing on decoder2 in the image segmentation network which is cascaded with the features of decoder1 to obtain the second segmentation result; wherein, decoder1 is based on the original unet and uses the CBAM mechanism.

[0104] Specifically, decoder1 is used to upsample the encoder features (for the convenience of description, the splicing features of encoder1 and encoder2 are referred to as encoder features) extracted by encoder1 and encoder2 into segmentation result 1, that is, the high-dimensional features extracted before are restored to the same size as the original image through the method of bilinear interpolation. Figure 1The size of the image is the same, but the original RGB three channels become a gray image (single channel, the label is also single channel, which is done to facilitate the evaluation of the segmentation result). Among them, decoder1 is based on the original unet using the CBAM mechanism, and decoder2 is the feature of decoder1 cascaded, and thus the segmentation result 2 is obtained. The process of obtaining the segmentation result 1 can be regarded as the result of the basic classifier 1, and similarly, the segmentation result 2 can be regarded as the result of the basic classifier 2. The mIoU of the data set in the two classifiers is calculated respectively, and the weighted average method is used to integrate the final result according to the calculation result. The final segmentation result is obtained. That is:

[0105] Seg=W1*seg1+W2*seg2;

[0106] Wherein, W1 represents the weight parameter of decoder1 in the two decoder modules; seg1 represents the first segmentation result obtained by decoder1 in the two decoder modules for image segmentation processing; W2 represents the weight parameter of decoder2 in the two decoder modules; seg2 represents the second segmentation result obtained by decoder2 in the two decoder modules for image segmentation processing; Seg represents the segmentation image data.

[0107] In the training stage, the mIoU of the data set under decoder1 and decoder2 needs to be calculated, and in the test or verification stage, these data do not need to be calculated.

[0108] The classification and identification module 24 is used for splicing processing based on the segmentation image data and the to-be-identified image data, and classifying and identifying the psoriasis image based on the spliced image.

[0109] In the specific implementation process of the present application, the splicing processing based on the segmentation image data and the to-be-identified image data, and the classification and identification of the psoriasis image based on the spliced image, includes: splicing the segmentation image data and the original image of the to-be-identified image data to obtain a spliced image; based on the sigmoid function, the spliced image is normalized to [0, 1], and based on the normalized result, the psoriasis image is classified and identified.

[0110] Specifically, based on the fact that there are thousands of skin diseases, it is difficult for a system / method to recognize all skin disease types. Therefore, a simple method is adopted here for psoriasis recognition, and the thousands of skin diseases are divided into three categories: one is psoriasis, and the other two are normal skin and non-psoriasis image data. Similarly, if a specified set of skin diseases is to be recognized, the skin disease types to be recognized can be selected, and image data of other skin diseases and normal skin image data can be added.

[0111] The segmented image data (gray image) and the original image (color image with RGB three channels) are sequentially spliced. The purpose of splicing here is to extract only the features of the lesion corresponding area when performing the classification task, so as to improve the accuracy of recognition. The classification network here can be any network with classification function, such as Resnet, VGG, etc. Here, deep learning is also used. The difference between here and segmentation is that the extracted features are mapped to a feature vector through a fully connected layer. Then, the sigmoid function is used to normalize it to [0, 1]. Finally, the class corresponding to the maximum value in the vector is taken as the final prediction result.

[0112] In the specific implementation of the present application, accurate classification and recognition of psoriasis images are achieved, and the value of the label is fully utilized to assist in the training and adjustment of network parameters, so as to better assist doctors in diagnosing psoriasis.

[0113] Those skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by programs instructing related hardware, and the programs can be stored in a computer readable storage medium, which can include read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk, etc.

[0114] In addition, the above describes in detail a psoriasis image intelligent recognition method and device based on ensemble learning provided by the embodiments of the present application, and the principles and implementation manners of the present application are described by using specific examples in this paper. The above description of the embodiments is only used to help understand the method of the present application and its core idea; at the same time, for those skilled in the art, according to the idea of the present application, the specific implementation manner and application range will be changed; in view of the above, the content of the specification should not be understood as a limitation of the present application.

Claims

1. A method for intelligent recognition of psoriasis images based on ensemble learning, characterized in that, The method includes: Obtain the image data to be identified; The image data to be identified is input into the feature extraction network in the ensemble learning model for feature extraction processing to obtain the extracted image features; Based on extracted image features, the image to be identified is segmented in the image segmentation network of the ensemble learning model to obtain segmented image data. The segmented image data and the image data to be identified are stitched together, and psoriasis image classification and recognition are performed based on the stitched image. The ensemble learning model integrates a feature extraction network and an image segmentation network; The feature extraction network consists of two encoder modules; the image segmentation network consists of two decoder modules. The step of inputting the image data to be identified into the feature extraction network of the ensemble learning model for feature extraction processing to obtain extracted image features includes: The image data to be identified is input into the feature extraction network in the ensemble learning model. The two encoder modules in the feature extraction network perform feature extraction respectively to obtain the features extracted by encoder1 and the features extracted by encoder2. The features extracted by encoder1 and encoder2 are concatenated to form the extracted image features. Among them, the two encoder modules are based on ResNet50 to extract features, and use the network structure before the fully connected layer of ResNet50 for feature extraction processing.

2. The intelligent image recognition method for psoriasis according to claim 1, characterized in that, The training process of the ensemble learning model is as follows: Obtain the training sample data required for training, and perform image preprocessing on the training sample data based on the OTU method to obtain preprocessed training sample data; The preprocessed training sample data is sequentially subjected to image enhancement processing involving rotation, scaling, and cropping to obtain enhanced training sample data. The enhanced training sample data is input into the ensemble learning model for training until the training converges or the training iterations are reached.

3. The intelligent image recognition method for psoriasis according to claim 2, characterized in that, The image preprocessing of the training sample data based on the OTU method to obtain preprocessed training sample data includes: The training sample data is converted to grayscale to obtain grayscale training sample data; Based on the structuring element, a morphological gray-level closing operation is performed on the gray-level training sample data to obtain the processed training sample data. The training sample data after the operation is processed by interactive threshold segmentation to obtain the binary segmentation result. The binary segmentation result is denoised based on the labeled connected components to obtain a denoised binary segmentation result. The noise-reduced binary segmentation result is used as a mask to reconstruct the hair region in the training sample data, thus obtaining the preprocessed training sample data.

4. The intelligent image recognition method for psoriasis according to claim 1, characterized in that, The step of performing image segmentation processing on the image to be identified in the image segmentation network of the ensemble learning model based on extracted image features to obtain segmented image data includes: The extracted image features are input into the image segmentation network in the ensemble learning model, and image segmentation processing is performed in the two decoder modules of the image segmentation network to form a first segmentation result and a second segmentation result. The first segmentation result and the second segmentation result are weighted to obtain segmented image data.

5. The intelligent image recognition method for psoriasis according to claim 4, characterized in that, The weighted processing of the first segmentation result and the second segmentation result is as follows: Seg = W1*seg1 + W2*seg2; Where W1 represents the weight parameters of decoder1 in the two decoder modules; seg1 represents the first segmentation result obtained by decoder1 in the two decoder modules through image segmentation processing; W2 represents the weight parameters of decoder2 in the two decoder modules; seg2 represents the second segmentation result obtained by decoder2 in the two decoder modules through image segmentation processing; and Seg represents the segmented image data.

6. The intelligent image recognition method for psoriasis according to claim 4, characterized in that, The image segmentation process is performed in two decoder modules of the image segmentation network to form a first segmentation result and a second segmentation result, including: Upsampling is performed in decoder1 of the image segmentation network to obtain the first segmentation result; In the image segmentation network, features of decoder1 are cascaded into decoder2 to perform image segmentation processing and obtain a second segmentation result. Among them, decoder1 uses the CBAM mechanism on top of the native unet.

7. The intelligent image recognition method for psoriasis according to claim 1, characterized in that, The step of stitching together the segmented image data and the image data to be identified, and then performing psoriasis image classification and recognition based on the stitched image, includes: The segmented image data is stitched together with the original image data of the image to be identified to obtain a stitched image. The stitched image is normalized to the range [0,1] using the Sigmoid function, and psoriasis image classification and recognition are performed based on the normalization result.

8. A psoriasis image intelligent recognition device based on ensemble learning, characterized in that, The device includes: Acquisition module: used to acquire image data to be recognized; Feature extraction module: used to input the image data to be identified into the feature extraction network in the ensemble learning model for feature extraction processing to obtain extracted image features; Image segmentation module: used to perform image segmentation processing on the image to be identified in the image segmentation network of the ensemble learning model based on extracted image features, to obtain segmented image data; Classification and recognition module: used to perform splicing processing based on the segmented image data and the image data to be recognized, and to perform psoriasis image classification and recognition based on the spliced ​​image; The ensemble learning model integrates a feature extraction network and an image segmentation network; The feature extraction network consists of two encoder modules; the image segmentation network consists of two decoder modules. The step of inputting the image data to be identified into the feature extraction network of the ensemble learning model for feature extraction processing to obtain extracted image features includes: The image data to be identified is input into the feature extraction network in the ensemble learning model. The two encoder modules in the feature extraction network perform feature extraction respectively to obtain the features extracted by encoder1 and the features extracted by encoder2. The features extracted by encoder1 and encoder2 are concatenated to form the extracted image features. Among them, the two encoder modules are based on ResNet50 to extract features, and use the network structure before the fully connected layer of ResNet50 for feature extraction processing.

Images