Method, medium and device for automatic segmentation of medical image lesion area based on improved lightweight convolutional network

Abstract

The present invention proposes a method, medium and device for automatic segmentation of lesion areas in medical images based on an improved lightweight convolutional network. First, a residual connection is introduced based on the inverted residual block in MobileNetV2 to construct a lightweight convolution module, so that the model can adapt to a limited sample data set; secondly, the feature pyramid network is designed to fuse feature maps of different resolutions with adaptive weights to improve the positioning accuracy of the lesion area; then, the channel attention weight and the spatial attention weight are adaptively calculated according to the input feature map to highlight the lesion area; then, the channel attention and the spatial attention are fused to pay more attention to the lesion area; finally, a loss function optimization strategy that comprehensively considers pixel gradient information, spatial information and multi-scale information is designed, so that the model pays more attention to samples that are difficult to classify. The present invention makes the segmentation of lesion areas more accurate and the model generalization ability stronger.

Description

Technical Field

[0001] The present invention belongs to the field of medical image processing, and in particular relates to a method, medium and device for automatically segmenting lesion areas in medical images based on an improved lightweight convolutional network. Background Art

[0002] Malignant tumors, also known as cancer, are serious organ diseases that lead to the death of patients. According to global data statistics in 2023, malignant tumors have become the second leading cause of death in the world. Every year, a large number of patients are diagnosed with malignant tumors, which poses a serious threat to human life and health. The current clinical method to reduce the mortality rate of malignant tumors mainly relies on early detection and early treatment.

[0003] Medical imaging technologies including computed tomography (CT) and magnetic resonance imaging (MRI) are one of the main ways to detect malignant tumors. Medical image segmentation is a key content in the field of medical imaging. It is of great significance to find the exact location and contour of the lesion area in the medical image and assist doctors in clinical lesion diagnosis, targeted drugs or partial resection surgery for tumors. However, traditional manual diagnosis methods of medical images are time-consuming and labor-intensive. Their accuracy depends on the professional knowledge and subjective judgment of experts. Uncertainty and error are difficult to avoid, which can easily cause medical accidents. At present, natural image segmentation technology based on convolutional neural network (CNN) has gradually matured, and deep learning models such as FCN (Fully CNN), U-Net, and V-Net have been widely used in the field of medical image segmentation. However, the current automatic segmentation technology for medical images is still not mature enough, facing the problem of limited sample data sets, and the segmentation performance of normal organs and diseased organs still needs to be improved. Summary of the invention

[0004] In view of the deficiencies in the prior art, the present invention provides a method, medium and device for automatic segmentation of lesion areas in medical images based on an improved lightweight convolutional network.

[0005] To achieve the above object, the present invention adopts the following technical solutions:

[0006] The method for automatic segmentation of medical image lesion areas based on an improved lightweight convolutional network is characterized by comprising:

[0007] Preprocess the medical imaging data and divide the preprocessed data into training sets and test sets;

[0008] Build a lightweight convolutional segmentation model based on the inverted residual block of MobileNetV2;

[0009] By setting inverted residual modules of different depths to obtain feature maps of different scales, a feature pyramid network is constructed and adaptive weights are introduced. Through top-down paths and lateral connections, feature maps of different scales are fused to obtain multi-scale features.

[0010] An adaptive channel attention mechanism is designed to calculate the attention weight of each channel of the pyramid network output feature, and the channel features are recalibrated accordingly. An adaptive spatial attention mechanism is designed to calculate the attention weight of each position of the pyramid network output feature, and the spatial features are recalibrated accordingly. The channel attention features and spatial attention features obtained after recalibration are fused and used as input to the convolutional segmentation model to obtain the segmentation results of the lesion area of ​​the medical image.

[0011] Design the loss function of the convolutional segmentation model, and use the divided training set and test set to train and test the convolutional segmentation model;

[0012] The medical image data to be segmented is input into the trained convolutional segmentation model to obtain the segmentation result of the medical image lesion area.

[0013] To optimize the above technical solutions, the specific measures taken also include:

[0014] Furthermore, the preprocessing of the medical image data includes normalization processing and denoising processing;

[0015] The normalization process is to normalize the medical image pixel values ​​to a uniform range of [0,1], as shown in the following formula:

[0016]

[0017] Where x represents the original medical image pixel, x max 、x min Represent the maximum and minimum values ​​of pixels respectively;

[0018] The denoising process adopts an adaptive window adjustment method adapted to different pixel intensities, as shown in the following formula:

[0019]

[0020] Where W(i, j) is the window size; m is the window adjustment parameter; CC(i, j), LC(i, j), IC(i, j) are the pixel comprehensive contrast, local contrast and intensity contrast respectively; I(i, j) is the gray value of pixel i, j in the image; α is a constant; μ is the neighborhood mean of pixel i, j; N represents the neighborhood size.

[0021] Furthermore, in the convolutional segmentation model constructed based on the inverted residual block of MobileNetV2, batch normalization and PReLu function activation operations are performed in both the dimension increase and convolution processes, batch normalization operations are performed on the dimension reduction, residual connections are introduced between the input features and the output features after convolution, and the PReLu activation function is introduced before the residual connection;

[0022] The image feature F′(X) extracted by the convolutional segmentation model is:

[0023] F′(X)=H(X)+δ(X);

[0024] Where H(X) is the output feature after convolution; X is the input feature, which is the feature extracted from the preprocessed medical image data; δ is the PReLU activation function.

[0025] Furthermore, the feature pyramid network is constructed and adaptive weights are introduced to fuse feature maps of different scales through a top-down path to obtain multi-scale features, specifically:

[0026] P i =α i Conv(C i+1 )+β i ·UpSample(N i );

[0027] Where P i represents the i-th layer feature map generated by top-down upsampling; C i+1 is the feature map of the i+1th convolution result from bottom to top; N i is the low-resolution feature map of the i-th layer; Conv is the convolution operation; UpSample is the upsampling operation; α i and β i is a learnable weight parameter, calculated through the channel attention mechanism:

[0028]

[0029] Where σ is the Sigmoid activation function.

[0030] Furthermore, the designed adaptive channel attention mechanism calculates the attention weight of each channel of the pyramid network output feature as:

[0031] S c =σ(W2δ(W1·AvgPool(F c )));

[0032] Where σ is the Sigmoid activation function; F cis the cth channel of the pyramid network output feature map; AvgPool is the average pooling operation; W1 and W2 are learnable parameter matrices; S c is the attention weight of the cth channel;

[0033] Recalibrate the channel features to obtain the recalibrated channel attention features F′ c :

[0034] F′ c =s c ·F c .

[0035] Furthermore, the designed adaptive spatial attention mechanism calculates the attention weight of each position of the pyramid network output feature as:

[0036] s ij =σ(W4δ(W3·Concat(AvgPool(P i ), MaxPool(P i ))));

[0037] Where Concat is a concatenation operation; MaxPool is a maximum pooling operation; W4 and W3 are learnable parameter matrices, and s ij is the attention weight of the (i, j)th position;

[0038] Recalibrate the spatial features to obtain the recalibrated spatial attention features F s ′:

[0039] F s ′=s ij ·P i .

[0040] Furthermore, the channel attention features and spatial attention features obtained after the fusion recalibration are specifically:

[0041]

[0042] In the formula, F fusion To fusion features, is the tensor product symbol.

[0043] Furthermore, the loss function of the convolutional segmentation model is:

[0044]

[0045] Where L is the total loss; w i is the weight of the i-th pixel; y i is the true label of the i-th pixel; p iis the probability that the i-th pixel predicted by the model is a positive class; α, β, γ, δ, η, θ are hyperparameters with values ​​in the range of [0, 1]; is the gradient information of the i-th pixel; I i is the feature information of the i-th pixel; I neighbor is the feature information of the neighboring pixels of the i-th pixel; multi-scale is the feature information of the i-th pixel at multiple scales.

[0046] Accordingly, the present invention proposes a computer-readable storage medium storing a computer program, characterized in that the computer program enables a computer to execute the method for automatic segmentation of medical image lesion areas based on an improved lightweight convolutional network as described above.

[0047] Accordingly, the present invention proposes an electronic device, characterized in that it includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method for automatic segmentation of medical image lesion areas based on an improved lightweight convolutional network as described above is implemented.

[0048] The beneficial effects of the present invention are as follows: the present invention introduces residual connections based on the inverted residual block in MobileNetV2 to construct a lightweight convolution module, so that the model can adapt to a limited sample data set; the feature pyramid network is designed to adaptively weight and fuse feature maps of different resolutions to improve the positioning accuracy of the lesion area; the channel attention weight and the spatial attention weight are adaptively calculated according to the input feature map to highlight the lesion area; the channel attention and the spatial attention are integrated to pay more attention to the lesion area; a loss function optimization strategy that comprehensively considers pixel gradient information, spatial information and multi-scale information is designed, so that the model pays more attention to samples that are difficult to classify. The present invention can effectively overcome the problem that the lesion data sample is small and the lesion area is difficult to distinguish from the normal organ, significantly improve the accuracy of lesion area segmentation, make lesion area segmentation and recognition more efficient, and the model generalization ability is stronger. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 This is a flow chart of the method for automatic segmentation of medical image lesion areas based on an improved lightweight convolutional network.

[0050] Figure 2 This is a schematic diagram of the inverted residual block structure in the improved MobileNetV2 of the present invention.

[0051] Figure 3 Schematic diagram of the feature pyramid network that introduces adaptive weights for the present invention.

[0052] Figure 4A comparison chart of the performance of two-dimensional liver organ segmentation in CT images on the LiTS dataset.

[0053] Figure 5 A comparison chart of the performance of two-dimensional liver organ segmentation in MRI images on the CHAOS dataset.

[0054] Figure 6 The figure shows the performance comparison of cell nucleus segmentation in mixed CT and MRI images on the DSB dataset. DETAILED DESCRIPTION

[0055] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0056] In one embodiment, the present invention proposes a method for automatic segmentation of medical image lesion areas based on an improved lightweight convolutional network, such as Figure 1 As shown, the specific operation steps include the following.

[0057] Step 1: Normalize the pixel values ​​of medical images to a uniform range of [0,1]

[0058] Where x represents the original medical image pixel, x max 、x min Represent the maximum and minimum values ​​of the image pixels respectively.

[0059] In order to enhance the contrast between tissues and organs in images, an adaptive window adjustment method that adapts to different pixel intensities is designed to denoise the image.

[0060]

[0061] Where W(i, j) is the window size; m is the window adjustment parameter, which is a constant and takes values ​​in the range of [1, 5]. CC(i, j), LC(i, j), and IC(i, j) are the pixel comprehensive contrast, local contrast, and intensity contrast, respectively. I(i, j) is the grayscale value of pixel i, j in the image. α is a constant and takes values ​​in the range of [1, 5]. μ is the neighborhood mean of pixel i, j. N represents the neighborhood size and is taken as the medium neighborhood size of 7×7.

[0062] Step 2: Use the three authoritative public datasets shown in Table 1 and randomly divide the datasets into 80% training sets and 20% test sets for training and testing evaluation of the segmentation model.

[0063] Table 1 Some important information of the three public data sets used in this invention

[0064] Segmentation target Input image modality Input image resolution Dataset source Labeled dataset liver CT 256×256 2017MICCAI LiTS 130 cases liver MRI 256×256 2019ISBI CHAOS 120 cases Nucleus mix 96×96 2018DSB Kaggle 670 cases

[0065] Step 3: If Figure 2 As shown in the figure, a lightweight convolutional segmentation model is built based on the inverted residual block in MobileNetV2. The fused deep separable convolution is based on the classic inverted residual block "first 1×1 dimensionality increase, then 3×3 convolution, and finally 1×1 dimensionality reduction". Batch normalization and PReLu function activation operations are performed on the dimensionality increase and convolution processes, and batch normalization operations are performed on the dimensionality reduction. Residual connections are introduced between the input features and the output features after convolution to reduce information loss. At the same time, the PReLu activation function is introduced before the residual connection to increase the nonlinear expression ability of the model.

[0066] Calculate the image features extracted by the model.

[0067] F′(X)=H(X)+δ(X);

[0068] Among them, F′(X) is the image feature extracted by the model; H(X) is the output feature after convolution; X is the input feature (i.e., the feature extracted based on the processed image data); δ is the PReLU activation function, which adds a linear transformation to the negative part of the ReLU function, which can effectively avoid the "death" phenomenon of neurons during training.

[0069] Step 4: Three feature maps of different scales are obtained through inverted residual blocks of different depths (these feature maps are 2 times the size of each other), a feature pyramid network is constructed, and adaptive weights are introduced. Feature maps of different resolutions are fused through top-down paths and lateral connections to obtain multi-scale features containing rich semantic information, thereby improving the positioning accuracy of the lesion area.

[0070] P i =α i Conv(C i+1 )+β i ·UpSample(N i );

[0071] Among them, P i represents the i-th layer feature map generated by top-down upsampling; C i+1 It is the feature map of the original bottom-up convolution result of layer i+1; N i is the low-resolution feature map of the i-th layer; Conv is the convolution operation; UpSample is the upsampling operation; α i and β i is a learnable weight parameter, calculated through the channel attention mechanism.

[0072]

[0073] Among them, σ is the Sigmoid activation function.

[0074] Step 5: First, design an adaptive channel attention mechanism to calculate the attention weight S of each channel c , adaptively learning the importance of each channel.

[0075] S c =σ(W2δ(W1·AvgPool(F c )));

[0076] Among them, F c is the cth channel of the feature pyramid network output feature map; AvgPool is the average pooling operation; W1 and W2 are learnable parameter matrices; S c is the attention weight of the cth channel.

[0077] Recalibrate the channel features to obtain the recalibrated channel attention features F′ c :

[0078] F′ c =s c ·F c .

[0079] Secondly, in order to capture the spatial characteristics of the lesion area, an adaptive spatial attention mechanism is designed to recalibrate the spatial features by learning the importance of each position. The attention weight of the (i, j)th position is calculated as:

[0080] s ij =σ(W4δ(W3·Concat(AvgPool(P i ), MaxPool(P i ))));

[0081] Among them, MaxPool is the maximum pooling operation; Concat is the concatenation operation; W4 and W3 are learnable parameter matrices, s ij is the attention weight of the (i, j)th position.

[0082] Recalibrated spatial attention feature F s ' is calculated as:

[0083] F s ′=s ij ·P i .

[0084] The fusion of channel attention features and spatial attention features can further enhance the model's ability to extract and distinguish image features, and improve the model's comprehensive learning and expression capabilities for image features.

[0085]

[0086] in, is the tensor product symbol.

[0087] F fusion Input into the model and obtain the final output of the model, which is the image lesion area segmentation result. The result is a feature map with the same size as the input image, and each pixel value represents the category or area to which the pixel belongs in the image.

[0088] Step 6: Consider the gradient information, spatial information and multi-scale information of the pixel and design the weight w i To adjust the loss function, focus on the hard-to-segment areas.

[0089]

[0090] Where L is the total loss; w i is the weight of the i-th pixel; y i is the true label of the i-th pixel (0 or 1); p i is the probability that the i-th pixel predicted by the model is a positive class; α, β, γ, δ, η, θ are hyperparameters with values ​​in the range of [0,1], which are used to adjust the range and decay rate of weights; is the gradient information of the i-th pixel; I i is the feature information of the i-th pixel; I neighbor is the feature information of the neighboring pixels of the i-th pixel; multi-scale is the feature information of the i-th pixel at multiple scales.

[0091] Step 7: After feature extraction, feature fusion and feature enhancement in steps 1 to 6 above, an efficient and accurate improved lightweight convolutional network model is obtained; the model is trained until the epoch is greater than the set threshold. The epoch thresholds of the training models based on the LiTS, CHAOS and DBS datasets are set to 30, 30 and 50 respectively.

[0092] Step 8: Input the test set into the trained network model to obtain the lesion area segmentation result.

[0093] In this embodiment, the effect evaluation uses four indicators: similarity coefficient (Dice), intersection over union (mIoU), precision (Precision) and recall (Recall) to evaluate the segmentation performance of the model.

[0094]

[0095] Among them, TP, FP, and FN are the number of true positives, false positives, and false negatives, respectively. The larger the values ​​of the four indicators, the higher the segmentation accuracy of the model.

[0096] Based on the above four evaluation methods, the segmentation performance of the lightweight convolutional segmentation model designed by the present invention is compared with that of the three mainstream segmentation algorithms, FCN, U-Net, and V-Net, on the above three data sets. Figure 4 , Figure 5 and Figure 6 The comparison between the present invention and the three algorithms shows that the convolutional segmentation model designed by the present invention has the best comprehensive performance in all four performance evaluation indicators.

[0097] Finally, the medical image data to be segmented is input into the trained convolutional segmentation model to obtain the segmentation result of the medical image lesion area.

[0098] In another embodiment, the present invention proposes a computer-readable storage medium storing a computer program, wherein the computer program enables a computer to execute the method for automatic segmentation of medical image lesion areas based on an improved lightweight convolutional network as described in Example 1.

[0099] In another embodiment, the present invention proposes an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method for automatic segmentation of medical image lesion areas based on an improved lightweight convolutional network as described in Example 1 is implemented.

[0100] In the embodiments disclosed in the present application, the computer storage medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. The computer storage medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the above. More specific examples of computer storage media can include electrical connections based on one or more lines, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above.

[0101] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in this application can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0102] The above are only preferred embodiments of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should be regarded as the protection scope of the present invention.

Claims

1. A method for automatic segmentation of medical image lesion areas based on an improved lightweight convolutional network, characterized in that: include: Preprocess the medical imaging data and divide the preprocessed data into training sets and test sets; Build a lightweight convolutional segmentation model based on the inverted residual block of MobileNetV2; By setting inverted residual modules of different depths to obtain feature maps of different scales, a feature pyramid network is constructed and adaptive weights are introduced. Through top-down paths and lateral connections, feature maps of different scales are fused to obtain multi-scale features. An adaptive channel attention mechanism is designed to calculate the attention weight of each channel of the pyramid network output feature, and the channel features are recalibrated accordingly. An adaptive spatial attention mechanism is designed to calculate the attention weight of each position of the pyramid network output feature, and the spatial features are recalibrated accordingly. The channel attention features and spatial attention features obtained after recalibration are integrated and used to input the convolutional segmentation model to obtain the segmentation results of the lesion area of ​​the medical image; Design the loss function of the convolutional segmentation model, and use the divided training set and test set to train and test the convolutional segmentation model; Input the medical image data to be segmented into the trained convolutional segmentation model to obtain the segmentation result of the medical image lesion area; Wherein, the preprocessing of the medical image data includes normalization processing and denoising processing; The normalization process is to normalize the medical image pixel values ​​to a uniform range of [0,1], as shown in the following formula: Where x represents the original medical image pixel, x max 、x min Represent the maximum and minimum values ​​of pixels respectively; The denoising process adopts an adaptive window adjustment method adapted to different pixel intensities, as shown in the following formula: Where W(i,j) is the window size; m is the window adjustment parameter; CC(i,j), LC(i,j), and IC(i,j) are the pixel comprehensive contrast, local contrast, and intensity contrast, respectively; I(i,j) is the grayscale value of pixel i,j in the image; α is a constant; μ is the neighborhood mean of pixel i,j; and N represents the neighborhood size.

2. The method for automatic segmentation of medical image lesion areas based on an improved lightweight convolutional network as claimed in claim 1, characterized in that: In the convolutional segmentation model constructed based on the inverted residual block of MobileNetV2, batch normalization and PReLu function activation operations are performed in both the dimension increase and convolution processes, batch normalization operations are performed on the dimension reduction, residual connections are introduced between the input features and the output features after convolution, and the PReLu activation function is introduced before the residual connection; The image feature F'(X) extracted by the convolutional segmentation model is: F'(X) = H(X) + δ(X); Where H(X) is the output feature after convolution; X is the input feature, which is the feature extracted from the preprocessed medical image data; δ is the PReLU activation function.

3. The method for automatic segmentation of medical image lesion areas based on an improved lightweight convolutional network as claimed in claim 1, characterized in that: The feature pyramid network is constructed and adaptive weights are introduced to fuse feature maps of different scales through top-down paths and lateral connections to obtain multi-scale features, specifically: P i =a i ·Conv(C i+1 )+b i ·UpSample(N i ); Where P i represents the i-th layer feature map generated by top-down upsampling; C i+1 is the feature map of the i+1th convolution result from bottom to top; N i is the low-resolution feature map of the i-th layer; Conv is the convolution operation; UpSample is the upsampling operation; α i and β i is a learnable weight parameter, calculated through the channel attention mechanism: Where σ is the Sigmoid activation function.

4. The method for automatic segmentation of medical image lesion areas based on an improved lightweight convolutional network as claimed in claim 1, characterized in that: The designed adaptive channel attention mechanism calculates the attention weight of each channel of the pyramid network output feature as follows: S c =σ(W2δ(W1·AvgPool(F c ))); Where σ is the Sigmoid activation function; F c is the cth channel of the pyramid network output feature map; AvgPool is the average pooling operation; W1 and W2 are learnable parameter matrices; S c is the attention weight of the cth channel; Recalibrate the channel features to obtain the recalibrated channel attention features F c ': F c '=s c ·F c 。 5. The method for automatic segmentation of medical image lesion areas based on an improved lightweight convolutional network as claimed in claim 4, characterized in that: The designed adaptive spatial attention mechanism calculates the attention weight of each position of the pyramid network output feature as: s ij =σ(W4δ(W3·Concat(AvgPool(P i ),MaxPool(P i )))); Where Concat is a concatenation operation; MaxPool is a maximum pooling operation; W4 and W3 are learnable parameter matrices, and s ij is the attention weight of the (i, j)th position; Recalibrate the spatial features to obtain the recalibrated spatial attention features F s ': F s ′=s ij ·P i 。 6. The method for automatic segmentation of medical image lesion areas based on an improved lightweight convolutional network as claimed in claim 5, characterized in that: The channel attention features and spatial attention features obtained after the fusion recalibration are specifically: In the formula, F fusion To fusion features, is the tensor product symbol.

7. The method for automatic segmentation of medical image lesion areas based on an improved lightweight convolutional network as claimed in claim 1, characterized in that: The loss function of the convolutional segmentation model is: Where L is the total loss; w i is the weight of the i-th pixel; y i is the true label of the i-th pixel; p i is the probability that the i-th pixel predicted by the model is a positive class; α, β, γ, δ, η, θ are hyperparameters with values ​​in the range of [0,1]; is the gradient information of the i-th pixel; I i is the feature information of the i-th pixel; I neighbor is the feature information of the neighboring pixels of the i-th pixel; I multi-scale is the feature information of the i-th pixel at multiple scales.

8. A computer-readable storage medium storing a computer program, characterized in that: The computer program enables the computer to execute the method for automatic segmentation of medical image lesion areas based on an improved lightweight convolutional network as described in any one of claims 1 to 7.

9. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the method for automatic segmentation of medical image lesion areas based on an improved lightweight convolutional network as described in any one of claims 1 to 7 is implemented.

Images