A method for medical image segmentation based on convolutional neural network

Abstract

The present invention discloses a method for medical image segmentation based on a convolutional neural network, the method comprising the following steps: preprocessing the medical image to obtain first image data and second image data; constructing an image segmentation model based on a convolutional neural network model to obtain a first image model and a Markov image model respectively; iterating the first image model, training the Markov image model according to the iteration result, and finally outputting the image segmentation result of the trained Markov image model. The method provided by the present invention introduces a fully connected conditional random field and a Markov network, and both the first image model and the Markov model are deep neural network models, which greatly enhances the learning ability of the neural network, establishes an accurate mapping from the original image to the grayscale image, can improve the image quality, reduce the doctor's misjudgment due to blurred images, and improve the doctor's judgment of the disease.

Description

Technical Field

[0001] The present invention relates to the field of medical technology, and in particular to a medical image segmentation method based on a convolutional neural network. Background Art

[0002] In actual image processing problems, image edge is one of the most basic features of an image, and often carries most of the information of an image. Image edge is often used in image processing and analysis technologies such as feature description, image recognition, image segmentation, image enhancement, and image compression, so as to further analyze and understand the image. It has a wide range of applications in the fields of image recognition, image segmentation, image enhancement, and image compression. We often need to detect and extract the edge image of an image. For example, the edge area of ​​orthopedic medical images is not clearly determined, which increases the difficulty for doctors to judge cases based on medical images.

[0003] The essence of edge detection is to use an algorithm to extract the boundary between the object and the background in the image. We define the edge as the boundary of the area where the grayscale changes sharply in the image. The change of the image grayscale can be reflected by the gradient of the image grayscale distribution, so we can use the local image differential technology to obtain the edge detection operator. The classic edge detection method is to construct an edge detection operator for a small neighborhood of pixels in the original image.

[0004] Neural network is a multidisciplinary research topic. With the continuous deepening of application technology and the continuous expansion of application scope, it is used to solve many problems that traditional science cannot solve. It has brought positive influences on people's learning of the world, discovering unknown categories, enhancing the level of modern scientific research, and thus driving productivity with science and technology and increasing the scale of people's economic development. It has become an internationally recognized cutting-edge frontier topic in the field of science. The exploration of neural networks is an important link in promoting the application of neural network technology and has extremely high practical engineering application value. Summary of the invention

[0005] The purpose of the present invention is to address the deficiencies in the prior art and provide a medical image segmentation method based on a convolutional neural network. The image edge grayscale value obtained by the method is more accurate and has a stronger sense of hierarchy.

[0006] The present invention provides a medical image segmentation method based on a convolutional neural network, the method being executed by a computer device and comprising the following steps:

[0007] 1) preprocessing the medical image to obtain first image data and second image data;

[0008] 2) constructing an image segmentation model based on the convolutional neural network model to obtain a first image model and a Markov image model respectively;

[0009] 3) Iterating the first image model, training the Markov image model according to the iteration result, and finally outputting the image segmentation result of the trained Markov image model.

[0010] Furthermore, the method for preprocessing the medical image includes the following steps: first, converting the medical image into a grayscale image, and the obtained grayscale image is called the first image data; second, performing image denoising or image enhancement processing on the first image data to obtain the second image data.

[0011] Furthermore, the method for converting the medical image into a grayscale image is: expanding any pixel point in the medical image into a coronal two-dimensional image, a sagittal two-dimensional image and a transverse two-dimensional image, and using the three expanded two-dimensional images as three channels of RGB to convert them into the grayscale image.

[0012] Furthermore, the method of constructing an image segmentation model based on a convolutional neural network model to obtain a first image model and a Markov image model respectively includes the following steps: first, constructing a VGG-16 deep neural convolutional network model; second, connecting the sofmax layer of the VGG-16 deep neural convolutional network model with a minimized fully connected conditional random field, and the connected model is called the first image model; then, connecting the sofmax layer of the VGG-16 deep neural convolutional network model with a Markov network, and the connected model is called the Markov image model.

[0013] Furthermore, in the VGG-16 deep neural convolutional network model: the convolution layer has a 3*3 convolution kernel, a step size of 1, and padding=same; the pooling layer has a 2*2 pooling kernel, a step size of 2; and the activation unit uses a Relu function.

[0014] Furthermore, the objective function of minimizing the fully connected conditional random field is:

[0015] Among them, E represents the edge between the pixel and other pixels; the unary potential function Ψu(x i ) is the output function of the VGG-16 deep neural convolutional network model; the pairwise potential function Ψp(x i ,x j ) is:

[0016] Among them, u(x i ,x j ) represents the compatibility measure between tags, K Grepresents a Gaussian kernel, M represents the number of Gaussian kernels, and the eigenvector f i Expressed by (x, y, R, G, B).

[0017] Furthermore, the objective function of the Markov network is:

[0018] Among them, E represents the edge between the pixel and other pixels; the unary potential function Ψ u (x i ) is the output function of the VGG-16 deep neural convolutional network model; the pairwise potential function Ψ p (x i ,x j ) is:

[0019] Among them, u(x i ,x j ) represents the compatibility measure between tags, K G represents a Gaussian kernel, M represents the number of Gaussian kernels, and the eigenvector f i Expressed by (x, y, R, G, B).

[0020] Furthermore, the method of iterating the first image model, training the Markov image model according to the iteration result, and finally outputting the image segmentation result includes the following steps: first, based on the second image data, using the first image model to output the nth image test result: inputting the n-1th image test result data into the nth image model to train the nth image model, and outputting the nth image test result from the nth image model; secondly, training the Markov model based on the nth image test result, and outputting the final image segmentation result from the Markov model.

[0021] Similarly, the present invention also provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the image segmentation method according to any one of the above claims.

[0022] Similarly, the present invention also provides a computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the processor executes the steps of the image segmentation method described in any one of the above claims.

[0023] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0024] The image segmentation method of the present invention is based on the learning of deep neural networks. By introducing fully connected conditional random fields and Markov networks, the first image model and the Markov model are both deep neural network models, which greatly enhances the learning ability of the neural network and establishes an accurate mapping from the original image to the grayscale image. The image information can be better extracted through the image enhancement technology method provided by this patent, which can improve the quality of the image, reduce the doctor's misjudgment due to blurred images, and improve the doctor's judgment of the disease. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The above and other objects, features and advantages of the embodiments of the present invention will become readily understood by reading the following detailed description with reference to the accompanying drawings. In the accompanying drawings, several embodiments of the present invention are shown in an exemplary and non-limiting manner, in which:

[0026] Figure 1 An exemplary process of an image segmentation method based on a convolutional neural network according to an embodiment of the present invention is shown.

[0027] Figure 2 Shows Figure 1 The flowchart of S1 is a schematic diagram of the method flow chart.

[0028] Figure 3 Shows Figure 1 The flowchart of S2 is a schematic diagram of the method flow chart.

[0029] Figure 4 Shows Figure 1 Schematic diagram of the flow chart of the S3 method. DETAILED DESCRIPTION

[0030] The principle and spirit of the present invention will be described below with reference to several exemplary embodiments. It should be understood that these embodiments are provided only to enable those skilled in the art to better understand and implement the present invention, and are not intended to limit the scope of the present invention in any way.

[0031] Embodiment 1: This embodiment provides an image segmentation method based on a convolutional neural network. The method performs image preprocessing before image segmentation. At the same time, a fully connected random field is introduced into the convolutional neural network to improve the edge clarity of the image, so that the edge decomposition area of ​​the image can be better distinguished, thereby assisting doctors in judging the condition. The flowchart is as follows Figure 1 As shown, the following steps are included:

[0032] S1, image preprocessing;

[0033] like Figure 2As shown, first, any pixel point in the medical image is expanded into a coronal two-dimensional image, a sagittal two-dimensional image and a transverse two-dimensional image, and the three expanded two-dimensional images are used as three channels of RGB to convert them into grayscale images, and these grayscale images are called first image data;

[0034] Secondly, performing Hanning window processing on the first image data, and performing image data denoising processing or image enhancement processing on the windowed image to obtain second image data;

[0035] S2, constructing an image segmentation model;

[0036] like Figure 3 As shown, first, a VGG-16 deep neural convolutional network model is constructed, in which: all convolution layers use 3*3 convolution kernels, the step size is 1, and padding=same; the pooling kernel of the pooling layer uses 2*2, the step size is 2; the activation unit uses the Relu function;

[0037] Secondly, the sofmax layer of the VGG-16 deep neural convolutional network model is connected to the minimized fully connected conditional random field, and the connected model is called the first image model. The objective function of the minimized fully connected conditional random field is:

[0038]

[0039] Among them, E represents the edge between the pixel and other pixels; the unary potential function Ψu(x i ) is the output function of the VGG-16 deep neural convolutional network model; the pairwise potential function Ψp(x i ,x j ) is calculated using the following formula:

[0040]

[0041] Among them, u(x i ,x j ) represents the compatibility measure between tags, K G represents a Gaussian kernel, M represents the number of Gaussian kernels, and the eigenvector f i It is represented by (x, y, R, G, B), that is, the pixel value (R, G, B) and the coordinate position (x, y) of the pixel point are calculated together as the feature vector.

[0042] The sofmax layer of the VGG-16 deep neural convolutional network model is connected to a Markov network, and the connection model is called a Markov image model, so that the objective function of the Markov image model is minimized, wherein the objective function is the same as the first image model:

[0043]

[0044] Among them, E represents the edge between the pixel and other pixels; the unary potential function Ψ u (x i ) is the output function of the VGG-16 deep neural convolutional network model; the pairwise potential function Ψ p (x i ,x j ) is calculated using the following formula:

[0045]

[0046] Among them, u(x i ,x j ) represents the compatibility measure between tags, K G represents a Gaussian kernel, M represents the number of Gaussian kernels, and the eigenvector f i It is represented by (x, y, R, G, B), that is, the pixel value (R, G, B) and coordinate position (x, y) of the pixel point are calculated together as the feature vector.

[0047] S3, iterative model;

[0048] like Figure 4 As shown, first, based on the second image data, the first image model is used to determine the nth image model. This includes: inputting the n-1th image test result data into the nth image model to train the nth image model, and outputting the nth image test result from the nth image model.

[0049] Secondly, a Markov model is trained based on the nth image test result, and a final image segmentation result is output from the Markov model, that is, the image segmentation result of the trained Markov image model is output.

[0050] An embodiment of the present invention further provides a computer device, including a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor executes the steps of the above-mentioned image segmentation method.

[0051] An embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the processor executes the steps of the above-mentioned image segmentation method.

[0052] The implementation may be distributed as circuits among multiple system components, such as multiple processors and memories, optionally including multiple distributed processing systems. Parameters, databases, and other data structures may be stored and managed separately, may be incorporated into a single memory or database, may be logically and physically organized in many different ways, and may be implemented in many different ways, including as data structures such as linked lists, hash tables, arrays, records, objects, or implicit storage mechanisms. The program may be part of a single program (e.g., a subroutine), an independent program, distributed across several memories and processors, or implemented in many different ways, such as in a library, such as a shared library (e.g., a dynamic link library (DLL)). For example, when executed by a circuit, a DLL may store instructions for performing any of the processing described above or shown in the accompanying drawings.

[0053] The methods and devices involved in the embodiments of the present invention can be implemented using standard programming techniques, using rule-based logic or other logic to implement various method steps. It should also be noted that the words "device" and "module" used herein and in the claims are intended to include implementations using one or more lines of software code and / or hardware implementations and / or devices for receiving input.

[0054] Any steps, operations or procedures described herein may be performed or implemented using one or more hardware or software modules, either alone or in combination with other devices. In one embodiment, the software module is implemented using a computer program product including a computer-readable medium containing computer program code, which can be executed by a computer processor to perform any or all of the described steps, operations or procedures.

[0055] The foregoing description of the implementation of the present invention has been given for purposes of illustration and description. The foregoing description is not intended to be exhaustive nor to limit the invention to the exact form disclosed, and various variations and modifications are possible in accordance with the above teachings or may be obtained from the practice of the invention. These embodiments are selected and described in order to illustrate the principles of the invention and its practical application, so that those skilled in the art can utilize the invention in various embodiments and with various modifications suitable for the particular use contemplated.

Claims

1. A medical image segmentation method based on convolutional neural network, characterized in that: The method is executed by a computer device and comprises the following steps: 1) Preprocessing the medical image to obtain first image data and second image data; 2) Constructing an image segmentation model based on the convolutional neural network model to obtain a first image model and a Markov image model respectively; 3) iterating the first image model, training the Markov image model according to the iteration result, and finally outputting the image segmentation result of the trained Markov image model; The method for preprocessing the medical image comprises the following steps: First, the medical image is converted into a grayscale image, and the obtained grayscale image is referred to as the first image data; Secondly, performing image denoising or image enhancement processing on the first image data to obtain the second image data; The method for constructing an image segmentation model based on a convolutional neural network model to obtain a first image model and a Markov image model comprises the following steps: First, construct the VGG-16 deep neural convolutional network model; Secondly, the sofmax layer of the VGG-16 deep neural convolutional network model is connected to the minimized fully connected conditional random field, and the connected model is called the first image model; Then, the sofmax layer of the VGG-16 deep neural convolutional network model is connected to the Markov network, and the connected model is called the Markov image model; In the VGG-16 deep neural convolutional network model: the convolution layer has a 3*3 convolution kernel, a step size of 1, and padding=same; the pooling layer has a 2*2 pooling kernel, a step size of 2; the activation unit uses the Relu function; The objective function of minimizing the fully connected conditional random field is: in, E represents the edge between a pixel and other pixels; The univariate potential function Ψu(x i ) is the output function of the VGG-16 deep neural convolutional network model; Pairwise potential function Ψp(x i ,x j ) is: in, u(x i ,x j ) represents the compatibility measure between tags, K G represents a Gaussian kernel, M represents the number of Gaussian kernels, and the eigenvector f i It is represented by (x, y, R, G, B); The objective function of the Markov network is: in, E represents the edge between a pixel and other pixels; The univariate potential function Ψu(x i ) is the output function of the VGG-16 deep neural convolutional network model; Pairwise potential function Ψp(x i ,x j ) is: in, u(x i ,x j ) represents the compatibility measure between tags, K G represents a Gaussian kernel, M represents the number of Gaussian kernels, and the eigenvector f i Expressed by (x, y, R, G, B).

2. The medical image segmentation method according to claim 1, characterized in that: The method for converting the medical image into a grayscale image is: expanding any pixel point in the medical image into a coronal two-dimensional image, a sagittal two-dimensional image and a transverse two-dimensional image, and using the three expanded two-dimensional images as three channels of RGB to convert them into the grayscale image.

3. The medical image segmentation method according to claim 1, characterized in that: The method of iterating the first image model, training the Markov image model according to the iteration result, and finally outputting the image segmentation result comprises the following steps: First, based on the second image data, the first image model is used to output the nth image test result: the n-1th image test result data is input into the nth image model to train the nth image model, and the nth image test result is output from the nth image model; Secondly, a Markov model is trained based on the nth image test result, and a final image segmentation result is output from the Markov model.

4. A computer device, characterized in that: The invention comprises a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor executes the steps of the image segmentation method according to any one of claims 1 to 3.

5. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor executes the steps of the image segmentation method according to any one of claims 1 to 3.

Images