Polyp image segmentation method, device, computer equipment and storage medium

Abstract

The present invention relates to the technical field of image segmentation, and discloses a polyp image segmentation method, device, computer equipment and storage medium. The method comprises: obtaining a polyp image training set and a polyp image verification set, performing grayscale processing on images of the polyp image training set to obtain a grayscale image set, using images of the polyp image training set to train a preset encoding convolutional neural network to obtain a first encoding feature map, a second encoding feature map, a third encoding feature map, a fourth encoding feature map and encoding convolutional neural network parameters, using the first encoding feature map, the second encoding feature map, the third encoding feature map, the fourth encoding feature map and grayscale images of the grayscale image set to train a preset decoding convolutional neural network to obtain decoding convolutional neural network parameters, obtaining a polyp segmentation model according to the encoding convolutional neural network parameters and the decoding convolutional neural network parameters, inputting a polyp image to be segmented into the polyp segmentation model for segmentation, and improving the segmentation accuracy of the polyp image to be segmented.

Description

Technical Field

[0001] The present invention relates to the field of image segmentation technology, and in particular to a polyp image segmentation method, device, computer equipment and storage medium. Background Art

[0002] Colonoscopy is the main method for colorectal cancer (CRC) screening. It can detect and remove colon polyps before they develop into cancer. The detection and segmentation of colorectal polyps is an important basis for doctors to diagnose colon cancer.

[0003] Traditional polyp image segmentation algorithms mainly rely on researchers to manually design polyp features to achieve segmentation tasks, which requires a lot of manpower and is inefficient. In addition, since polyps of the same type have different appearances such as size, texture and color, in colonoscopy images, the boundary between polyps and surrounding mucosa is usually very blurred. Manually designing polyp features to achieve polyp image segmentation cannot accurately segment polyp images through blurred boundaries. Summary of the invention

[0004] The embodiments of the present invention provide a polyp image segmentation method, apparatus, computer equipment and storage medium to improve the accuracy of polyp segmentation.

[0005] In order to solve the above technical problems, the present application embodiment provides a polyp image segmentation method, including:

[0006] Obtain a polyp image training set and a polyp image verification set;

[0007] Performing grayscale processing on the images in the polyp image training set to obtain a grayscale image set;

[0008] Using the images in the polyp image training set to train a preset encoding convolutional neural network to obtain a first encoding feature map, a second encoding feature map, a third encoding feature map, a fourth encoding feature map and encoding convolutional neural network parameters;

[0009] Using the first coding feature map, the second coding feature map, the third coding feature map, the fourth coding feature map, and the grayscale image in the grayscale image set to train a preset decoding convolutional neural network to obtain decoding convolutional neural network parameters;

[0010] According to the encoding convolutional neural network parameters and the decoding convolutional neural network parameters, a polyp segmentation model to be verified is obtained;

[0011] Using the polyp image verification set to verify the polyp segmentation model to be verified, obtaining verification result information, and determining whether the polyp segmentation model to be verified is a polyp segmentation model according to the verification result information;

[0012] If yes, when obtaining the to-be-segmented polyp image, the to-be-segmented polyp image is input into the polyp segmentation model for segmentation to obtain a polyp image segmentation result graph.

[0013] In order to solve the above technical problems, the embodiment of the present application further provides a polyp image segmentation device, comprising:

[0014] An image acquisition module, used for acquiring a polyp image training set and a polyp image verification set;

[0015] A grayscale processing module, used for performing grayscale processing on the images in the polyp image training set to obtain a grayscale image set;

[0016] A coding convolutional neural network parameter acquisition module, used to train a preset coding convolutional neural network using images in the polyp image training set to obtain a first coding feature map, a second coding feature map, a third coding feature map, a fourth coding feature map and coding convolutional neural network parameters;

[0017] A decoding convolutional neural network parameter acquisition module, used to train a preset decoding convolutional neural network using the first encoding feature map, the second encoding feature map, the third encoding feature map, the fourth encoding feature map, and the grayscale image in the grayscale image set to obtain decoding convolutional neural network parameters;

[0018] A polyp segmentation model acquisition module to be verified, used to obtain the polyp segmentation model to be verified according to the encoding convolutional neural network parameters and the decoding convolutional neural network parameters;

[0019] A verification module, used to verify the polyp segmentation model to be verified using the images in the polyp image verification set, obtain verification result information, and determine whether the polyp segmentation model to be verified is a polyp segmentation model according to the verification result information;

[0020] The segmentation module is used for, when obtaining the polyp image to be segmented, inputting the polyp image to be segmented into the polyp segmentation model for segmentation to obtain a polyp image segmentation result map.

[0021] In order to solve the above technical problems, an embodiment of the present application also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above polyp image segmentation method when executing the computer program.

[0022] In order to solve the above technical problems, an embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above polyp image segmentation method are implemented.

[0023] The polyp image segmentation method, apparatus, computer equipment and storage medium provided by the embodiments of the present invention obtain a polyp image training set and a polyp image verification set, grayscale process the images in the polyp image training set to obtain a grayscale image set, use the images in the polyp image training set to train a preset encoding convolutional neural network to obtain a first encoding feature map, a second encoding feature map, a third encoding feature map, a fourth encoding feature map and encoding convolutional neural network parameters, use the first encoding feature map, the second encoding feature map, the third encoding feature map, the fourth encoding feature map and the grayscale images in the grayscale image set to train a preset decoding convolutional neural network to obtain decoding convolutional neural network parameters, and obtain a to-be-verified image according to the encoding convolutional neural network parameters and the decoding convolutional neural network parameters. A polyp segmentation model is provided. The polyp segmentation model to be verified is verified by using the polyp image verification set to obtain verification result information, and based on the verification result information, it is determined whether the polyp segmentation model to be verified is a polyp segmentation model. If so, when obtaining the polyp image to be segmented, the polyp image to be segmented is input into the polyp segmentation model for segmentation to obtain a polyp image segmentation result diagram. The present invention trains a preset encoding convolutional neural network and a preset decoding convolutional neural network by using images in a polyp image verification set and images in a grayscale image set to obtain a polyp segmentation image to be verified, and verifies the polyp segmentation model to be verified by using images in the polyp verification set to obtain a polyp segmentation model, thereby improving the training accuracy of the polyp segmentation model and thereby improving the segmentation accuracy of the polyp segmentation model for the polyp image to be segmented. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0025] Figure 1 is an exemplary system architecture diagram to which the present application may be applied;

[0026] Figure 2 is a flow chart of an embodiment of the polyp image segmentation method of the present application;

[0027] Figure 3 is a structural schematic diagram of an embodiment of a polyp image segmentation device according to the present application;

[0028] Figure 4 It is a structural diagram of an embodiment of a computer device according to the present application. DETAILED DESCRIPTION

[0029] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0030] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0031] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0032] See also Figure 1 ,like Figure 1 As shown, the system architecture 100 may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0033] Users can use terminal devices 101 , 102 , 103 to interact with server 105 via network 104 to receive or send messages, etc.

[0034] Terminal devices 101, 102, 103 can be various electronic devices with display screens and supporting web browsing, including but not limited to smart phones, tablet computers, e-book readers, MP3 players (Moving Picture E interface display perts Group Audio Layer III, Moving Picture Experts Compression Standard Audio Layer 3), MP4 (Moving Picture E interface display perts Group Audio Layer IV, Moving Picture Experts Compression Standard Audio Layer 4) players, laptop computers and desktop computers, etc.

[0035] The server 105 may be a server that provides various services, such as a background server that provides support for web pages displayed on the terminal devices 101 , 102 , and 103 .

[0036] It should be noted that the polyp image segmentation method provided in the embodiment of the present application is executed by a server, and accordingly, the polyp image segmentation device is arranged in the server.

[0037] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. According to the implementation requirements, there may be any number of terminal devices, networks and servers. The terminal devices 101, 102, 103 in the embodiment of the present application may specifically correspond to application systems in actual production.

[0038] See also Figure 2 , Figure 2 A polyp image segmentation method provided by an embodiment of the present invention is shown. The method is applied in Figure 1 The server in is used as an example to illustrate the details as follows:

[0039] S201: Obtain a polyp image training set and a polyp image verification set.

[0040] Specifically, the polyp image training set and the polyp image verification set are obtained from the publicly available polyp datasets Kvasir and CVC-ClinicDB. The Kvasir dataset contains 1,000 polyp images, the size of which ranges from 332x487 pixels to 1920x1072 pixels, and the regional shape and color of the polyps in each polyp image are different. The CVC-ClinicDB dataset contains 612 polyp images, the size of which is 384x288 pixels, and the regional shape and color of the polyps in each polyp image are different. By using the 1,000 polyp images in the Kvasir dataset The 612 polyp images in the dataset CVC-ClinicDB are divided according to the preset division rules to obtain the polyp image training set and the polyp image verification set, wherein the preset division rule is set according to the actual application scenario and is not specifically limited here. For example, 900 polyp images out of the 1000 polyp images in the dataset Kvasir are divided into the polyp image training set, and the remaining 100 polyp images are used as the polyp image verification set; 550 polyp images out of the 612 polyp images in the dataset CVC-ClinicDB are used as the polyp training set, and the remaining 62 polyp images are used as the polyp image verification set.

[0041] S202: grayscale processing is performed on the images in the polyp image training set to obtain a grayscale image set.

[0042] Specifically, the images in the polyp training set are gray-scaled by using image processing techniques such as image averaging or image gray-scaling to obtain a gray-scale image set.

[0043] Optionally, the images in the polyp image verification set are gray-scaled by using image processing techniques such as image averaging or image gray-scaling to obtain a gray-scale image verification set.

[0044] Optionally, in step S202, before grayscale processing is performed on the images in the polyp image training set to obtain the grayscale image set, the polyp segmentation method further includes:

[0045] The images in the polyp image training set are preprocessed, wherein the preprocessing methods include but are not limited to horizontal flipping and vertical flipping, random rotation, image resizing, normalization, and grayscale transformation.

[0046] Exemplarily, the process of preprocessing the images in the polyp image training set is as follows:

[0047] Optionally, based on a preset probability value, the images in the polyp image training set are flipped horizontally and vertically. In this embodiment, the preset probability value is preferably set to 50%.

[0048] Optionally, based on a preset rotation range value, the images in the polyp image training set are randomly rotated. In this embodiment, the preset rotation range value is preferably set to 0° to 90°.

[0049] Optionally, based on a preset resolution, the image resolution of the images in the polyp image training set is reset to fix the same resolution. In this embodiment, the preset resolution is preferably set to 352x352 pixels.

[0050] Optionally, the images in the polyp image training set are normalized.

[0051] Optionally, grayscale conversion is performed on the images in the polyp image training set to convert the images in the polyp image training set into single-channel grayscale images.

[0052] S203: Using images in the polyp image training set to train a preset encoding convolutional neural network to obtain a first encoding feature map, a second encoding feature map, a third encoding feature map, a fourth encoding feature map and encoding convolutional neural network parameters.

[0053] Specifically, the preset encoding convolutional neural network is implemented based on the HardNet network, which includes a first encoding block, a second encoding block, a third encoding block and a fourth encoding block, wherein the first encoding block, the second encoding block, the third encoding block and the fourth encoding block are connected in sequence, exemplarily, the first encoding block includes a convolution layer conv3*3, a convolution layer conv3*3 (step size = 2), a maximum pooling layer MaxPooling (step size = 2), an 8*HarDBIk layer, a convolution layer conv1*1 and a maximum pooling layer MaxPooling (step size = 2), and are connected in sequence, wherein the second maximum pooling layer MaxPooling (step size = 2) outputs a first encoding feature map;

[0054] The second encoding block includes an 8*HarDBIk layer, a convolutional layer conv1*1, an 8*HarDBIk layer, a convolutional layer conv1*1 and a maximum pooling layer MaxPooling (step size = 2), which are connected in sequence, wherein the maximum pooling layer MaxPooling (step size = 2) outputs a second encoding feature map;

[0055] The third encoding block includes an 8*HarDBIk layer, a convolution layer conv1*1 and a maximum pooling layer MaxPooling (step size = 2), which are connected in sequence, wherein the maximum pooling layer MaxPooling (step size = 2) outputs a third encoding feature map;

[0056] The fourth encoding block includes an 8*HarDBIk layer and a convolutional layer conv1*1, which are connected in sequence, wherein the convolutional layer conv1*1 outputs a fourth encoding feature map;

[0057] Among them, the output of the second maximum pooling layer MaxPooling (step size = 2) of the first coding block is connected to the input of the first 8*HarDBIk layer of the second coding block, the output of the maximum pooling layer MaxPooling (step size = 2) of the second coding block is connected to the input of the 8*HarDBIk layer of the third coding block, and the output of the maximum pooling layer MaxPooling (step size = 2) of the third coding block is connected to the input of the 8*HarDBIk layer of the third coding block;

[0058] The method adopted by the maximum pooling layer MaxPooling (step length = 2) is maximum pooling, that is, the maximum value of the image area of ​​the polyp image in the polyp image training set is selected as the value after pooling of the image area;

[0059] The HarDBIk layer specifically includes a convolutional layer conv3*3, a convolutional layer conv3*3, a maximum pooling layer MaxPooling, a convolutional layer conv3*3, ..., a convolutional layer conv3*3, a Droput layer, and a convolutional layer conv1*1, and they are connected in sequence, wherein "..." represents n convolutional layers conv3*3, n is a positive integer greater than 0, the Droput layer is used to avoid overfitting of the HarDBIk layer during training, the convolutional layer conv3*3 is a convolutional layer including at least one convolutional kernel of 3*3, the convolutional layer conv1*1 is a convolutional layer including at least one convolutional kernel of 1*1, and the 8*HarDBIk layer is 8 HarDBIk layers connected in sequence;

[0060] Specifically, the first encoding feature map is a feature map output by the first encoding block after training the images in the polyp training set, the second encoding feature map is a feature map output by the second encoding block after training the first encoding feature map, the third encoding feature map is a feature map output by the third encoding block after training the second encoding feature map, and the fourth encoding feature map is a feature map output by the fourth encoding block after training the third encoding feature map. The preset encoding convolutional neural network is trained multiple times with the images in the polyp training set to obtain encoding convolutional neural network parameters, wherein the encoding convolutional neural network parameters are network parameters such as weights and bias values ​​in the preset encoding convolutional neural network.

[0061] S204: Using the first coding feature map, the second coding feature map, the third coding feature map, the fourth coding feature map and the grayscale image in the grayscale image set to train a preset decoding convolutional neural network to obtain decoding convolutional neural network parameters.

[0062] Specifically, the preset decoding convolutional neural network includes a first decoding block, a second decoding block, a third decoding block and a fourth decoding block, wherein the first decoding block, the second decoding block, the third decoding block and the fourth decoding block are connected in sequence, exemplarily, the first decoding block includes an upsampling layer, a convolution layer conv3*3, a BN (Batch Normalization) layer and a RELU layer, and are connected in sequence, wherein the upsampling layer adopts a bilinear difference method, which is used to restore the image resolution of the fourth encoding feature map and restore the spatial information in the fourth encoding feature map, and the RELU layer adopts a Relu activation function to increase the nonlinear segmentation ability of the first decoding block for the fourth encoding feature map, and improve the accuracy of segmenting the polyps in the polyp image to be segmented, and the BN (Batch Normalization) layer normalizes the feature map output by the convolution layer conv3*3, and the second decoding block includes a first multi-scale attention module and a first neural network module; the third decoding module includes a first filtering enhancement module, a second multi-scale attention module and a second neural network module; the fourth decoding module includes a second filtering enhancement module, a third multi-scale attention module and a third neural network module;

[0063] Among them, the first neural network module, the second neural network module and the third neural network module have the same network structure, namely, a connection layer, an upsampling layer, a convolution layer conv3*3, a BN (Batch Normalization) layer and a RELU layer, and are connected in sequence, wherein the connection layer is a concatenate layer, which is used to fuse the input features, and the first filtering enhancement module is used to linearly transform the second coded feature map and the grayscale image in the grayscale image set, extract the change intensity in the first coded feature map, improve the contrast of the target area and its boundary with the background area in the second coded feature map, and superimpose the change intensity on the second coded feature map to restore part of the feature information in the second coded feature, and the second filtering enhancement module is used to linearly transform the first coded feature map and the grayscale image in the grayscale image set. The first multi-scale attention module uses a first decoding feature map as input, and uses a sigmoid function to calculate the first decoding feature map, and then uses the sigmoid function to calculate the first decoding feature map. The first multi-scale attention module uses a first decoding feature map as input, and uses the sigmoid function to calculate the first decoding feature map. The first multi-scale attention module uses a first decoding feature map as input, and the second multi-scale attention module uses a first decoding feature map as input, and the first ... The weight value is inverted. For example, the weight value is 3, and the inverted weight value is (-1*3+1). The inverted weight value is multiplied by the first decoded feature map to obtain a feature map. The cat function is used to multiply the inverted weight value by the first decoded feature map to obtain a feature map. The feature map and the first decoded feature map are spliced ​​to output the feature map output by the first multi-scale attention module. The second multi-scale attention module uses the feature map output by the first filtering enhancement module as input, and calculates the weight value through the sigmoid function. The weight value is inverted. For example, the weight value is 3, and the inverted weight value is (-1*3+1). The inverted weight value is multiplied by the first The feature map output by the filtering enhancement module is multiplied to obtain a feature map. Through the cat function, the inverted weight value is multiplied by the feature map output by the first filtering enhancement module to obtain a feature map. The feature map and the feature map output by the first filtering enhancement module are spliced ​​to output the feature map output by the second multi-scale attention module. The third multi-scale attention module takes the feature map output by the second filtering enhancement module as input, calculates the weight value through the sigmoid function, and inverts the weight value. For example, if the weight value is 3, the inverted weight value is (-1*3+1). The inverted weight value is multiplied by the feature map output by the second filtering enhancement module to obtain a feature map. Through the cat function,The inverted weight value is multiplied with the feature map output by the second filtering enhancement module to obtain a feature map and the feature map output by the first filtering enhancement module to obtain a feature map output by the third multi-scale attention module, wherein the decoding convolutional neural network parameters are network parameters such as weights and bias values ​​in the preset decoding convolutional neural network.

[0064] S205: Obtaining a polyp segmentation model to be verified according to the encoding convolutional neural network parameters and the decoding convolutional neural network parameters.

[0065] Specifically, the encoding convolutional neural network parameters are used as the parameters of the preset encoding convolutional neural network, the decoding convolutional neural network parameters are used as the parameters of the preset decoding convolutional neural network, and the preset encoding convolutional neural network and the preset decoding convolutional neural network constitute the polyp segmentation model to be verified.

[0066] S206: Using the polyp image verification set to verify the polyp segmentation model to be verified, obtaining verification result information, and determining whether the polyp segmentation model to be verified is a polyp segmentation model based on the verification result information.

[0067] Specifically, the verification result information may be a similarity value set. The polyp segmentation model to be verified uses the trained encoding convolutional neural network parameters and decoding convolutional neural network to extract features of the polyp area of ​​the images in the polyp image verification set, and outputs a set of prediction images (i.e., binary images of the polyp area). The prediction images in the prediction image set are calculated with the corresponding label images in the preset polyp image set to obtain a similarity value set. If the number of similarity values ​​in the similarity value set that is greater than a preset threshold is greater than a preset number threshold, the polyp segmentation model to be verified is determined to be a polyp segmentation model. Otherwise, the polyp segmentation model to be verified is trained, and the encoding convolutional neural network parameters and the decoding convolutional neural network parameters are updated until the number of similarity values ​​in the similarity value set that is greater than a preset threshold is greater than a preset number threshold, wherein the preset threshold and the preset number threshold are obtained based on analysis of historical experience data and are not specifically limited here.

[0068] S207: If yes, when obtaining the image of the polyp to be segmented, the image of the polyp to be segmented is input into the polyp segmentation model for segmentation to obtain a polyp image segmentation result map.

[0069] Specifically, the output feature map of the fourth decoding module in the preset decoding convolutional neural network is the polyp image segmentation result map.

[0070] In this embodiment, by obtaining a polyp image training set and a polyp image verification set, grayscale processing is performed on the images in the polyp image training set to obtain a grayscale image set, and the images in the polyp image training set are used to train a preset encoding convolutional neural network to obtain a first encoding feature map, a second encoding feature map, a third encoding feature map, a fourth encoding feature map, and encoding convolutional neural network parameters, and the first encoding feature map, the second encoding feature map, the third encoding feature map, the fourth encoding feature map, and the grayscale images in the grayscale image set are used to train a preset decoding convolutional neural network to obtain decoding convolutional neural network parameters, and a polyp segmentation model to be verified is obtained according to the encoding convolutional neural network parameters and the decoding convolutional neural network parameters, and the polyp image verification is used. The polyp segmentation model to be verified is verified in a set to obtain verification result information, and according to the verification result information, it is determined whether the polyp segmentation model to be verified is a polyp segmentation model. If so, when obtaining the polyp image to be segmented, the polyp picture to be segmented is input into the polyp segmentation model for segmentation to obtain a polyp image segmentation result diagram. The present invention trains a preset encoding convolutional neural network and a preset decoding convolutional neural network by using images in a polyp image verification set and images in a grayscale image set to obtain a polyp segmentation image to be verified, and verifies the polyp segmentation model to be verified by using images in a polyp verification set to obtain a polyp segmentation model, thereby improving the training accuracy of the polyp segmentation model and further improving the segmentation accuracy of the polyp segmentation model for the polyp image to be segmented.

[0071] In some optional implementations of this embodiment, in step S202, the polyp image training set includes at least one polyp image to be trained, and grayscale processing is performed on the images in the polyp image training set to obtain the grayscale image set, which includes:

[0072] Based on a preset scaling ratio, each polyp image to be trained is scaled to obtain an expanded sample to be trained corresponding to each polyp image to be trained.

[0073] Specifically, the preset scaling ratio may be to randomly magnify each polyp image to be trained by 1.25 times, keep it unchanged, or reduce it by 0.75 times.

[0074] The augmented samples to be trained and the polyp images to be trained are used as a polyp image training set.

[0075] In this embodiment, based on a preset scaling ratio, each polyp image to be trained is scaled to obtain an expanded sample to be trained corresponding to each polyp image to be trained, thereby increasing the training samples and improving the accuracy of network parameter training of a preset encoding convolutional neural network and a preset decoding convolutional neural network, thereby improving the accuracy of polyp segmentation.

[0076] In some optional implementations of this embodiment, in step S203, the preset encoding convolutional neural network includes a first encoding block, a second encoding block, a third encoding block, and a fourth encoding block, and the preset encoding convolutional neural network is trained using images in a polyp image training set to obtain a first encoding feature map, a second encoding feature map, a third encoding feature map, and a fourth encoding feature map, including:

[0077] The first coding block is used to extract features from images in the polyp image training set to obtain a first coding feature map.

[0078] The second encoding block is used to perform feature extraction on the first encoding feature map to obtain a second encoding feature map.

[0079] The third encoding block is used to perform feature extraction on the second encoding feature map to obtain a third encoding feature map.

[0080] The fourth encoding block is used to perform feature extraction on the third encoding feature map to obtain a fourth encoding feature map and encoding convolutional neural network parameters.

[0081] In this embodiment, the first coding block, the second coding block, the third coding block and the fourth coding block are trained using images in the polyp image training set to improve the accuracy of model parameter training, thereby improving the accuracy of polyp segmentation.

[0082] In some optional implementations of this embodiment, in step S204, the preset decoding convolutional neural network includes a first decoding block, a second decoding block, a third decoding block, and a fourth decoding block, and the preset decoding convolutional neural network is trained using the first encoding feature map, the second encoding feature map, the third encoding feature map, the fourth encoding feature map, and the grayscale image in the grayscale image set, and obtaining the decoding convolutional neural network parameters includes the following steps S2040 to S2043:

[0083] Step S2040: Use the first decoding block to perform feature processing on the fourth encoding feature map to obtain a first decoding feature map.

[0084] Step S2041: Use the second decoding block to perform feature processing on the first decoding feature map and the third encoding feature map to obtain a second decoding feature map.

[0085] Step S2042: Use the third decoding block to perform feature processing on the second decoding feature map, the second encoding feature map and the grayscale image in the grayscale image set to obtain a third decoding feature map.

[0086] Step S2043: Use the fourth decoding block to perform feature processing on the third decoding feature map, the first encoding feature map and the grayscale image in the grayscale image set to obtain decoding convolutional neural network parameters.

[0087] Specifically, the fourth decoding block performs feature processing on the third decoding feature map, the first encoding feature map and the grayscale image in the grayscale image set, and outputs a feature map as a polyp image segmentation result map, and at the same time, obtains decoding convolutional neural network parameters.

[0088] In this embodiment, the first decoding block, the second decoding block, the third decoding block and the fourth decoding block are trained by using the first coding feature map, the second coding feature map, the third coding feature map, the fourth coding feature map and the grayscale images in the grayscale image set to improve the accuracy of model parameter training, thereby improving the accuracy of polyp segmentation.

[0089] In some optional implementations of this embodiment, in step S2041, the second decoding block includes a first multi-scale attention module and a first neural network module, and the second decoding block is used to perform feature processing on the first decoding feature map and the third encoding feature map to obtain a second decoding feature map including:

[0090] The first multi-scale attention module is used to perform feature processing on the first decoding feature map and the third encoding feature map to obtain a first feature splicing map.

[0091] Specifically, the first multi-scale attention module is used, the first decoding feature map is used as a weight, and a dot multiplication operation and a residual connection are performed on the third-layer encoding feature map to obtain a first feature splicing map.

[0092] The first neural network module is used to perform feature processing on the first feature splicing graph to obtain a second decoding feature graph.

[0093] In this embodiment, the first multi-scale attention module is used to perform feature processing on the first decoding feature map and the third encoding feature map to establish the target area and its boundary with the background area, thereby improving the accuracy of polyp segmentation.

[0094] In some optional implementations of this embodiment, in step S2042, the third decoding block includes a first filtering enhancement module, a second multi-scale attention module, and a second neural network module, and the third decoding block is used to perform feature processing on the second decoding feature map, the second encoding feature map, and the grayscale image in the grayscale image set, and the third decoding feature map includes:

[0095] The first filtering enhancement module is used to perform feature processing on the second decoding feature map, the second encoding feature map and the grayscale image in the grayscale image set to obtain a second decoding feature enhancement map, a second encoding feature enhancement map and a first grayscale image enhancement map.

[0096] A second multi-scale attention module is used to perform feature processing on the second decoding feature enhancement map, the second encoding feature enhancement map and the first grayscale image enhancement map to obtain a second feature splicing map.

[0097] Specifically, a second multi-scale attention module is used, and the second decoded feature enhancement map is used as a weight. The second encoded feature enhancement map and the first grayscale image enhancement map are subjected to a dot multiplication operation and a residual connection to obtain a second feature splicing map.

[0098] The second neural network module is used to perform feature processing on the second feature concatenation graph to obtain a third decoding feature graph.

[0099] In this embodiment, the second decoding feature enhancement map, the second encoding feature enhancement map and the first grayscale image enhancement map are trained through the second multi-scale attention module to establish the target area and its boundary with the background area; and the second decoding feature map, the second encoding feature map and the grayscale image in the grayscale image set are trained through the first filtering enhancement module to restore the lost feature information and improve the accuracy of polyp segmentation.

[0100] In some optional implementations of this embodiment, in step S2043, the fourth decoding block includes a second filtering enhancement module, a third multi-scale attention module and a third neural network module, and the fourth decoding block is used to train the third decoding feature map, the first encoding feature map and the grayscale image in the grayscale image set, and the decoding convolutional neural network parameters include:

[0101] The second filtering enhancement module is used to train the third decoding feature map, the first encoding feature map and the grayscale image in the grayscale image set to obtain the third decoding feature enhancement map, the first encoding feature enhancement map and the second grayscale image enhancement map.

[0102] A third multi-scale attention module is used to perform feature processing on the third decoding feature enhancement map, the first encoding feature enhancement map, and the second grayscale image enhancement map to obtain a third feature splicing map.

[0103] The third neural network module is used to perform feature processing on the third feature splicing graph to obtain decoding convolutional neural network parameters.

[0104] In this embodiment, the third decoding feature enhancement map, the first encoding feature enhancement map and the second grayscale image enhancement map are trained through the third multi-scale attention module to establish the target area and its boundary with the background area; and the third decoding feature map, the first encoding feature map and the grayscale image in the grayscale image set are trained through the second filtering enhancement module to restore the lost feature information and improve the accuracy of polyp segmentation.

[0105] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0106] Figure 3The principle block diagram of the polyp image segmentation device corresponding to the polyp image segmentation method in the above embodiment is shown. Figure 3 As shown, the polyp image segmentation device includes an image acquisition module 30, a grayscale processing module 31, an encoding convolutional neural network parameter acquisition module 32, a decoding convolutional neural network parameter acquisition module 33, a polyp segmentation model acquisition module 34 to be verified, a verification module 35 and a segmentation module 36.

[0107] The detailed description of each functional module is as follows:

[0108] The image acquisition module 30 is used to acquire a polyp image training set and a polyp image verification set.

[0109] The grayscale processing module 31 is used to perform grayscale processing on the images in the polyp image training set to obtain a grayscale image set.

[0110] The encoding convolutional neural network parameter acquisition module 32 is used to train the preset encoding convolutional neural network using images in the polyp image training set to obtain a first encoding feature map, a second encoding feature map, a third encoding feature map, a fourth encoding feature map and encoding convolutional neural network parameters.

[0111] The decoding convolutional neural network parameter acquisition module 33 is used to train the preset decoding convolutional neural network using the first encoding feature map, the second encoding feature map, the third encoding feature map, the fourth encoding feature map and the grayscale image in the grayscale image set to obtain the decoding convolutional neural network parameters.

[0112] The polyp segmentation model acquisition module 34 is used to obtain the polyp segmentation model to be verified according to the encoding convolutional neural network parameters and the decoding convolutional neural network parameters.

[0113] The verification module 35 is used to verify the polyp segmentation model to be verified using the images in the polyp image verification set, obtain verification result information, and determine whether the polyp segmentation model to be verified is a polyp segmentation model based on the verification result information.

[0114] The segmentation module 36 is used to input the to-be-segmented polyp image into the polyp segmentation model for segmentation when acquiring the to-be-segmented polyp image, and obtain a polyp image segmentation result map.

[0115] Furthermore, the polyp image training set includes at least one polyp image to be trained, and the polyp image segmentation device includes:

[0116] The scaling processing module is used to perform scaling processing on each polyp image to be trained based on a preset scaling ratio to obtain an expanded sample to be trained corresponding to each polyp image to be trained.

[0117] The polyp image training set determination module is used to use the to-be-trained expanded samples and the to-be-trained polyp images as the polyp image training set.

[0118] Further, the preset encoding convolutional neural network includes a first encoding block, a second encoding block, a third encoding block and a fourth encoding block, and the encoding convolutional neural network parameter acquisition module 32 includes:

[0119] A first training module, configured to extract features from images in a polyp image training set using a first coding block to obtain a first coding feature map;

[0120] A second training module is used to extract features from the first coding feature map using a second coding block to obtain a second coding feature map;

[0121] A third training module is used to extract features from the second coding feature map using a third coding block to obtain a third coding feature map;

[0122] The fourth training model is used to extract features from the third encoded feature map using a fourth encoding block to obtain a fourth encoded feature map and encoded convolutional neural network parameters.

[0123] Further, the preset decoding convolutional neural network includes a first decoding block, a second decoding block, a third decoding block and a fourth decoding block, and the decoding convolutional neural network parameter acquisition module 33 includes:

[0124] The fifth training module is used to use the first decoding block to perform feature processing on the fourth encoding feature map to obtain a first decoding feature map.

[0125] The sixth training module is used to use the second decoding block to perform feature processing on the first decoding feature map and the third encoding feature map to obtain a second decoding feature map.

[0126] The seventh training module is used to use the third decoding block to perform feature processing on the second decoding feature map, the second encoding feature map and the grayscale image in the grayscale image set to obtain a third decoding feature map.

[0127] The eighth training module is used to use the fourth decoding block to perform feature processing on the third decoding feature map, the first encoding feature map and the grayscale image in the grayscale image set to obtain decoding convolutional neural network parameters.

[0128] Furthermore, the second decoding block includes a first multi-scale attention module and a first neural network module, and the sixth training module includes:

[0129] A first feature splicing map obtaining module, used for performing feature processing on the first decoding feature map and the third encoding feature map using a first multi-scale attention module to obtain a first feature splicing map;

[0130] The second decoding feature map acquisition module is used to use the first neural network module to perform feature processing on the first feature splicing map to obtain a second decoding feature map.

[0131] Furthermore, the third decoding block includes a first filtering enhancement module, a second multi-scale attention module and a second neural network module, and the seventh training module includes:

[0132] The ninth training module is used to use the first filtering enhancement module to perform feature processing on the second decoding feature map, the second encoding feature map and the grayscale image in the grayscale image set to obtain the second decoding feature enhancement map, the second encoding feature enhancement map and the first grayscale image enhancement map.

[0133] The second feature splicing map obtaining module is used to use the second multi-scale attention module to perform feature processing on the second decoding feature enhancement map, the second encoding feature enhancement map and the first grayscale image enhancement map to obtain the second feature splicing map.

[0134] The third decoding feature map acquisition module is used to use the second neural network module to perform feature processing on the second feature splicing map to obtain a third decoding feature map.

[0135] Further, the fourth decoding block includes a second filtering enhancement module, a third multi-scale attention module and a third neural network module, and the eighth training module includes:

[0136] The tenth training module is used to use the second filtering enhancement module to perform feature processing on the third decoding feature map, the first encoding feature map and the grayscale image in the grayscale image set to obtain the third decoding feature enhancement map, the first encoding feature enhancement map and the second grayscale image enhancement map.

[0137] The third feature splicing map obtaining module is used to use the third multi-scale attention module to perform feature processing on the third decoding feature enhancement map, the first encoding feature enhancement map and the second grayscale image enhancement map to obtain a third feature splicing map.

[0138] The decoding convolutional neural network parameter acquisition module is used to use the third neural network module to perform feature processing on the third feature splicing image to obtain decoding convolutional neural network parameters.

[0139] The specific definition of the polyp image segmentation device can be found in the definition of the polyp image segmentation method above, which will not be repeated here. Each module in the above-mentioned polyp image segmentation device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0140] To solve the above technical problems, the present application also provides a computer device. Figure 4 , Figure 4 This is a basic structural block diagram of the computer device in this embodiment.

[0141] The computer device 4 includes a memory 41, a processor 42, and a network interface 43 that are interconnected through a system bus. It should be noted that the figure only shows a computer device 4 with components connected to the memory 41, the processor 42, and the network interface 43, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (Application Specific Integrated Circuit, ASIC), programmable gate arrays (Field-Programmable Gate Array, FPGA), digital processors (Digital Signal Processor, DSP), embedded devices, etc.

[0142] The computer device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may interact with a user through a keyboard, a mouse, a remote controller, a touch pad, or a voice control device.

[0143] The memory 41 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or D interface display memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 41 can be an internal storage unit of the computer device 4, such as a hard disk or memory of the computer device 4. In other embodiments, the memory 41 can also be an external storage device of the computer device 4, such as a plug-in hard disk equipped on the computer device 4, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. Of course, the memory 41 can also include both the internal storage unit of the computer device 4 and its external storage device. In this embodiment, the memory 41 is generally used to store the operating system and various application software installed on the computer device 4, such as the program code for controlling electronic files, etc. In addition, the memory 41 can also be used to temporarily store various types of data that have been output or are to be output.

[0144] The processor 42 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips in some embodiments. The processor 42 is generally used to control the overall operation of the computer device 4. In this embodiment, the processor 42 is used to run the program code stored in the memory 41 or process data, such as running the program code for controlling the electronic file.

[0145] The network interface 43 may include a wireless network interface or a wired network interface. The network interface 43 is generally used to establish a communication connection between the computer device 4 and other electronic devices.

[0146] The present application also provides another embodiment, namely, providing a computer-readable storage medium, wherein the computer-readable storage medium stores an interface display program, and the interface display program can be executed by at least one processor to enable the at least one processor to perform the steps of the polyp image segmentation method as described above.

[0147] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0148] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to perform equivalent replacement of some of the technical features therein. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.

Claims

1. A polyp image segmentation method, characterized in that: The polyp image segmentation method comprises: Obtain a polyp image training set and a polyp image verification set; Performing grayscale processing on the images in the polyp image training set to obtain a grayscale image set; Using the images in the polyp image training set to train a preset encoding convolutional neural network to obtain a first encoding feature map, a second encoding feature map, a third encoding feature map, a fourth encoding feature map and encoding convolutional neural network parameters; Using the first coding feature map, the second coding feature map, the third coding feature map, the fourth coding feature map, and the grayscale image in the grayscale image set to train a preset decoding convolutional neural network to obtain decoding convolutional neural network parameters; According to the encoding convolutional neural network parameters and the decoding convolutional neural network parameters, a polyp segmentation model to be verified is obtained; Using the polyp image verification set to verify the polyp segmentation model to be verified, obtaining verification result information, and determining whether the polyp segmentation model to be verified is a polyp segmentation model according to the verification result information; If yes, when obtaining the image of the polyp to be segmented, the image of the polyp to be segmented is input into the polyp segmentation model for segmentation to obtain a polyp image segmentation result image; The preset coding convolutional neural network includes a first coding block, a second coding block, a third coding block and a fourth coding block. The preset coding convolutional neural network is trained using images in the polyp image training set to obtain a first coding feature map, a second coding feature map, a third coding feature map, a fourth coding feature map and coding convolutional neural network parameters including: Using the first coding block to perform feature extraction on the images in the polyp image training set to obtain the first coding feature map; Using the second coding block to perform feature extraction on the first coding feature map to obtain the second coding feature map; Using the third coding block to perform feature extraction on the second coding feature map to obtain the third coding feature map; Using the fourth encoding block to perform feature extraction on the third encoding feature map to obtain the fourth encoding feature map and the encoding convolutional neural network parameters; The preset decoding convolutional neural network includes a first decoding block, a second decoding block, a third decoding block and a fourth decoding block. The first encoding feature map, the second encoding feature map, the third encoding feature map, the fourth encoding feature map and the grayscale image in the grayscale image set are used to train the preset decoding convolutional neural network, and the decoding convolutional neural network parameters obtained include: Using the first decoding block to perform feature processing on the fourth encoding feature map to obtain a first decoding feature map; Using the second decoding block to perform feature processing on the first decoding feature map and the third encoding feature map to obtain a second decoding feature map; Using the third decoding block to perform feature processing on the second decoding feature map, the second encoding feature map and the grayscale image in the grayscale image set to obtain a third decoding feature map; The fourth decoding block is used to perform feature processing on the third decoding feature map, the first encoding feature map and the grayscale image in the grayscale image set to obtain decoding convolutional neural network parameters.

2. The polyp image segmentation method according to claim 1, characterized in that: The polyp image training set includes at least one polyp image to be trained, and grayscale processing is performed on the images in the polyp image training set to obtain a grayscale image set, which includes: Based on a preset scaling ratio, scaling is performed on each of the polyp images to be trained to obtain an expanded sample to be trained corresponding to each of the polyp images to be trained; The expanded samples to be trained and the polyp images to be trained are used as a polyp image training set.

3. The polyp image segmentation method according to claim 1, characterized in that: The second decoding block includes a first multi-scale attention module and a first neural network module, and the second decoding block is used to perform feature processing on the first decoding feature map and the third encoding feature map to obtain a second decoding feature map, including: Using the first multi-scale attention module to perform feature processing on the first decoding feature map and the third encoding feature map to obtain a first feature splicing map; The first neural network module is used to perform feature processing on the first feature splicing graph to obtain a second decoding feature graph.

4. The polyp image segmentation method according to claim 1, characterized in that: The third decoding block includes a first filtering enhancement module, a second multi-scale attention module and a second neural network module, and the third decoding block is used to perform feature processing on the second decoding feature map, the second encoding feature map and the grayscale image in the grayscale image set to obtain the third decoding feature map, including: Using the first filtering enhancement module to perform feature processing on the second decoding feature map, the second encoding feature map, and the grayscale image in the grayscale image set to obtain a second decoding feature enhancement map, a second encoding feature enhancement map, and a first grayscale image enhancement map; Using the second multi-scale attention module to perform feature processing on the second decoded feature enhancement map, the second encoded feature enhancement map, and the first grayscale image enhancement map to obtain a second feature splicing map; The second neural network module is used to perform feature processing on the second feature splicing graph to obtain a third decoding feature graph.

5. The polyp image segmentation method according to claim 1, characterized in that: The fourth decoding block includes a second filtering enhancement module, a third multi-scale attention module and a third neural network module. The fourth decoding block is used to perform feature processing on the third decoding feature map, the first encoding feature map and the grayscale image in the grayscale image set to obtain decoding convolutional neural network parameters including: Using the second filtering enhancement module to perform feature processing on the third decoding feature map, the first encoding feature map and the grayscale image in the grayscale image set to obtain a third decoding feature enhancement map, a first encoding feature enhancement map and a second grayscale image enhancement map; Using a third multi-scale attention module to perform feature processing on the third decoded feature enhancement map, the first encoded feature enhancement map, and the second grayscale image enhancement map to obtain a third feature splicing map; The third neural network module is used to perform feature processing on the third feature splicing graph to obtain the decoding convolutional neural network parameters.

6. A polyp image segmentation device, characterized in that: The polyp image segmentation device comprises: An image acquisition module, used for acquiring a polyp image training set and a polyp image verification set; A grayscale processing module, used for performing grayscale processing on the images in the polyp image training set to obtain a grayscale image set; A coding convolutional neural network parameter acquisition module, used to train a preset coding convolutional neural network using images in the polyp image training set to obtain a first coding feature map, a second coding feature map, a third coding feature map, a fourth coding feature map and coding convolutional neural network parameters; A decoding convolutional neural network parameter acquisition module, used to train a preset decoding convolutional neural network using the first encoding feature map, the second encoding feature map, the third encoding feature map, the fourth encoding feature map, and the grayscale image in the grayscale image set to obtain decoding convolutional neural network parameters; A polyp segmentation model acquisition module to be verified, used to obtain the polyp segmentation model to be verified according to the encoding convolutional neural network parameters and the decoding convolutional neural network parameters; A verification module, used to verify the polyp segmentation model to be verified using the images in the polyp image verification set, obtain verification result information, and determine whether the polyp segmentation model to be verified is a polyp segmentation model according to the verification result information; A segmentation module, for inputting the to-be-segmented polyp image into the polyp segmentation model for segmentation when acquiring the to-be-segmented polyp image, and obtaining a polyp image segmentation result image; The preset encoding convolutional neural network includes a first encoding block, a second encoding block, a third encoding block and a fourth encoding block, and the encoding convolutional neural network parameter acquisition module includes: A first training module, configured to extract features from images in a polyp image training set using a first coding block to obtain a first coding feature map; A second training module is used to extract features from the first coding feature map using a second coding block to obtain a second coding feature map; A third training module is used to extract features from the second coding feature map using a third coding block to obtain a third coding feature map; A fourth training model, used for performing feature extraction on the third encoding feature map by using a fourth encoding block to obtain a fourth encoding feature map and encoding convolutional neural network parameters; The preset decoding convolutional neural network includes a first decoding block, a second decoding block, a third decoding block and a fourth decoding block, and the decoding convolutional neural network parameter acquisition module includes: a fifth training module, configured to perform feature processing on the fourth encoding feature map by using the first decoding block to obtain a first decoding feature map; A sixth training module, configured to perform feature processing on the first decoding feature map and the third encoding feature map using the second decoding block to obtain a second decoding feature map; a seventh training module, configured to perform feature processing on the second decoding feature map, the second encoding feature map, and the grayscale image in the grayscale image set using the third decoding block to obtain a third decoding feature map; The eighth training module is used to use the fourth decoding block to perform feature processing on the third decoding feature map, the first encoding feature map and the grayscale image in the grayscale image set to obtain decoding convolutional neural network parameters.

7. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the polyp image segmentation method according to any one of claims 1 to 5 is implemented.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the polyp image segmentation method according to any one of claims 1 to 5 is implemented.

Images