A gastrointestinal endoscopic image polyp detection system, terminal and storage medium based on artificial intelligence

Abstract

The present invention discloses a gastrointestinal endoscope image polyp detection system, terminal and storage medium based on artificial intelligence. It includes an image sampling module: used to collect gastrointestinal endoscope pictures with complete lesion areas, and divide the collected pictures into training sets and test sets; an image preprocessing module: preprocessing the pictures in the training set output by the image sampling module; a gastrointestinal endoscope image polyp detection neural network module: extracting high-dimensional features from the image input to the network model through the encoding network, and then completing feature optimization and denoising through the polyp lesion feature enhancement module, and finally restoring semantic information through the decoding network, and outputting the location and edge of the polyp; a polyp detection module: inputting the gastrointestinal endoscope picture to be detected into the trained model, and outputting the detection result. The present invention intelligently identifies and detects polyps and other lesions in endoscopic images by building a polyp detection model, and at the same time achieves a higher accuracy rate by continuously training the neural network model.

Description

Technical Field

[0001] The present invention belongs to the technical field of gastrointestinal endoscopic image polyp detection, and specifically relates to a gastrointestinal endoscopic image polyp detection system, terminal and storage medium based on artificial intelligence. Background Art

[0002] Deep convolutional neural networks have been successfully applied in the field of vision. Due to their powerful representation capabilities, fast reasoning, and filter sharing characteristics, deep convolutional neural network models provide state-of-the-art performance for many computer vision tasks such as image classification, image segmentation, and image detection. Recent studies have shown that deep convolutional neural networks can automatically learn hierarchical structures directly from data to obtain increasingly complex features. With the development of this technology, object detection methods based on deep learning have achieved great success in medical image analysis. By learning from input image samples, the network model continuously adjusts the parameter matrices of each layer to perform recognition and detection on real data.

[0003] Polyps are protrusions on the surface of the colon. Polyps appear in different shapes, from flat to pedunculated. Early diagnosis of colorectal cancer can be achieved if colorectal polyps, which can be a precursor to cancer, are detected and removed before they develop into malignancies. The gold standard screening method for polyp detection and removal is optical colonoscopy. Colonoscopy is a successful preventive procedure. But colonoscopy is an operator-dependent procedure, where human factors such as fatigue and lack of concentration can lead to missed polyps during long and back-to-back procedures. Computer-assisted polyp detection can assist colonoscopists in the task of polyp detection and can improve concentration during the procedure. A reliable polyp detection system must handle the variability of polyps in shape, size, color, and texture. Such a system has the potential to save lives. Summary of the invention

[0004] In order to solve the problems in the background technology, the present invention provides a gastrointestinal endoscopic image polyp detection system, terminal and storage medium based on artificial intelligence. The image features are extracted from the gastrointestinal endoscopic image through a coding network that integrates multi-scale context information, providing effective information for the identification and positioning of polyps; the high-dimensional features extracted from the coding network are refined by the polyp lesion feature enhancement module, and the features associated with the lesion area are further emphasized after feature transformation, thereby increasing the accuracy of recognition and detection; the effective features extracted at different levels are summarized through a decoding network with multi-level feature fusion, feature fusion is completed, and the area of ​​lesions such as polyps in the original image is restored from the features, thereby completing the identification and positioning of polyps.

[0005] The technical solution is as follows:

[0006] 1. Provided is a gastrointestinal endoscopic image polyp detection system based on artificial intelligence, including:

[0007] Image sampling module: used to collect gastrointestinal endoscopic images with complete polyp lesion areas, good image brightness and intensity, and few or no interference factors, and divide the collected images into training sets and test sets;

[0008] Image preprocessing module: preprocess the images in the training set output by the image sampling module;

[0009] Gastrointestinal endoscopic image polyp detection neural network module: including encoding network, polyp lesion feature enhancement module and decoding network; the encoding network is mainly composed of four sequentially connected context information aggregation modules; the decoding network is mainly composed of four sequentially connected convolutional layers, and an upsampling operation is performed after each convolutional layer. The output features after upsampling are fused with the output features of the context information aggregation module in the encoding network through residual connection and then input into the next convolutional layer;

[0010] The image input to the gastrointestinal endoscopy image polyp detection neural network module is processed through the encoding network to extract high-dimensional features, and then the polyp lesion feature enhancement module completes feature optimization and denoising. Finally, the decoding network restores the semantic information and outputs the location and edge of the polyp.

[0011] Neural network training module: The images output by the image preprocessing module are input into the gastrointestinal endoscopy image polyp detection neural network module for autonomous training, and the network model parameters with the best accuracy are saved to complete the training; the images in the test set output by the image sampling module are input into the trained neural network model to test the generalization effect of the network and verify the recognition and detection accuracy of the neural network model;

[0012] Polyp detection module: load the model file output by the neural network training module to obtain the trained gastrointestinal endoscopy image polyp detection neural network model, input the gastrointestinal endoscopy image to be detected into the gastrointestinal endoscopy image polyp detection neural network model, output the position and edge of the polyp in the gastrointestinal endoscopy image, and complete the detection.

[0013] The neural network module for polyp detection in gastrointestinal endoscopy images includes an encoding network for integrating multi-scale context information, a polyp lesion feature enhancement module, and a decoding network for multi-level feature fusion. The encoding network for integrating multi-scale context information is composed of continuous convolution layers, pooling layers, and activation functions. It extracts and accumulates feature information through a continuously expanding receptive field for the detection and positioning of polyps. The polyp lesion feature enhancement module uses an attention algorithm and is composed of two attention modules, one extracting a spatial attention operator from the dimension of space, and the other extracting a channel attention operator from the dimension of the channel; the two operators guide the network model to focus on the area of ​​the polyp lesion, thereby filtering out redundant information, making the detection and recognition more accurate. The decoding network for multi-level feature fusion is composed of multiple continuous convolution layers and upsampling layers. By learning and adjusting parameters, the position of polyps in the original image is continuously restored from the complex feature information of high latitudes, and the semantic information of the feature map is continuously supplemented through jump connections, thereby improving the accuracy of positioning detection.

[0014] The encoding network consists of four context information aggregation modules, each of which contains multiple parallel convolutional layers. The convolution kernels of each convolutional layer are of different sizes, which brings different sizes of receptive fields. This allows the context information aggregation module to capture the features of target objects of different sizes. After passing through the four context information aggregation modules, the features of the image are compressed and integrated into high latitudes, which is the basis for the model to identify and detect polyp lesions.

[0015] The polyp lesion feature enhancement module uses continuous fully connected layers, convolutional layers, Softmax layers, as well as transposition, dot multiplication and other operations to further transform the output of the encoding network. Through automatically learned parameters, the network model focuses on areas related to lesions, thereby further enhancing favorable feature information and increasing the accuracy of recognition and detection.

[0016] The decoding network of multi-level feature fusion uses cross-connections to fuse feature information from different scales. Shallow features contain more detailed information, and deep features contain more semantic information. Multi-scale features are fused to complete the identification and positioning of polyps.

[0017] In the image preprocessing module, the preprocessing process is specifically as follows: firstly, the polyp lesion areas contained in all the images are marked, then the resolution of all the images is normalized, and the format is converted, then the data is augmented, and finally the data is rotated and mirrored.

[0018] 2. Provided is a terminal including a memory and a processor

[0019] The memory is used to store computer programs;

[0020] The processor is used to implement the functions of the above-mentioned artificial intelligence-based gastrointestinal endoscopic image polyp detection system when executing the computer program.

[0021] 3. Provides a computer-readable storage medium

[0022] The storage medium stores a computer program, and when the computer program is executed by the processor, the functions of the above-mentioned artificial intelligence-based gastrointestinal endoscopic image polyp detection system are realized.

[0023] The beneficial effects of the present invention are:

[0024] 1. Compared with the prior art, the present invention detects polyps in endoscopic images by building a gastrointestinal endoscopic image polyp detection neural network, and achieves a higher accuracy rate by continuously training the constructed convolutional neural network model, so that the judgment accuracy of the network model is not lower than the judgment level of experienced doctors.

[0025] 2. The present invention adopts an attention mechanism to focus on the area of ​​polyp lesions, thereby filtering out redundant information and further improving the accuracy of recognition and detection.

[0026] 3. The total number of parameters of the neural network for detecting polyps in gastrointestinal endoscopic images of the present invention is within a reasonable range, the calculation time is fast, and targets such as polyps in the image can be quickly identified. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 It is a schematic diagram of the structure of the neural network module for detecting polyps in gastrointestinal endoscopic images of the present invention;

[0028] Figure 2 It is a schematic diagram of a context information aggregation module of the present invention;

[0029] Figure 3 It is a schematic diagram of the polyp lesion feature enhancement module of the present invention.

[0030] Figure 4 It is a schematic diagram of the process of the present invention. DETAILED DESCRIPTION

[0031] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0032] like Figure 1The figure shows the neural network module for polyp detection in gastrointestinal endoscopic images of the present invention. The neural network module for polyp detection in gastrointestinal endoscopic images consists of an encoding network integrating multi-scale context information, a polyp lesion feature enhancement module, and a decoding network for multi-level feature fusion. Among them, the encoding network consists of four context information aggregation modules; the polyp lesion feature enhancement module uses an attention algorithm and consists of two attention modules, one extracting a spatial attention operator from the dimension of space, and the other extracting a channel attention operator from the dimension of the channel. The decoding network consists of four consecutive convolutional layers, and an upsampling operation is performed after each convolutional layer. The upsampled output features are fused with the features from the encoding layer through a residual connection, and then input into the next convolutional layer, and the detection results are output after four times. Each convolutional layer performs convolution calculation on the input, and the input of the model is an endoscopic image. The input image first passes through the encoding network to extract a high-dimensional feature vector, and then passes through the polyp lesion feature enhancement module to complete feature optimization and denoising, and finally restores the semantic information through the decoding network to detect the position and edge of the polyp.

[0033] The existing network structure is limited by a single receptive field, and each layer can only see a fixed-size area. This simple structure cannot capture polyp lesions of varying sizes and rich variations. Figure 2 The context information aggregation module shown in the figure uses convolution kernels of different sizes in parallel to form receptive field branches of different sizes, capturing the characteristic information of polyps of different sizes. It is used as the basis for the network model to detect lesions. Specifically, we connect 3×3, 5×5, and 7×7 convolution kernels in parallel, add all the results, aggregate the features through a maximum pooling layer, and reduce the spatial size of the features to complete the encoding. The 1×1 convolution kernel is used to adjust the feature channel.

[0034] Due to the complex features of the lesion area, many traditional neural networks are difficult to identify. In addition, due to uncertain factors such as contrast and lighting in the image, noise features will be introduced, interfering with detection. We designed Figure 3 The polyp lesion feature enhancement module shown uses the attention mechanism to focus the network on the lesion area, strengthen the features and filter the noise area. Specifically, the high-dimensional features output by the encoding network are transformed through two fully connected layers, transposed and multiplied with themselves. The result is passed through a softmax function to obtain an attention matrix, which guides the network to focus on the lesion area. Finally, the original input high-dimensional features are multiplied by the attention matrix and further transformed through a 3×3 convolution layer to obtain enhanced optimized features and filter the influence of noise information. The calculation formula of the polyp lesion feature enhancement module is as follows:

[0035] M=I*{I*[MLP2(MLP1(I))]T}

[0036] O = Conv3 × 3 (M)

[0037] Among them, I represents the input high-dimensional features, MLP1 and MLP2 are two layers of fully connected layer transformation functions, M is the intermediate feature vector, and O represents the output features of the module.

[0038] The multi-level feature fusion decoding network uses residual connections to combine the fine image features from the encoding network with the high-dimensional semantic features in the decoder to extract more expressive features for restoring the location information of the lesion in the original image.

[0039] The method for building a gastrointestinal endoscopic image polyp detection module of the present invention comprises the following steps: step one: building a gastrointestinal endoscopic image polyp detection neural network model, the network model comprising an encoding network, a polyp lesion feature enhancement module, and a decoding network; step two: collecting 800 gastrointestinal endoscopic images, selecting images with good brightness and intensity, relatively complete lesion areas, and few or no interference factors as samples, and then preprocessing 600 of the images. During the processing, the lesion areas contained in all images are first marked, and the resolution of all images is normalized, and format conversion is performed, and then data augmentation is performed, and the data is rotated, mirrored, and other transformations are performed, and finally 1000 images are obtained as a training data set; step three: the 1000 processed gastrointestinal endoscopic images are sent to the network model for computer autonomous training. During the training process, hyperparameters such as learning rate are adjusted, and the parameters of the convolutional neural network model with the best accuracy are saved to complete the training; Step 4: The remaining 200 gastrointestinal endoscopy images are sent to the convolutional neural network model to test the generalization effect of the network and verify the recognition and detection accuracy of the model. The test results show that the detection accuracy is over 90%. The real-time performance of the algorithm with the highest accuracy reaches 35FPS, and the system can quickly identify targets such as polyps in the image. The model parameter volume is 35M, which meets the computer storage size requirements.

[0040] The detection result output by our CNN output layer is a binary image of the same size as the original image, with the pixel value of the background area being 0 and the pixel value of the lesion area being 255. This binary image indicates the location and edge information of the lesion, achieving the effect of semantic segmentation.

[0041] like Figure 4 Shown is a schematic diagram of the process of the present invention:

[0042] 1) First, we build a high-quality endoscopic image database through the collected gastrointestinal endoscopic image data, and then open the database to doctors. Doctors and experts will label and outline the lesion area. After labeling, the image and label will be preprocessed together, and data augmentation will be completed.

[0043] 2) Send the input data to the built encoding network, and after passing through the lesion feature enhancement module and the decoding network, output the detection results, calculate the accuracy, and complete the training. Then send the trained model to the test database for testing. If the accuracy is not enough, return to adjust the parameters and retrain. If the accuracy meets the requirements, then add the test data set to the training database to further improve the model performance.

[0044] 3) Then test the speed performance of the model. If the model speed is slow, compress the model and return to retrain until the speed requirement is met. The model that meets the accuracy and speed requirements is sent to the online application to complete the auxiliary diagnosis, and the new data is added to the endoscopic image database for a new round of iteration.

Claims

1. A gastrointestinal endoscopic image polyp detection system based on artificial intelligence, characterized in that: include: Image sampling module: used to collect complete gastrointestinal endoscopic images of the polyp lesion area and divide the collected images into training sets and test sets; Image preprocessing module: preprocess the images in the training set output by the image sampling module; Neural network module for polyp detection in gastrointestinal endoscopy images: including encoding network, polyp lesion feature enhancement module and decoding network; The encoding network consists of four sequentially connected context information aggregation modules; The decoding network consists of four convolutional layers connected in sequence. After each convolutional layer, an upsampling operation is performed. The upsampled output features are fused with the output features of the context information aggregation module in the encoding network through a residual connection and then input into the next convolutional layer. The image input to the gastrointestinal endoscopy image polyp detection neural network module is processed through the encoding network to extract high-dimensional features, and then the polyp lesion feature enhancement module completes feature optimization and denoising. Finally, the decoding network restores the semantic information and outputs the location and edge of the polyp. Neural network training module: The images output by the image preprocessing module are input into the gastrointestinal endoscopy image polyp detection neural network module for autonomous training, and the network model parameters with the best accuracy are saved to complete the training; Polyp detection module: load the model file output by the neural network training module to obtain the trained gastrointestinal endoscopic image polyp detection neural network model, input the gastrointestinal endoscopic image to be detected into the trained gastrointestinal endoscopic image polyp detection neural network model, output the position and edge of the polyp in the gastrointestinal endoscopic image, and complete the detection.

2. The artificial intelligence-based gastrointestinal endoscopic image polyp detection system according to claim 1, characterized in that: In the gastrointestinal endoscopic image polyp detection neural network module, each context information aggregation module consists of five convolutional layers, an activation layer and a maximum pooling layer, and the second convolutional layer, the third convolutional layer and the fourth convolutional layer are connected in parallel; the input of the context information aggregation module is respectively input into the second convolutional layer, the third convolutional layer and the fourth convolutional layer after passing through the first convolutional layer, and the outputs of the second convolutional layer, the third convolutional layer and the fourth convolutional layer are added and then input into the maximum pooling layer through the fifth convolutional layer and the ReLU activation layer in turn.

3. The artificial intelligence-based gastrointestinal endoscopic image polyp detection system according to claim 2, characterized in that: The convolution kernels of the second convolution layer, the third convolution layer and the fourth convolution layer are 3×3, 5×5 and 7×7 respectively, and the convolution kernels of the first convolution layer and the fifth convolution layer are both 1×1.

4. The artificial intelligence-based gastrointestinal endoscopic image polyp detection system according to claim 1, characterized in that: In the gastrointestinal endoscopic image polyp detection neural network module, the polyp lesion feature enhancement module includes two attention modules: a channel attention operator and a spatial attention operator; The high-dimensional features output by the encoding network are transposed after the feature transformation of the two fully connected layers in the channel attention operator, and then multiplied with the high-dimensional features of the original input. The multiplication result is passed through the softmax function of the spatial attention operator to obtain an attention matrix. After the attention matrix is ​​multiplied by the high-dimensional features of the original input, further feature transformation is completed through a 3×3 convolution layer to obtain the enhanced optimized features as the output of the polyp lesion feature enhancement module.

5. The artificial intelligence-based gastrointestinal endoscopic image polyp detection system according to claim 1, characterized in that: In the gastrointestinal endoscopic image polyp detection neural network module, the decoding network includes four sequentially connected convolution modules, each convolution module includes a convolution layer and an upsampling layer, and the upsampling operation is performed after the convolution layer; The output of the first convolution module is fused with the output of the fourth context information aggregation module through a residual connection and then input into the second convolution module; the output of the second convolution module is fused with the output of the third context information aggregation module through a residual connection and then input into the third convolution module; the output of the third convolution module is fused with the output of the second context information aggregation module through a residual connection and then input into the fourth convolution module; the output of the fourth convolution module is fused with the output of the first context information aggregation module through a residual connection and then used as the output of the decoding network.

6. The artificial intelligence-based gastrointestinal endoscopic image polyp detection system according to claim 1, characterized in that: In the image preprocessing module, the preprocessing process is specifically as follows: firstly, the polyp lesion areas contained in all the images are marked, then the resolution of all the images is normalized, and the format is converted, then the data is augmented, and finally the data is rotated and mirrored.

7. A terminal, characterized in that: including memory and processor; The memory is used to store computer programs; The processor is used to implement the functions of the artificial intelligence-based gastrointestinal endoscopic image polyp detection system as described in any one of claims 1 to 6 when executing the computer program.

8. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by the processor, the function of the artificial intelligence-based gastrointestinal endoscopic image polyp detection system as described in any one of claims 1 to 6 is realized.

Images