Endoscopic mucosal lesion recognition and classification method, system, device and medium

Abstract

The present invention discloses a method, system, device and medium for identifying and classifying lesions of endoscopic mucosal images. The method includes: reading a video signal output by an endoscopic device to obtain a mucosal image; calculating multiple quality feature values ​​of the mucosal image and determining the image quality of the mucosal image accordingly; predicting normal mucosal areas, unclear mucosal areas, abnormal mucosal areas and abnormal mucosal area classification results in mucosal images that meet preset image quality standards; and fusing the predicted results onto an image to generate an identification classification map. The present invention improves the accuracy of abnormal mucosal areas and classification by screening out unclear images and then identifying clear areas in clear images, thereby improving the accuracy and reference of the lesion identification and classification method of endoscopic mucosal images for assistance; through classification prediction, the analysis of abnormal mucosal areas is realized, and classification based on clear image areas makes the classification accurate.

Description

Technical Field

[0001] The present invention relates to the field of image recognition technology, and in particular to a method, system, device and medium for identifying and classifying lesions of endoscopic mucosal images. Background Art

[0002] Digestive endoscopy is an important tool for lesion identification and classification and treatment of digestive system diseases. In recent years, with the advancement of endoscopic equipment and artificial intelligence technology, the accuracy and efficiency of endoscopic examinations have been significantly improved. At present, digestive endoscopy technology includes traditional endoscopy, capsule endoscopy, artificial intelligence-assisted endoscopy, etc. Among them, the application of artificial intelligence (AI) technology is particularly eye-catching, which can significantly improve the detection and classification accuracy of lesion areas.

[0003] Existing endoscopic examination technology mainly relies on the doctor's experience and skills, and there is a certain degree of subjectivity and errors in lesion identification and classification. Although AI technology has improved the accuracy of lesion identification and classification in some aspects, it cannot guarantee that every frame and every area in every frame of the endoscopic video is clear, so the identification and classification of AI technology often have inaccurate problems, which makes the overall accuracy low.

[0004] In addition, there are many classification standards for gastrointestinal mucosal lesions, such as V / S, JNET, JES, NICE classification, etc. Few experts can fully master all classification standards, so it is necessary to design a method for lesion classification.

[0005] Therefore, there is an urgent need to develop a lesion recognition and classification method and system for endoscopic mucosal images that can accurately identify and classify lesions. Summary of the invention

[0006] In view of the above problems, the present invention provides a method, system, device and medium for identifying and classifying lesions in endoscopic mucosal images.

[0007] The technical solution adopted by the present invention to solve the technical problem is as follows:

[0008] In a first aspect, the present invention provides a method for identifying and classifying lesions in endoscopic mucosal images, comprising:

[0009] S1, reading the video signal output by the endoscope device to obtain a mucosal image;

[0010] S2, calculating a plurality of quality feature values ​​of the mucosal image and determining the image quality of the mucosal image accordingly;

[0011] S3, predicting the normal mucosal area, unclear mucosal area, abnormal mucosal area and abnormal mucosal area classification results in the mucosal images that meet the preset image quality standards;

[0012] S4. Fusion the predicted results onto an image to generate a recognition classification map.

[0013] In a preferred embodiment, the calculation of multiple quality feature values ​​of the mucosal image is specifically as follows: the first image quality detection module calculates the first quality feature value and the second quality feature value of the mucosal image; the second image quality detection module calculates the third quality feature value of the mucosal image, and the third image quality detection module calculates the fourth quality feature value and the fifth quality feature value of the mucosal image;

[0014] The first quality eigenvalue is the gradient mean, the second quality eigenvalue is the gradient variance, the third quality eigenvalue is the high frequency energy, the fourth quality eigenvalue is the mean of the local variance, and the fifth quality eigenvalue is the variance of the local variance.

[0015] In a preferred embodiment, the step of calculating multiple quality feature values ​​of the mucosal image includes:

[0016] The first image quality detection module uses the Sobel gradient operator to calculate the gradient amplitude of the mucosal image in the horizontal and vertical directions, synthesizes the gradient amplitude map, calculates the mean and variance of the gradient amplitude to obtain the gradient mean and gradient variance;

[0017] The second image quality detection module performs Fourier transform on the mucosal image to obtain an image after frequency domain transformation, and calculates the energy of the high-frequency component of the image after frequency domain transformation to obtain high-frequency energy;

[0018] The third image quality detection module divides the mucosal image into multiple sub-image blocks, calculates the pixel value variance of each sub-image block, calculates the pixel value variance of each sub-image block as the local variance, and counts the mean of the variances of all sub-image blocks and the variance of the sub-image block variances to obtain the mean of the local variance and the variance of the local variance.

[0019] In a preferred embodiment, S3 specifically includes: predicting the normal mucosal area, unclear mucosal area, abnormal mucosal area and abnormal mucosal area classification results in the mucosal image that meets the preset image quality standards through a multi-task cascade network.

[0020] In a preferred embodiment, the multi-task cascade network includes a first network module for generating a backbone feature map, a second network module for generating a coarse-grained candidate target area, and a third network module for refining the edge features of the candidate target area to obtain a classification result and a segmentation mask; the third network module includes a first cascade position prediction network module, a first cascade mask prediction network module, a second cascade position prediction network module, a second cascade mask prediction network module, a third cascade position prediction network module and a third cascade mask prediction network module arranged in sequence; the segmentation mask is used as a prediction result for predicting normal mucosal areas, unclear mucosal areas, and abnormal mucosal areas in mucosal images that meet preset image quality standards, and the classification result is used as a prediction result for predicting the typing results of abnormal mucosal areas.

[0021] In a preferred embodiment, S3 specifically includes:

[0022] The first network module generates a plurality of backbone feature graphs;

[0023] The second network module performs convolution calculation on the backbone feature map finally obtained by the first network module to obtain an intermediate feature map;

[0024] The second network module convolves the intermediate feature map to generate a foreground or background classification probability of the candidate area, and convolves the intermediate feature map to generate a bounding box regression parameter of the candidate area;

[0025] The second network module fuses the foreground or background classification probability of the candidate region and the bounding box regression parameter of the candidate region to obtain a coarse-grained candidate target region;

[0026] The first cascade position prediction network module generates preliminary classification results and preliminary bounding boxes based on the coarse-grained candidate target regions and all backbone feature maps;

[0027] The first cascade mask prediction network module generates a preliminary segmentation mask by performing convolution and deconvolution calculations based on the preliminary classification results, preliminary bounding boxes, and all backbone feature maps;

[0028] The second cascade position prediction network module generates a refined classification result and a refined bounding box based on the preliminary classification result, the preliminary bounding box, the preliminary segmentation mask, and all backbone feature maps;

[0029] The second cascade mask prediction network module generates a refined segmentation mask through convolution and deconvolution calculations based on the refined classification results, refined bounding boxes, and all backbone feature maps;

[0030] The third cascade position prediction network module generates the final classification result and the final bounding box through convolution and full convolution calculations based on the refined classification results, refined bounding boxes, refined segmentation masks, and all backbone feature maps;

[0031] The third cascade mask prediction network module generates the final segmentation mask through deconvolution calculation based on the final bounding box.

[0032] In a preferred embodiment, the S4 specifically includes:

[0033] Perform edge contour smoothing on the final segmentation mask of each channel;

[0034] Perform edge detection on the final segmentation mask after edge contour smoothing;

[0035] Using the edge detection results, extract the contour of each channel;

[0036] The contours of all channels and the final classification results are fused into one image to generate a recognition classification map.

[0037] In a second aspect, the present invention provides a lesion recognition and classification system for endoscopic mucosal images, comprising:

[0038] An acquisition module, used for reading the video signal output by the endoscope device to obtain a mucosal image;

[0039] A calculation and judgment module, used for calculating a plurality of quality feature values ​​of the mucosal image and judging the image quality of the mucosal image accordingly;

[0040] A prediction module, used to predict a normal mucosal area, an unclear mucosal area, an abnormal mucosal area, and a classification result of an abnormal mucosal area in a mucosal image that meets a preset image quality standard;

[0041] The fusion module is used to fuse the prediction results of the prediction module into an image to generate a recognition classification map.

[0042] In a third aspect, the present invention provides an electronic device, characterized in that it comprises: a memory; one or more processors; one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include instructions for executing any one of the methods for lesion identification and classification of endoscopic mucosal images described in the first aspect.

[0043] In a fourth aspect, the present invention provides that the computer-readable storage medium includes instructions, and when the instructions are executed on a computer, the computer executes the lesion identification and classification method of endoscopic mucosal images as described in the first aspect.

[0044] The lesion identification and classification method, system, device and medium of endoscopic mucosal images of the present invention analyzes multiple quality feature values ​​to eliminate mucosal images that do not meet the preset image quality standards, further determines the unclear mucosal area based on the clear image, performs typing prediction for the abnormal mucosal area, and integrates the results into an image to generate an identification classification map. The present invention improves the accuracy of abnormal mucosal areas and typing by screening out unclear images and identifying clear areas in clear images, thereby improving the accuracy and reference of the lesion identification and classification method of endoscopic mucosal images. The present invention realizes the analysis of abnormal mucosal areas through typing prediction, and typing is accurate based on clear image areas. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0046] Figure 1 This is a flow chart of the method for identifying and classifying lesions in endoscopic mucosal images provided in Example 1.

[0047] FIG. 2 ( a ) is a specific flow chart of S2 in the method for identifying and classifying lesions in endoscopic mucosal images provided in Example 2.

[0048] FIG2( b ) is a specific flow chart of S3 in the method for identifying and classifying lesions in endoscopic mucosal images provided in Example 2.

[0049] FIG2( c ) is a specific flow chart of S4 in the method for identifying and classifying lesions in endoscopic mucosal images provided in Example 2.

[0050] Figure 3 This is a flow chart of the method for identifying and classifying lesions in endoscopic mucosal images provided in Example 3.

[0051] Figure 4 This is a flow chart of a method for identifying and classifying lesions in endoscopic mucosal images provided in Example 4.

[0052] Figure 5 This is a framework diagram of the lesion recognition and classification system for endoscopic mucosal images provided in Example 5.

[0053] Figure 6 This is a framework diagram of the electronic device provided in Example 6. DETAILED DESCRIPTION

[0054] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0055] Existing endoscopic examination technology mainly relies on the experience and skills of doctors, and there are certain subjectivity and lesion identification and classification errors. Although AI technology has improved the accuracy of lesion identification and classification in some aspects, it occasionally causes problems such as unclear images, which makes AI identification and classification inaccurate, so that AI results may affect doctors' judgment. In addition, there are many classification standards for gastrointestinal mucosal lesions, such as V / S, JNET, JES, NICE classification, etc. Few experts can fully master all classification standards, so it is necessary to design a method that can classify lesions to assist doctors in classification. To this end, the present invention provides a lesion identification and classification method, system, device and medium for endoscopic mucosal images.

[0056] The following embodiments of the present invention can be performed separately, and the embodiments can also be performed in combination with each other, and the embodiments of the present invention do not specifically limit this. In the embodiments of the present invention, "first", "second", etc. are used to describe various components, but these components should not be limited by these terms. These terms are only used to distinguish one component from another component. The "and / or" mentioned in the present invention refers to any and all combinations including one or more related listed items.

[0057] Below, the lesion identification and classification method, system, device and medium of endoscopic mucosal images and their technical effects are described.

[0058] Embodiment 1

[0059] This embodiment provides a method for identifying and classifying lesions in endoscopic mucosal images. Figure 1 A schematic diagram of a method for identifying and classifying lesions in endoscopic mucosal images provided in this embodiment, the method comprising:

[0060] S1, reading the video signal output by the endoscope device to obtain a mucosal image;

[0061] S2, calculating a plurality of quality feature values ​​of the mucosal image and determining the image quality of the mucosal image based on the values;

[0062] S3, predicting the normal mucosal area, unclear mucosal area, abnormal mucosal area and the predicted abnormal mucosal area classification results in the mucosal images that meet the preset image quality standards;

[0063] S4. Fusion the predicted results onto an image to generate a recognition classification map.

[0064] It can be understood that the mucosal image meeting the preset image quality standard in S3 has at least one of a normal mucosal area, an unclear mucosal area, and an abnormal mucosal area;

[0065] The prediction result of predicting the normal mucosa region, the unclear mucosa region and the abnormal mucosa region in the mucosa image meeting the preset image quality standard may be that there is no at least one of the unclear mucosa region, the normal mucosa region and the abnormal mucosa region;

[0066] The unclear mucosal area may be an area with unclear image quality, or an area that does not belong to a normal mucosal area and cannot be determined as an abnormal mucosal area.

[0067] In this embodiment, the abnormal mucosal area and the abnormal mucosal area typing results are predicted simultaneously. In other embodiments, the normal mucosal area, unclear mucosal area and abnormal mucosal area may be predicted first, and if an abnormal mucosal area exists, the abnormal mucosal area typing results may be predicted.

[0068] Usually, unclear mucosal areas and abnormal mucosal areas are marked in the recognition classification map in S4, and the unmarked areas are all normal mucosal areas, which are also specially marked compared with the normal mucosal areas. This method of not marking the normal mucosal areas can ensure the clarity of the image and reduce the fusion marking operation.

[0069] Here, it is not limited to merging the prediction results of several mucosal images into one image in S4. For example, the prediction results of one mucosal image may be fused into one image, or the prediction results of several mucosal images of adjacent frames in several videos may be fused into one image. In this embodiment, the prediction result of each mucosal image corresponds to one recognition classification map.

[0070] The lesion identification and classification method of endoscopic mucosal images of this embodiment integrates image quality assessment, analyzes multiple quality feature values ​​to eliminate mucosal images that do not meet the preset image quality standards, further determines the unclear mucosal area based on the clear image, performs typing prediction for abnormal mucosal areas, and integrates the results into one image to generate an identification classification map. This embodiment improves the accuracy of abnormal mucosal areas and typing by screening out unclear images, identifying clear areas in clear images, and identifying clear areas as normal or abnormal, thereby improving the accuracy and reference of the lesion identification and classification method of endoscopic mucosal images. This embodiment realizes the analysis of abnormal mucosal areas through typing prediction, and makes typing accurate based on clear image areas.

[0071] Embodiment 2

[0072] The existing AI technology used in digestive endoscopy has the following main disadvantages:

[0073] Lesion recognition and classification accuracy: Although AI technology has improved the detection rate of certain lesions, it is still insufficient in the classification and accurate segmentation of complex mucosal lesions. The boundaries of some lesion areas are fuzzy and difficult to identify, and there are many types of lesions, which increases the difficulty of lesion recognition and classification, resulting in a high error rate in lesion recognition and classification.

[0074] Image quality: In actual operation, the clarity of the image will be affected by factors such as endoscopic operation and patient movement, resulting in blurred images in some areas, which is not conducive to accurate lesion identification and classification.

[0075] In terms of time efficiency: Current endoscopic examinations rely on the doctor's experience and skills. The speed of lesion identification and classification is slow and easily affected by subjective factors. Although the application of AI can increase the speed, it still needs further optimization to achieve real-time and efficient lesion identification and classification.

[0076] In terms of typing and classification: there are many classification standards for gastrointestinal mucosal lesions, such as V / S, JNET, JES, NICE classification, etc. Few experts can fully master all classification standards, so it is necessary to design a method for lesion typing.

[0077] Based on the above problems, compared with the first embodiment, this embodiment provides a detailed method for identifying and classifying lesions in endoscopic mucosal images, including:

[0078] S1. Use a video capture card to transmit digital / analog video signals such as HDMI, DVI, SDI, and S-Video to a computer, read the video signals output by the endoscope device through OpenCV, transcode them frame by frame into RGB image format, and obtain video frame images, that is, obtain several mucosal images.

[0079] S2. Calculate the quality characteristic value of the mucosal image using the image quality detection module, classify the mucosal image based on the quality characteristic value using a two-class classification model, and obtain a determination result of the image quality of the mucosal image.

[0080] It can be understood that the results of determining the image quality of the mucosal image are divided into two categories: one is a clear image, that is, a mucosal image that meets the preset image quality standard; the other is an unclear image, that is, a mucosal image that does not meet the preset image quality standard.

[0081] The image quality detection module includes a first image quality detection module, a second image quality detection module and a third image quality detection module. The first image quality detection module is used to calculate the first quality characteristic value (gradient mean) and the second quality characteristic value (gradient variance) of the mucosal image, the second image quality detection module is used to calculate the third quality characteristic value (high-frequency energy) of the mucosal image, and the third image quality detection module is used to calculate the fourth quality characteristic value (mean of local variance) and the fifth quality characteristic value (variance of local variance) of the mucosal image.

[0082] Specifically, the first quality eigenvalue and the second quality eigenvalue are gradient features, specifically, the first quality eigenvalue is the gradient mean, and the second quality eigenvalue is the gradient variance; the third quality eigenvalue is the frequency domain feature, specifically, the high-frequency energy; the fourth quality eigenvalue and the fifth quality eigenvalue are local variance features, specifically, the fourth quality eigenvalue is the mean of the local variance, and the fifth quality eigenvalue is the variance of the local variance.

[0083] Referring to FIG2(a), the first image quality detection module is used to calculate the gradient of the mucosal image. Specifically, the Sobel gradient operator is used to calculate the gradient amplitude in the horizontal direction and the vertical direction respectively. Then, the gradient amplitude map is synthesized, and the mean and variance of the gradient amplitude are calculated. The mean and variance of the gradient amplitude are used to quantify the edge clarity, and the first quality feature value (gradient mean) and the second quality feature value (gradient variance) are obtained. The horizontal gradient amplitude is used to calculate the gradient amplitude in the vertical direction. Indicates the gradient magnitude in the vertical direction. express,

[0084] (1)

[0085] in, Represents mucosal images, Represents the horizontal coordinate of the mucosal image, represents the horizontal coordinate of the mucosal image, that is, the mucosal image can be expressed as .

[0086] The high-frequency energy of the mucosal image is calculated using the second image quality detection module, specifically:

[0087] Perform Fourier transform on the mucosal image to obtain the image after frequency domain transformation. The formula is as follows:

[0088] (2)

[0089] in, represents the image after frequency domain transformation, represents the horizontal coordinate of the image after frequency domain transformation, Represents the ordinate of the image after frequency domain transformation, represents the width of the mucosal image, represents the height of the mucosal image, represents the grayscale mucosal image in the spatial domain, Represents an imaginary number;

[0090] Calculate the energy of the high-frequency component of the image after frequency domain transformation, that is, high-frequency energy, which is used to quantify the detail information of the image. High-frequency energy is the third quality eigenvalue. The calculation formula is as follows:

[0091] (3)

[0092] in, Represents the high frequency domain.

[0093] Calculating the fourth quality characteristic value and the fifth quality characteristic value of the mucosal image by using the third image quality detection module includes:

[0094] The mucosal image is divided into multiple sub-image blocks, for example, 64 sub-image blocks according to 8×8, and the variance of the pixel values ​​of each sub-image block (also called local variance or sub-image block variance) is calculated. The formula is as follows:

[0095] (4)

[0096] in, represents the pixel value variance of the sub-image block, represents the number of pixels in the sub-image block, Represents the sub-image block pixel values, Represents the average pixel value of the sub-image block;

[0097] The mean of all sub-image block variances and the variance of the sub-image block variances are counted to quantify the clarity of the local area (sub-image block), and the fourth quality eigenvalue (the mean of the local variance) and the fifth quality eigenvalue (the variance of the local variance) are obtained.

[0098] Using the first fusion determination module, it is determined whether the image quality meets the preset image quality standard according to the first quality characteristic value, the second quality characteristic value, the third quality characteristic value, the fourth quality characteristic value and the fifth quality characteristic value, and an image quality result (clear / unclear) is obtained. Specifically,

[0099] The extracted gradient features, frequency domain features and local variance features are standardized through the first fusion judgment module, and the first quality feature value, the second quality feature value, the third quality feature value, the fourth quality feature value and the fifth quality feature value after the standardized processing are input into the SVM model, and the SVM model outputs an image quality classification result, and the image quality classification result is in compliance with the preset image quality standard (the image is clear) or not in compliance with the preset image quality standard (the image is unclear).

[0100] S3. The mucosal image that meets the preset image quality standard is used as the input of the pre-trained multi-task cascade network. The output of the multi-task cascade network is the abnormal mucosal area and its classification results, the normal mucosal area and / or the unclear mucosal area, that is, the prediction result is obtained through the multi-task cascade network prediction.

[0101] The S3 includes constructing a multi-task cascade network and training the multi-task cascade network to predict the mucosal image in S2 that meets the preset image quality standard to obtain a prediction result, and the prediction result corresponds to the optimized final segmentation mask and the final classification result below. The final classification result is used as the classification result of the multi-task cascade network, and the final segmentation mask is used as the segmentation result of the multi-task cascade network.

[0102] Specifically, the multi-task cascade network includes: a first network module for feature extraction and generating a backbone feature map; a second network module for generating a coarse-grained candidate target area; a third network module for refining edge features of the candidate target area to obtain a classification result (class label of the target area) and a segmentation mask, and the third network module includes three position prediction network modules and three mask prediction network modules, and the position prediction network modules and the mask prediction network modules are interspersed and connected in series. The segmentation mask finally output by the third network module is used as a prediction result of the normal mucosal area, unclear mucosal area, and abnormal mucosal area in the mucosal image that meets the preset image quality standard, and the classification result finally obtained by the third network module is used as a prediction result of the abnormal mucosal area typing result. The first network module, the second network module, and the third network module are connected in sequence, and the output end of the first network module is connected to the third network module. See Figure 2(b) for details.

[0103] The first network module is used for feature extraction. As a specific embodiment, it is composed of 4 groups of residual network modules with attention mechanism, and the mucosal image that meets the preset image quality standard is input to the first network module. The 4 groups of residual network modules with attention mechanism respectively obtain the first backbone feature map, the second backbone feature map, the third backbone feature map, and the fourth backbone feature map. It can be understood that the second backbone feature map is obtained based on the first backbone feature map, the third backbone feature map is obtained based on the second backbone feature map, and the fourth backbone feature map is obtained based on the third backbone feature map. Here, the fourth backbone feature map is used as the backbone feature map finally obtained by the first network module.

[0104] The second network module is used to receive the fourth backbone feature map output by the first network module. The second network module uses 3 3 convolutional layers to generate intermediate feature maps. The second network module also has two sets of branch networks, namely the first 1 1 convolutional layer and the second 1 1 convolutional layer, input the intermediate feature map into the first 1 1 convolutional layer, generates the foreground / background classification probability of the candidate area, and outputs the first two-dimensional tensor (shape is H,W,2H, W, 2H,W,2, H and W are the height and width of the feature map); the intermediate feature map is input to the second 1 1 convolution layer, obtain the bounding box regression parameters of the candidate area, and output the second two-dimensional tensor (the shape is H, W, 4H, W, 4H, W, 4, H and W are the height and width of the feature map, and the last 4 represents the four coordinates of the bounding box). The output results of the two groups of branch networks are fused to obtain a list of candidate areas, that is, a coarse-grained candidate target area. The coarse-grained candidate target area corresponds to a coarse-grained normal mucosa area, a coarse-grained unclear mucosa area, and a coarse-grained abnormal mucosa area.

[0105] The third network module includes three cascaded position prediction network modules and three cascaded mask prediction network modules, which are connected in series with each other, and the three cascaded position prediction network modules and the three cascaded mask prediction network modules are interspersed. Each cascaded position prediction network module is connected to a cascaded mask prediction network module, that is, the third network module includes a first cascaded position prediction network module, a first cascaded mask prediction network module, a second cascaded position prediction network module, a second cascaded mask prediction network module, a third cascaded position prediction network module, and a third cascaded mask prediction network module. The third network module refines the edge features of the candidate target area to obtain a classification result and a segmentation mask. The third network module includes a number of convolutional layers, a full convolutional layer, and a deconvolutional layer arranged in sequence, the full convolutional layer outputs the classification result, and the deconvolutional layer outputs the segmentation mask.

[0106] The segmentation operation in the third network module corresponds to segmenting normal mucosal areas, unclear mucosal areas and abnormal mucosal areas, and the classification operation in the third network module corresponds to classifying abnormal mucosal areas.

[0107] Through the first cascade position prediction network module and the first cascade mask prediction network module: the input of the first cascade position prediction network module includes a candidate region list and multiple (for example, 4) backbone feature maps generated by the first network module. In the first cascade position prediction network module, firstly, the corresponding feature regions are extracted from the backbone feature maps according to the candidate region list output by the second network module; these feature regions are then processed through multiple convolutional layers and pooling layers to generate preliminary classification results and preliminary bounding boxes, and the output is a preliminary classification result (category label) and a preliminary bounding box. The input of the first cascade mask prediction network module includes the output of the first cascade position prediction network module (preliminary classification results and preliminary bounding boxes) and 4 backbone feature maps. In the first cascade mask prediction network module, the preliminary classification results and the 4 backbone feature maps are processed through multiple convolutional layers and deconvolutional layers to generate a preliminary segmentation mask, and the output is a preliminary segmentation mask.

[0108] Through the second cascade position prediction network module and the second cascade mask prediction network module: the input of the second cascade position prediction network module includes the preliminary classification result, preliminary bounding box and preliminary segmentation mask of the first cascade mask prediction network module, and the 4 backbone feature maps of the first network module. In the second cascade position prediction network module, similar to the structure of the first cascade position prediction network module, the preliminary result is further processed and refined to generate a refined classification result and a refined bounding box, and the output is the refined classification result and the refined bounding box. The input of the second cascade mask prediction network module includes the output of the second cascade position prediction network module (refined classification result and refined bounding box) and 4 backbone feature maps. In the second cascade mask prediction network module, the refined classification result and the 4 backbone feature maps are processed through multiple convolutional layers and deconvolutional layers to generate a more refined segmentation mask, which is output as a refined segmentation mask.

[0109] Through the third cascade position prediction network module and the third cascade mask prediction network module: the input of the third cascade position prediction network module includes the refined classification result of the second cascade position prediction network module, the refined bounding box and the refined segmentation mask of the second cascade mask prediction network module, and 4 backbone feature maps. In the third cascade position prediction network module, the results are further processed and refined, and the final classification result and the final bounding box are generated by multi-convolution layer and full convolution layer processing, and the output is the final classification result and the final bounding box. The input of the third cascade mask prediction network module includes the output of the third cascade position prediction network module (the final bounding box). In the third cascade mask prediction network module, the final bounding box is processed by the deconvolution layer to generate the final segmentation mask, and the output is the final segmentation mask. The final segmentation mask is used as the prediction result of the normal mucosal area, the unclear mucosal area, and the abnormal mucosal area in the mucosal image that meets the preset image quality standard, and the final classification result is used as the prediction result of the abnormal mucosal area classification result.

[0110] The multi-task cascade network is used as a mucosal partition prediction network model. For example, for the mucosal images of gastroscopy, the mucosal partition prediction network model outputs the V / S typing result; for the mucosal images of colonoscopy, the mucosal partition prediction network model outputs the JNET typing result; the specific method depends on the preset training data and is not limited by the present invention.

[0111] For the multi-task cascade network training, specifically, a large number of endoscopic images are collected and annotated to ensure that each image contains detailed segmentation and classification information of the mucosal area. Here, the target images can be selected according to actual needs and the specified classification categories can be annotated. For example, the enlarged mucosal images of the stomach and esophagus are annotated, and the annotated categories are labeled according to the VS classification and JES classification.

[0112] S4. Smooth the edge contour of the final segmentation mask, then perform edge detection, use the edge detection result to extract the contour, fuse the extracted contour and the regional target category label into an image, and generate a recognition classification map.

[0113] Referring to FIG. 2( c ), S4 is specifically as follows:

[0114] S4.1, performing edge contour smoothing processing on the final segmentation mask of each channel, that is, performing corrosion + expansion processing on the multi-channel mask image to smooth the edge contour;

[0115] S4.2, performing edge detection on the final segmentation mask after edge contour smoothing, specifically using the Sobel operator or the Canny edge detection algorithm;

[0116] S4.3. Using the edge detection results, extract the contour of each channel using the findContours function in OpenCV.

[0117] S4.4. Fusion the results of all channels (contours and final classification results) into one image to generate a recognition classification map.

[0118] The lesion recognition and classification method of endoscopic mucosal images in this embodiment improves the segmentation and classification accuracy of mucosal areas in endoscopic images through a multi-stage deep learning network. In particular, the recognition effect in complex lesion areas is far superior to the existing technology, thereby achieving high-precision lesion recognition and classification.

[0119] The lesion recognition and classification method of endoscopic mucosal images in this embodiment can control the quality of images used for prediction. It integrates image quality assessment by analyzing multiple quality feature values ​​to eliminate mucosal images that do not meet preset image quality standards, further determines unclear mucosal areas based on clear images, double-screens unclear areas, determines abnormal mucosal areas and abnormal mucosal area classification results for clear areas, thereby improving prediction accuracy and improving the reliability of intelligent lesion recognition and classification.

[0120] This embodiment realizes the analysis of abnormal mucosal areas through classification prediction, and classification is performed based on clear image areas to make the classification accurate.

[0121] This embodiment generates an identification and classification map, which includes the identification and labeling of normal mucosal areas, unclear mucosal areas, abnormal mucosal areas, and abnormal mucosal area classification results. The highly accurate identification and classification map facilitates doctors to diagnose the disease and improves the doctor's work efficiency.

[0122] Embodiment 3

[0123] See also Figure 3 Compared with the second embodiment, S3 of this embodiment is different from the second embodiment, where S3 specifically includes:

[0124] Use segmentation algorithms such as U-Net or Mask R-CNN to segment the targets in the mucosal images that meet the preset image quality standards to obtain bounding boxes;

[0125] Based on the bounding box, classification networks such as Resnet or DenseNet are used to predict the normal mucosal area, unclear mucosal area, abnormal mucosal area and abnormal mucosal area classification results in the mucosal images that meet the preset image quality standards.

[0126] The lesion identification and classification method of endoscopic mucosal images in this embodiment can classify lesions, and has high accuracy in identifying normal mucosal areas, unclear mucosal areas, abnormal mucosal areas, and classifications. Based on this, the doctor's lesion identification and classification efficiency and accuracy can be improved.

[0127] Embodiment 4

[0128] Compared with the second embodiment, this embodiment provides detailed processing for images that do not meet the preset image quality standard, see Figure 4 , including:

[0129] Mucosal images that do not meet the preset image quality standards are classified into two categories: mucosal images that can be optimized and mucosal images that are difficult to optimize. For the mucosal images that can be optimized, intelligent image clarity optimization is performed. For the optimized images, it can be judged again whether they meet the preset image quality standards. If they do, steps S3 and S4 are performed.

[0130] As an example, the intelligent optimization method may be:

[0131] Clear processing of the optimizable mucosal image, that is, enhancing the local contrast of the image area where the suspended impurity features are located, to generate a clear image;

[0132] Based on the field of view scene enhancement processing of the sharpened image, that is, enhancing the feature information of the field of view scene features, an optimized image is generated.

[0133] As another example, the intelligent optimization method may be:

[0134] For the deblurring of the mucosal image that can be optimized, the specific image is divided into multiple image blocks; each of the image blocks is analyzed and marked, the original clear image block and the image block to be optimized are marked, and the conventional image local deblurring method is used to remove the blurred area in each image block to be optimized.

[0135] In some embodiments, for mucosal images that do not meet preset image quality standards or optimized images, normal mucosal areas, unclear mucosal areas, and abnormal mucosal areas in the image are predicted separately. If there is an abnormal mucosal area, the classification result of the abnormal mucosal area is predicted.

[0136] The lesion recognition and classification method of endoscopic mucosal images in the present embodiment can take into account and fully utilize the mucosal images in the video signal, can classify the lesions, and has a high recognition accuracy for normal mucosal areas, unclear mucosal areas, abnormal mucosal areas, and classification. Based on this, the doctor's lesion recognition and classification efficiency and accuracy can be improved.

[0137] Embodiment 5

[0138] See also Figure 5 This embodiment provides a lesion recognition and classification system for endoscopic mucosal images, including:

[0139] An acquisition module 10 is used to read the video signal output by the endoscope device to obtain a mucosal image;

[0140] A calculation and judgment module 20, used for calculating a plurality of quality feature values ​​of the mucosal image and judging the image quality of the mucosal image accordingly;

[0141] A prediction module 30, for predicting a normal mucosal area, an unclear mucosal area, an abnormal mucosal area, and a classification result of an abnormal mucosal area in a mucosal image that meets a preset image quality standard;

[0142] The fusion module 40 is used to fuse the prediction results of the prediction module 30 into an image to generate a recognition classification map.

[0143] In this embodiment, the calculation and judgment module 20 is used to calculate the first quality characteristic value and the second quality characteristic value of the mucosal image using the first image quality detection module, to calculate the third quality characteristic value of the mucosal image using the second image quality detection module, to calculate the fourth quality characteristic value and the fifth quality characteristic value of the mucosal image using the third image quality detection module, and to judge the image quality of the mucosal image based on the first quality characteristic value, the second quality characteristic value, the third quality characteristic value, the fourth quality characteristic value and the fifth quality characteristic value.

[0144] The first quality eigenvalue is the gradient mean, the second quality eigenvalue is the gradient variance, the third quality eigenvalue is the high frequency energy, the fourth quality eigenvalue is the mean of the local variance, and the fifth quality eigenvalue is the variance of the local variance.

[0145] It can be understood that the calculation and judgment module 20 includes a first image quality detection module, a second image quality detection module, a third image quality detection module and a judgment unit;

[0146] The first image quality detection module is used to calculate the gradient amplitude of the mucosal image in the horizontal and vertical directions using the Sobel gradient operator, then synthesize the gradient amplitude map, calculate the mean and variance of the gradient amplitude to obtain the gradient mean and gradient variance;

[0147] The second image quality detection module is used to perform Fourier transform on the mucosal image to obtain an image after frequency domain transformation, and calculate the energy of the high-frequency component of the image after frequency domain transformation to obtain high-frequency energy;

[0148] The third image quality detection module is used to divide the mucosal image into multiple sub-image blocks, calculate the pixel value variance of each sub-image block, calculate the pixel value variance of each sub-image block as the local variance, and count the mean of the variances of all sub-image blocks and the variance of the sub-image block variances to obtain the mean of the local variance and the variance of the local variance;

[0149] A judging unit is used to judge the image quality of the mucosal image according to the gradient mean, the gradient variance, the high frequency energy, the mean of the local variance and the variance of the local variance.

[0150] The prediction module 30 includes a multi-task cascade network unit. The multi-task cascade network unit carries a multi-task cascade network, and the multi-task cascade network unit is used to predict the normal mucosal area, unclear mucosal area, abnormal mucosal area and abnormal mucosal area classification results in the mucosal image that meets the preset image quality standard using the multi-task cascade network.

[0151] The multi-task cascade network includes a first network module for generating a backbone feature map, a second network module for generating a coarse-grained candidate target area, and a third network module for refining the edge features of the candidate target area to obtain a classification result and a segmentation mask; the third network module includes a first cascade position prediction network module, a first cascade mask prediction network module, a second cascade position prediction network module, a second cascade mask prediction network module, a third cascade position prediction network module and a third cascade mask prediction network module arranged in sequence; the segmentation mask output by the third network module is used as a prediction result of a normal mucosal area, an unclear mucosal area, and an abnormal mucosal area in a mucosal image that meets a preset image quality standard, and the classification result output by the third network module is used as a prediction result of an abnormal mucosal area typing result.

[0152] The first network module is used to generate a plurality of backbone feature graphs;

[0153] The second network module is used to convolve the backbone feature map finally obtained by the first network module to obtain an intermediate feature map, to convolve the intermediate feature map to generate a foreground or background classification probability of the candidate area, to convolve the intermediate feature map to generate a bounding box regression parameter of the candidate area, and the second network module is used to fuse the foreground or background classification probability of the candidate area and the bounding box regression parameter of the candidate area to obtain a coarse-grained candidate target area;

[0154] The first cascade position prediction network module is used to generate a preliminary classification result and a preliminary bounding box according to the coarse-grained candidate target area and all backbone feature maps, and to perform convolution and deconvolution calculations according to the preliminary classification result, the preliminary bounding box and all backbone feature maps to generate a preliminary segmentation mask;

[0155] The second cascade position prediction network module is used to generate a refined classification result and a refined bounding box according to the preliminary classification result, the preliminary bounding box, the preliminary segmentation mask and all backbone feature maps;

[0156] The second cascade mask prediction network module is used to generate a refined segmentation mask through convolution and deconvolution calculations according to the refined classification results, the refined bounding box and all backbone feature maps;

[0157] The third cascade position prediction network module is used to generate the final classification result and the final bounding box through convolution and full convolution calculation according to the refined classification result, the refined bounding box, the refined segmentation mask and all backbone feature maps;

[0158] The third cascade mask prediction network module is used to generate a final segmentation mask according to the final bounding box through deconvolution calculation.

[0159] The fusion module 40 includes:

[0160] A smoothing processing unit, used for performing edge contour smoothing on the final segmentation mask of each channel;

[0161] An edge detection unit, used for performing edge detection on the final segmentation mask after edge contour smoothing;

[0162] A contour extraction unit, used to extract the contour of each channel using the edge detection result;

[0163] The fusion unit is used to fuse the results of all channels (contours and final classification results) into one image to generate a recognition classification map.

[0164] When the endoscopic mucosal image lesion recognition and classification system is implemented, the endoscopic mucosal image lesion recognition and classification method in any of the above embodiments can be referred to to implement the endoscopic mucosal image lesion recognition and classification, and the specific implementation steps will not be repeated.

[0165] The lesion recognition and classification system of endoscopic mucosal images in this embodiment can not only classify lesions, but also has high recognition accuracy for normal mucosal areas, unclear mucosal areas, abnormal mucosal areas, and classification. Based on this, it can improve the doctor's lesion recognition and classification efficiency and accuracy.

[0166] The technicians in the field to which the present invention belongs can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0167] Embodiment 6

[0168] This embodiment provides an electronic device that can be directly connected to an endoscope device.

[0169] The method of the present invention can realize an electronic device, such as Figure 6 The electronic device includes: a memory; one or more processors; one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include instructions for executing the lesion identification and classification method of endoscopic mucosal images according to any of the above embodiments.

[0170] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0171] The memory may include various types of storage units, such as system memory, read-only memory (ROM), and permanent storage. In addition, the memory may include any combination of computer-readable storage media, such as semiconductor memory chips, magnetic disks, and optical disks.

[0172] The memory stores executable codes, and when the executable codes are processed by the processor, the processor can execute part or all of the methods described above.

[0173] Embodiment 7

[0174] In this embodiment, a computer-readable storage medium is provided, and the computer-readable storage medium includes instructions. When the instructions are executed on a computer, the computer executes the lesion identification and classification method of endoscopic mucosal images described in any of the above embodiments.

[0175] It should be noted that in the above embodiments, the description of each embodiment has its own emphasis, and for parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0176] The present invention is described with reference to flowcharts and / or block diagrams of methods, systems, apparatuses, and media according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as a combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded computer, or other programmable data processing device to produce a machine, so that instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0177] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

Claims

1. A method for identifying and classifying lesions in endoscopic mucosal images, characterized in that: include: S1, reading the video signal output by the endoscope device to obtain a mucosal image; S2, calculating a plurality of quality feature values ​​of the mucosal image and determining the image quality of the mucosal image accordingly; S3, predicting the normal mucosal area, unclear mucosal area, abnormal mucosal area and abnormal mucosal area classification results in the mucosal images that meet the preset image quality standards; S4, merging the predicted results into an image to generate a recognition classification map; S3 specifically includes: predicting the normal mucosal area, unclear mucosal area, abnormal mucosal area and abnormal mucosal area classification results in the mucosal image that meets the preset image quality standard through a multi-task cascade network; The multi-task cascade network includes a first network module for generating a backbone feature map, a second network module for generating a coarse-grained candidate target area, and a third network module for refining edge features of the candidate target area to obtain a classification result and a segmentation mask; the third network module includes a first cascade position prediction network module, a first cascade mask prediction network module, a second cascade position prediction network module, a second cascade mask prediction network module, a third cascade position prediction network module, and a third cascade mask prediction network module, which are arranged in sequence; the segmentation mask is used as a prediction result of predicting a normal mucosal area, an unclear mucosal area, and an abnormal mucosal area in a mucosal image that meets a preset image quality standard, and the classification result is used as a prediction result of predicting a typing result of an abnormal mucosal area; The S3 specifically includes: The first network module generates a plurality of backbone feature graphs; The second network module performs convolution calculation on the backbone feature map finally obtained by the first network module to obtain an intermediate feature map; The second network module convolves the intermediate feature map to generate a foreground or background classification probability of the candidate area, and convolves the intermediate feature map to generate a bounding box regression parameter of the candidate area; The second network module fuses the foreground or background classification probability of the candidate region and the bounding box regression parameter of the candidate region to obtain a coarse-grained candidate target region; The first cascade position prediction network module generates preliminary classification results and preliminary bounding boxes based on the coarse-grained candidate target regions and all backbone feature maps; The first cascade mask prediction network module generates a preliminary segmentation mask by performing convolution and deconvolution calculations based on the preliminary classification results, preliminary bounding boxes, and all backbone feature maps; The second cascade position prediction network module generates a refined classification result and a refined bounding box based on the preliminary classification result, the preliminary bounding box, the preliminary segmentation mask, and all backbone feature maps; The second cascade mask prediction network module generates a refined segmentation mask through convolution and deconvolution calculations based on the refined classification results, refined bounding boxes, and all backbone feature maps; The third cascade position prediction network module generates the final classification result and the final bounding box through convolution and full convolution calculations based on the refined classification results, refined bounding boxes, refined segmentation masks, and all backbone feature maps; The third cascade mask prediction network module generates the final segmentation mask through deconvolution calculation based on the final bounding box.

2. The method for identifying and classifying lesions in endoscopic mucosal images according to claim 1, characterized in that: The calculation of the multiple quality feature values ​​of the mucosal image is specifically as follows: the first image quality detection module calculates the first quality feature value and the second quality feature value of the mucosal image; The second image quality detection module calculates a third quality characteristic value of the mucosal image, and the third image quality detection module calculates a fourth quality characteristic value and a fifth quality characteristic value of the mucosal image; The first quality eigenvalue is the gradient mean, the second quality eigenvalue is the gradient variance, the third quality eigenvalue is the high frequency energy, the fourth quality eigenvalue is the mean of the local variance, and the fifth quality eigenvalue is the variance of the local variance.

3. The method for identifying and classifying lesions in endoscopic mucosal images according to claim 2, characterized in that: The step of calculating a plurality of quality feature values ​​of the mucosal image comprises: The first image quality detection module uses the Sobel gradient operator to calculate the gradient amplitude of the mucosal image in the horizontal and vertical directions, synthesizes the gradient amplitude map, calculates the mean and variance of the gradient amplitude to obtain the gradient mean and gradient variance; The second image quality detection module performs Fourier transform on the mucosal image to obtain an image after frequency domain transformation, and calculates the energy of the high-frequency component of the image after frequency domain transformation to obtain high-frequency energy; The third image quality detection module divides the mucosal image into multiple sub-image blocks, calculates the pixel value variance of each sub-image block, calculates the pixel value variance of each sub-image block as the local variance, and counts the mean of the variances of all sub-image blocks and the variance of the sub-image block variances to obtain the mean of the local variance and the variance of the local variance.

4. The method for identifying and classifying lesions in endoscopic mucosal images according to claim 1, characterized in that: The S4 specifically includes: Perform edge contour smoothing on the final segmentation mask of each channel; Perform edge detection on the final segmentation mask after edge contour smoothing; Using the edge detection results, extract the contour of each channel; The results of all channels are fused into one image to generate a recognition classification map.

5. A lesion recognition and classification system for endoscopic mucosal images, characterized in that: include: An acquisition module, used for reading the video signal output by the endoscope device to obtain a mucosal image; A calculation and judgment module, used for calculating a plurality of quality feature values ​​of the mucosal image and judging the image quality of the mucosal image accordingly; A prediction module, used to predict normal mucosal areas, unclear mucosal areas, abnormal mucosal areas, and abnormal mucosal area classification results in mucosal images that meet preset image quality standards; the prediction module is specifically used to predict normal mucosal areas, unclear mucosal areas, abnormal mucosal areas, and abnormal mucosal area classification results in mucosal images that meet preset image quality standards through a multi-task cascade network; A fusion module is used to fuse the prediction results of the prediction module into an image to generate a recognition classification map; The multi-task cascade network includes a first network module for generating a backbone feature map, a second network module for generating a coarse-grained candidate target area, and a third network module for performing edge feature refinement on the candidate target area to obtain a classification result and a segmentation mask; the third network module includes a first cascade position prediction network module, a first cascade mask prediction network module, a second cascade position prediction network module, a second cascade mask prediction network module, a third cascade position prediction network module, and a third cascade mask prediction network module, which are arranged in sequence; the segmentation mask is used as a prediction result of predicting a normal mucosal area, an unclear mucosal area, and an abnormal mucosal area in a mucosal image that meets a preset image quality standard, and the classification result is used as a prediction result of predicting a typing result of an abnormal mucosal area; the first network module is used to generate a plurality of backbone feature maps; the second network module is used to perform convolution calculation on the backbone feature map finally obtained by the first network module to obtain an intermediate feature map; The second network module is used to convolve the intermediate feature map to generate a foreground or background classification probability of the candidate area, and convolve the intermediate feature map to generate a bounding box regression parameter of the candidate area; The second network module is used to fuse the foreground or background classification probability of the candidate area and the bounding box regression parameters of the candidate area to obtain a coarse-grained candidate target area; the first cascade position prediction network module is used to generate a preliminary classification result and a preliminary bounding box based on the coarse-grained candidate target area and all backbone feature maps; the first cascade mask prediction network module is used to perform convolution and deconvolution calculations based on the preliminary classification result, the preliminary bounding box and all backbone feature maps to generate a preliminary segmentation mask; the second cascade position prediction network module is used to generate a preliminary segmentation mask based on the preliminary classification result, the preliminary bounding box and the preliminary segmentation mask. The first cascade mask prediction network module is used to generate a refined segmentation mask according to the refined classification result, the refined bounding box and all the backbone feature maps through convolution and deconvolution calculations; the second cascade position prediction network module is used to generate a final classification result and a final bounding box according to the refined classification result, the refined bounding box, the refined segmentation mask and all the backbone feature maps through convolution and full convolution calculations; the third cascade mask prediction network module is used to generate a final segmentation mask according to the final bounding box through deconvolution calculations.

6. An electronic device, characterized in that: include: Memory; one or more processors; One or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include instructions for executing any of the methods for lesion identification and classification of endoscopic mucosal images according to claims 1-4.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes instructions, and when the instructions are executed on a computer, the computer executes the lesion identification and classification method for endoscopic mucosal images according to any one of claims 1 to 4.

Images