Gastrointestinal endoscope-based ai-assisted lesion localization method and system

Abstract

The present application relates to the technical field of auxiliary identification positioning, in particular to an AI-assisted lesion positioning method and system based on a gastrointestinal endoscope. Based on segmentation boxes of different sizes, a gastric ulcer lesion area and a corresponding confidence level in a gastrointestinal endoscope image are identified. The completeness value of the gastric ulcer lesion area is determined. According to the number of times that a pixel point at each position is divided into the gastric ulcer lesion area under segmentation boxes of different sizes, in combination with the confidence level and the completeness value of the gastric ulcer lesion area to which the pixel point is divided, the region belonging value of each pixel point is obtained. Based on the region belonging value, the pixel points are merged to obtain a preliminary screening lesion area. The region belonging value of the pixel points and the distribution of the region in the preliminary screening lesion area are analyzed to determine the performance bias degree of each preliminary screening lesion area. Based on the performance bias degree, a target gastric ulcer area is determined from the preliminary screening lesion area. The present application improves the identification accuracy of the target gastric ulcer area.

Description

Technical Field

[0001] This invention relates to the field of assisted identification and localization technology, specifically to an AI-assisted lesion localization method and system based on gastrointestinal endoscopy. Background Technology

[0002] Artificial intelligence (AI) can be used to assist in the identification of gastric ulcer lesions, improving the accuracy and efficiency of diagnosis. For example, AI algorithms can help doctors analyze endoscopic images more accurately, pinpointing small, early-stage, or easily overlooked ulcer lesions, effectively reducing the risk of missed diagnoses. At the same time, AI algorithms can also quickly process massive amounts of image data, reducing the workload of doctors, shortening diagnosis time, and helping to standardize diagnostic criteria.

[0003] Currently, the common method for identifying gastric ulcer lesions with the assistance of AI algorithms is to directly identify the gastric ulcer lesion area in gastrointestinal endoscopy images using deep neural networks. However, since gastric ulcers and chronic gastritis have many similar morphological features, the accuracy of directly identifying gastric ulcer lesions using deep neural networks is relatively low. Summary of the Invention

[0004] To address the technical problem of low accuracy in directly identifying gastric ulcer lesions using deep neural networks, the present invention aims to provide an AI-assisted lesion localization method and system based on gastroscopy and colonoscopy. The specific technical solution adopted is as follows:

[0005] In a first aspect, embodiments of the present invention provide an AI-assisted lesion localization method based on gastrointestinal endoscopy, the method comprising:

[0006] Based on segmentation boxes of different sizes, the regions of gastric ulcer lesions in gastrointestinal endoscopy images and their corresponding confidence levels are identified.

[0007] The integrity value of the gastric ulcer lesion area is determined based on the integrity of the area within the segmentation box.

[0008] Based on the number of times each pixel at each location is classified into a gastric ulcer lesion region under different sized segmentation boxes, and combined with the confidence and integrity values ​​of the gastric ulcer lesion regions to which the pixel is classified, the region affiliation value of each pixel at each location in the gastrointestinal endoscopy image is obtained; based on the region affiliation value, pixels at different locations are merged to obtain the initial screening lesion region;

[0009] Analyze the regional values ​​and distribution of pixels within the initial screening lesion area to determine the degree of performance bias for each initial screening lesion area;

[0010] Based on the degree of performance bias, the target gastric ulcer area is determined from the initial screening lesion area.

[0011] Further, determining the integrity value of the gastric ulcer lesion region based on the regional integrity of the gastric ulcer lesion region within the segmentation frame includes:

[0012] Determine the overall distribution distance between gastric ulcer lesion regions within the segmentation frame; the overall distribution distance characterizes the distance between the boundary points of the gastric ulcer lesion regions.

[0013] The integrity value of the gastric ulcer lesion region within the segmentation box is determined by combining the overall distribution distance of the gastric ulcer lesion region, the area of ​​the gastric ulcer lesion region, and the number of regions.

[0014] Further, the method of obtaining the region belonging value of each pixel in the gastrointestinal endoscopy image based on the number of times each pixel is classified into a gastric ulcer lesion region under different sized segmentation boxes, combined with the confidence and integrity values ​​of the gastric ulcer lesion region to which the pixel is classified, includes:

[0015] The reference confidence level of a pixel is determined based on the confidence level and integrity value of the gastric ulcer lesion region to which the pixel is assigned.

[0016] By combining the reference confidence of each pixel at each location with the number of times the pixel was classified into the gastric ulcer lesion area under different sized segmentation boxes, the region belonging value of each pixel at each location in the gastrointestinal endoscopy image is obtained.

[0017] Further, the step of merging pixels at different locations based on the region's value to obtain the initial screening lesion region includes:

[0018] Adjacent pixels whose region values ​​are greater than a preset merging threshold are merged to obtain the initial screening lesion region.

[0019] Furthermore, the analysis of the regional values ​​and distribution of pixels within the initial screening lesion area to determine the performance bias of each initial screening lesion area includes:

[0020] Based on the regional values ​​and distribution of pixels within the initial screening lesion area, the severity of the disease in the gastrointestinal endoscopic images is determined; the initial screening lesion area is compared with other initial lesion areas to determine the severity performance value of the initial screening lesion area; using the severity of the disease as a weight, the severity performance value of each initial screening lesion area is weighted to obtain the performance bias of each initial screening lesion area.

[0021] Furthermore, determining the severity of the condition in the gastrointestinal endoscopic images based on the regional values ​​and distribution of pixels within the initially screened lesion area includes:

[0022] Based on the number and area proportion of the initial screening lesion regions, the scale of the initial screening lesion regions in the gastrointestinal endoscopic images is determined; combining the scale of the manifestation with the regional conformity of the initial screening lesion regions, the severity of the disease in the gastrointestinal endoscopic images is determined; the regional conformity characterizes the overall representation of the regional values ​​of the pixels in the initial screening lesion regions.

[0023] Furthermore, the comparison of the initial screening lesion area with other initial lesion areas to determine the severity value of the initial screening lesion area includes:

[0024] Using any initially screened lesion area as the target lesion area, the sum of the differences in the regional conformity between the target lesion area and other initially screened lesion areas is calculated as the initial performance value of the target lesion area.

[0025] By combining the initial performance value of the target lesion area with the regional conformity of the target lesion area, the severity performance value of the target lesion area is determined.

[0026] Furthermore, the region conformity is: the average value of the region belonging to the pixels in the initial screening lesion region.

[0027] Furthermore, the determination of the target gastric ulcer region from the initially screened lesion region based on the degree of performance bias includes:

[0028] The lesion areas with a pre-defined bias greater than the preset screening threshold are selected as the final target gastric ulcer areas.

[0029] Secondly, an AI-assisted lesion localization system based on gastrointestinal endoscopy is provided, the system comprising the following modules:

[0030] The initial recognition module is used to identify the gastric ulcer lesion area and its corresponding confidence level in gastrointestinal endoscopy images based on segmentation boxes of different sizes;

[0031] The integrity analysis module is used to determine the integrity value of the gastric ulcer lesion area based on the integrity of the area within the segmentation box.

[0032] The region screening module is used to obtain the region affiliation value of each pixel in the gastrointestinal endoscopy image based on the number of times each pixel is classified into a gastric ulcer lesion region under different sized segmentation boxes, combined with the confidence and integrity values ​​of the gastric ulcer lesion regions to which the pixel is classified; based on the region affiliation value, pixels at different locations are merged to obtain the preliminary screening lesion regions.

[0033] The region analysis module is used to analyze the region values ​​of pixels within the initial screening lesion area and the distribution of the region, and to determine the degree of performance bias of each initial screening lesion area.

[0034] The region determination module is used to identify the target gastric ulcer region from the initial screening lesion region based on the degree of performance bias.

[0035] Thirdly, embodiments of the present invention provide an electronic device, including a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, it implements the various possible implementations of the first aspect.

[0036] Fourthly, embodiments of the present invention provide a computer program product comprising: computer program code, which, when executed on a computer, causes the computer to perform the method described in the first aspect or any possible implementation thereof.

[0037] Fifthly, embodiments of the present invention provide a computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to perform the various possible implementations of the first aspect.

[0038] The embodiments of the present invention have at least the following beneficial effects:

[0039] This invention first obtains the identified gastric ulcer lesion regions under segmentation boxes of different sizes. Because segmentation boxes of different sizes have different focuses in feature analysis of gastrointestinal images—smaller segmentation boxes focus more on identifying details, while larger segmentation boxes focus more on identifying the overall distribution—the confidence levels corresponding to the gastric ulcer lesion regions in the gastrointestinal images obtained under different segmentation box sizes are not the same. Furthermore, this invention analyzes the regional integrity of the initially identified gastric ulcer lesion regions within the segmentation boxes, thereby obtaining the integrity values ​​of the gastric ulcer lesion regions under different segmentation box sizes. Then, it determines the region affiliation of pixels within the gastric ulcer lesion regions, achieving feature analysis combining segmentation boxes of multiple sizes to obtain the initial screening lesion regions in the gastrointestinal images. Furthermore, by comparing different initial screening lesion regions in the gastrointestinal images, the regional affiliation values ​​of pixels within the initial screening lesion regions and the distribution of the regions are analyzed to determine the performance bias of each initial screening lesion region, thereby screening out the target gastric ulcer lesion regions. This invention analyzes the features of gastric ulcer lesion areas identified by segmentation frames of different sizes, thereby achieving more accurate identification of actual gastric ulcer lesion areas in gastrointestinal images and avoiding the neglect of some lesion areas. Attached Figure Description

[0040] To more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0041] Figure 1 A flowchart illustrating an AI-assisted lesion localization method based on gastrointestinal endoscopy, provided in one embodiment of the present invention;

[0042] Figure 2 This is a flowchart of a method for determining an integrity value according to an embodiment of the present invention;

[0043] Figure 3 A flowchart of a method for determining the degree of performance bias provided in one embodiment of the present invention;

[0044] Figure 4 This is a system block diagram of an AI-assisted lesion localization system based on gastrointestinal endoscopy, provided as an embodiment of the present invention.

[0045] Figure 5 This is a schematic diagram of the structure of a computer device provided in one embodiment of the present invention. Detailed Implementation

[0046] To further illustrate the technical means and effects of the present invention in achieving its intended purpose, the following detailed description, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of an AI-assisted lesion localization method and system based on gastrointestinal endoscopy proposed in accordance with the present invention.

[0047] In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments may be combined in any suitable form.

[0048] In the description of the embodiments of the present invention, unless otherwise stated, " / " means "or". For example, A / B can mean A or B. The "and / or" in the text is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist simultaneously, and B exists alone. In addition, in the description of the embodiments of the present invention, "multiple" means two or more.

[0049] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as implying or suggesting relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature.

[0050] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0051] The embodiments of the present invention will now be described with reference to the accompanying drawings. Those skilled in the art will recognize that, with technological advancements and the emergence of new scenarios, the technical solutions provided by the embodiments of the present invention are also applicable to similar technical problems.

[0052] This invention provides a specific implementation method and system for AI-assisted lesion localization based on gastroscopy and colonoscopy. This method is applicable to lesion localization scenarios assisted by gastroscopy and colonoscopy. In this scenario, a doctor inserts a thin, flexible endoscope into the patient's mouth, through the mouth and esophagus, into the stomach, and obtains real-time images of the stomach.

[0053] The following description, in conjunction with the accompanying drawings, details the specific scheme of the AI-assisted lesion localization method and system based on gastrointestinal endoscopy provided by this invention.

[0054] Please see Figure 1 The diagram illustrates a flowchart of an AI-assisted lesion localization method based on gastroscopy and colonoscopy, according to an embodiment of the present invention. The method includes the following steps:

[0055] Step S100: Obtain the gastric ulcer lesion area and corresponding confidence level in the gastrointestinal endoscopy image.

[0056] Real-time images of the stomach are acquired and used as reference images for identifying gastric ulcer lesions; these are referred to as gastrointestinal images.

[0057] First, a deep learning model is used to initially identify the location of gastric ulcer lesions in the gastrointestinal images corresponding to the gastroscopy and colonoscopy.

[0058] In this embodiment of the invention, the U-Net network is used to locate and identify gastric ulcer lesions in gastrointestinal endoscopy images. The input image enters the U-Net network and sequentially passes through two 3×3 convolutional layers and a 2×2 pooling layer for feature extraction. The right side of the U-Net model is the decoding path, which uses upsampling to restore the feature map to the original image size. The decoding layer consists of two 3×3 convolutional layers and a 2×2 deconvolutional layer, which reduce the number of channels and restore the image resolution. The encoder part contains rich spatial detail information. To avoid this information being lost in deep networks, the U-Net network employs a skip connection mechanism. That is, each encoder layer directly passes the feature map to the corresponding decoder through skip connections, effectively preserving the image features after downsampling. Finally, the network uses a 1×1 convolutional layer to map the number of channels in the feature map to the required number of classes, obtains the confidence level for different classes, and outputs the segmentation result map. This can be understood as each pixel in a gastrointestinal endoscopy image having its own corresponding confidence level, which represents the pixel's classification into different categories. In this embodiment of the invention, a binary classification task can be set: background and foreground, where the foreground is the gastric ulcer lesion area, meaning that the pixel can be classified into two categories. It should be noted that in this embodiment of the invention, the confidence level ranges from [0,1]. The higher the confidence level, the more likely the corresponding pixel belongs to the gastric ulcer lesion area.

[0059] Understandably, after segmenting the gastric ulcer lesion region in the gastrointestinal endoscopy image, each gastric ulcer lesion region has its own corresponding confidence level, which is the average confidence level of all pixels in that gastric ulcer lesion region. Each size of the segmentation box can segment the corresponding gastric ulcer lesion region in the gastrointestinal endoscopy image; that is, the identified gastric ulcer lesion region corresponding to each size of segmentation box may be different.

[0060] To more accurately identify gastric ulcer lesion areas, this invention improves the feature extraction module in the U-Net network by introducing a multi-scale contextual attention module. This module segments the input image with bounding boxes of different sizes, extracting richer and more accurate image feature information.

[0061] In this embodiment of the invention, the sizes of the segmentation boxes are set to: 3×3, 5×5, 7×7, 9×9, and 11×11. It should be noted that the cropping interval for a single-size segmentation box in the image is set to 3.

[0062] Step S200: Determine the integrity value of the gastric ulcer lesion area within the segmentation frame based on the integrity status of the area within the segmentation frame.

[0063] Since segmentation boxes of different sizes may incorporate features of gastrointestinal endoscopic images at different dimensions, the confidence level of the model output may be biased. The more singular and complete the gastric ulcer lesion region within the segmentation box, the easier it is for the model to segment accurately, potentially resulting in higher recognition performance. The singularity and completeness of the gastric ulcer lesion region within the segmentation box can be determined by using the distance distribution of the lesion region's edge points within the segmentation box as a reference.

[0064] In a preferred embodiment of the present invention, the gastric ulcer lesion areas can be screened first based on confidence levels; gastric ulcer lesion areas with confidence levels greater than a preset threshold are selected, and in subsequent steps, only gastric ulcer lesion areas with confidence levels greater than the preset threshold are analyzed. In this embodiment of the present invention, the preset threshold value is 0.8. In other embodiments, the implementer can adjust this value according to the actual situation. When the implementer has stricter requirements for the analysis of gastric ulcer lesion areas, the preset threshold can be appropriately increased; when the implementer has relatively relaxed requirements for the analysis of gastric ulcer lesion areas, the preset threshold can be appropriately decreased.

[0065] First, after obtaining the gastric ulcer lesion area in the gastrointestinal endoscopy image through step S100, multiple boundary points of the gastric ulcer lesion area are obtained.

[0066] Please see Figure 2 As shown, in some possible implementations, the determination of the integrity value of the gastric ulcer lesion region within the segmentation box in step S200 above can be achieved through the following steps S210 to S220:

[0067] Step S210: Determine the overall distribution distance between the gastric ulcer lesion areas within the segmentation frame.

[0068] The overall distribution distance is between the regional boundary points that characterize the gastric ulcer lesion area;

[0069] In some embodiments, the overall distribution distance is obtained by calculating the normalized value of the average distance between all region boundary points in the currently analyzed segmentation box and the nearest region boundary point, which is used as the overall distribution distance of the segmentation box. It should be noted that each region boundary point has a nearest region boundary point in the segmentation box, unless there is only one region boundary point in the segmentation box. If there is only one region boundary point in the segmentation box, the overall distribution distance in the current segmentation box is 1.

[0070] Step S220: Combine the overall distribution distance of the gastric ulcer lesion area within the segmentation box, the area of ​​the gastric ulcer lesion area, and the number of areas to determine the integrity value of the gastric ulcer lesion area within the segmentation box.

[0071] Among them, the area of ​​the gastric ulcer lesion region within the segmentation box is positively correlated with the integrity value; the overall distribution distance and number of gastric ulcer lesions within the segmentation box are negatively correlated with the integrity value, and the integrity value is the normalized value.

[0072] Obtain the number of gastric ulcer lesion regions within the segmentation box, and calculate the area of ​​each gastric ulcer lesion region within the segmentation box, taking the proportion of the area within the segmentation box as the region area proportion.

[0073] In some embodiments, taking a segmentation frame j of arbitrary size as an example, the integrity value of the gastric ulcer lesion area within the segmentation frame j is... The calculation formula is: ;in, The area percentage of the region defined by the dividing box j; The number of regions in the dividing box j; denoted as , where is the overall distribution distance of the segmentation box j; norm is the normalization function.

[0074] In the formula for calculating the completeness value, the number of regions in the segmentation box j is included. The fewer the number of regions contained within a segmentation box, the lower the probability that the gastric ulcer lesion region within that segmentation box is intact. The overall distribution distance represents the distance between the boundary points of regions within a segmentation box. The smaller the distance between the boundary points of regions within a segmentation box, the higher the probability that each boundary point corresponds to a gastric ulcer lesion region. Therefore, compared to a segmentation box containing multiple gastric ulcer lesion regions, the fewer gastric ulcer lesion regions within a segmentation box, the greater the integrity value of the gastric ulcer lesion region within that segmentation box.

[0075] In embodiments of the present invention, the maximum-minimum value normalization method can be used to... Normalization is performed so that the normalized integrity value ranges from [0, 1].

[0076] Step S300: Based on the number of times each pixel at each location is classified into the gastric ulcer lesion region under different sized segmentation boxes, and combined with the confidence and integrity values ​​of the gastric ulcer lesion region to which the pixel is classified, the region belonging value of each pixel at each location in the gastrointestinal endoscopy image is obtained; based on the region belonging value, pixels at different locations are merged to obtain the initial screening lesion region.

[0077] The more singular and complete the gastric ulcer lesion area contained within the segmentation box, the more accurate the confidence score of the gastric ulcer lesion area. Therefore, for the extent of gastric ulcer lesions in gastrointestinal images, the greater the singularity and completeness of the gastric ulcer lesion area within the segmentation box, the more accurate and reliable the confidence score obtained from this segmentation box, and the greater its reference value when fusing confidence score data from multiple segmentation boxes.

[0078] Using any pixel as the target pixel, the number of times the target pixel is divided into all gastric ulcer lesion regions under different sized segmentation boxes is obtained. Then, the number of times the target pixel is divided into all gastric ulcer lesion regions is normalized by the max-min normalization algorithm to obtain the normalization count.

[0079] The higher the integrity value, the greater the reference value of the corresponding pixel. Similarly, the higher the confidence level of a gastric ulcer lesion area, the greater the reference value of belonging to that gastric ulcer lesion area. Therefore, the reference confidence level of a pixel can be determined by combining the integrity value and the confidence level.

[0080] In some embodiments, the reference confidence of a pixel is determined based on the confidence and integrity values ​​of the gastric ulcer lesion region to which the pixel is assigned: the average of the product of the integrity value and the confidence value of the gastric ulcer lesion region when the target pixel is assigned to different gastric ulcer lesion regions is used as the reference confidence of the target pixel.

[0081] Furthermore, by combining the reference confidence level and the normalization number of each pixel at each location, the region classification value of each pixel in the gastrointestinal endoscopy image is obtained. Both the reference confidence level and the normalization number are positively correlated with the region classification value of each pixel in the gastrointestinal endoscopy image.

[0082] In some embodiments, the product of the reference confidence of the target pixel and the number of times the target pixel is classified into all gastric ulcer lesion regions is calculated as the region belonging value of the target pixel.

[0083] The more times a pixel is classified into the gastric ulcer lesion area under different sized bounding boxes, the greater the degree of agreement that the pixel belongs to the gastric ulcer lesion area; at the same time, the higher the reference confidence of the pixel, the greater the probability that the pixel belongs to the gastric ulcer lesion area.

[0084] The larger the region value of a pixel, the more likely it is to be a gastric ulcer lesion area. If the region values ​​of connected pixels are all large, then the actual gastric ulcer lesion area in the gastroscopy and colonoscopy image can be formed by the adjacent pixels.

[0085] Therefore, based on the region's value, pixels at different locations are merged to obtain the initial screening lesion region: adjacent pixels with a region's value greater than a preset merging threshold are merged to obtain the initial screening lesion region. In this embodiment of the invention, the preset merging threshold is 0.6; in other embodiments, this value can be adjusted by the implementer according to the actual situation. When two pixels are located within each other's eight neighbors, they are considered adjacent pixels.

[0086] Step S400: Analyze the regional values ​​and distribution of pixels within the initial screening lesion area to determine the degree of performance bias for each initial screening lesion area.

[0087] Gastric ulcers often develop from gastritis, and the lesions of the two conditions share many similar morphological features. Images of shallow gastric ulcers may resemble those of chronic gastritis, leading to model confusion. The more severe the gastritis, the more likely it is to cause a gastric ulcer. Therefore, analyzing the distribution of initially screened lesion areas in gastrointestinal endoscopic images helps determine the severity of the condition and serves as a reference for identifying target gastric ulcer areas.

[0088] Please see Figure 3 In some possible implementations, step S400 above can be achieved through steps S410 to S420:

[0089] Step S410: Based on the region values ​​of the pixels within the initially screened lesion area and the distribution of the region, determine the severity of the condition in the gastrointestinal endoscopic images.

[0090] The scale of the lesions in the gastrointestinal endoscopic images is determined based on the number of regions and the area ratio of the initial screening lesions.

[0091] The number of lesion regions in the initial screening of gastrointestinal images and the area ratio of the initial screening lesion regions in the gastrointestinal images were obtained.

[0092] When the area of ​​the initially screened lesion region is relatively large, it indicates that the lesion region appears more clearly and is of a larger scale in the gastrointestinal endoscopic image. However, when there are many initially screened lesion regions, since the area of ​​the gastrointestinal endoscopic image is fixed, the area of ​​the initially screened lesion regions is relatively small with a larger number of regions, making them less clear than when there are fewer regions. Therefore, when determining the scale of the initially screened lesion region in the gastrointestinal endoscopic image based on the number of regions and their area proportion, the area proportion is positively correlated with the scale of the lesion, while the number of regions is negatively correlated with the scale of the lesion.

[0093] In some embodiments, the ratio of the area of ​​the primary screening lesion region to the number of regions is used as the scale of the primary screening lesion region in the gastrointestinal endoscopic image.

[0094] When the lesion area initially screened has a large area in the gastrointestinal endoscopy image, it is easier to locate the lesion in the current frame of the gastrointestinal endoscopy image.

[0095] Furthermore, by combining the magnitude of the observed changes with the regional consistency of the initially screened lesion area, the severity of the condition in the gastrointestinal endoscopic images is determined. The regional consistency characterizes the overall representation of the regional values ​​of pixels within the initially screened lesion area.

[0096] The average regional conformity of each initially screened lesion region in the gastrointestinal endoscopy images is calculated as the overall conformity.

[0097] The method for obtaining the regional conformity of the initial screening lesion area is: the average value of the regional values ​​of the pixels in the initial screening lesion area.

[0098] The extent of the lesion and the overall consistency with the initial screening lesion area were both positively correlated with the severity of the disease in the gastrointestinal images.

[0099] In some embodiments, the severity of the disease is the product of the size of the initial screening lesion area and the overall conformity.

[0100] The greater the overall consistency among multiple initial screening lesion areas, the larger the area represented by the initial screening lesion area in the gastroscopy and colonoscopy images. Under normal circumstances, the more severe the gastric ulcer, the larger its area will appear in the image. Therefore, in this embodiment of the invention, the severity of the condition in the gastroscopy and colonoscopy images is determined by combining the scale of the lesion with the overall consistency of the initial screening lesion areas.

[0101] Step S420: Compare the initial screening lesion area with other initial lesion areas to determine the severity value of the initial screening lesion area.

[0102] The more severe the condition shown in gastroscopy and colonoscopy images, the more likely the area of ​​gastric ulcer lesion with a higher overall consistency in the images is to be the actual gastric ulcer lesion. By comparing the overall consistency of different initial screening lesion areas in gastroscopy and colonoscopy images, the greater the overall consistency, the greater the likelihood of it being the actual gastric ulcer lesion.

[0103] Using any initially screened lesion area as the target lesion area, the sum of the differences in the regional conformity between the target lesion area and other initially screened lesion areas is calculated as the initial performance value of the target lesion area.

[0104] The severity score of the target lesion region is determined by combining the initial performance value and the regional conformity of the target lesion region. Both the initial performance value and the regional conformity of the target lesion region are positively correlated with the severity score of the target lesion region. The regional conformity of the target lesion region is the average value of the regional values ​​of the pixels within the target lesion region.

[0105] In some embodiments, the product of the initial performance value of the target lesion region and the regional conformity of the target lesion region is calculated as the severity performance value of the target lesion region.

[0106] The higher the severity score of the initial screening lesion area, and the greater the severity of the condition in the gastroscopy and colonoscopy images, the greater the probability that the images will show a severe condition, and the more likely the images will show gastric ulcer lesions that have developed from gastritis. In particular, the initial screening lesion area, due to its higher severity score, will show a more consistent pattern with gastric ulcer lesions than other initial screening lesion areas, and its lesion size is more likely to be that of a gastric ulcer.

[0107] Step S430: Using the severity of the condition as a weight, the severity performance value of each initial screening lesion area is weighted to obtain the performance bias of each initial screening lesion area.

[0108] In some embodiments, the product of disease severity and severity performance value of the initial screening lesion area is calculated as the performance bias of the initial screening lesion area.

[0109] Taking the g-th initial screening lesion area as an example, the severity of the disease in the g-th initial screening lesion area... The calculation formula is: ;in, This represents the initial performance value of the g-th primary screening lesion region in the b-th frame of the gastrointestinal endoscopy image; The region conformity of the g-th initial screening lesion region in the b-th frame of the gastrointestinal endoscopy image; is the severity value of the g-th initial screening lesion region in the b-th frame of the gastrointestinal endoscopy image; The severity of the condition is represented by the b-th frame of the gastrointestinal endoscopy image.

[0110] Step S500: Based on the degree of performance bias, the target gastric ulcer region is determined from the initial screening lesion region.

[0111] The performance bias is normalized using a max-min normalization algorithm, resulting in a normalized value range of [0,1]. When the normalized performance bias is greater than a preset screening threshold, the initially screened lesion area is determined as the final target gastric ulcer area. In other words, initially screened lesion areas with a normalized performance bias greater than the preset screening threshold are designated as the final target gastric ulcer areas. In this embodiment, the preset screening threshold is 0.8; in other embodiments, this value can be adjusted by the implementer according to actual conditions.

[0112] The above method is used to identify and acquire target gastric ulcer regions in different gastroscopy and colonoscopy images. The gastroscopy and colonoscopy images to be analyzed are then transmitted and stored in conjunction with the corresponding identified target gastric ulcer regions.

[0113] The coordinates of the target gastric ulcer area in different gastrointestinal endoscopic images are obtained using SQL queries. A mask is then created to mark the gastric ulcer lesion area in the image and visualize it in the form of an image.

[0114] Please see Figure 4 , Figure 4 This invention provides a system block diagram of an AI-assisted lesion localization system based on gastrointestinal endoscopy, comprising:

[0115] The initial recognition module is used to identify the gastric ulcer lesion area and its corresponding confidence level in gastrointestinal endoscopy images based on segmentation boxes of different sizes;

[0116] The integrity analysis module is used to determine the integrity value of the gastric ulcer lesion area based on the integrity of the area within the segmentation box.

[0117] The region screening module is used to obtain the region affiliation value of each pixel in the gastrointestinal endoscopy image based on the number of times each pixel is classified into a gastric ulcer lesion region under different sized segmentation boxes, combined with the confidence and integrity values ​​of the gastric ulcer lesion regions to which the pixel is classified; based on the region affiliation value, pixels at different locations are merged to obtain the preliminary screening lesion regions.

[0118] The region analysis module is used to analyze the region values ​​of pixels within the initial screening lesion area and the distribution of the region, and to determine the degree of performance bias of each initial screening lesion area.

[0119] The region determination module is used to identify target gastric ulcer regions from the initially screened lesion regions based on the degree of performance bias. Optionally, the transmission medium can be a wired link, such as, but not limited to, coaxial cable, optical fiber, and digital subscriber line, or a wireless link, such as, but not limited to, Wireless Fidelity (WIFI), Bluetooth, and mobile device networks.

[0120] It should be noted that the device provided in the above embodiments is only an example of the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the computer device can be divided into different functional modules to complete all or part of the functions described above.

[0121] Figure 5 This is a schematic diagram of the structure of a computer device provided in an embodiment of the present invention. For example, as shown... Figure 4 As shown, the computer device 600 includes: a memory 610, a processor 620, and a computer program 630 stored in the memory 610 and running on the processor 620, wherein when the processor 620 executes the computer program 630, the computer device can execute any of the aforementioned AI-assisted lesion localization methods based on gastrointestinal endoscopy.

[0122] Furthermore, embodiments of the present invention also protect an apparatus that may include a memory and a processor, wherein the memory stores executable program code, and the processor is used to call and execute the executable program code to execute the AI-assisted lesion localization method based on gastrointestinal endoscopy provided in the embodiments of the present invention.

[0123] In this embodiment of the invention, the device can be divided into functional modules according to the above method example. For example, each module can correspond to a separate function, or two or more functions can be integrated into one processing module. The integrated module can be implemented in hardware. It should be noted that the module division in this embodiment is illustrative and is only a logical functional division. In actual implementation, there may be other division methods.

[0124] When each module is divided according to its function, the device may also include a signal uploading module, a determination module, and an adjustment module. It should be noted that all relevant content of each step involved in the above method embodiments can be referenced from the functional descriptions of the corresponding functional modules, and will not be repeated here.

[0125] It should be understood that the device provided in the embodiments of the present invention is used to execute the above-described AI-assisted lesion localization method based on gastrointestinal endoscopy, and therefore can achieve the same effect as the above-described implementation method.

[0126] When using integrated units, the device may include a processing module and a storage module. When applied to a device, the processing module can be used to control and manage the device's operations. The storage module can be used to support the device in executing program code, etc. The processing module may be a processor or a controller, which can implement or execute various exemplary logic blocks, modules, and circuits as described in this disclosure. The processor may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of Digital Signal Processing (DSP) and a microprocessor, etc., and the storage module may be a memory.

[0127] In addition, the device provided in the embodiments of the present invention may specifically be a chip, component or module. The chip may include a connected processor and a memory. The memory is used to store instructions. When the processor calls and executes the instructions, the chip can execute the AI-assisted lesion localization method based on gastrointestinal endoscopy provided in the above embodiments.

[0128] This invention also provides a computer-readable storage medium storing computer program code. When the computer program code is run on a computer, the computer executes the aforementioned method steps to implement the AI-assisted lesion localization method based on gastrointestinal endoscopy provided in the above embodiments.

[0129] This invention also provides a computer program product that, when run on a computer, causes the computer to perform the aforementioned steps to implement the AI-assisted lesion localization method based on gastrointestinal endoscopy provided in the above embodiments.

[0130] In this invention, the apparatus, computer-readable storage medium, computer program product, or chip provided in the embodiments are all used to execute the corresponding methods described above. Therefore, the beneficial effects they achieve can be referred to the beneficial effects in the corresponding methods described above, and will not be repeated here. Through the above description of the embodiments, those skilled in the art will understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In the embodiments provided by this invention, it should be understood that the disclosed apparatus and method can be implemented in other ways.

[0131] The device embodiments described above are merely illustrative. For example, the division of modules or units is only a logical functional division. In actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0132] It should also be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0133] It should be noted that the order of the above embodiments of the present invention is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0134] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0135] The above content is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the protection scope of the present invention.

Claims

1. An AI-assisted lesion localization method based on gastrointestinal endoscopy, characterized in that, The method includes the following steps: Based on segmentation boxes of different sizes, the regions of gastric ulcer lesions in gastrointestinal endoscopy images and their corresponding confidence levels are identified. The integrity value of the gastric ulcer lesion area is determined based on the integrity of the area within the segmentation box. Based on the number of times each pixel at each location is classified into a gastric ulcer lesion region under different sized segmentation boxes, and combined with the confidence and integrity values ​​of the gastric ulcer lesion regions to which the pixel is classified, the region affiliation value of each pixel at each location in the gastrointestinal endoscopy image is obtained; based on the region affiliation value, pixels at different locations are merged to obtain the initial screening lesion region; Analyze the regional values ​​and distribution of pixels within the initial screening lesion area to determine the degree of performance bias for each initial screening lesion area; Based on the degree of performance bias, the target gastric ulcer area is determined from the initial screening lesion area.

2. The AI-assisted lesion localization method based on gastrointestinal endoscopy according to claim 1, characterized in that, The determination of the integrity value of the gastric ulcer lesion area based on the regional integrity of the gastric ulcer lesion area within the segmentation frame includes: Determine the overall distribution distance between gastric ulcer lesion regions within the segmentation frame; the overall distribution distance characterizes the distance between the boundary points of the gastric ulcer lesion regions. The integrity value of the gastric ulcer lesion region within the segmentation box is determined by combining the overall distribution distance of the gastric ulcer lesion region, the area of ​​the gastric ulcer lesion region, and the number of regions.

3. The AI-assisted lesion localization method based on gastrointestinal endoscopy according to claim 1, characterized in that, The method involves determining the region affiliation of each pixel in the gastrointestinal endoscopic image based on the number of times it is classified into a gastric ulcer lesion region under different sized bounding boxes, combined with the confidence and integrity values ​​of the gastric ulcer lesion region to which the pixel is classified. This includes: The reference confidence level of a pixel is determined based on the confidence level and integrity value of the gastric ulcer lesion region to which the pixel is assigned. By combining the reference confidence of each pixel at each location with the number of times the pixel was classified into the gastric ulcer lesion area under different sized segmentation boxes, the region belonging value of each pixel at each location in the gastrointestinal endoscopy image is obtained.

4. The AI-assisted lesion localization method based on gastrointestinal endoscopy according to claim 1, characterized in that, The process of merging pixels at different locations based on the region's value to obtain the initial screening lesion region includes: Adjacent pixels whose region values ​​are greater than a preset merging threshold are merged to obtain the initial screening lesion region.

5. The AI-assisted lesion localization method based on gastrointestinal endoscopy according to claim 1, characterized in that, The analysis of the pixel values ​​and distribution within the initial screening lesion area determines the degree of performance bias for each initial screening lesion area, including: Based on the regional values ​​and distribution of pixels within the initial screening lesion area, the severity of the disease in the gastrointestinal endoscopic images is determined; the initial screening lesion area is compared with other initial lesion areas to determine the severity performance value of the initial screening lesion area; using the severity of the disease as a weight, the severity performance value of each initial screening lesion area is weighted to obtain the performance bias of each initial screening lesion area.

6. The AI-assisted lesion localization method based on gastrointestinal endoscopy according to claim 5, characterized in that, The determination of the severity of the condition in the gastrointestinal endoscopic images based on the region values ​​and distribution of pixels within the initially screened lesion area includes: Based on the number and area proportion of the initial screening lesion regions, the scale of the initial screening lesion regions in the gastrointestinal endoscopic images is determined; combining the scale of the manifestation with the regional conformity of the initial screening lesion regions, the severity of the disease in the gastrointestinal endoscopic images is determined; the regional conformity characterizes the overall representation of the regional values ​​of the pixels in the initial screening lesion regions.

7. The AI-assisted lesion localization method based on gastrointestinal endoscopy according to claim 6, characterized in that, The comparison of the initial screening lesion area with other initial lesion areas to determine the severity value of the initial screening lesion area includes: Using any initially screened lesion area as the target lesion area, the sum of the differences in the regional conformity between the target lesion area and other initially screened lesion areas is calculated as the initial performance value of the target lesion area. By combining the initial performance value of the target lesion area with the regional conformity of the target lesion area, the severity performance value of the target lesion area is determined.

8. The AI-assisted lesion localization method based on gastrointestinal endoscopy according to claim 6, characterized in that, The region conformity is defined as the average value of the region to which the pixels in the initial screening lesion region belong.

9. The AI-assisted lesion localization method based on gastrointestinal endoscopy according to claim 5, characterized in that, The process of identifying the target gastric ulcer region from the initial screening lesion region based on the degree of performance bias includes: The lesion areas with a pre-defined bias greater than the preset screening threshold are selected as the final target gastric ulcer areas.

10. An AI-assisted lesion localization system based on gastrointestinal endoscopy, characterized in that, The system includes the following modules: The initial recognition module is used to identify the gastric ulcer lesion area and its corresponding confidence level in gastrointestinal endoscopy images based on segmentation boxes of different sizes; The integrity analysis module is used to determine the integrity value of the gastric ulcer lesion area based on the integrity of the area within the segmentation box. The region screening module is used to determine the region to which each pixel in the gastrointestinal endoscopy image belongs based on the number of times it is classified into the gastric ulcer lesion region under different sized segmentation boxes, combined with the confidence and integrity values ​​of the gastric ulcer lesion region to which the pixel is classified. Based on the value of the region, pixels at different locations are merged to obtain the initial screening lesion region; The region analysis module is used to analyze the region values ​​of pixels within the initial screening lesion area and the distribution of the region, and to determine the degree of performance bias of each initial screening lesion area. The region determination module is used to identify the target gastric ulcer region from the initial screening lesion region based on the degree of performance bias.

Images