A capsule endoscopy lesion recognition method and system based on artificial intelligence

Through an artificial intelligence-based method, the bounding box confidence and disease type probability of capsule endoscopic images are calculated using the annotation, clustering and attention mechanism, which solves the problem of large amount of image data and identification accuracy and speed in capsule endoscopy examination. , achieving efficient lesion recognition.

CN118781439BActive Publication Date: 2024-11-22GUANGDONG GENERAL HOSPITAL +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411266284.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-09-11
Publication Date
2024-11-22
Estimated Expiration
2044-09-11

AI Technical Summary

Technical Problem

The amount of image data generated by capsule endoscopy is huge, resulting in a large amount of workload for reading films, which requires doctors to analyze for a long time, and the images may be disturbed by fluid or food residues, making it difficult to take into account both the accuracy and speed of recognition.

Method used

Using an artificial intelligence-based method, by obtaining images taken by capsule endoscopy, annotating and clustering, the bounding box confidence and disease type probability corresponding to the grid are calculated, and non-maximum suppression filtering is used to identify the lesions. Target box.

Benefits of technology

It achieves a high recognition speed while ensuring recognition accuracy, improves the efficiency and accuracy of automatic reading of capsule endoscopic images, and reduces the requirements for doctors' professional abilities.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118781439B_ABST
    Figure CN118781439B_ABST
Patent Text Reader

Abstract

The present invention relates to the field of artificial intelligence, and specifically to a capsule endoscope lesion recognition method and system based on artificial intelligence, including clustering the annotation frames, obtaining a priori frames and grid sizes according to the clustering results; obtaining a prediction range based on the priori frames; converting the image to be recognized taken by the capsule endoscope into a preset size through convolution, and then dividing it into a plurality of grids of the grid size, calculating the attention of all grids in each bounding box corresponding to each grid, and obtaining the confidence of each bounding box through the attention; calculating the probability of the corresponding disease type within the prediction range centered on the grid; filtering the bounding box according to the disease type probability and confidence, filtering the remaining bounding boxes by using the IoU threshold using non-maximum suppression, and using the filtered bounding box as the recognized lesion target box. The present invention improves the recognition accuracy and reduces false positives while ensuring the recognition speed.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and in particular to an artificial intelligence-based capsule endoscopy lesion recognition method and system. Background Art

[0002] Examination of the small intestine has always been a difficult point. Although enteroscopy is available, the operation is complicated and the patient acceptance is low. Compared with enteroscopy, capsule endoscopy (CE) is a miniature camera device that patients can swallow to facilitate doctors to observe and evaluate the internal conditions of the digestive tract. This technology is mainly used to examine small intestinal diseases such as Crohn's disease, small intestinal bleeding, polyps, and early colorectal cancer. Its main advantages include being non-invasive, simple to operate, and less invasive to patients. Patients can undergo the examination without anesthesia and discomfort, and capsule endoscopy can provide images of areas that are difficult to reach with traditional endoscopes, allowing doctors to more comprehensively examine the entire digestive tract.

[0003] However, the amount of image data generated by capsule endoscopy is huge. It can capture several images per second. It takes about 10 hours from swallowing to excretion of the capsule endoscope. A capsule endoscopy can generate tens of thousands of images. The workload of reading the images is extremely large, and professional doctors are required to perform long-term analysis and interpretation, which is not only time-consuming but also requires high professional ability of doctors. Moreover, the images may be interfered by body fluids or food residues, making diagnosis more difficult. In addition, the uneven movement speed of the capsule may cause the image of some areas to be missing or unclear, which brings challenges to accurate diagnosis. However, recognition accuracy and recognition speed are contradictory. How to have a high recognition speed for tens of thousands of images captured by capsule endoscopy while ensuring high accuracy is the key to automatic reading of images captured by capsule endoscopy. Summary of the invention

[0004] In order to solve the above problems, the present invention provides a capsule endoscopy lesion recognition method based on artificial intelligence, the method comprising the following steps:

[0005] Acquire an image taken by a capsule endoscope, annotate the image after deduplication, cluster the annotated frames, obtain a priori frames and a grid size according to the clustering results; and obtain a prediction range based on the priori frames;

[0006] The image to be identified taken by the capsule endoscope is converted into a preset size through convolution, and then divided into a plurality of grids of the grid size, and for each grid, the attention of all grids in each bounding box corresponding to the grid is calculated, and the confidence of each bounding box is obtained through the attention; the probability of the corresponding disease type within the prediction range centered on the grid is calculated;

[0007] The bounding boxes are filtered according to the disease type probability and confidence, and the remaining bounding boxes are filtered using the IoU threshold using non-maximum suppression, and the filtered bounding boxes are used as the identified lesion target boxes.

[0008] Preferably, clustering the annotation boxes is specifically performed as follows:

[0009] Select one annotation box from all annotation boxes, calculate the ratio of the length of other annotation boxes to the length of the selected annotation box, and the ratio of the width of other annotation boxes to the width of the selected annotation box. If both ratios are within the set range, the other annotation boxes and the selected annotation box are classified into one category; and so on, until all annotation boxes are clustered;

[0010] The number of clusters is determined. If the number of clusters exceeds a preset value, several clusters with the smallest labeled boxes are deleted until the number of clusters does not exceed the preset value.

[0011] Preferably, the prior frame and grid size are obtained according to the clustering results, specifically:

[0012] For each cluster, calculate the average length and width of the annotation box in the cluster, and obtain the prior box corresponding to the cluster based on the two average values;

[0013] Get the minimum length and width of all prior boxes, which is a multiple of the grid length.

[0014] Preferably, the prediction range is obtained based on the prior frame, specifically:

[0015] Calculate the union of all prior boxes and use the union as the prediction range.

[0016] Preferably, the attention of all grids in each bounding box corresponding to each grid is calculated for each grid, and the confidence of each bounding box is obtained through the attention, specifically:

[0017] Each grid in the bounding box is passed through a linear projection layer to obtain an embedding vector, and the learning token is added. The content of the learning token is learned through the attention mechanism, and the confidence of the bounding box is obtained through the content of the learning token.

[0018] Preferably, filtering the bounding box according to the disease type probability and confidence level is specifically as follows:

[0019] The maximum value of the disease type probability is obtained. If the maximum value meets the conditions and the confidence is greater than the confidence threshold, the bounding box is retained; if the maximum value does not meet the conditions and / or the confidence is not greater than the confidence threshold, the bounding box is not retained.

[0020] Preferably, the IoU threshold is specifically determined as follows:

[0021] Determine whether the annotation boxes have an intersection. If not, set the IoU threshold to 0; otherwise, calculate the average IoU value of all annotation boxes with intersections, and use the average value as the IoU threshold.

[0022] In another aspect, the present invention further provides a capsule endoscopy lesion recognition system based on artificial intelligence, the system comprising the following modules:

[0023] A priori frame and grid acquisition module is used to acquire images taken by capsule endoscopy, annotate the images after deduplication, cluster the annotated frames, obtain the priori frame and grid size according to the clustering results; and obtain the prediction range based on the priori frame;

[0024] A feature extraction module, used for converting the image to be identified taken by the capsule endoscope into a preset size through convolution, and then dividing it into a plurality of grids of the grid size, calculating the attention of all grids in each bounding box corresponding to the grid for each grid, and obtaining the confidence of each bounding box through the attention; calculating the probability of the corresponding disease type within the prediction range centered on the grid;

[0025] The bounding box screening module is used to filter the bounding boxes according to the disease type probability and confidence, and use the IoU threshold to filter the remaining bounding boxes in a non-maximum suppression manner, and use the filtered bounding boxes as the identified lesion target boxes.

[0026] Preferably, clustering the annotation boxes is specifically performed as follows:

[0027] Select one annotation box from all annotation boxes, calculate the ratio of the length of other annotation boxes to the length of the selected annotation box, and the ratio of the width of other annotation boxes to the width of the selected annotation box. If both ratios are within the set range, the other annotation boxes and the selected annotation box are classified into one category; and so on, until all annotation boxes are clustered;

[0028] The number of clusters is determined. If the number of clusters exceeds a preset value, several clusters with the smallest labeled boxes are deleted until the number of clusters does not exceed the preset value.

[0029] Preferably, the prior frame and grid size are obtained according to the clustering results, specifically:

[0030] For each cluster, calculate the average length and width of the annotation box in the cluster, and obtain the prior box corresponding to the cluster based on the two average values;

[0031] Get the minimum length and width of all prior boxes, which is a multiple of the grid length.

[0032] Preferably, the prediction range is obtained based on the prior frame, specifically:

[0033] Calculate the union of all prior boxes and use the union as the prediction range.

[0034] Preferably, the attention of all grids in each bounding box corresponding to each grid is calculated for each grid, and the confidence of each bounding box is obtained through the attention, specifically:

[0035] Each grid in the bounding box is passed through a linear projection layer to obtain an embedding vector, and the learning token is added. The content of the learning token is learned through the attention mechanism, and the confidence of the bounding box is obtained through the content of the learning token.

[0036] Preferably, filtering the bounding box according to the disease type probability and confidence level is specifically as follows:

[0037] The maximum value of the disease type probability is obtained. If the maximum value meets the conditions and the confidence is greater than the confidence threshold, the bounding box is retained; if the maximum value does not meet the conditions and / or the confidence is not greater than the confidence threshold, the bounding box is not retained.

[0038] Preferably, the IoU threshold is specifically determined as follows:

[0039] Determine whether the annotation boxes have an intersection. If not, set the IoU threshold to 0; otherwise, calculate the average IoU value of all annotation boxes with intersections, and use the average value as the IoU threshold.

[0040] In addition, the present invention also provides a computer program product, which implements the above method when executed by a processor.

[0041] In view of the problem that there are many images taken by capsule endoscopy and there are many interferences in the images, the capsule endoscopy lesion recognition method based on artificial intelligence provided by the present invention ensures the accuracy of recognition while maintaining a high recognition speed. Specifically, the confidence of each bounding box corresponding to the grid is calculated by using the attention mechanism, and the prediction range is obtained according to the prior frame. The probability of the disease type corresponding to each grid is calculated according to the prediction range, which improves the accuracy of disease recognition and can achieve a faster recognition speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 This is a flow chart of Embodiment 1;

[0043] Figure 2 Schematic diagram of the prior box and grid;

[0044] Figure 3 It is a schematic diagram of the relationship between the preset range and the prior box;

[0045] Figure 4 This is a structural diagram of the second embodiment. DETAILED DESCRIPTION

[0046] In this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the process, method, article or device including the elements. In the present invention, if it involves the collection of personal privacy data, such as face, mobile phone usage information, etc., the individual's permission will be obtained in advance, including but not limited to verbal reminders, posters, mobile phone reminders, etc.; if it involves conflicts with laws and regulations, it will be produced or used within the scope permitted by laws and regulations.

[0047] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0048] Embodiment 1, the present invention provides a capsule endoscopy lesion recognition method based on artificial intelligence, such as Figure 1 As shown, the method comprises the following steps:

[0049] S1, obtaining an image taken by a capsule endoscope, annotating the image after deduplication, clustering the annotated frames, obtaining a priori frames and a grid size according to the clustering results; and obtaining a prediction range based on the priori frames;

[0050] When constructing a training set, it is necessary to annotate the pictures taken by the capsule endoscope. Labelme is preferred as the annotation tool. Since the frequency of capsule endoscope shooting is relatively fast, and several pictures are taken within 1 second, the images need to be deduplicated before annotation, and pictures with a similarity exceeding a certain level need to be deleted. After annotation, there will be multiple annotation boxes. The prior bounding box is obtained by clustering the annotation boxes. The prior bounding box is also called the anchor box. In one embodiment, the clustering method is determined by the 1-IoU method, where IoU represents the intersection-over-union ratio. The IoU of the two annotation boxes is calculated, and the 1-IoU is used to determine whether the two annotation boxes are clustered into one category. However, there is a problem with this clustering. It only ignores the shape of the annotation box based on the overlap ratio of the two annotation boxes. In another embodiment, the clustering of the annotation boxes is specifically as follows:

[0051] S11, select one annotation box from all the annotation boxes, calculate the ratio of the length of the other annotation boxes to the length of the selected annotation box, and the ratio of the width of the other annotation boxes to the width of the selected annotation box, if both ratios are within the set range, then classify the other annotation boxes and the selected annotation box into one category; and so on, until all the annotation boxes are clustered;

[0052] The shapes of the two annotation boxes can be limited by the ratio of the lengths and widths of the two annotation boxes. For example, the length of annotation box A is 1 and the width is 1, and the length of another annotation box B is 1 and the width is 1.1. Assuming that the set range is 0.9-1.1, annotation box A and annotation box B will be clustered into one category. Compared with clustering based on IoU, this embodiment can better cluster annotation boxes of similar shapes into one category.

[0053] S12, determining the number of clusters. If the number of clusters exceeds a preset value, deleting several clusters with the smallest marked boxes until the number of clusters does not exceed the preset value.

[0054] The purpose of clustering is to obtain bounding boxes. If too many bounding boxes are set, the calculation speed will be affected. The number of clusters is further determined. The number of clusters is limited by setting the number of clusters, that is, by a preset value. If the number of clusters exceeds the preset value, several clusters with smaller number of marked boxes will be deleted until the number of clusters does not exceed the preset value. In a more specific embodiment, the preset value is 9.

[0055] In another embodiment, S12 may be replaced by: determining the number of clusters, if the number of clusters is greater than 9 or less than 4, adjusting the range set in S11, and re-executing S11 until the number of clusters is in [4,9].

[0056] After clustering the annotation boxes, a priori box (anchor) is further obtained according to the clustering results. In a specific embodiment, the priori box and grid size are obtained according to the clustering results, specifically:

[0057] For each cluster, calculate the average length and width of the annotation box in the cluster, and obtain the prior box corresponding to the cluster based on the two average values;

[0058] Since the annotation boxes in a cluster are similar, the size of the prior box is determined according to the average length and average width of the annotation boxes in the cluster. Assuming that there are 10 annotation boxes in the first cluster, and the average length and width of these 10 annotation boxes are 1 and 1 respectively, the length and width of the prior box are set to 1 and 1 respectively.

[0059] It should be noted that since the images taken by capsule endoscopy are relatively large, the original images are usually processed, including but not limited to adjusting the contrast, convolution and reducing the size, etc. If the above annotation box is an annotation of the original image, it needs to be converted proportionally to the annotation box after the size is reduced. If it is annotating the image after the size is reduced, no conversion is required.

[0060] In another embodiment, after the average value is used as the length and width of the prior frame, it does not necessarily include an integer number of pixels or an integer number of grids. In this case, the prior frame needs to be adjusted so that the prior frame includes an integer number of pixels and an integer number of grids to facilitate subsequent processing.

[0061] Get the minimum length and width of all prior boxes, the minimum value is a multiple of the grid length, where the schematic diagram of the prior box and the grid is as follows Figure 2 shown.

[0062] In order to ensure the relationship between the grid and the prior box, that is, the prior box includes an integer number of grids, when setting the grid, it is necessary to ensure that the length of the grid satisfies the minimum value of the length and width of all prior boxes is an integer multiple of the grid length.

[0063] In the Yolo series of target detection models, each grid corresponds to a one-dimensional vector, including the position (x, y, w, h) of all bounding boxes centered on this grid, the confidence of each bounding box, and the probability of each classification. In the Yolo series of target detection models, the classification probability is obtained through feature extraction of the Yolo model, that is, operations such as convolution. However, due to the influence of the field of view of the convolution, the confidence and classification probability are not very accurate, and there may be missed detections. In order to improve the recognition accuracy, the present invention uses an attention mechanism to calculate the confidence of each bounding box when using a trained model. At the same time, in order to expand the receptive field, the probability of the disease type corresponding to the preset range centered on the grid is calculated, where the prediction range is determined according to the prior frame. Specifically: the union of all prior frames is calculated, and the union is used as the prediction range. Figure 3 As shown in the figure, each dotted box is a priori box, and the solid box is the prediction range. The union of all prior boxes as the prediction range can ensure that if a lesion appears in a partial area of ​​a prediction box, the receptive field can be expanded through the prediction range, thus improving the prediction accuracy of the disease type.

[0064] S2, converting the image to be identified taken by the capsule endoscope into a preset size through convolution, and then dividing it into a plurality of grids of the grid size, calculating the attention of all grids in each bounding box corresponding to each grid, and obtaining the confidence of each bounding box through the attention; calculating the probability of the corresponding disease type within the prediction range centered on the grid;

[0065] When recognizing an image to be recognized taken by a capsule endoscope, the image is converted to a preset size through convolution, and then divided into multiple grids according to the grid size. Each grid corresponds to multiple bounding boxes. The attention of the bounding box is calculated by the grids contained in the bounding box, and the confidence of each bounding box is calculated through the attention mechanism. Among them, the size of the bounding box is the same as the size of the prior box, and the number is also the same. The bounding box here can also be expressed as a prior box, but in order to distinguish the prior box obtained according to the annotation box in S1 and the bounding box actually used in S2, it is distinguished by name here; however, the actual bounding box is the same as the prior box. For example, if 5 prior boxes are obtained in S1, each grid will correspond to 5 bounding boxes of the same size as the prior box.

[0066] The attention mechanism can obtain more comprehensive information about each bounding box. Compared with simple convolution operations, the confidence of each bounding box calculated by the attention mechanism is more accurate. There are many ways to obtain the confidence of each bounding box by paying attention to all the grids of the bounding box. For example, according to the ViT model, the grid in the bounding box is subjected to attention mechanism to obtain CLS Token, and then the confidence is calculated by MLP; or after paying attention to the grid, the tensor is obtained and the confidence is further calculated by convolution and MLP.

[0067] In a more detailed embodiment, the attention of all grids in each bounding box corresponding to each grid is calculated for each grid, and the confidence of each bounding box is obtained through the attention, specifically:

[0068] Each grid in the bounding box is embedded through a linear projection layer, and a learning token (CLSToken) is added. The content of the learning token is learned through the attention mechanism, and the confidence of the bounding box is obtained through the content of the learning token. When identifying the image to be identified, the confidence represents the possibility that the bounding box contains a lesion.

[0069] Each grid also corresponds to a prediction of each disease, that is, the probability of the disease type. In order to expand the receptive field, the present invention uses the disease probability within the prediction range centered on the grid as the probability of the disease type, such as small intestinal bleeding, polyps, etc., and calculates the corresponding disease type probability within the prediction range centered on the grid. Specific methods include but are not limited to using convolutional networks to extract features, using MLP classification, etc.

[0070] Among them, the disease type probability refers to the probability of the disease to be identified. For example, if five types of lesions are to be identified, there are five disease types and the probability of each disease.

[0071] S3, filter the bounding boxes according to the disease type probability and confidence, use the IoU threshold to filter the remaining bounding boxes using non-maximum suppression, and use the filtered bounding boxes as the identified lesion target boxes.

[0072] After obtaining each bounding box (x, y, w, h) corresponding to each grid, the confidence of each bounding box, and the probability of each classification, the bounding boxes are filtered by non-maximum suppression to remove redundant bounding boxes. In order to improve the processing speed and avoid false positives, the bounding boxes are first filtered according to the probability and confidence of the disease type. If the predicted probabilities of all disease types are very low, then the possibility of the existence of lesions in all bounding boxes corresponding to this grid is very small, and the confidence of a bounding box is very low, so this bounding box can also be directly removed. In a specific embodiment, the filtering of bounding boxes according to the probability and confidence of disease type is specifically as follows:

[0073] The maximum value of the disease type probability is obtained. If the maximum value meets the conditions and the confidence is greater than the confidence threshold, the bounding box is retained; if the maximum value does not meet the conditions and / or the confidence is not greater than the confidence threshold, the bounding box is not retained.

[0074] For example, there are only two types of diseases, small intestinal bleeding and polyps, and the probabilities of the disease types corresponding to a grid are 0.1 and 0.2 respectively. Since the corresponding disease type probability within the prediction range centered on the grid has a large receptive field, but the corresponding disease type probability is still very small, it indicates that the probability of the disease existing in this grid is very small, and all bounding boxes corresponding to this grid can be removed. If the probability of one of the diseases is very high, such as the probability of polyps is 0.9, the confidence of all bounding boxes corresponding to the grid is further determined. If the confidence is greater than the confidence threshold, the bounding box is retained.

[0075] Non-maximum suppression (NMS) requires setting an IoU threshold, which is directly related to the final filtering effect. In practice, it is found that the probability of overlapping bounding boxes appearing at adjacent positions of the same image taken by capsule endoscopy is very low. The IoU threshold can be set very low or even 0. In a specific embodiment, the IoU threshold is specifically determined as follows:

[0076] Determine whether the annotation boxes have an intersection. If not, set the IoU threshold to 0; otherwise, calculate the average IoU value of all annotation boxes with intersections, and use the average value as the IoU threshold.

[0077] S2 and S3 are the process of using the image to be identified. Before use, it also includes the training of the model. During training, the confidence calculation of the bounding box corresponding to the grid is calculated based on the IoU between the predicted box and the labeled box and the probability that the bounding box contains a lesion. In the use stage, since there is no labeled box, the IoU is no longer calculated.

[0078] Embodiment 2, the present invention also improves a capsule endoscopy lesion recognition system based on artificial intelligence, such as Figure 4As shown, the system includes the following modules:

[0079] A priori frame and grid acquisition module is used to acquire images taken by capsule endoscopy, annotate the images after deduplication, cluster the annotated frames, obtain the priori frame and grid size according to the clustering results; and obtain the prediction range based on the priori frame;

[0080] A feature extraction module, used for converting the image to be identified taken by the capsule endoscope into a preset size through convolution, and then dividing it into a plurality of grids of the grid size, calculating the attention of all grids in each bounding box corresponding to the grid for each grid, and obtaining the confidence of each bounding box through the attention; calculating the probability of the corresponding disease type within the prediction range centered on the grid;

[0081] The bounding box filtering module is used to filter the bounding boxes according to the disease type probability and confidence, and use the IoU threshold to filter the remaining bounding boxes by non-maximum suppression, and use the filtered bounding boxes as the identified lesion target boxes.

[0082] Preferably, clustering the annotation boxes is specifically performed as follows:

[0083] Select one annotation box from all annotation boxes, calculate the ratio of the length of other annotation boxes to the length of the selected annotation box, and the ratio of the width of other annotation boxes to the width of the selected annotation box. If both ratios are within the set range, the other annotation boxes and the selected annotation box are classified into one category; and so on, until all annotation boxes are clustered;

[0084] The number of clusters is determined. If the number of clusters exceeds a preset value, several clusters with the smallest labeled boxes are deleted until the number of clusters does not exceed the preset value.

[0085] Preferably, the prior frame and grid size are obtained according to the clustering results, specifically:

[0086] For each cluster, calculate the average length and width of the annotation box in the cluster, and obtain the prior box corresponding to the cluster based on the two average values;

[0087] Get the minimum length and width of all prior boxes, which is a multiple of the grid length.

[0088] Preferably, the prediction range is obtained based on the prior frame, specifically:

[0089] Calculate the union of all prior boxes and use the union as the prediction range.

[0090] Preferably, the attention of all grids in each bounding box corresponding to each grid is calculated for each grid, and the confidence of each bounding box is obtained through the attention, specifically:

[0091] Each grid in the bounding box is passed through a linear projection layer to obtain an embedding vector, and the learning token is added. The content of the learning token is learned through the attention mechanism, and the confidence of the bounding box is obtained through the content of the learning token.

[0092] Preferably, filtering the bounding box according to the disease type probability and confidence level is specifically as follows:

[0093] The maximum value of the disease type probability is obtained. If the maximum value meets the conditions and the confidence is greater than the confidence threshold, the bounding box is retained; if the maximum value does not meet the conditions and / or the confidence is not greater than the confidence threshold, the bounding box is not retained.

[0094] Preferably, the IoU threshold is specifically determined as follows:

[0095] Determine whether the annotation boxes have an intersection. If not, set the IoU threshold to 0; otherwise, calculate the average IoU value of all annotation boxes with intersections, and use the average value as the IoU threshold.

[0096] Embodiment 3: The present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method described above is implemented.

[0097] Through the description of the above implementation methods, technicians in this field can clearly understand that each implementation method can be implemented by adding a necessary general hardware platform, and of course can also be implemented by combining hardware and software. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a computer product, and the present invention can be in the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0098] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it, and other embodiments may also be used. Although the present invention has been described in detail with reference to the aforementioned embodiments, ordinary technicians in this field should understand that they can still modify the technical solutions recorded in the aforementioned embodiments, or replace some of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A capsule endoscopy lesion recognition method based on artificial intelligence, characterized in that: The method comprises the following steps: Acquire an image taken by a capsule endoscope, annotate the image after deduplication, cluster the annotated frames, obtain a priori frames and a grid size according to the clustering results; and obtain a prediction range based on the priori frames; The image to be identified taken by the capsule endoscope is converted into a preset size through convolution, and then divided into a plurality of grids of the grid size, and for each grid, the attention of all grids in each bounding box corresponding to the grid is calculated, and the confidence of each bounding box is obtained through the attention; the probability of the corresponding disease type within the prediction range centered on the grid is calculated; Filter the bounding boxes according to the disease type probability and confidence, use the IoU threshold to filter the remaining bounding boxes by non-maximum suppression, and use the filtered bounding boxes as the identified lesion target boxes; The clustering of the annotation boxes is specifically as follows: Select one annotation box from all annotation boxes, calculate the ratio of the length of other annotation boxes to the length of the selected annotation box, and the ratio of the width of other annotation boxes to the width of the selected annotation box. If both ratios are within the set range, the other annotation boxes and the selected annotation box are classified into one category; and so on, until all annotation boxes are clustered; Determine the number of clusters. If the number of clusters exceeds the preset value, delete the clusters with the smallest marked boxes until the number of clusters does not exceed the preset value. The attention of all grids in each bounding box corresponding to each grid is calculated for each grid, and the confidence of each bounding box is obtained through the attention, specifically: Each grid in the bounding box is passed through a linear projection layer to obtain an embedding vector, and the learning token is added. The content of the learning token is learned through the attention mechanism, and the confidence of the bounding box is obtained through the content of the learning token.

2. The method according to claim 1, characterized in that The prior frame and grid size are obtained according to the clustering results, specifically: For each cluster, calculate the average length and width of the annotation box in the cluster, and obtain the prior box corresponding to the cluster based on the two average values; Get the minimum length and width of all prior boxes, which is a multiple of the grid length.

3. The method according to claim 1, characterized in that The prediction range is obtained based on the prior frame, specifically: Calculate the union of all prior boxes and use the union as the prediction range.

4. The method according to claim 1, characterized in that The filtering of the bounding box according to the disease type probability and confidence is specifically as follows: The maximum value of the disease type probability is obtained. If the maximum value meets the conditions and the confidence is greater than the confidence threshold, the bounding box is retained; if the maximum value does not meet the conditions and / or the confidence is not greater than the confidence threshold, the bounding box is not retained.

5. The method according to claim 1, characterized in that The specific method for determining the IoU threshold is: Determine whether the annotation boxes have an intersection. If not, set the IoU threshold to 0; otherwise, calculate the average IoU value of all annotation boxes with intersections, and use the average value as the IoU threshold.

6. A capsule endoscopy lesion recognition system based on artificial intelligence, characterized in that: The system includes the following modules: A priori frame and grid acquisition module is used to acquire images taken by capsule endoscopy, annotate the images after deduplication, cluster the annotated frames, obtain the priori frame and grid size according to the clustering results; and obtain the prediction range based on the priori frame; A feature extraction module, used for converting the image to be identified taken by the capsule endoscope into a preset size through convolution, and then dividing it into a plurality of grids of the grid size, and calculating the attention of all grids in each bounding box corresponding to each grid, and obtaining the confidence of each bounding box through the attention; Calculate the probability of the corresponding disease type within the prediction range centered on the grid; The bounding box filtering module is used to filter the bounding boxes according to the disease type probability and confidence, and use the IoU threshold to filter the remaining bounding boxes by non-maximum suppression, and use the filtered bounding boxes as the identified lesion target boxes; The clustering of the annotation boxes is specifically as follows: Select one annotation box from all annotation boxes, calculate the ratio of the length of other annotation boxes to the length of the selected annotation box, and the ratio of the width of other annotation boxes to the width of the selected annotation box. If both ratios are within the set range, the other annotation boxes and the selected annotation box are classified into one category; and so on, until all annotation boxes are clustered; Determine the number of clusters. If the number of clusters exceeds the preset value, delete the clusters with the smallest marked boxes until the number of clusters does not exceed the preset value. The attention of all grids in each bounding box corresponding to each grid is calculated for each grid, and the confidence of each bounding box is obtained through the attention, specifically: Each grid in the bounding box is passed through a linear projection layer to obtain an embedding vector, and the learning token is added. The content of the learning token is learned through the attention mechanism, and the confidence of the bounding box is obtained through the content of the learning token.

7. A computer storage device, wherein a computer program is stored on the storage device, characterized in that: When the computer program is executed by a processor, the computer program implements the method according to any one of claims 1 to 5.