Inflammatory bowel disease image detection method, device and equipment based on weak supervised learning

Abstract

The invention belongs to the technical field of image detection, and discloses an inflammatory bowel disease image detection method, device and equipment based on weak supervised learning, and the method comprises the steps: carrying out the segmentation of a full-view image of an inflammatory bowel disease through different scales, and segmenting the full-view image into a plurality of image blocks which are equal in size and are not overlapped at each scale; filtering the image blocks according to image quality to obtain a plurality of filtered image blocks; extracting features of the filtered image blocks by adopting a feature extraction model to obtain first feature vectors corresponding to the filtered image blocks; all the first feature vectors are converted into second feature vectors through self-attention; and classifying the second feature vector by using a linear classifier to obtain a detection result. The model is light in weight, low in equipment operation requirement and high in reasoning efficiency, the performance similar to that of a full supervision method can be obtained on the basis that pixel labeling is not needed, training convergence is fast, and training cost is low.

Description

Technical Field

[0001] The present invention belongs to the technical field of image detection, and specifically relates to an inflammatory bowel disease image detection method, device and equipment based on weakly supervised learning. Background Art

[0002] Inflammatory bowel disease (IBD) is a chronic nonspecific inflammatory disease that primarily affects the ileum, rectum, and colon, and is characterized by persistent gastrointestinal inflammation. Currently, mucosal biopsy is an important method for initial screening of IBD. However, the prevalence of IBD is slowly increasing, and there is a shortage of pathologists. Therefore, automatic IBD detection is needed for IBD biopsy pathology sections.

[0003] At present, the IBD pathology image assessment method based on deep learning mainly uses the mainstream convolutional neural network to divide the whole-field images (WSI) with huge resolution (the resolution can generally reach 100,000*100,000) into multiple small image blocks, and use these divided image blocks for training and classification. A huge amount of video memory is required during operation. In addition, annotation information of each image block (also called pixel-level annotation) is also required during training to achieve fully supervised training of the deep learning model. However, labeling each image block often requires a lot of manpower.

[0004] Therefore, current automatic detection methods for IBD images have high training costs and high operating requirements. Summary of the invention

[0005] The purpose of the present invention is to provide an inflammatory bowel disease image detection method, device, equipment and storage medium based on weakly supervised learning, which is used to solve the problems of high training cost and high operation requirements of automatic detection methods of IBD images.

[0006] The first aspect of the present invention discloses an inflammatory bowel disease image detection method based on weakly supervised learning, comprising:

[0007] Segmenting the full-field image of inflammatory bowel disease at different scales, and at each scale, segmenting the full-field image into a plurality of image blocks of equal size and non-overlapping;

[0008] Filter the image block according to image quality to obtain a plurality of filtered image blocks;

[0009] Extracting features of the filtered image block using a feature extraction model to obtain a first feature vector corresponding to the filtered image block;

[0010] Use self-attention to transform all first eigenvectors into second eigenvectors;

[0011] The second feature vector is classified using a linear classifier to obtain a detection result.

[0012] In some embodiments, the using self-attention to convert all first feature vectors into second feature vectors includes:

[0013] After calculating the self-attention of the first eigenvector at the same scale, global maximum pooling is used to compress it to obtain the self-attention features corresponding to each scale;

[0014] Attention pooling is used to compress all self-attention features to obtain the second feature vector.

[0015] In some embodiments, after calculating the self-attention for the first feature vector at the same scale, the method further includes:

[0016] Obtaining a self-attention value corresponding to each of the filtered image blocks according to the self-attention vector obtained by calculating the self-attention;

[0017] According to the self-attention value corresponding to the filtered image block, setting the image parameter of the filtered image block;

[0018] A heat map of the suspected lesion area is generated based on the image parameters of all filtered image patches.

[0019] In some embodiments, filtering the image block according to image quality to obtain a plurality of filtered image blocks includes:

[0020] Using a threshold segmentation algorithm, filtering the image blocks in which the proportion of white areas in the image exceeds a preset threshold, to obtain a filter set;

[0021] The image blocks with artifacts are filtered in the filter set using a pre-trained deep convolutional neural network model to obtain a plurality of filtered image blocks.

[0022] In some embodiments, the step of training the feature extraction model comprises:

[0023] In the training data set, image blocks are randomly selected for data augmentation, and multiple augmented views corresponding to the selected image blocks are obtained;

[0024] Extracting first features of each enhanced view using an encoder with shared parameters;

[0025] Performing nonlinear projection on the first feature using a fully connected layer to obtain a second feature of each enhanced view;

[0026] Based on the second feature of the enhanced view, a loss value is calculated according to the similarity between the enhanced views;

[0027] Parameters of the feature extraction model are optimized using momentum update according to the loss value.

[0028] In some embodiments, the using a linear classifier to classify the second feature vector to obtain a detection result includes:

[0029] Using a linear classifier to classify the second feature vector to obtain the probability of each classification;

[0030] According to the probability of each classification, the whole-slice image classification results and activity assessment results are obtained.

[0031] The second aspect of the present invention discloses an inflammatory bowel disease image detection device based on weakly supervised learning, comprising:

[0032] A segmentation module, used for segmenting the full-field image of inflammatory bowel disease at different scales, and at each scale, segmenting the full-field image into a plurality of image blocks of equal size and non-overlapping;

[0033] A quality filtering module, used for filtering the image blocks according to image quality to obtain a plurality of filtered image blocks;

[0034] A feature extraction module, used to extract features of the filtered image block using a feature extraction model to obtain a first feature vector corresponding to the filtered image block;

[0035] A self-attention conversion module, for converting all first feature vectors into second feature vectors using self-attention;

[0036] A classification module is used to classify the second feature vector using a linear classifier to obtain a detection result.

[0037] In some embodiments, the self-attention conversion module includes a first conversion unit and a second conversion unit. The first conversion unit is used to calculate the self-attention of the first eigenvector at the same scale and then use the global maximum pooling to compress it to obtain the self-attention features corresponding to each scale; the second conversion unit is used to use attention pooling to compress all self-attention features to obtain the second eigenvector.

[0038] The third aspect of the present invention discloses an electronic device, comprising a memory storing executable program code and a processor coupled to the memory; the processor calls the executable program code stored in the memory to execute the inflammatory bowel disease image detection method based on weak supervised learning disclosed in the first aspect.

[0039] The fourth aspect of the present invention discloses a computer-readable storage medium, which stores a computer program, wherein the computer program enables a computer to execute the inflammatory bowel disease image detection method based on weakly supervised learning disclosed in the first aspect.

[0040] The beneficial effect of the present invention is that the full-view image is first segmented at different scales, and then the first feature vectors are extracted from the segmented image blocks respectively, and then the self-attention mechanism is used to convert all the extracted first feature vectors into second feature vectors that can be used for classification, and finally a linear classifier is used to obtain the detection result. By combining the feature extraction of a single image block with the self-attention calculation of the features extracted from the image block, the model is lightweight, the equipment operation requirements are low, and the reasoning efficiency is high. Moreover, without the need for pixel annotation, performance similar to that of the full supervision method can be obtained, and the training convergence is fast and the training cost is low. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The accompanying drawings herein show specific examples of the technical solutions described in the present invention, and together with the specific implementation methods, constitute a part of the specification, and are used to explain the technical solutions, principles and effects of the present invention.

[0042] Unless otherwise specified or defined, the same figure marks in different drawings represent the same or similar technical features, and the same or similar technical features may also be represented by different figure marks.

[0043] Figure 1 It is a flow chart of an inflammatory bowel disease image detection method based on weakly supervised learning disclosed in an embodiment of the present invention;

[0044] Figure 2 is a schematic diagram of image segmentation and quality filtering according to an embodiment of the present invention;

[0045] Figure 3 is a flow chart of training a feature extraction model according to an embodiment of the present invention;

[0046] Figure 4 It is a functional framework diagram of a feature extraction model for training according to an embodiment of the present invention;

[0047] Figure 5 yes Figure 1 Flow chart of step S400 in the embodiment;

[0048] Figure 6 is a flow chart of generating a heat map of a suspected lesion area according to an embodiment of the present invention;

[0049] Figure 7 is a schematic diagram of comparison between a heat map of a suspected lesion area and an original map annotated by a pathologist according to an embodiment of the present invention;

[0050] Figure 8 Schematic diagram of the training process and reasoning process of a linear classifier according to an embodiment of the present invention;

[0051] Fig. 9 This is a schematic diagram of training loss-iteration steps in an embodiment of the present invention;

[0052] Fig.10 1 is a graph of ROC and Recall curves according to an embodiment of the present invention;

[0053] Fig.11 is a structural schematic diagram of an inflammatory bowel disease image detection device based on weakly supervised learning according to an embodiment of the present invention;

[0054] Fig.12 It is a structural schematic diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0055] Unless otherwise specified or defined, all technical and scientific terms used herein have the same meaning as those generally understood by those skilled in the art. In the case of combining the technical solution of the present invention with realistic scenarios, all technical and scientific terms used herein may also have meanings corresponding to the purpose of implementing the technical solution of the present invention. The "first, second..." used herein is only used to distinguish the names and does not represent a specific quantity or order. The term "and / or" used herein includes any and all combinations of one or more related listed items.

[0056] It should be noted that when a component is considered to be "fixed to" another component, it can be directly fixed to the other component or there can be a central component; when an component is considered to be "connected to" another component, it can be directly connected to the other component or there can be a central component at the same time; when an component is considered to be "installed on" another component, it can be directly installed on the other component or there can be a central component at the same time. When an component is considered to be "set on" another component, it can be directly set on the other component or there can be a central component at the same time.

[0057] Unless otherwise specified or defined, the "said" and "the" used in this document refer to the technical features or technical contents mentioned or described before the corresponding position, and the technical features or technical contents may be the same as or similar to the technical features or technical contents mentioned therein. In addition, the terms "including" and "having" and any variations thereof used in this document are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units inherent to these processes, methods, products or devices.

[0058] In order to reduce the training cost and operation requirements of automatic IBD image detection, the present invention segments the full-view image of IBD at different scales, then filters the image blocks with unqualified image quality, extracts the features of each filtered image block, uses self-attention to convert the extracted features into feature representations that can be used for classification, and then uses a linear classifier for classification. Without the need for pixel annotation, it can provide performance similar to that of the fully supervised method, with fast training convergence and low training cost, and the model is lightweight, with low video memory consumption and low operation requirements.

[0059] In order to facilitate the understanding of the present invention, specific embodiments of the present invention will be described in more detail below with reference to the accompanying drawings.

[0060] The embodiment of the present invention discloses an inflammatory bowel disease image detection method based on weakly supervised learning, which can be implemented by computer programming as an independent detection system. The execution subject of the method can be an electronic device such as a computer, a laptop, a tablet computer, or a control chip embedded in an electronic device, and the present invention is not limited to this.

[0061] like Figure 1 As shown, the method comprises the following steps:

[0062] Step S100: segmenting the full-field image of inflammatory bowel disease at different scales, and segmenting the full-field image into a plurality of image blocks of equal size and non-overlapping at each scale;

[0063] The whole-field image (hereinafter referred to as WSI) of inflammatory bowel disease refers to a high-resolution image (generally with a resolution of 100,000*100,000) generated by digitizing tissue sections. WSI files are large and require computing devices with high-performance storage and processing capabilities. Therefore, the whole-field image is usually segmented into multiple image blocks at different scales, and at each scale, these image blocks are equal in size and do not overlap. These image blocks can then be used for training and classification. For example, using a 50*50 segmentation scale, a 100,000*100,000 resolution full-field image is segmented into 2000*2000 image blocks. Obviously, the WSI can also be preprocessed before segmentation, such as using histogram equalization to improve image clarity or using Gaussian filtering, median filtering, and other methods to reduce noise.

[0064] Step S200: filtering the image blocks according to the image quality to obtain a plurality of filtered image blocks;

[0065] Since the WSI has a large resolution, most areas are white and contain artifacts caused by human beings during preparation, it is necessary to perform quality filtering on the original image blocks to reduce noise interference and speed up the extraction efficiency of the feature extraction model.

[0066] Specifically, a threshold segmentation algorithm is first used to filter image blocks in which the proportion of white areas in the image background exceeds a preset threshold (such as 90%), and the remaining image blocks constitute a filter set; then, a pre-trained deep convolutional neural network model is used to perform image recognition on each image block in the filter set. If an artifact is identified in the image block, the image block is removed from the filter set, and what remains in the filter set is the filtered image block.

[0067] refer to Figure 2 In this embodiment, the OTSU threshold segmentation algorithm is used to filter the image blocks with mostly white background, and the trained VGG-18 model is used to filter the image blocks with artifacts. The image blocks are input to the VGG-18 model to obtain the quality score. The specific expression is: P = f(patch), where f is the trained VGG-18 model, patch is the image block filtered by OTSU, and P is the quality score. When the quality score is less than the preset score, it is determined that the image block has artifacts and needs to be removed.

[0068] Step S300: extracting features of the filtered image block using a feature extraction model to obtain a first feature vector corresponding to the filtered image block;

[0069] There is no restriction on the architecture of the feature extraction model, and a convolutional neural network can be used. Since the WSI is scaled and quality filtered, the size of a single image block is not large. Compared with the feature extraction model directly extracting the features of the WSI, the weight and bias parameters of the feature extraction model for extracting a single image block are not large, the feature extraction model is lightweight, has low video memory consumption, and low inference cost.

[0070] In this embodiment, the feature extraction model is a ResNet50 model. During feature extraction, all filtered image blocks are embedded as vectors, input into the feature extraction model, and feature vectors of each filtered image block are extracted. When multiple WSIs are detected at the same time, all feature vectors of filtered image blocks from the same WSI are divided into a large group, denoted as G, and the feature vectors of filtered image blocks from the same scale in G are divided into a small group, denoted as g. The expression for feature extraction is: V = F (patch), where V is the first feature vector formed by ResNet50 embedding, F is the ResNet50 model, and patch is the filtered image block. After the ResNet50 model of this embodiment is trained, the size of the quantized ResNet50 model is only 32M iB, and the OpenVINO reasoning framework can even be used to infer in a pure CPU environment that does not rely on a GPU. In addition, when training the feature extraction model, it is only necessary to learn the feature representation of a single small image block, with low training cost, fast convergence, and high efficiency.

[0071] Specifically, Figure 3 As shown, the steps of training the feature extraction model include:

[0072] Step S310: randomly selecting image blocks in the training data set for data enhancement, and obtaining multiple enhanced views corresponding to the selected image blocks;

[0073] refer to Figure 4 During training, an image block is randomly sampled from the training data set as the input image, and multiple data augmentation operations are performed on the input image. In this embodiment, random cropping, rotation, and color jittering are selected to generate multiple enhanced views corresponding to the same image block. These enhanced views can be regarded as different representations (xi, xj) of the sampled image block, but they share the same semantic information.

[0074] Step S320: extracting first features of each enhanced view using an encoder with shared parameters;

[0075] These enhanced views are input into a parameter-sharing encoder (e.g., convolutional neural network ResNet50) using multiple branches. The encoder maps the original enhanced view into a high-dimensional feature space and extracts the first feature of each enhanced view. All first features can be exemplarily represented as (hi, hj).

[0076] Step S330: performing nonlinear projection on the first feature using a fully connected layer to obtain a second feature of each enhanced view;

[0077] The extracted first feature is nonlinearly projected through a fully connected layer. This nonlinear projection layer helps to map the feature into a space that is more suitable for contrastive learning, so as to obtain the second feature of each enhanced view. All the second features can be exemplarily represented as (zi, zj).

[0078] Step S340: Based on the second feature of the enhanced view, a loss value is calculated according to the similarity between the enhanced views;

[0079] Step S350: Adopt momentum to update and optimize the parameters of the feature extraction model according to the loss value.

[0080] According to the second features of each enhanced view, the similarity between different enhanced views is calculated to obtain a similarity value, that is, a loss value. For example, for two enhanced views of the same image, we hope that their second features are close to each other in the feature space; and for enhanced views of different images, we hope that their second features are far away from each other in the feature space. This embodiment uses the NT-Xent loss function to measure this similarity.

[0081] After the loss value is obtained, momentum update is used to train the feature extraction model, optimize the parameters of the feature extraction model, and continue to optimize after replacing the randomly sampled image blocks until the feature extraction model converges. Among them, momentum update is a conventional method in this field and will not be described in detail here.

[0082] Step S400: using self-attention to convert all first eigenvectors into second eigenvectors;

[0083] When a convolutional neural network is used to directly use a large number of image blocks for classification, the huge number of parameters formed by the convolutional neural network needs to occupy a huge amount of video memory, and the operating equipment requirements are high. Therefore, the present invention first extracts the first feature vector of each filtered image block, and then performs a self-attention operation. The more relevant the filtered image block is to the IBD, the higher the self-attention value is, and the second feature vector that can be used for classification is obtained, which can reduce the equipment operation requirements.

[0084] like Figure 5 As shown, the specific steps include:

[0085] Step S410: After calculating the self-attention of the first feature vector at the same scale, use the global maximum pooling to compress it to obtain the self-attention features corresponding to each scale;

[0086] For each first feature vector at each scale, self-attention calculation is performed to obtain a self-attention vector, and then the self-attention vector is compressed using global maximum pooling to obtain the self-attention features corresponding to each scale. Among them, self-attention calculation and global maximum pooling operations are conventional operations in this field and will not be described in detail here.

[0087] In this embodiment, for the group g obtained in step S300, the self-attention correlation of all the first feature vectors in g is calculated, and the calculation method is expressed as: h = HSA (g), where HSA represents the self-attention correlation operation. Then use the global maximum pooling to compress h to obtain the self-attention feature x that can represent the corresponding scale. The calculation method is expressed as: x = GAP (h), where GAP represents the global maximum pooling operation. Repeat the above operation for g of all scales to obtain the self-attention features x1, x2, and x3 corresponding to each scale.

[0088] Step S420: Use attention pooling to compress all self-attention features to obtain a second feature vector.

[0089] For all self-attention features of WSI, attention pooling is used to perform compression operations to obtain the second feature vector corresponding to WSI. When multiple WSIs are detected at the same time, the second feature vector corresponding to each WSI is obtained.

[0090] In this embodiment, attention pooling is used to compress feature vectors x1, x2, and x3 of all scales to obtain a second feature vector b that can be used for classification. Attention pooling is a technology that combines attention mechanism and pooling operation, and is a conventional technology used in deep learning models.

[0091] In some embodiments, after calculating the self-attention for the first feature vector at the same scale, such as Figure 6 As shown, it also includes:

[0092] Step S411: Obtain the self-attention value corresponding to each filtered image block according to the self-attention vector obtained by calculating the self-attention;

[0093] The self-attention vector includes the self-attention values ​​corresponding to multiple filtered image blocks. By searching in the self-attention vector, the self-attention value corresponding to each filtered image block can be obtained. The self-attention value can represent the correlation between the filtered image block and the IBD.

[0094] Step S412: setting image parameters of the filtered image block according to the self-attention value corresponding to the filtered image block;

[0095] Step S413: Generate a heat map of the suspected lesion area according to the image parameters of all filtered image blocks.

[0096] The self-attention value corresponding to each filtered image block is obtained, and the segmentation interval to which the self-attention value belongs is determined according to the preset numerical range of each segmentation interval. Each segmentation interval corresponds to a different image parameter, so the image parameters of the filtered image block can be determined. After the image parameters of all filtered image blocks are determined, the filtered image blocks are displayed in the image according to the image parameters to generate a heat map of the suspected lesion area.

[0097] refer to Figure 7 , Figure 7 The left part is the heat map of the suspected lesion area generated by this embodiment, which is different from the original image annotated by the pathologist ( Figure 7 The right part in the middle has a high degree of coincidence, indicating that the present invention can accurately output possible lesion areas. The outputted heat map of suspected lesion areas can provide a reference for pathologists.

[0098] Step S500: using a linear classifier to classify the second feature vector to obtain a detection result.

[0099] After the first eigenvector is converted into the second eigenvector using self-attention in step S400, the second eigenvector can accurately represent the IBD, and then the linear classifier MLP is used to classify the second eigenvector to obtain the classification results of the whole slice image, such as whether there is IBD and the area where IBD exists. In this embodiment, the activity assessment results are also obtained, such as mild activity period, moderate activity period, severe activity period, remission period, etc. Specifically, the second eigenvector is first classified using a linear classifier to obtain the probability of each classification, such as the probability of belonging to a normal area, a mild IBD area, a severe IBD area, etc.; then these probabilities are correspondingly counted and summarized, and the full slice image classification results and activity assessment results can be obtained.

[0100] The training process and reasoning process of this embodiment are as follows Figure 8 As shown, the difference between the training process and the inference process is that in the training process, after obtaining the full-slice image classification result, the full-slice image classification loss is calculated using the cross entropy function according to the full-slice image-level labels formed by the pre-annotated WSI and the full-slice image classification result, and the momentum update is used to optimize the parameters of the linear classifier until the linear classifier converges. Therefore, this embodiment does not require pixel annotation, and only image-level annotation can be used to complete the model training.

[0101] This example only requires one RTX4090 GPU (24GB video memory) to complete the training of about 1400 WSI images with a resolution of about (100,000*100,000) in a short time (20 minutes). Training loss-iteration number is as follows Fig. 9 As shown, from Fig. 9It can be seen from the figure that training can be completed in step-60, with fast convergence and low training cost. The ROC curve and Recall curve of the verification evaluation are shown in Figure 2. Fig.10 After training, the model size is only 32M i B after quantization, and OpenVINO (a toolkit designed to optimize and deploy deep learning models) reasoning framework can even be used in a pure CPU environment without relying on GPU.

[0102] In summary, by segmenting the full-view image, extracting the features of the segmented image blocks, and then using the self-attention mechanism to calculate the self-attention of the extracted features, and then performing global maximum pooling and attention pooling, the features of all image blocks are converted into feature representations that can be used for classification, and finally a linear classifier is used to obtain the detection results. This makes the model lightweight, has low equipment operation requirements, and high inference efficiency. Moreover, without the need for pixel annotation, it can achieve performance similar to that of the full supervision method, with fast training convergence and low training cost.

[0103] like Fig.11 As shown, based on the above-mentioned inflammatory bowel disease image detection method based on weakly supervised learning, an embodiment of the present invention discloses an inflammatory bowel disease image detection device based on weakly supervised learning, comprising:

[0104] A segmentation module 600 is used to segment the full-view image of inflammatory bowel disease at different scales, and to segment the full-view image into a plurality of image blocks of equal size and non-overlapping at each scale;

[0105] A quality filtering module 610, configured to filter the image block according to image quality to obtain a plurality of filtered image blocks;

[0106] A feature extraction module 620 is used to extract features of the filtered image block using a feature extraction model to obtain a first feature vector corresponding to the filtered image block;

[0107] A self-attention conversion module 630, configured to convert all first feature vectors into second feature vectors using self-attention;

[0108] The classification module 640 is used to classify the second feature vector using a linear classifier to obtain a detection result.

[0109] In some embodiments, the self-attention conversion module includes a first conversion unit and a second conversion unit. The first conversion unit is used to calculate the self-attention of the first eigenvector at the same scale and then use the global maximum pooling to compress it to obtain the self-attention features corresponding to each scale; the second conversion unit is used to use attention pooling to compress all self-attention features to obtain the second eigenvector.

[0110] like Fig.12 As shown, an embodiment of the present invention discloses an electronic device, including a memory 401 storing executable program codes and a processor 402 coupled to the memory 401;

[0111] The processor 402 calls the executable program code stored in the memory 401 to execute the inflammatory bowel disease image detection method based on weakly supervised learning described in the above embodiments.

[0112] An embodiment of the present invention further discloses a computer-readable storage medium storing a computer program, wherein the computer program enables a computer to execute the inflammatory bowel disease image detection method based on weakly supervised learning described in the above embodiments.

[0113] The purpose of the above embodiments is to exemplarily reproduce and deduce the technical solution of the present invention, and to fully describe the technical solution, purpose and effect of the present invention. Its purpose is to make the public understand the disclosed content of the present invention more thoroughly and comprehensively, and it does not limit the scope of protection of the present invention.

[0114] The above embodiments are not exhaustive enumerations of the present invention, and there may be multiple other implementations not listed. Any replacement and improvement made without violating the concept of the present invention shall fall within the protection scope of the present invention.

Claims

1. A method for detecting inflammatory bowel disease images based on weakly supervised learning, characterized in that: include: Segmenting the full-field image of inflammatory bowel disease at different scales, and at each scale, segmenting the full-field image into a plurality of image blocks of equal size and non-overlapping; Filter the image block according to image quality to obtain a plurality of filtered image blocks; Extracting features of the filtered image block using a feature extraction model to obtain a first feature vector corresponding to the filtered image block; Use self-attention to transform all first eigenvectors into second eigenvectors; The second feature vector is classified using a linear classifier to obtain a detection result.

2. The inflammatory bowel disease image detection method based on weakly supervised learning according to claim 1, characterized in that: The method of converting all first feature vectors into second feature vectors by using self-attention includes: After calculating the self-attention of the first eigenvector at the same scale, global maximum pooling is used to compress it to obtain the self-attention features corresponding to each scale; Attention pooling is used to compress all self-attention features to obtain the second feature vector.

3. The inflammatory bowel disease image detection method based on weakly supervised learning according to claim 2, characterized in that: After calculating the self-attention for the first eigenvector at the same scale, it also includes: Obtaining a self-attention value corresponding to each of the filtered image blocks according to the self-attention vector obtained by calculating the self-attention; According to the self-attention value corresponding to the filtered image block, setting the image parameter of the filtered image block; A heat map of the suspected lesion area is generated based on the image parameters of all filtered image patches.

4. The inflammatory bowel disease image detection method based on weakly supervised learning according to claim 1, characterized in that: The filtering the image blocks according to the image quality to obtain a plurality of filtered image blocks comprises: Using a threshold segmentation algorithm, filtering the image blocks in which the proportion of white areas in the image exceeds a preset threshold, to obtain a filter set; The image blocks with artifacts are filtered in the filter set using a pre-trained deep convolutional neural network model to obtain a plurality of filtered image blocks.

5. The inflammatory bowel disease image detection method based on weakly supervised learning according to claim 1, characterized in that: The steps of training the feature extraction model include: In the training data set, image blocks are randomly selected for data augmentation, and multiple augmented views corresponding to the selected image blocks are obtained; Extracting first features of each enhanced view using an encoder with shared parameters; Performing nonlinear projection on the first feature using a fully connected layer to obtain a second feature of each enhanced view; Based on the second feature of the enhanced view, a loss value is calculated according to the similarity between the enhanced views; Parameters of the feature extraction model are optimized using momentum update according to the loss value.

6. The inflammatory bowel disease image detection method based on weakly supervised learning according to claim 1, characterized in that: The using a linear classifier to classify the second feature vector to obtain a detection result includes: Using a linear classifier to classify the second feature vector to obtain the probability of each classification; According to the probability of each classification, the whole-slice image classification results and activity assessment results are obtained.

7. An inflammatory bowel disease image detection device based on weakly supervised learning, characterized in that: include: A segmentation module, used for segmenting the full-field image of inflammatory bowel disease at different scales, and at each scale, segmenting the full-field image into a plurality of image blocks of equal size and non-overlapping; A quality filtering module, used for filtering the image blocks according to image quality to obtain a plurality of filtered image blocks; A feature extraction module, used to extract features of the filtered image block using a feature extraction model to obtain a first feature vector corresponding to the filtered image block; A self-attention conversion module, for converting all first feature vectors into second feature vectors using self-attention; A classification module is used to classify the second feature vector using a linear classifier to obtain a detection result.

8. The inflammatory bowel disease image detection device based on weakly supervised learning according to claim 7, characterized in that: The self-attention conversion module includes a first conversion unit and a second conversion unit, wherein the first conversion unit is used to calculate the self-attention of the first feature vector at the same scale and then use the global maximum pooling to compress it to obtain the self-attention features corresponding to each scale; The second conversion unit is used to use attention pooling to compress all self-attention features to obtain the second feature vector.

9. An electronic device, characterized in that: It includes a memory storing executable program code and a processor coupled to the memory; the processor calls the executable program code stored in the memory to execute the inflammatory bowel disease image detection method based on weakly supervised learning as described in any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program enables a computer to execute the inflammatory bowel disease image detection method based on weakly supervised learning according to any one of claims 1 to 6.

Images