Methods, systems, media, and equipment for detecting abnormalities in esophageal endoscopic images.

Abstract

This invention relates to a method, system, medium, and device for anomaly detection in esophageal endoscopic images, comprising: acquiring and preprocessing esophageal endoscopic images; reconstructing the preprocessed esophageal endoscopic images using a trained image reconstruction model; obtaining an anomaly score for each esophageal endoscopic image based on the reconstruction error between the reconstructed image and the original image; and determining whether the esophageal endoscopic image is abnormal based on a set threshold and the anomaly score. The image reconstruction model is a multi-scale feature fusion clustering memory variational autoencoder, trained using healthy images from the esophageal endoscopic images as the training set. The model can be constructed and trained using only normal samples, and the accuracy of anomaly detection is greatly improved through multi-scale feature fusion, memory modules, clustering, and other techniques.

Description

Technical Field

[0001] This invention relates to the field of computer-aided diagnostic technology, specifically to methods, systems, media, and devices for detecting abnormalities in esophageal endoscopy images. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] Esophageal endoscopy is an important tool for examining and locating diseases such as esophageal tumors. Through esophageal endoscopic images, doctors can visually examine the location, extent, and morphology of lesions in the digestive tract mucosa, thereby making accurate judgments. In large-scale screening, for diseases such as early-stage esophageal cancer that lack obvious clinical symptoms, computer-aided diagnostic technology is often relied upon. This technology uses computers to identify details in esophageal endoscopic images, assisting doctors in diagnosis and reducing their workload.

[0004] Computer-aided diagnostic technology processes raw medical image data using computers to identify and output possible results. Taking commonly used deep neural networks as an example, their training generally requires a large amount of labeled, class-balanced datasets; otherwise, overfitting and other problems are highly likely. For medical images, this is often extremely costly or difficult to obtain. For instance, in a large-scale physical examination or screening, a large number of healthy esophageal endoscopy images are often collected, while only a very small number of images of diseased individuals are collected. Images of some low-incidence diseases may not be collected at all. This means that traditional deep learning models struggle to learn the characteristics of these diseases, potentially misclassifying them as normal images in the future. Furthermore, all images must be labeled by professional physicians before they can be used for training, which is also a very costly task. Summary of the Invention

[0005] To address the technical problems mentioned above, this invention provides a method, system, medium, and device for anomaly detection in esophageal endoscopy images. The model can be constructed and trained using only normal samples, and the accuracy of anomaly detection is greatly improved through multi-scale feature fusion, memory modules, clustering, and other techniques.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] The first aspect of the present invention provides a method for detecting abnormalities in esophageal endoscopic images, comprising the following steps:

[0008] Esophageal endoscopy images are acquired and preprocessed. The preprocessed esophageal endoscopy images are reconstructed using a trained image reconstruction model. An anomaly score is obtained for each esophageal endoscopy image based on the reconstruction error between the reconstructed image and the original image.

[0009] The esophageal endoscopy image is determined to be abnormal based on the set threshold and abnormal score.

[0010] The image reconstruction model is a clustering memory variational autoencoder with multi-scale feature fusion, and is trained using healthy images from esophageal endoscopy images as the training set.

[0011] Clustering memory variational autoencoders with multi-scale feature fusion include:

[0012] The multi-scale encoder module is used to extract features from esophageal endoscopy images at different resolutions. It has multiple encoders, each of which outputs the variance σ and mean μ of a Gaussian distribution.

[0013] The clustering memory module includes multiple memory vectors of the same dimension, with the dimension of the memory vectors being the same as the dimension of the encoded features. The input is the variance σ and mean μ of each encoder output, and the output is the weighted sum of the memory vectors.

[0014] The multi-scale feature fusion module fuses the output of the clustering memory module to obtain the fused variance σ and mean μ, which are used as the sampling distribution of the decoder module.

[0015] The decoder module takes a vector randomly sampled from a Gaussian distribution obtained from the multi-scale feature fusion module as input, and decodes it through a multi-layer neural network to obtain an image with the same resolution as the original image.

[0016] The clustering memory module is a two-dimensional matrix with multiple memory vectors of the same dimension. The memory vectors only remember the features of normal samples, and the output is only a weighted sum of the features of normal samples.

[0017] The clustering memory module contains a clustering algorithm that optimizes the distribution of memory vectors in the feature space using a scattering matrix.

[0018] Preprocessing includes splitting the dataset into a training set and a test set, with the training set containing only healthy images.

[0019] Image reconstruction models are trained using a training set and a loss function, which includes:

[0020] Reconstruction error loss is used to ensure the similarity between the original image and the reconstructed image;

[0021] The regularization term is the KL divergence between the encoded Gaussian distribution and the standard normal distribution;

[0022] Clustering loss function is used to optimize the distribution of memory vectors in the clustering memory module.

[0023] An anomaly score is obtained for each esophageal endoscopy image based on the reconstruction error between the reconstructed image and the original image, specifically:

[0024] Adjust the resolution of the esophageal endoscope image to be detected and input it into the encoder of the trained image reconstruction model.

[0025] Once the reconstructed image is obtained, the reconstruction error between the image and the original image is calculated, which is the clustering loss function.

[0026] The reconstruction error is normalized to obtain the anomaly score for each image to be detected, calculated using the following formula: Where e i e min and e max These represent the reconstruction error of the sample, the smallest reconstruction error among all samples, and the largest reconstruction error among all samples, respectively.

[0027] A second aspect of the present invention provides a system for implementing the above-described method, comprising:

[0028] The anomaly score module is configured to: acquire and preprocess esophageal endoscopy images, reconstruct the preprocessed esophageal endoscopy images using a trained image reconstruction model, and obtain an anomaly score for each esophageal endoscopy image based on the reconstruction error between the reconstructed image and the original image.

[0029] The image judgment module is configured to: determine whether the esophageal endoscopy image is abnormal based on a set threshold and an abnormality score;

[0030] The image reconstruction model is a clustering memory variational autoencoder with multi-scale feature fusion, and is trained using healthy images from esophageal endoscopy images as the training set.

[0031] A third aspect of the present invention provides a computer-readable storage medium.

[0032] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the above-described method for detecting abnormalities in esophageal endoscopic images.

[0033] A fourth aspect of the present invention provides a computer device.

[0034] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps in the above-described method for detecting abnormalities in esophageal endoscopy images.

[0035] Compared with existing technologies, one or more of the above technical solutions have the following beneficial effects:

[0036] 1. It is an unsupervised algorithm that only requires healthy images from esophageal endoscopy to complete the model training, without needing images with lesions. This reduces the difficulty of data collection and labeling costs, and it can detect all images with features different from healthy images. That is, it has a good detection effect on all abnormal esophageal states, effectively improving the problem that traditional classification models cannot cover all esophageal image states.

[0037] 2. Multi-scale feature fusion technology was used during encoding. By changing the size of the input image, features of healthy esophageal images at various scales were extracted, thereby obtaining more feature information and achieving better anomaly detection results.

[0038] 3. A clustering memory module is introduced into the encoded features, instead of direct decoding. The decoder input is always a weighted sum of memory vectors, but the memory module only contains features from healthy images, which further reduces the model's reconstruction performance for abnormal images and inhibits the model's generalization ability.

[0039] 4. The clustering algorithm in the clustering memory module can optimize the distribution of memory vectors in the feature space, better remember the features of healthy images, and improve the anomaly detection effect. Attached Figure Description

[0040] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0041] Figure 1 This is a schematic diagram of the abnormal detection process of esophageal endoscopy images provided in one or more embodiments of the present invention;

[0042] Figure 2 This is a schematic diagram of a clustering memory variational autoencoder network structure used in the anomaly detection process of esophageal endoscopy images provided by one or more embodiments of the present invention. Detailed Implementation

[0043] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0044] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0045] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0046] As described in the background section, when using deep learning models for computer-aided diagnosis, it relies on a large amount of labeled raw image data. On the one hand, the data needs to be labeled by doctors, and on the other hand, images of some uncommon diseases are difficult to obtain, resulting in high costs and low accuracy in assisted diagnosis.

[0047] Anomaly detection is the process of identifying anomalous samples among a large number of normal samples when there is an imbalance between positive and negative samples. The training set contains only normal samples without labels, meaning a binary classification model is built using only one type of data. Anomaly detection models can perfectly solve the problems of difficult dataset collection and high annotation costs faced in esophageal endoscopy image classification tasks. A binary classification model can be built using only healthy images, and it can be applied to the initial screening of large-scale endoscopic images, assisting doctors in diagnosis, and further annotation of data.

[0048] Currently, there are four main types of anomaly detection models: distribution-based, reconstruction-based, pseudo-anomaly enhancement-based, and distillation learning-based. Experiments have shown that esophageal endoscopy images have the characteristic that healthy images are better reconstructed than abnormal images. Therefore, the following examples adopt the reconstruction-based method.

[0049] Anomaly detection methods based on reconstruction mainly use autoencoders or variational autoencoders as their basic structure. They are trained using only normal data so that the system can only learn the features of normal data. As a result, the reconstruction error of normal data is lower than that of abnormal data, and thus abnormal data can be detected.

[0050] However, many studies have shown that deep neural networks have extremely strong generalization capabilities, and can learn features from similar data even if the data did not appear in the training. However, some healthy and diseased esophageal endoscopy images are very similar. Therefore, the most advanced solution to this problem is to add a memory module between the encoder and decoder to suppress the generalization ability of the model. However, optimizing the distribution of memory vectors in the memory module is difficult. If the optimization strategy is not ideal, the model will not be able to fully remember the features of normal samples, resulting in excessive reconstruction error of normal samples, or it will learn abnormal features, resulting in excessively small re-encounter error of abnormal samples.

[0051] Therefore, the following embodiments provide a method, system, medium, and device for anomaly detection in esophageal endoscopy images. The model can be constructed and trained under the condition that only normal samples are available. Furthermore, the accuracy of anomaly detection is greatly improved through multi-scale feature fusion, memory modules, clustering, and other techniques.

[0052] Example 1:

[0053] like Figures 1-2 As shown, the method for detecting abnormalities in esophageal endoscopic images includes the following steps:

[0054] Acquire esophageal endoscopy images and perform preprocessing;

[0055] Obtain the abnormality score for each esophageal endoscopy image;

[0056] Set an appropriate threshold, and use this threshold and the abnormality score of the image to determine whether the esophageal endoscopy image is abnormal;

[0057] Specifically, the abnormality score of each esophageal endoscope image is obtained by reconstructing the esophageal endoscope image using a clustering memory variational autoencoder trained with multi-scale feature fusion, and obtaining the abnormality score of each esophageal endoscope image based on the reconstruction error.

[0058] Specifically:

[0059] S1: Acquire esophageal endoscopy images, then preprocess the endoscopy images to obtain training and test sets;

[0060] S2: Initialize the neural network framework for training;

[0061] S3: Input the training set images obtained in step S1 into the neural network framework, and use the loss function to complete the training of the neural network;

[0062] S4: Use the neural network trained in step S3 to calculate the anomaly score for each esophageal endoscopy image in the test set or to be detected.

[0063] S5: Set an appropriate threshold based on the proportion of abnormal images or actual needs, and classify each image according to this threshold and the abnormal score obtained in step S4 to detect abnormal images.

[0064] Step S1 specifically includes:

[0065] S11: Collect esophageal endoscopy images, which can be collected in large-scale physical examinations or by directly using public datasets;

[0066] S12: Preprocess the collected esophageal endoscopy images, including removing blurry images and adjusting all images to an appropriate size;

[0067] S13: Divide the data into a training set and a test set, where the training set contains only healthy images;

[0068] S14: If there are not enough training images, data augmentation can be performed, that is, operations such as rotating or mirroring the images can be performed to increase the number of images.

[0069] Step S2 specifically includes:

[0070] S21: Constructing the neural network framework, specifically, constructing a multi-scale feature fusion clustering memory variational autoencoder. The constructed multi-scale feature fusion clustering memory variational autoencoder consists of four parts: a multi-scale encoder module, a clustering memory module, a multi-scale feature fusion module, and a decoder module. The specific structure is as follows: Figure 2 As shown.

[0071] The multi-scale encoder module has multiple encoders of different sizes for feature extraction from input images at different resolutions. Each encoder independently outputs a Gaussian distribution with variance σ and mean μ. Compared to ordinary variational autoencoders, multi-scale encoding can extract features from input images at different resolutions, thus achieving multi-scale feature extraction and obtaining more complete feature information from the original image.

[0072] The clustering memory module is a two-dimensional matrix composed of multiple memory vectors of the same dimension, identical to the dimension of the encoded features. Each encoder is followed by two identical clustering memory modules to map the encoded variance σ and mean μ. The output of the clustering memory module is a weighted sum of the memory vectors. Since the training set contains only normal samples, the memory vectors can only remember the features of normal samples, and the output can only be a weighted sum of normal sample features, thus suppressing the model's generalization ability and reducing its reconstruction performance for outlier samples. The clustering memory module also incorporates a clustering algorithm, optimizing the distribution of memory vectors in the feature space through a scattering matrix. This makes the features stored in the memory module more extensive and accurate, further improving the model's performance.

[0073] The multi-scale feature fusion module uses a splicing operation to fuse the output features of all clustering memory modules, obtaining the fused Gaussian distribution variance σ and mean μ, which serve as the sampling distribution for the decoder.

[0074] The decoder takes a vector randomly sampled from a Gaussian distribution obtained by the multi-scale feature fusion module as input. This vector is then decoded through a multi-layer neural network to obtain an image with the same resolution as the original image. The decoder's network structure ensures that the image obtained after decoding the fused features has the same dimensionality as the original image.

[0075] S22: Complete the construction and initialization of a clustering memory variational autoencoder with multi-scale feature fusion. Select appropriate parameters, including network dimension, number of network layers, convolutional kernel size, stride, activation function, latent variable feature dimension, and number of memory vectors, to complete the construction of the neural network model and initialize it randomly.

[0076] Step S3 specifically includes:

[0077] S31: Adjust the resolution of the input raw esophageal endoscopy image to obtain multiple images of different resolutions that are adapted to the dimensions of the multi-scale encoder, and input them into encoders of different scales respectively;

[0078] S32: The encoder neural network calculates and transmits the input image. The encoder consists of multiple layers of neural networks. Each layer of neural network consists of multiple neurons. Each neuron contains parameters weight ω, bias b and activation function f.

[0079] The formula for calculating neurons is y = f(∑ωx). i +), where x i y is the output of each neuron in the previous layer, which is also the input of this neuron. y is the output of this neuron, which will be passed to the next layer of the neural network as the input of each neuron in the next layer.

[0080] S33: Each encoder will ultimately output two feature vectors: σ and μ, representing the variance and mean of a Gaussian distribution, respectively. These feature vectors are then fed into the clustering memory module; assuming the input feature vector is z.

[0081] First, calculate its cosine similarity with each memory vector: Where m j For memory vectors;

[0082] Then, the weights of each memory vector are obtained through the Softmax operation:

[0083]

[0084] Finally, the output of the memory module is obtained by weighted summation, which is the feature vector after memory mapping:

[0085]

[0086] S33: The mapped feature vectors enter the feature fusion module, where a concatenation operation is used to fuse multiple feature vectors of different scales, ultimately obtaining a Gaussian distribution with variance σ and mean μ. Random sampling is performed in this Gaussian distribution to obtain the vector to be decoded.

[0087] S34: Input the vector to be decoded obtained in step S33 into the decoder, and obtain the reconstructed esophageal endoscopy image after decoding. The decoder is also composed of a multi-layer neural network. It should be noted that the parameter settings of the neural network and convolution kernel of the decoder should be such that the resolution of the reconstructed image obtained by decoding is consistent with that of the original image.

[0088] S35: Calculate the loss function. The loss function of this model consists of three parts.

[0089] The first part is the reconstruction error loss, used to ensure that the original image and the reconstructed image are as similar as possible. Assuming the image resolution is m×n, the formula for calculating the reconstruction error loss function is: Where x ij and y ij These are the pixels of the original image and the reconstructed image, respectively.

[0090] The second part of the loss function is the regularization term, which is the KL divergence between the encoded Gaussian distribution and the standard normal distribution. Because variational autoencoders assume the prior distribution of the latent variables is a standard normal distribution, the encoded distribution should be as similar as possible to the standard normal distribution. The formula for this term in the loss function is:

[0091]

[0092] Where σ and μ are the variance and mean of the Gaussian distribution obtained from the encoding, respectively.

[0093] The third term of the loss function is the clustering loss function, which is used to optimize the distribution of memory vectors in the clustering memory module. The purpose of the clustering operation is to improve the sparsity of the memory vectors in space and make them closer to the original feature vectors. The memory vectors are used as cluster centers, and the original feature vectors are used as samples to be clustered. The scattering matrix is ​​used to measure the clustering results.

[0094] Assume the cluster centers of the two classes are m respectively. i and m j Then the scattering matrix between these two types is The total inter-class scattering matrix is:

[0095]

[0096] Where N is the total number of vectors; the centroid of all classes is... Where K is the number of memory vectors; for a single class, assuming the vector to be clustered (i.e., the feature vector of the input memory module) is z, then the intra-class scattering matrix is... Where n j The number of eigenvectors clustered into this class; the total in-class scattering matrix of all classes is Using the scattering matrix mentioned above, the formula for calculating the clustering loss function can be obtained as follows:

[0097] The model's total loss function is the weighted sum of the three loss functions mentioned above. It should be noted that, due to the use of a multi-scale feature extraction method, multiple Gaussian distributions of features will be obtained. The KL divergence will then be calculated with the standard normal distribution for each feature. Therefore, the second loss function will have multiple terms.

[0098] The formula for calculating the total loss function is as follows: Where T is the number of multi-scale encoders, and λ1, λ2 and λ3 are the weight parameters for balancing the three loss functions.

[0099] S36: Train the neural network using the loss function obtained in S35. Repeat steps S31 to S35 using the data in the training set, continuously optimizing the parameters in the neural network along the direction of gradient descent of the loss function, so that the value of the loss function continuously shrinks until it converges to a certain value and tends to remain constant.

[0100] Step S4 specifically includes:

[0101] S41: Adjust the resolution of the esophageal endoscope image to be detected to obtain multiple images of different resolutions that are adapted to the dimensions of the multi-scale encoder, and input them into the different scale encoders of the trained model respectively.

[0102] S42: Obtain the image reconstructed by the model, and calculate the reconstruction error between the original image and the original image, which is the first term in the loss function;

[0103] S43: After obtaining the reconstruction errors of all samples to be detected, normalize these data to obtain the anomaly score corresponding to each image to be detected. The calculation formula is as follows: Where e i e min and e max These represent the reconstruction error of the sample, the smallest reconstruction error among all samples, and the largest reconstruction error among all samples, respectively.

[0104] Step S5 specifically includes:

[0105] S51: Set a threshold. The threshold can be set according to the proportion of known abnormalities in the detected samples or the needs of the actual task. For example, if it is known that 70% of the esophageal endoscopy images are healthy, the 0.7 quantile of the abnormality score can be set as the threshold; if the task goal is to assist doctors in detecting patients with diseases, a smaller threshold should be set to filter out as many abnormal samples as possible; if the task goal is to filter normal images for dataset expansion and further training, a larger threshold should be set to make full use of every image.

[0106] S52: Classify the detected images based on the anomaly score obtained in step S43 and the threshold set in step S51. Determine samples with anomaly scores higher than the threshold as abnormal and samples with anomaly scores lower than the threshold as normal, thus completing the anomaly detection task.

[0107] The above process is an unsupervised algorithm, which only requires healthy esophageal endoscopy images to complete the model training, greatly reducing the difficulty of data collection and annotation costs; it can detect all images with features different from healthy images, that is, it has a good detection effect on all esophageal diseases, effectively improving the problem that traditional classification models cannot cover all esophageal diseases.

[0108] Multi-scale feature fusion technology was used during encoding. By changing the size of the input image, features of healthy esophageal images at various scales were extracted, thereby obtaining more feature information and achieving better anomaly detection results.

[0109] A clustering memory module is introduced for the encoded features, eliminating direct decoding. The decoder's input is always a weighted sum of memory vectors, but the memory module only contains features from healthy images, further reducing the model's reconstruction performance for abnormal images and suppressing its generalization ability. Furthermore, the clustering algorithm in the clustering memory module optimizes the distribution of memory vectors in the feature space, better remembering features from healthy images and improving anomaly detection.

[0110] Example 2:

[0111] A system for implementing the above method includes:

[0112] 11. An anomaly detection system for endoscopic images, characterized in that it comprises:

[0113] The anomaly score module is configured to: acquire and preprocess esophageal endoscopy images, reconstruct the preprocessed esophageal endoscopy images using a trained image reconstruction model, and obtain an anomaly score for each esophageal endoscopy image based on the reconstruction error between the reconstructed image and the original image.

[0114] The image judgment module is configured to: determine whether the esophageal endoscopy image is abnormal based on a set threshold and an abnormality score;

[0115] The image reconstruction model is a clustering memory variational autoencoder with multi-scale feature fusion, and is trained using healthy images from esophageal endoscopy images as the training set.

[0116] The above system is an unsupervised algorithm that only requires healthy esophageal endoscopy images to complete the model training, which greatly reduces the difficulty of data collection and annotation costs. Moreover, it can detect all images with features different from healthy images, that is, it has a good detection effect on all esophageal diseases, effectively improving the problem that traditional classification models cannot cover all esophageal diseases.

[0117] Example 3:

[0118] This embodiment provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps in the abnormal detection method for esophageal endoscopy images as described in Embodiment 1 above.

[0119] The above method is an unsupervised algorithm that only requires healthy esophageal endoscopy images to complete the model training, which greatly reduces the difficulty of data collection and labeling costs. Moreover, it can detect all images with features different from healthy images, that is, it has a good detection effect on all esophageal diseases, effectively improving the problem that traditional classification models cannot cover all esophageal diseases.

[0120] Example 4:

[0121] This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps in the abnormal detection method of esophageal endoscopy images as described in Embodiment 1 above.

[0122] The above method is an unsupervised algorithm that only requires healthy esophageal endoscopy images to complete the model training, which greatly reduces the difficulty of data collection and labeling costs. Moreover, it can detect all images with features different from healthy images, that is, it has a good detection effect on all esophageal diseases, effectively improving the problem that traditional classification models cannot cover all esophageal diseases.

[0123] The steps or modules involved in Embodiments 2 to 4 above correspond to those in Embodiment 1. For specific implementation details, please refer to the relevant description section of Embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood as including any medium capable of storing, encoding, or carrying an instruction set for execution by a processor and enabling the processor to perform any of the methods in this invention.

[0124] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method of detecting an abnormality of an esophagus endoscope image, characterized by, The method comprises the following steps: obtaining an esophagoscope image and preprocessing, reconstructing the preprocessed esophagoscope image by using a trained image reconstruction model, and obtaining an abnormal score of each esophagoscope image according to a reconstruction error between the reconstructed image and the original image; determining whether the esophagoscope image is abnormal according to a set threshold value and the abnormal score; wherein the image reconstruction model is a multi-scale feature fusion clustering memory variational autoencoder, and the training is completed by taking healthy images in the esophagoscope image as a training set; the multi-scale feature fusion clustering memory variational autoencoder comprises: A multi-scale encoder module for extracting features in an endoscopic image of an esophagus at different resolutions, having a plurality of encoders each outputting a variance and a mean of a Gaussian distribution The clustering memory module comprises a plurality of memory vectors of the same dimension, the dimension of the memory vector is the same as the dimension of the encoded feature, the input is the variance of each encoder output and the mean , and the output is the weighted sum of the memory vectors; the clustering memory module is a two-dimensional matrix with a plurality of memory vectors of the same dimension, the memory vector only remembers the features of normal samples, and the output is only the weighted sum of the features of normal samples; a multi-scale feature fusion module that fuses the outputs of the clustering memory modules to obtain a fused variance and mean as a sampling distribution of the decoder module; a decoder module, which inputs a vector randomly sampled from a Gaussian distribution obtained from the multi-scale feature fusion module, and decodes through a multi-layer neural network to obtain an image with the same resolution as the original image.

2. The abnormality detection method of an esophageal endoscopic image according to claim 1, characterized by, The clustering memory module has a clustering algorithm, and the optimization of the distribution of the memory vector in the feature space is completed by a scattering matrix.

3. The method of detecting an abnormality of an esophageal endoscopic image according to Claim 1, wherein The preprocessing includes dividing the training set and the test set, wherein the training set only contains healthy images.

4. The method of detecting an abnormality of an esophageal endoscopic image according to Claim 1, wherein The image reconstruction model is trained by using the training set and a loss function, and the loss function comprises: a reconstruction error loss for ensuring the similarity between the original image and the reconstructed image; a regularization term, which is the KL divergence between the Gaussian distribution obtained by encoding and the standard normal distribution; a clustering loss function for optimizing the distribution of the memory vector in the clustering memory module.

5. The method of detecting an abnormality of an esophageal endoscopic image according to Claim 1, wherein The abnormal score of each esophagoscope image is obtained according to the reconstruction error between the reconstructed image and the original image, specifically: adjusting the resolution of the esophagoscope image to be detected, and inputting it into the encoder of the trained image reconstruction model respectively; obtaining the image reconstructed by the model, and calculating the reconstruction error with the original image, i.e. the clustering loss function; The obtained reconstruction error is normalized to obtain an anomaly score corresponding to each image to be detected, and the formula is: wherein , and are the reconstruction error of the sample, the minimum reconstruction error in all samples, and the maximum reconstruction error in all samples, respectively.

6. An abnormality detection system for an esophageal endoscope image, characterized by comprise: an abnormal score module configured to obtain an esophagoscope image and preprocess, reconstruct the preprocessed esophagoscope image by using a trained image reconstruction model, and obtain an abnormal score of each esophagoscope image according to a reconstruction error between the reconstructed image and the original image; an image determination module configured to determine whether the esophagoscope image is abnormal according to a set threshold value and the abnormal score; wherein the image reconstruction model is a multi-scale feature fusion clustering memory variational autoencoder, and the training is completed by taking healthy images in the esophagoscope image as a training set; the multi-scale feature fusion clustering memory variational autoencoder comprises: A multi-scale encoder module for extracting features in an endoscopic image of an esophagus at different resolutions, having a plurality of encoders each outputting a variance and a mean of a Gaussian distribution The clustering memory module comprises a plurality of memory vectors of the same dimension, the dimension of the memory vector is the same as the dimension of the encoded feature, the input is the variance of each encoder output and the mean , and the output is the weighted sum of the memory vectors; the clustering memory module is a two-dimensional matrix with a plurality of memory vectors of the same dimension, the memory vector only remembers the features of normal samples, and the output is only the weighted sum of the features of normal samples; a multi-scale feature fusion module that fuses the outputs of the clustering memory modules to obtain a fused variance and mean as a sampling distribution of the decoder module; a decoder module, which inputs a vector randomly sampled from a Gaussian distribution obtained from the multi-scale feature fusion module, and decodes through a multi-layer neural network to obtain an image with the same resolution as the original image.

7. A computer-readable storage medium, characterized in that, A computer program is stored thereon, which is executed by a processor to implement the steps in the esophagoscope image abnormality detection method according to any one of claims 1-5.

8. A computer device, comprising: A memory, a processor, and a computer program stored on the memory and executable on the processor are included, and the processor implements the steps in the esophagoscope image abnormality detection method according to any one of claims 1-5 when executing the program.

Images