Automatic atrophic gastritis identification system based on generative adversarial network

Abstract

The invention belongs to the technical field of medical image automatic identification, and particularly relates to an atrophic gastritis automatic identification system based on a generative adversarial network, which comprises a data acquisition module used for acquiring a gastroscope image of a required part, and marking the part of the gastroscope image, the atrophy condition and the atrophy area information to obtain a gastroscope image data set; the double sampling module is used for carrying out preprocessing and double sampling on the gastroscope image data set to obtain a balanced gastroscope image data set; the model training module is used for respectively training a Murmuraea bamboo typing model and an atrophy area identification model by utilizing the balanced gastroscope image data set; the combined recognition module is used for processing the gastroscope image to be detected by utilizing the Murmun bamboo typing model and the atrophic area recognition model, carrying out combined recognition and determining the severity of the atrophic gastritis in the gastroscope image; according to the method, automatic identification of atrophic gastritis can be carried out on the gastroscope image, and the method has high accuracy to assist doctors.

Description

Technical Field

[0001] The present invention belongs to the technical field of automatic recognition of medical images, and in particular relates to an automatic recognition system for atrophic gastritis based on a generative adversarial network. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] Gastric cancer usually develops from atrophic gastritis. Patients with late-stage atrophic gastritis have a higher probability of developing gastric cancer. Early detection and treatment of atrophic gastritis can help prevent the occurrence of gastric cancer. However, the level of endoscopists varies greatly, and manual diagnosis of atrophic gastritis requires a lot of time and energy. It is necessary to use an intelligent system to automatically identify atrophic gastritis to assist medical staff in diagnosis. Nowadays, deep learning is increasingly used in the field of medical image classification and segmentation. Deep learning can be used to automatically identify patients, reduce the burden on doctors, and help inexperienced doctors make more accurate diagnoses of atrophic gastritis. However, the existing technology does not have a complete system for automatically identifying atrophic gastritis based on gastroscopy images to automatically judge the severity of atrophic gastritis and automatically identify atrophic areas.

[0004] For the gastroscopy data sets that can be collected, non-atrophic gastritis data is often more than atrophic gastritis data, and the probability of disease in different parts is different, and the difficulty of gastroscopy data collection is different. The amount of data in each part also varies greatly, which leads to data imbalance. When there is an imbalance between positive and negative samples and data between different parts in the gastroscopy data set, in order to ensure that the prediction results of the entire model are as accurate as possible, the prediction results of the model will be tilted towards the category with more data, resulting in incorrect predictions.

[0005] Data expansion for classes with less data is a method to solve data imbalance, but it is difficult to collect data for classes with less data on atrophic gastritis. Manual data expansion requires doctors to spend a lot of manpower and time costs, which is not feasible.

[0006] Generative adversarial networks are an unsupervised learning method in the field of computer vision that can be used to generate the desired images. In recent years, generative adversarial networks have received high attention and wide application in various fields. In the medical field, they have been applied to medical image generation and medical image enhancement. However, the existing technology only uses simple data enhancement methods, such as flipping, rotation, and translation, to expand the unbalanced atrophic gastritis data. These image data obtained through spatial transformation cannot ensure the variability of the data and cannot effectively solve the problem of data imbalance. Summary of the invention

[0007] In order to solve the above problems, the present invention proposes an automatic recognition system for atrophic gastritis based on a generative adversarial network, which can automatically recognize atrophic gastritis in gastroscopy images with high accuracy and can better assist doctors in diagnosis.

[0008] According to some embodiments, the present invention adopts the following technical solutions: An automatic recognition system for atrophic gastritis based on generative adversarial networks, comprising: A data acquisition module is used to acquire gastroscopic images of the required parts, and mark the parts of the gastroscopic images, whether they are atrophic, and the atrophic area information to obtain a gastroscopic image data set; The dual sampling module is used to preprocess the gastroscope image dataset and perform dual sampling based on feature screening of the generative adversarial network to obtain a balanced gastroscope image dataset; A model training module, used to train the Kimura Takemoto classification model and the atrophy area recognition model respectively using a balanced gastroscopy image dataset; The joint recognition module is used to process the gastroscopic image to be tested using the Kimura Takemoto classification model and the atrophic area recognition model respectively, and to perform joint recognition based on the outputs of the two models to determine the severity of atrophic gastritis in the gastroscopic image.

[0009] As an optional embodiment, the gastroscopic images of the required parts include the gastroscopic images of the gastric antrum, gastric angle, lower part of the lesser curvature of the gastric body, middle and upper part of the lesser curvature of the gastric body, cardia, gastric fundus and greater curvature of the gastric body.

[0010] As an optional embodiment, the dual sampling module includes a preprocessing module for preprocessing the gastroscopic image data set, removing blurred, defocused, and haloed images in the gastroscopic image data set, as well as text, thumbnails, and black edges in the images, and cropping the image to the main part of the image.

[0011] As an optional implementation, the dual sampling module includes a judgment module, which is configured to further subdivide closed and open atrophic gastritis into two categories, atrophic and non-atrophic, to form subdivision categories, set a threshold u, and if the number of images in the subdivision category is greater than g×(1+u), the subdivision category is a class that requires undersampling, and the images in the subdivision category are partitioned and undersampled; if the number of images in the subdivision category is less than g×(1-u), the subdivision category is a class that requires oversampling, and the images in the subdivision category are oversampled based on feature screening based on a generative adversarial network; if the number of images in the subdivision category is greater than or equal to g×(1-u) and less than or equal to g×(1+u), no undersampling or oversampling operation is required, g is the ratio of the total number of gastroscopic images to n, and n is the total number of subdivision categories.

[0012] As a further embodiment, the dual sampling module includes an undersampling module configured to: extract features using a network model for each image in the class requiring undersampling to obtain a feature vector; Calculate the Euclidean distance between every two eigenvectors, and for each eigenvector, select the m smallest values ​​of the distance between the eigenvector and other eigenvectors; The reciprocals of the selected m distance values ​​are added together to obtain the score of the image, the images are sorted according to the scores, and the images are divided into a reserved area and an under-sampled area according to a set ratio; The images in the under-sampled area are randomly under-sampled, and the number of under-sampled images retained is g×(1-u). The images retained by under-sampling and the images in the reserved area are added to the training set.

[0013] As a further embodiment, the dual sampling module, including the oversampling module, is configured to: construct a generator and a discriminator, wherein the generator is used to generate fake samples, and the discriminator is used to distinguish between samples generated by the generator and real samples; Train the generator and discriminator. During the training process, the image is first adjusted and enhanced; Perform feature screening, extract features from the images generated by the trained generator and discriminator and the original images, obtain the feature vectors of the generated images and the feature vectors of the original images, calculate the distance between the feature vector of the generated image and the feature vector of each original image, select the smallest n distance values, add the reciprocals of the selected n distance values ​​to get the score of the image, set a threshold h, if the image score is less than h, then the image cannot be used as an image for training, if the image score is greater than or equal to h, then the image is added to the training set; Repeat the above feature screening process until the number of training set images is greater than or equal to g×(1-u).

[0014] As an optional implementation, the Kimura Takemoto typing model includes a feature extraction layer, an image block embedding layer, a position embedding layer, an encoder and a multi-layer perceptron connected in sequence, wherein the feature extraction layer is used to extract features from the resized balanced gastroscopy image data set image, the image block embedding layer receives the feature extraction result, and performs convolution and flattening processing on it, the position embedding layer is used to merge the output of the image block embedding layer with the category label and then add position embedding, the encoder is used to standardize the output of the position embedding layer after the discard operation, and extract the category label, and the multi-layer perceptron is used to make predictions based on the extracted results.

[0015] As an optional implementation, when training the Kimura Takemoto typing model, each image marked as atrophy but predicted to be non-atrophy is divided into a plurality of small grid images, the order of the image blocks in the grid is randomly disrupted, the disrupted images are added to the original training set, and the Kimura Takemoto typing model is trained again.

[0016] As an optional implementation, the atrophy area recognition model includes a plurality of transformer layers of different sizes connected in sequence, each transformer layer includes an efficient self-attention layer, a hybrid fully connected feedforward network layer and an overlapping image block merging layer, and the obtained multiple multi-scale features are input into a multi-layer perceptron layer to obtain a prediction output; When training the atrophy area recognition model, each image marked as atrophy but with a predicted atrophy area smaller than a set value and its mask image are divided into multiple grids of small blocks. The order of the image blocks in the grid is randomly disrupted, and the disrupted images are added to the original training set, and the atrophy area recognition model is trained again.

[0017] As an optional implementation, the joint recognition module is used to input the gastroscopic image to be tested into the Kimura Takemoto classification model to obtain the image-level classification prediction result, and input the gastroscopic image to be tested into the atrophic region recognition model to obtain the pixel-level atrophic region and the atrophic region area; When the output result of the Kimura Takemoto classification model is non-atrophy, and the output of the atrophy area recognition model shows that the area of ​​the atrophy area is smaller than the set value, it is judged that there is no atrophic gastritis; when the output result of the Kimura Takemoto classification model is atrophy, or the output of the atrophy area recognition model shows that the area of ​​the atrophy area is larger than the set value, it is judged that there is atrophic gastritis; If the patient is diagnosed with atrophic gastritis, the severity of the atrophic gastritis is determined based on the output results of the Kimura Takemoto classification model.

[0018] Compared with the prior art, the present invention has the following beneficial effects: The present invention can automatically identify atrophic gastritis in gastroscopy images to assist doctors in judging the severity of atrophic gastritis, thereby improving the accuracy and stability of identification and greatly reducing the manpower and time costs required for doctors' diagnosis.

[0019] In view of the characteristics of the atrophic gastritis dataset, the present invention simultaneously uses feature screening oversampling and partition undersampling methods based on generative adversarial networks to solve the previously difficult problem of data imbalance, increase the proportion of data in categories with smaller data volumes, and effectively improve the accuracy, stability, and robustness of model predictions.

[0020] The present invention regards each piece of collected and labeled data as very precious. In order to avoid wasting effective information lost in the undersampling process, partition undersampling is used to retain more effective information, and a judgment mechanism is set to decide whether to oversample or undersample such data.

[0021] In view of the problem that it is more difficult to collect data for categories with less data volume, the present invention uses an oversampling method to automatically generate data, which can greatly reduce the manpower and time costs of professional doctors in collecting data; and adds a feature filter to the oversampling method to ensure the effectiveness of the training set images for training.

[0022] The present invention adds a mechanism to prevent erroneous prediction as non-atrophy in the Kimura Takemoto typing model and the atrophy area identification model, which not only strengthens the model's learning of samples with incorrect predictions, but also effectively reduces the missed diagnosis rate. A joint recognition module is used for auxiliary recognition. The basis for recognition comes from the Kimura Takemoto typing on the one hand, and from the atrophy area predicted by the model on the other hand. The recognition result is more comprehensive, which further reduces the missed diagnosis rate and improves the stability of the model.

[0023] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0025] Figure 1 It is an overall flow chart of the automatic identification system of atrophic gastritis based on generative adversarial network in an embodiment of the present invention; Figure 2 Schematic diagram of the process of the dual sampling module in an embodiment of the present invention; Figure 3 Schematic diagram of the process of partition undersampling in an embodiment of the present invention; Figure 4 It is a structural diagram of a feature screening oversampling model based on a generative adversarial network in an embodiment of the present invention; Figure 5 It is a structural diagram of the Kimura Takemoto classification model in an embodiment of the present invention; Figure 6 Schematic diagram of the process of the joint identification module in an embodiment of the present invention. DETAILED DESCRIPTION

[0026] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0027] It should be noted that the following detailed descriptions are all illustrative and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0028] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0029] In the absence of conflict, the embodiments in this application and the features in the embodiments may be combined with each other.

[0030] Embodiment 1 An automatic recognition system for atrophic gastritis based on generative adversarial networks, such as Figure 1 As shown, including: A data acquisition module is used to acquire gastroscopic images of the required parts, and mark the parts of the gastroscopic images, whether they are atrophic, and the atrophic area information to obtain a gastroscopic image data set; A double sampling module is used to preprocess and double sample the gastroscope image dataset to obtain a balanced gastroscope image dataset; A model training module, used to train the Kimura Takemoto classification model and the atrophy area recognition model respectively using a balanced gastroscopy image dataset; The joint recognition module is used to input the gastroscopic images to be tested into the Kimura Takemoto classification model and the atrophic area recognition model respectively, and input the outputs of the two models into the joint recognition model together, and automatically identify the severity of atrophic gastritis in the gastroscopic images according to the model output results to assist doctors in diagnosis.

[0031] Among them, obtaining a gastroscopy image dataset includes: Gastroscopic images of the gastric antrum, gastric angle, lower part of the lesser curvature of the gastric body, upper and middle part of the lesser curvature of the gastric body, cardia, gastric fundus and greater curvature of the gastric body are obtained, and professional physicians with rich experience annotate the location of the gastroscopic images, whether they are atrophied, and information about the atrophic area. The location of the gastroscopic images and whether they are atrophied are obtained by image-level annotation of the images, and the atrophic area information is obtained by pixel-level annotation of the images.

[0032] According to Kimura Takemoto classification method, the gastroscopic image parts are matched with the classification, and the corresponding relationship is C1: gastric antrum, C2: gastric angle, lower part of lesser curvature of gastric body, C3: middle and upper part of lesser curvature of gastric body, O1: cardia, O2: gastric fundus, O3: greater curvature of gastric body, and the gastroscopic image data set is obtained.

[0033] Among them, Kimura Takemoto classification method can be used to identify atrophic gastritis, which divides atrophic gastritis into closed type (C) and open type (O). The distinction is based on the fact that the atrophic boundary of the closed type does not reach the cardia, while the atrophic boundary of the open type extends above the cardia, and there are differences in the level and color of the gastric mucosa on both sides of the atrophic boundary. According to the location of the atrophic boundary, the closed type and the open type are each divided into three subtypes, namely C1, C2, C3 and O1, O2, O3.

[0034] The gastroscope image dataset is preprocessed and double sampled to obtain a balanced gastroscope image dataset, including: Firstly, the blurred, defocused, haloed images as well as the text, thumbnails, and black edges in the gastroscopic image dataset are removed, and the image is cropped to the main part.

[0035] Experiments have found that the gastroscopy image dataset has a strong dependence on data balance. Using only undersampling will waste useful information in classes with more data, and using only oversampling will reduce the model's generalization ability for classes with less data. Figure 2 As shown in the figure, the double sampling methods of undersampling and oversampling are used in turn to achieve data balance. The undersampling method is partition undersampling, and the oversampling method is feature screening oversampling based on the generative adversarial network. The gastroscopic image dataset is divided into a training set, a validation set, and a test set in a ratio of 8:1:1. The training set is double sampled, and the validation set and the test set remain unchanged to obtain a balanced gastroscopic image dataset.

[0036] The steps for using the undersampling and oversampling methods sequentially are: C1, C2, C3, O1, O2, and O3 are further subdivided into two categories: atrophy and non-atrophy, with a total of 12 subdivided categories. Let g = the total number of gastroscopic images / 12, and set the threshold u, where u = 0.2. If the number of images in a subdivided category is greater than g×(1+u), the subdivided category is considered to be a class that requires undersampling, and the images in the corresponding category are partitioned and undersampled; if the number of images in a subdivided category is less than g×(1-u), it is considered to be a class that requires oversampling, and the corresponding images are oversampled based on feature screening based on the generative adversarial network; if the number of images in a subdivided category is greater than or equal to g×(1-u) and less than or equal to g×(1+u), no undersampling or oversampling operation is required.

[0037] The process of partition undersampling is as follows Figure 3 As shown, for each image in the class that needs to be undersampled, the EfficientNet network is used to extract features, and the last layer of the network is removed for output to obtain a feature vector.

[0038] Calculate the Euclidean distance between every two eigenvectors, and for each eigenvector, select the m smallest values ​​of the distance between the eigenvector and other eigenvectors.

[0039] The reciprocals of the selected m distance values ​​are added together to obtain the score of the image. The images of this type are sorted according to the scores, and the images are divided into a reserved area and an undersampled area in a 1:1 ratio. The images with the top 0.5 scores are divided into the reserved area, and the images with the bottom 0.5 scores are divided into the undersampled area.

[0040] The images in the under-sampled area are randomly under-sampled, and the number of under-sampled images retained is g×(1-u). The images retained by under-sampling and the images in the reserved area are added to the training set.

[0041] The steps of feature screening oversampling based on generative adversarial networks, such as Figure 4 As shown: Based on the StyleGAN2 ADA network, a feature screening oversampling model is constructed. The StyleGAN2 ADA network is different from the traditional generative adversarial network. It can use a small amount of data to generate high-quality artificial images, thereby better solving the problem of sample imbalance in the atrophic gastritis dataset. The StyleGAN2 ADA generator G and discriminator D are constructed. The generator generates fake samples, and the discriminator is used to distinguish between samples generated by the generator and real samples.

[0042] When training the discriminator, the hidden variable is input into the generator to obtain a fake sample image, the real sample image and the fake sample image are subjected to the same enhancement changes, the changed image is input into the discriminator, and the discriminator is trained using the non-saturated logistic loss, and the calculation formula is: ; When training the generator, the latent variable is input into the generator to obtain a fake sample image, the fake sample image is enhanced and changed, the changed image is input into the discriminator, and the generator is trained using non-saturated logistic loss.

[0043] During the training process of the discriminator and generator, the images of the classes that need to be oversampled are first adjusted to 256×256 pixels, and the images are enhanced including color transformation, pixel operation, graphic transformation, noise addition, image spatial filtering and cropping. The adaptive discriminator enhancement technology is used to adaptively adjust the probability of image enhancement transformation. This technology effectively avoids overfitting and can achieve good results on data sets with a small number of images.

[0044] Construct a feature filter. The steps of feature screening are as follows: use the trained generative adversarial network to generate images whose number is g-the number of images of this type, input the generated image and the original image into the efficientnet network, remove the last layer of the network and output the feature vector, and obtain the feature vector of the generated image and the feature vector of the original image. Calculate the distance between the feature vector of the generated image and the feature vector of each original image, and select the smallest n distance values. Add the reciprocals of the selected n distance values ​​to get the score of the image, set the threshold h, if the score of the image is less than h, the image cannot be used as an image for training, if the score of the image is greater than or equal to h, the image is added to the training set.

[0045] Repeat the above feature screening operation until the number of images in the training set is greater than or equal to g×(1-u).

[0046] Train the Kimura Takemoto classification model, such as Figure 5 As shown, including: The balanced gastroscopic image dataset is divided into 7 categories according to {no atrophy, C1 atrophy, C2 atrophy, C3 atrophy, O1 atrophy, O2 atrophy, O3 atrophy}. The image size of the balanced gastroscopic image dataset is adjusted to 224×224×3 and input into the resnet50 layer for feature extraction. The size of the first-layer feature matrix is ​​14×14×1024. The first-layer feature matrix is ​​input into the image block embedding layer, which includes a 1×1 convolution and flattening process. The size of the second-layer feature matrix is ​​196×768. The second-layer feature matrix is ​​merged with the category label and then position embedded. The size of the third-layer feature matrix is ​​197×768. After the third-layer feature matrix is ​​discarded, it is input into the transformer encoder and standardized. The size of the fourth-layer feature matrix is ​​197×768. The category label of size 1×768 in the fourth-layer feature matrix is ​​extracted and finally input into the multi-layer perceptron to obtain the prediction result.

[0047] Each image marked as atrophy but predicted as non-atrophy is divided into small blocks of 3×3 grids, and the order of the image blocks in the grid is randomly disrupted. The disrupted images are added to the original training set, and the Kimura Takemoto classification model is trained again. Gastroscopic images have distinct structural characteristics. Adding the disrupted images to the original training set can better train the differences in mucosal characteristics of atrophic gastritis and effectively reduce the missed diagnosis rate.

[0048] In this embodiment, the evaluation index of the Kimura Takemoto classification model may be accuracy, precision, recall and / or F score.

[0049] Training atrophy region recognition model, including: The image size of the balanced gastroscope image dataset was adjusted to 256×256×3, and the image was divided into multiple image blocks of 4×4×3 size, and then passed through 4 transformer layers of different sizes. Each transformer layer contains an efficient self-attention layer, a hybrid fully connected feedforward network layer, and an overlapping image block merging layer. The output multi-scale feature sizes of each transformer layer are 64×64×C1, 32×32×C2, 16×16×C3, and 8×8×C4. The obtained 4 multi-scale features are input into the multi-layer perceptron layer to obtain a 64×64×6 prediction output.

[0050] Each image marked as atrophy but predicted to have atrophy area of ​​0 and its mask image are divided into small blocks of 3×3 grids. The order of the image blocks in the grid is randomly shuffled, and the shuffled images are added to the original training set, and the atrophy area recognition model is trained again.

[0051] The evaluation indicators of the atrophy area identification model include intersection over union ratio, Dice coefficient and ROC curve.

[0052] The gastroscopy image to be tested is input into the Kimura Takemoto classification model and the atrophy area recognition model, and the outputs of the two models are input into the joint recognition model together, such as Figure 6 As shown, the identification results of atrophic gastritis are obtained to assist doctors in diagnosis, including: The gastroscopic image to be tested is input into the Kimura Takemoto classification model to obtain the image-level classification prediction result, and the gastroscopic image to be tested is input into the atrophy area recognition model to obtain the pixel-level atrophy area and the atrophy area area.

[0053] Construct a joint recognition model, input the obtained results into the joint recognition model, and obtain the multi-level atrophic gastritis recognition results. The steps are as follows: First determine whether there is atrophy. When the output result of the Kimura Takemoto classification model is non-atrophy, and the output of the atrophy area recognition model is 0, it is determined that there is no atrophic gastritis. When the output result of the Kimura Takemoto classification model is atrophy, or the output of the atrophy area recognition model is not 0, it is determined that there is atrophic gastritis.

[0054] If a patient is diagnosed with atrophic gastritis, the severity of the atrophic gastritis is determined using the output of the Kimura Takemoto classification model, and the severity of the atrophic gastritis is classified into mild (C1-C2), moderate (C3-O1), and severe (O2-O3).

[0055] It should be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of an all-hardware embodiment, an all-software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of one or more computer-usable storage media (including but not limited to disk storage, CD - ROM , optical storage, etc.).

[0056] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0057] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0058] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0059] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made by those skilled in the art within the spirit and principle of the present invention without creative labor shall be included in the protection scope of the present invention.

Claims

1. An automatic identification system for atrophic gastritis based on generative adversarial networks, characterized in that: include: A data acquisition module is used to acquire gastroscopic images of the required parts, and mark the parts of the gastroscopic images, whether they are atrophic, and the atrophic area information to obtain a gastroscopic image data set; The dual sampling module is used to preprocess the gastroscope image dataset and perform dual sampling based on feature screening of the generative adversarial network to obtain a balanced gastroscope image dataset; A model training module, used to train the Kimura Takemoto classification model and the atrophy area recognition model respectively using a balanced gastroscopy image dataset; The joint recognition module is used to process the gastroscopic image to be tested using the Kimura Takemoto classification model and the atrophic area recognition model respectively, and to perform joint recognition based on the outputs of the two models to determine the severity of atrophic gastritis in the gastroscopic image.

2. The automatic identification system for atrophic gastritis based on generative adversarial networks as claimed in claim 1, characterized in that: The endoscopic images of the required parts include the antrum, angle of the stomach, lower part of the lesser curvature of the gastric body, upper and middle part of the lesser curvature of the gastric body, cardia, fundus and greater curvature of the gastric body.

3. The automatic identification system for atrophic gastritis based on generative adversarial network as claimed in claim 1, characterized in that: The dual sampling module includes a preprocessing module for preprocessing the gastroscope image data set, removing blurred, defocused, haloed images and text, thumbnails, black edges in the gastroscope image data set, and cropping the image to the main part of the image.

4. The automatic identification system for atrophic gastritis based on generative adversarial network as claimed in claim 3, characterized in that: The dual sampling module includes a judgment module, which is configured to subdivide closed and open atrophic gastritis into two categories, atrophy and non-atrophy, to form subdivision categories, set a threshold u, and if the number of images in the subdivision category is greater than g×(1+u), the subdivision category is a category that requires undersampling, and the images in the subdivision category are partitioned and undersampled; if the number of images in the subdivision category is less than g×(1-u), the subdivision category is a category that requires oversampling, and the images in the subdivision category are oversampled based on feature screening of the generative adversarial network; if the number of images in the subdivision category is greater than or equal to g×(1-u) and less than or equal to g×(1+u), no undersampling or oversampling operation is required, g is the ratio of the total number of gastroscopic images to n, and n is the total number of subdivision categories.

5. The automatic identification system for atrophic gastritis based on generative adversarial network as claimed in claim 4, characterized in that: The dual sampling module, including the undersampling module, is configured to: extract features using a network model for each image in the class requiring undersampling to obtain a feature vector; Calculate the Euclidean distance between every two eigenvectors, and for each eigenvector, select the m smallest values ​​of the distance between the eigenvector and other eigenvectors; The reciprocals of the selected m distance values ​​are added together to obtain the score of the image, the images are sorted according to the scores, and the images are divided into a reserved area and an under-sampled area according to a set ratio; The images in the under-sampled area are randomly under-sampled, and the number of under-sampled images retained is g×(1-u). The images retained by under-sampling and the images in the reserved area are added to the training set.

6. The automatic identification system for atrophic gastritis based on generative adversarial network as claimed in claim 4, characterized in that: The dual sampling module, including the oversampling module, is configured to: construct a generator and a discriminator, wherein the generator is used to generate fake samples, and the discriminator is used to distinguish between samples generated by the generator and real samples; Train the generator and discriminator. During the training process, the image is first adjusted and enhanced; Perform feature screening, extract features from the images generated by the trained generator and discriminator and the original images, obtain the feature vectors of the generated images and the feature vectors of the original images, calculate the distance between the feature vector of the generated image and the feature vector of each original image, select the smallest n distance values, add the reciprocals of the selected n distance values ​​to get the score of the image, set a threshold h, if the image score is less than h, then the image cannot be used as an image for training, if the image score is greater than or equal to h, then the image is added to the training set; Repeat the above feature screening process until the number of training set images is greater than or equal to g×(1-u).

7. The automatic identification system for atrophic gastritis based on generative adversarial network as claimed in claim 1, characterized in that: The Kimura Takemoto typing model includes a feature extraction layer, an image block embedding layer, a position embedding layer, an encoder and a multi-layer perceptron connected in sequence, wherein the feature extraction layer is used to extract features from the resized balanced gastroscope image data set image, the image block embedding layer receives the feature extraction result, and performs convolution and flattening processing on it, the position embedding layer is used to merge the output of the image block embedding layer with the category label and then add the position embedding, the encoder is used to standardize the output of the position embedding layer after the discarding operation, and extract the category label, and the multi-layer perceptron is used to make predictions based on the extracted results.

8. The automatic identification system for atrophic gastritis based on generative adversarial network as claimed in claim 1 or 7, characterized in that When training the Kimura Takemoto typing model, each image marked as atrophy but predicted as non-atrophy is divided into multiple grids of small image blocks, the order of the image blocks in the grid is randomly disrupted, the disrupted images are added to the original training set, and the Kimura Takemoto typing model is trained again.

9. The automatic identification system for atrophic gastritis based on generative adversarial network as claimed in claim 1, characterized in that: The atrophy area recognition model includes a plurality of transformer layers of different sizes connected in sequence, each transformer layer includes a high-efficiency self-attention layer, a hybrid fully connected feedforward network layer and an overlapping image block merging layer, and the obtained multiple multi-scale features are input into a multi-layer perceptron layer to obtain a prediction output; When training the atrophy area recognition model, each image marked as atrophy but with a predicted atrophy area smaller than a set value and its mask image are divided into multiple grids of small blocks. The order of the image blocks in the grid is randomly disrupted, and the disrupted images are added to the original training set, and the atrophy area recognition model is trained again.

10. The automatic identification system for atrophic gastritis based on generative adversarial network as claimed in claim 1, characterized in that: The joint recognition module is used to input the gastroscopic image to be tested into the Kimura Takemoto classification model to obtain the image-level classification prediction result, and input the gastroscopic image to be tested into the atrophic region recognition model to obtain the pixel-level atrophic region and the atrophic region area; When the output result of the Kimura Takemoto classification model is non-atrophy, and the output of the atrophy area recognition model shows that the area of ​​the atrophy area is smaller than the set value, it is judged that there is no atrophic gastritis; when the output result of the Kimura Takemoto classification model is atrophy, or the output of the atrophy area recognition model shows that the area of ​​the atrophy area is larger than the set value, it is judged that there is atrophic gastritis; If the patient is diagnosed with atrophic gastritis, the severity of the atrophic gastritis is determined based on the output results of the Kimura Takemoto classification model.

Images