Gastroscopic image classification method, device and storage medium based on ResNet-50 time compression

Abstract

The present application discloses a gastroscope image classification method, device and storage medium based on ResNet‑50 time compression, which relates to the field of medical image intelligent processing technology. The method comprises: obtaining a gastroscope image; preprocessing the gastroscope image; inputting the preprocessed gastroscope image into the ResNet‑50 network model, wherein each residual structure in the first three residual structures of the ResNet‑50 network model is followed by a branch classifier, and the last residual structure is followed by a trunk classifier; the classification results of each classifier in the ResNet‑50 network model are obtained in sequence, and the final classification result of the gastroscope image is determined according to the obtained classification results. The problem in the prior art that human judgment may lead to misdiagnosis and a lot of time is consumed due to the influence of the doctor's experience level when observing the gastroscope image with human eyes is solved, and the classification result can be automatically output after the gastroscope image is input into the ResNet‑50 network model, which saves the test time without basically affecting the classification accuracy.

Description

Technical Field

[0001] The present invention relates to a gastroscope image classification method, device and storage medium based on ResNet-50 time compression, and belongs to the technical field of medical image intelligent processing. Background Art

[0002] Duodenal ulcer is one of the common and frequently occurring diseases in the Chinese population, and is a common type of peptic ulcer. It is prone to occur in winter and spring when the climate changes greatly. The incidence rate in men is significantly higher than that in women. It is closely related to abnormal gastric acid secretion, Helicobacter pylori (H.pylori) infection, nonsteroidal anti-inflammatory drugs (NSAIDs), irregular lifestyle and diet, work and external pressure, smoking, drinking, and mental and psychological factors. Duodenal ulcers mostly occur in the duodenal bulb (95%), mostly in the anterior wall, followed by the posterior wall, inferior wall, and superior wall. Since gastroscopy technology is effective in diagnosing duodenal ulcers, it has been recommended as the main diagnostic method for duodenal ulcers. Specifically, gastroscopy can directly observe and photograph the duodenal mucosa, and can also take live tissue for pathological examination under direct vision. It is the main method for diagnosing duodenal ulcers. The gastroscopy method has certain advantages, but human factors such as inconsistent experience levels of doctors or special circumstances such as negligence will directly affect the final diagnosis of duodenal ulcers. At the same time, observing gastroscopy images with the human eye will also consume a lot of time. Summary of the invention

[0003] The object of the present invention is to provide a gastroscopy image classification method, device and storage medium based on ResNet-50 time compression, so as to solve the problems existing in the prior art.

[0004] In order to achieve the above object, the present invention provides the following technical solutions:

[0005] According to a first aspect, an embodiment of the present invention provides a method for classifying gastroscopy images based on ResNet-50 time compression, the method comprising:

[0006] Obtain gastroscopy images;

[0007] Preprocessing the gastroscopy image;

[0008] Inputting the preprocessed gastroscopy image into a ResNet-50 network model, wherein each of the first three residual structures of the ResNet-50 network model is followed by a branch classifier, and the last residual structure is followed by a trunk classifier;

[0009] The classification results of each classifier in the ResNet-50 network model are obtained in turn, and the final classification result of the gastroscopy image is determined according to the obtained classification results.

[0010] Optionally, sequentially acquiring the classification results of each classifier in the ResNet-50 network model, and determining the final classification result of the gastroscopy image according to the acquired classification results, includes:

[0011] Get the classification results of each classifier in turn;

[0012] After obtaining the classification results of each classifier, the uncertainty is calculated based on the currently obtained classification results;

[0013] When the calculated uncertainty is less than a preset threshold, the final classification result of the gastroscopy image is output according to the current classification result, and the acquisition of subsequent classification results of each classifier is stopped.

[0014] Optionally, the method further includes:

[0015] When the calculated uncertainty is greater than the preset threshold, the step of sequentially obtaining the classification results of each classifier is performed again.

[0016] Optionally, each of the first two branch classifiers includes 4 convolutional layers, 1 pooling layer and 1 fully connected layer, the third branch classifier includes 1 pooling layer and 1 fully connected layer, and the trunk classifier includes 1 pooling layer and 1 fully connected layer.

[0017] Optionally, the preprocessing of the gastroscopy image includes:

[0018] The gastroscope image is scaled, cropped, randomly horizontally flipped, and standardized to obtain the standardized gastroscope image.

[0019] In a second aspect, a method for classifying gastroscopy images based on ResNet-50 time compression is provided, the method comprising:

[0020] Obtain sample gastroscopy images, wherein the sample gastroscopy images include N1 duodenal ulcer images and N2 normal duodenal images, where N1 and N2 are both positive integers;

[0021] Preprocessing the sample gastroscopy images;

[0022] The initialized ResNet-50 network model is trained by the preprocessed sample gastroscopic images to obtain a trained ResNet-50 network model. A branch classifier is set after each of the first three residual structures of the trained ResNet-50 network model, and a trunk classifier is set after the last residual structure. The trained ResNet-50 network model is used to output the classification result of the gastroscopic image when the gastroscopic image is input.

[0023] Optionally, the initializing the ResNet-50 network model by training the preprocessed sample gastroscopy images includes:

[0024] The backbone of the initialized ResNet-50 network model is trained using labeled sample gastroscopy images, and the classification results are obtained through the backbone classifier. The calculated classification results and the real labels corresponding to the samples are calculated using the cross entropy loss function to calculate the loss, and the ResNet-50 network model is fitted to the real distribution of gastroscopy images.

[0025] Optionally, the initializing the ResNet-50 network model for training by using the preprocessed sample gastroscopy images further includes:

[0026] The probability distribution predicted by the trunk classifier is distilled to each branch classifier. The branch self-distillation uses unlabeled task data. The classification results of the trunk classifier and the classification results of the branch classifier are compared using the relative entropy loss function KL divergence for loss calculation.

[0027] In a third aspect, a gastroscopy image classification device based on ResNet-50 time compression is provided, the device comprising a memory and a processor, the memory storing at least one program instruction, the processor loading and executing the at least one program instruction to implement the method described in the first aspect or the second aspect.

[0028] In a fourth aspect, a computer storage medium is provided, wherein at least one program instruction is stored in the storage medium, and the at least one program instruction is loaded and executed by a processor to implement the method as described in the first aspect or the second aspect.

[0029] By obtaining a gastroscope image; preprocessing the gastroscope image; inputting the preprocessed gastroscope image into the ResNet-50 network model, each residual structure in the first three residual structures of the ResNet-50 network model is provided with a branch classifier, and the last residual structure is provided with a trunk classifier; the classification results of each classifier in the ResNet-50 network model are obtained in sequence, and the final classification result of the gastroscope image is determined according to the obtained classification results. The problem that human judgment may cause misdiagnosis and take a lot of time when observing gastroscope images with human eyes in the prior art due to the influence of the doctor's experience level and the like is solved, and the classification result can be automatically output after the gastroscope image is input into the ResNet-50 network model, and the effect of saving test time without basically affecting the classification accuracy is achieved. That is, the present application does not require manual participation in the classification detection process, and while reducing human factors, it can provide doctors with efficient diagnostic references and reduce morbidity and mortality.

[0030] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention and implement it according to the contents of the specification, the following is a detailed description of the preferred embodiments of the present invention with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 A method flow chart of a gastroscopy image classification method based on ResNet-50 time compression provided by an embodiment of the present invention;

[0032] Figure 2 A schematic diagram of the structure of a ResNet-50 network model provided for one embodiment of the present invention;

[0033] Figure 3 A method flow chart of a gastroscopy image classification method based on ResNet-50 time compression provided by an embodiment of the present invention;

[0034] Figure 4 A schematic diagram of test results of the classification method of the present application and the classification method of the existing ResNet-50 network model provided for one embodiment of the present invention. DETAILED DESCRIPTION

[0035] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0036] In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and "third" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance.

[0037] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0038] In addition, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0039] Please refer to Figure 1 , which shows a method flow chart of a gastroscopy image classification method based on ResNet-50 time compression provided by an embodiment of the present application, such as Figure 1 As shown, the method includes:

[0040] Step 101, obtaining sample gastroscopy images, wherein the sample gastroscopy images include N1 images of duodenal ulcer and N2 images of normal duodenum, where N1 and N2 are both positive integers;

[0041] Among them, N1 and N2 can be the same or different, and there is no limitation on this. In order to ensure the training accuracy, N1 and N2 are usually larger values. For example, in this embodiment, N1=N2=800.

[0042] Step 102, preprocessing the sample gastroscopy image;

[0043] The preprocessing of the input image includes scaling, cropping, random horizontal flipping and standardization. Scaling is to scale the size of all different input images to 256*256*3, and then crop them to 224*224*3, in order to cut off the black redundant parts of the four corners of the gastroscopy image. The random flipping is controlled by the parameter P, which is the probability of image flipping. Finally, data standardization is performed to obtain the characteristics of the final input image.

[0044] Step 103, training the initialized ResNet-50 network model by using the preprocessed sample gastroscopic images to obtain a trained ResNet-50 network model, wherein each of the first three residual structures of the trained ResNet-50 network model is followed by a branch classifier, and the last residual structure is followed by a trunk classifier, and the trained ResNet-50 network model is used to output the classification result of the gastroscopic image when the gastroscopic image is input.

[0045] After obtaining the preprocessed sample gastroscopic images, the preprocessed sample gastroscopic images are input into the initialized ResNet-50 network model for training to obtain the trained ResNet-50 network model. A branch classifier is set after each of the first three residual structures of the trained ResNet-50 network model, and a trunk classifier is set after the last residual structure. The trained ResNet-50 network model is used to output the classification result of the gastroscopic image when the gastroscopic image is input.

[0046] Among them, each of the first two branch classifiers includes 4 convolutional layers, 1 pooling layer and 1 fully connected layer, the third branch classifier includes 1 pooling layer and 1 fully connected layer, and the trunk classifier includes 1 pooling layer and 1 fully connected layer.

[0047] In this example, the ResNet-50 network model is initialized to consist of 50 weighted layers, and the convolution kernels used are 1*1, 3*3, and 1*1. Figure 2 , ResNet-50 is mainly divided into 6 parts, namely conv1, conv2_x, conv3_x, conv4_x, conv5_x and the last adaptive global pooling layer and fully connected layer. A branch classifier is added after conv2_x, conv3_x and conv4_x, and a trunk classifier is added after conv5_x. The last fully connected layer of each classifier contains 2 neurons, and the last two results are distribution probabilities, that is, the probability of being classified as a normal duodenum and the probability of duodenal ulcer.

[0048] Optionally, when training the initialized ResNet-50 network model, the trunk and branches are trained separately. When training the trunk, labeled task data should be used. The classification results are obtained through the trunk classifier. The classification results and the true labels corresponding to the samples are calculated using the cross entropy loss function, and the entire network is fitted with the true distribution of gastroscopy images. After a certain number of iterations, the model weight parameters that achieve the best results are saved. When training the branches, unlabeled task data is used, and the probability distribution predicted by the trunk classifier is distilled to the branch classifier, that is, the classification result p of the trunk classifier is t With branch classifier p s The classification results use KL divergence (ie, relative entropy loss function) for loss calculation, and the final loss is equal to the sum of all KL divergences. The formula is as follows:

[0049]

[0050]

[0051] in, refers to the classification result of the i-th branch classifier, p t refers to the classification result of the backbone classifier, and L refers to the number of all classifiers.

[0052] After the above training, the trained ResNet-50 network model can be obtained. When the gastroscopy image needs to be judged, the gastroscopy image can be input into the trained ResNet-50 network model to obtain the classification result.

[0053] Please refer to Figure 3 , which shows a method flow chart of a gastroscopy image classification method based on ResNet-50 time compression provided by an embodiment of the present application, such as Figure 3 As shown, the method includes:

[0054] Step 301, obtaining a gastroscopy image;

[0055] Step 302, preprocessing the gastroscopy image;

[0056] The side preprocessing described in this step is similar to the preprocessing described in step 102, and will not be repeated here.

[0057] Step 303, inputting the preprocessed gastroscopy image into a ResNet-50 network model, wherein each of the first three residual structures of the ResNet-50 network model is provided with a branch classifier, and the last residual structure is provided with a trunk classifier;

[0058] Step 304, obtaining the classification results of each classifier in the ResNet-50 network model in turn, and determining the final classification result of the gastroscopy image according to the obtained classification results.

[0059] The ResNet-50 network model includes four classifiers. In actual implementation, each classifier can output a classification result. Therefore, in actual implementation, this step may include:

[0060] First, obtain the classification results of each classifier in turn;

[0061] After the preprocessed gastroscopy images are input into the ResNet-50 network model, the four classifiers in the ResNet-50 network model can output the corresponding classification results in turn.

[0062] Second, after obtaining the classification results of each classifier, the uncertainty is calculated based on the currently obtained classification results;

[0063] First, through the first - layer classifier, the final classification distribution probability \(P = [p1, p2]\) is obtained, where \(p1\) is the probability that the picture is a normal duodenum, \(p2\) is the probability that the picture is a duodenal ulcer, and \(p1 + p2=1\). Calculate the uncertainty of classification according to the probability, and the calculation formula is as follows.

[0064] Third, when the calculated uncertainty is less than the preset threshold, the final classification result of the gastroscope picture is output according to the current classification result, and the classification results of each subsequent classifier are no longer obtained.

[0065] Set the preset threshold as threshold. If the calculated uncertainty is less than threshold, it is determined that the current classification is credible and stopped in advance. If \(p1\geq p2\), it means the model judges that the image is a normal duodenum image. When \(p1 < p2\), it means the model judges that the image is a duodenal ulcer image.

[0066] Fourth, when the calculated uncertainty is greater than the preset threshold, the step of sequentially obtaining the classification results of each classifier is executed again.

[0067] If the calculated uncertainty is greater than threshold, it is determined that the current classification is not credible and continue to execute until the next classifier is encountered for re - judgment.

[0068] According to the experimental results, in order to reduce more time while ensuring the accuracy rate, it is recommended that the preset threshold be taken around 0.98. If the preset threshold is too high, the image basically terminates after passing through one - layer classifier. Although a lot of time is reduced, the accuracy rate will be reduced to a certain extent. If the preset threshold is too low, the image basically goes through the entire model before classification. Although the accuracy rate is guaranteed, the time consumed is larger than that of the unimproved ResNet - 50 model.

[0069] Figure 4 For the comparison table of the time, F - value, and accuracy rate consumed when selecting different models, it can be seen that using ResNet - 50 in the existing scheme takes more time. The average time for single - sample testing is 239 milliseconds, the number of parameters is 164584910, and the accuracy rate reaches 0.91. For the ResNet - 50 network model in this application, the single - sample testing time is 129 milliseconds, the number of parameters is 77581814. Compared with the ResNet - 50 network model in the existing scheme, it reduces nearly half of the time for only a single sample, the number of parameters is half of the original, and the accuracy rate reaches 0.90, close to the original model. This shows that the method for classifying gastroscope pictures based on time compression of ResNet - 50 not only consumes less time but also can be close to the performance of the original model in terms of accuracy rate.

[0070] In summary, by obtaining a gastroscope image; preprocessing the gastroscope image; inputting the preprocessed gastroscope image into the ResNet-50 network model, each residual structure in the first three residual structures of the ResNet-50 network model is provided with a branch classifier, and the last residual structure is provided with a trunk classifier; the classification results of each classifier in the ResNet-50 network model are obtained in sequence, and the final classification result of the gastroscope image is determined according to the obtained classification results. The problem that human judgment may cause misdiagnosis and take a lot of time when observing gastroscope images with human eyes in the prior art due to the influence of the doctor's experience level is solved, and the classification result can be automatically output after the gastroscope image is input into the ResNet-50 network model, and the effect of saving test time without basically affecting the classification accuracy is achieved. That is, the present application does not require manual participation in the classification detection process, and while reducing human factors, it can provide doctors with efficient diagnostic references and reduce morbidity and mortality.

[0071] In addition, multiple classifiers are set in the ResNet-50 network model in the present application, and judgment is made according to the classification results of each classifier, and the classification results can be terminated and output in advance when the uncertainty is low. That is, the output can be made in advance in the first, second, and third classifiers to ensure that the time is greatly reduced, and it can also be output in the fourth classifier to ensure the accuracy of image classification.

[0072] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0073] The above-mentioned embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A gastroscopy image classification method based on ResNet-50 time compression, characterized in that: The method comprises: Obtain gastroscopy images; Preprocessing the gastroscopy image; Inputting the preprocessed gastroscopy image into a ResNet-50 network model, wherein each of the first three residual structures of the ResNet-50 network model is followed by a branch classifier, and the last residual structure is followed by a trunk classifier; Obtaining the classification results of each classifier in the ResNet-50 network model in turn, and determining the final classification result of the gastroscopy image according to the obtained classification results; Wherein, the step of sequentially obtaining the classification results of each classifier in the ResNet-50 network model and determining the final classification results of the gastroscopic image according to the obtained classification results comprises: sequentially obtaining the classification results of each classifier; after obtaining the classification results of each classifier, calculating uncertainty according to the currently obtained classification results; when the calculated uncertainty is less than a preset threshold, outputting the final classification result of the gastroscopic image according to the current classification result, and stopping obtaining the classification results of each subsequent classifier; when the calculated uncertainty is greater than the preset threshold, executing again the step of sequentially obtaining the classification results of each classifier; The ResNet-50 network model is initialized and trained using the preprocessed sample gastroscopy images, including: Using labeled sample gastroscopy images to train the backbone of the initialized ResNet-50 network model, obtaining classification results through the backbone classifier, performing loss calculation between the calculated classification results and the real labels corresponding to the samples using a cross entropy loss function, and fitting the ResNet-50 network model to the real distribution of gastroscopy images; The probability distribution predicted by the trunk classifier is distilled to each branch classifier. The branch self-distillation uses unlabeled task data. The classification results of the trunk classifier and the classification results of the branch classifier are calculated using the relative entropy loss function KL divergence; The uncertainty of the classification is calculated based on the probability, and the calculation formula is as follows:

2. The method according to claim 1, characterized in that Each of the first two branch classifiers includes 4 convolutional layers, 1 pooling layer and 1 fully connected layer, the third branch classifier includes 1 pooling layer and 1 fully connected layer, and the trunk classifier includes 1 pooling layer and 1 fully connected layer.

3. The method according to claim 1, characterized in that The preprocessing of the gastroscopy image comprises: The gastroscope image is scaled, cropped, randomly horizontally flipped, and standardized to obtain the standardized gastroscope image.

4. The method according to claim 1, characterized in that The method comprises: Obtain sample gastroscopy images, wherein the sample gastroscopy images include N1 duodenal ulcer images and N2 normal duodenal images, where N1 and N2 are both positive integers; Preprocessing the sample gastroscopy images; The initialized ResNet-50 network model is trained by the preprocessed sample gastroscopic images to obtain a trained ResNet-50 network model. A branch classifier is set after each of the first three residual structures of the trained ResNet-50 network model, and a trunk classifier is set after the last residual structure. The trained ResNet-50 network model is used to output the classification result of the gastroscopic image when the gastroscopic image is input.

5. A gastroscopy image classification device based on ResNet-50 time compression, characterized in that: The device includes a memory and a processor, wherein at least one program instruction is stored in the memory, and the processor implements the method according to any one of claims 1 to 4 by loading and executing the at least one program instruction.

6. A computer storage medium, characterized in that: The storage medium stores at least one program instruction, and the at least one program instruction is loaded and executed by the processor to implement the method according to any one of claims 1 to 4.

Images