Endoscope image identification method, electronic device, and storage medium

Abstract

The present invention provides an endoscope image identification method, an electronic device, and a storage medium. The endoscope image identification method includes a step of performing disease type prediction for a plurality of disease categories for a plurality of original images using a first neural network model, a step of constructing a test sample set for the plurality of disease categories based on the disease type prediction results for the plurality of original images, a step of performing disease type classification for each of the test sample sets for the plurality of disease categories using a second neural network model, and a step of superimposing the plurality of disease type classification results to obtain a case diagnosis result, in which the second neural network model weights and combines a plurality of image features in the test sample set to obtain the disease type classification result. The endoscope image identification method weights and combines a plurality of image features of the test sample set to improve the accuracy of disease type classification.

Description

[Technical Field]

[0001] This application claims priority from a Chinese patent application filed on June 23, 2021, bearing application number 202110695472.3 and entitled "Endoscopic image identification method, electronic device and storage medium," the entire contents of which, including the specification, claims, accompanying drawings and abstract of the above-mentioned Chinese patent application, are incorporated herein by reference.

[0002] [Technical field] The present invention relates to the field of medical device imaging, and more particularly to an endoscopic image identification method, electronic device and storage medium based on deep learning. [Background technology]

[0003] Capsule endoscopy is an effective diagnostic and treatment tool for examining patients for gastrointestinal pathologies, integrating devices such as a camera, LED light, and wireless communication module. During an examination, a patient swallows the capsule endoscope, which captures images as it moves through the digestive tract and transmits them outside the patient's body. The images acquired by the capsule endoscope are analyzed to identify lesions in the digestive tract. Capsule endoscopy has the advantage of being less painful for patients than conventional endoscopy and being able to examine the entire digestive tract, making it a widely used innovative technology.

[0004] Capsule endoscopes collect a large number of images (e.g., tens of thousands) during an examination, making image reading a laborious and time-consuming process. With technological advances, lesion identification using image processing and computer vision techniques has attracted widespread attention. However, in conventional endoscopic image identification methods, lesion identification is performed on each image collected by a capsule endoscope via a convolutional neural network to obtain a diagnosis. Even if the accuracy rate of an endoscopic image identification method is as high as 90%, if any of the images collected from a patient's digestive tract contain incorrect lesion identification results, it can lead to an incorrect case diagnosis.

[0005] Therefore, there is still a need to further improve endoscopic image identification methods to improve the accuracy of case diagnosis based on a large number of images. Summary of the Invention

[0006] In order to solve the above technical problems, the object of the present invention is to provide an endoscopic image identification method, an electronic device and a storage medium, which performs disease type prediction for a plurality of original images according to a single image, and then performs disease type identification for a plurality of image features of a test sample set based on the disease type prediction result, thereby improving the accuracy of identification.

[0007] According to a first aspect of the present invention, there is provided an endoscopic image identification method, the method comprising the steps of: using a first neural network model to perform disease type prediction for each of a plurality of disease categories for a plurality of original images; constructing test sample sets for the plurality of disease categories, each containing a predetermined number of image features of the original images based on the disease type prediction results for the plurality of original images; using a second neural network model to perform disease type classification for each of the test sample sets for the plurality of disease categories; and superimposing the plurality of disease type classification results to obtain a case diagnosis result, wherein the second neural network model weights and combines the plurality of image features in the test sample set to obtain the disease type classification result.

[0008] Preferably, the first neural network model is a convolutional neural network model, which inputs a single image of the plurality of original images and outputs image features and classification probabilities of the plurality of disease categories.

[0009] Preferably, the second neural network model is a recurrent neural network model, which inputs a plurality of image features in the test sample set and outputs a disease type identification result corresponding to the test sample set.

[0010] Preferably, the second neural network model includes: a first fully connected layer that performs a dimension reduction process on each of the multiple image features in the test sample set; a bidirectional long-short-term memory layer that predicts hidden states of the multiple image features that have undergone the dimension reduction process according to forward and backward directions; and an attention mechanism that weights and combines the hidden states of the multiple image features to form final features, and the second neural network model obtains a disease type identification result based on the final features.

[0011] Preferably, the first fully connected layer includes a plurality of fully connected units each performing a dimensionality reduction process on a corresponding image feature.

[0012] Preferably, the bidirectional long short-term memory layer includes a plurality of forward long short-term memory units and a plurality of backward long short-term memory units, each of which performs forward prediction for a corresponding image feature, and each of which performs backward prediction for a corresponding image feature.

[0013] Preferably, the weighted combining step includes weighting and summing the hidden states of the plurality of image features, wherein the weight coefficients of the plurality of image features indicate the impact on disease classification of the corresponding disease category.

[0014] TIFF0007656097000001.tif38157

[0015] Preferably, the step of constructing test sample sets for the plurality of disease categories includes a step of selecting, for each different disease category among the plurality of disease categories, image features of a predetermined number of original images having the highest classification probability from the plurality of original images to form a test sample set.

[0016] Preferably, the predetermined number is an integer within the range of 2-128.

[0017] Preferably, the plurality of images are acquired by one of a fiber optic endoscope, a self-propelled capsule endoscope, and a passive capsule endoscope.

[0018] According to a second aspect of the present invention, there is provided an electronic device including a memory and a processor, wherein the memory stores a computer program executable by the processor, and when the processor executes the program, the electronic device realizes the steps of the above-mentioned endoscopic image identification method based on deep learning.

[0019] According to a third aspect of the present invention, there is provided a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, realizes the steps of the above-described endoscopic image identification method based on deep learning.

[0020] According to the endoscopic image identification method, electronic device, and storage medium of the present invention, a first neural network model is used to perform disease prediction, a second neural network model is used to perform disease identification, and the second neural network weights and combines multiple image features in a test sample set to obtain a disease identification result, thereby improving the accuracy of disease identification.Furthermore, multiple disease identification results are obtained based on multiple test sample sets corresponding to multiple disease categories, and a case diagnosis result is obtained by overlaying the identification results of the multiple disease categories.

[0021] In a preferred embodiment, the second neural network model includes a bidirectional long-short-term memory layer for predicting the hidden states of multiple image features according to forward and backward directions, and combining image features from previous and subsequent times to perform disease type identification, thereby further improving the accuracy of disease type identification.

[0022] In a preferred embodiment, each test sample set includes a predetermined number of image features of original images, for example, 2 to 128, so that both the accuracy of disease type identification and the calculation time for disease type categories can be considered. [Brief explanation of the drawings]

[0023] [Figure 1] FIG. 1 shows a schematic diagram of the capsule endoscope system. [Figure 2] FIG. 2 shows a schematic cross-sectional view of an example of a capsule endoscope. [Figure 3] FIG. 3 shows a flowchart of an endoscopic image identification method according to an embodiment of the present invention. [Figure 4] FIG. 4 shows a schematic block diagram of an endoscopic image identification method according to an embodiment of the present invention. [Figure 5] FIG. 5 shows a schematic block diagram of a first neural network model in an endoscopic image recognition method according to an embodiment of the present invention. [Figure 6] FIG. 6 shows a schematic block diagram of a second neural network model in an endoscopic image recognition method according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0024] The present invention will be described in detail below with reference to specific embodiments shown in the accompanying drawings. However, these embodiments do not limit the present invention, and any structural, method, or functional changes made by those skilled in the art according to these embodiments are all included in the protection scope of the present invention.

[0025] 1 shows a schematic structural diagram of a capsule endoscopy system, which includes, for example, a host computer 104, a magnetic ball 105, a three-axis displacement base 106, a magnetic ball holder 107, and a wireless receiving device 108.

[0026] The magnetic ball holder 107 includes a first end connected to the three-axis displacement base 106 and a second end connected to the magnetic ball 105. The three-axis displacement base 106 can translate, for example, along three mutually perpendicular coordinate axes. The magnetic ball holder 107 translates together with the three-axis displacement base 106, allowing the magnetic ball 105 to rotate relative to the magnetic ball holder 107 in horizontal and vertical planes. For example, a motor and a lead screw are used to drive the translation of the three-axis displacement base 106, and a motor and a belt are used to drive the rotation of the magnetic ball 105. This allows the orientation of the magnetic ball 105 to be changed with five degrees of freedom. The magnetic ball 105 is, for example, a permanent magnet with opposing north and south poles. When the orientation of the magnetic ball 105 changes, an external magnetic field is generated whose position and orientation change accordingly.

[0027] During an examination, a patient 101 swallows the capsule endoscope 10 and lies down on a bed 102, for example. The capsule endoscope 10 moves along the digestive tract. As described below, the capsule endoscope 10 contains a permanent magnet. The host computer 104 sends operational commands to the three-axis displacement base 106 and the magnetic ball holder 107 to control the posture of the magnetic ball 105. The external magnetic field generated by the magnetic ball 105 acts on the permanent magnet, thereby controlling the position and orientation of the capsule endoscope 10 within the patient's digestive tract. The capsule endoscope 10 takes images as it moves through the digestive tract and transmits the images to a wireless receiving device 108 located outside the patient's body. The host computer 104, connected to the wireless receiving device 108, acquires the images collected by the capsule endoscope 10 and analyzes them to identify lesions in the digestive tract.

[0028] 2 is a schematic cross-sectional view of an example of a capsule endoscope 10. The capsule endoscope 10 includes a housing 11 and a circuit assembly disposed within the housing 11.

[0029] The housing 11 is formed of a polymeric material, such as plastic, and includes a transparent end for providing illumination and imaging optical paths. The circuit assembly includes an image sensor 12, a first circuit board 21, a permanent magnet 15, a battery 16, a second circuit board 22, and a wireless transmitter 17, arranged in order along the main axis of the housing 11. The image sensor 12 is located on the opposite side of the housing 11 from the transparent end and is mounted, for example, at the center of the first circuit board 21. The first circuit board 21 also mounts multiple LEDs 13 surrounding the image sensor 12. The second circuit board 22 mounts the wireless transmitter 17. The first and second circuit boards 21 and 22 are connected via a flexible circuit board 23, with the permanent magnet 15 and battery 16 sandwiched between them. The flexible circuit board 23 or an additional circuit board provides contact between the positive and negative terminals of the battery.

[0030] Additionally, the circuit assembly may also include a limiting block 18 fixedly connected to the second circuit board 22 for engaging the flexible circuit board 23 or the housing 11 .

[0031] When the capsule endoscope 10 is in the imaging state, the LEDs 13 are turned on to emit light from the end of the housing, and the image sensor 12 acquires an image of the patient's digestive tract from the end of the housing. The image data is transmitted to the wireless transmitter 17 via the flexible substrate 23 and then to the wireless receiver 108 outside the patient's body, allowing the host computer 104 to acquire images for lesion analysis.

[0032] 3 and 4 show a flowchart and a schematic block diagram, respectively, of an endoscopic image identification method according to an embodiment of the present invention.

[0033] In the capsule endoscopy system shown in Fig. 1, a magnetic ball is used to control the position and orientation of the capsule endoscope, which collects a large number of raw images of the patient's digestive tract at different positions and orientations, and the host computer further executes the endoscopic image identification method shown in Fig. 3 to obtain a case diagnosis result. The above capsule endoscopy system includes a self-propelled capsule endoscope for collecting images of the digestive tract, which is just one method of acquiring raw images. In other embodiments, the raw images may be images of the digestive tract acquired via a fiber optic endoscope, images of the digestive tract collected by a passive capsule endoscope, etc.

[0034] In step S01, a first neural network model is used to perform disease type prediction for a single image of an original image to obtain image features and classification probabilities of disease types of the original image, where the classification probabilities refer to the probability that the single image is classified into different disease types. In this embodiment, the first neural network model is, for example, a convolutional neural network (abbreviated as CNN) model.

[0035] Referring to FIG. 5, the first neural network model may include, for example, multiple convolutional layers, at least one pooling layer, at least one fully connected layer, and at least one normalized index layer (e.g., a softmax layer). In a convolutional neural network model, different image features can be extracted through convolution operations. The multiple convolutional layers can sequentially extract low-level and high-level image features. The pooling layer downsamples the image features (i.e., the low-level and high-level image features) to compress the image feature data and parameters while maintaining the invariance of the image features. Each node in the fully connected layer is connected to all nodes in the previous layer and integrates and classifies the final features (i.e., the downsampled image features) extracted in the previous layer. The normalized index layer maps the output of the previous layer (e.g., the fully connected layer) to a probability value in the interval (0, 1) to obtain the classification probability of the corresponding disease type. It can be understood that the first neural network model can be obtained through training with a labeled training sample set. A single original image collected during inspection is used as the input of the first neural network model, image features are extracted from the pooling layer, and classification probabilities are calculated from the normalized index layer.

[0036] The endoscopic image identification method of the present invention is not limited to a specific convolutional neural network (CNN) model and can use general network models such as ResNet, DenseNet, and MobileNet. For example, the applicant disclosed a convolutional neural network model applicable to this step in Chinese Patent Application No. 202110010379.4. As mentioned above, a capsule endoscope can collect tens of thousands of original images during a patient examination. The input of the first neural network model is a single image of at least a portion of the original images, and the single image can be processed to obtain a corresponding disease category and classification probability. The disease categories include at least one of erosion, bleeding, ulcer, polyp, protuberance, telangiectasia, vascular malformation, diverticulum, and parasite. In this example, a total of nine disease categories are listed. However, it should be understood that the number of disease categories that can be identified by the first neural network model is related to the training sample set, and the present invention is not limited to a specific number of disease categories.

[0037] In step S02, for different disease categories, image features of a plurality of images with the highest disease classification probability are selected from the original images to form a test sample set.

[0038] For multiple disease categories, the multiple original images for which disease type prediction has been performed are ranked according to classification probability, and the image features of the original images with the highest classification probability for the corresponding disease category are selected to form respective test sample sets. The image features in the test sample sets are preferably image features output by the pooling layer. The number of images S in the test sample set for each disease category may be a predetermined number, for example, any integer between 2 and 128, taking into consideration both the accuracy of disease identification and calculation time. In this embodiment, the number of disease categories N=9, and the number of images S in each test sample set for each disease category=10. In other embodiments, the number of disease categories and the test sample sets for each disease category can be adjusted according to actual needs.

[0039] For example, referring to FIG. 4, collected images are input to a first neural network model (i.e., a convolutional neural network model) to perform disease type prediction. The first neural network model processes each collected image and obtains the probability that the collected image will be classified into a different disease type category based on the image features of each collected image. Thus, collected images classified into Category 1, Image 1, Image 2, Image 3, ... Image M, are obtained, and image samples with the highest classification probability are selected to obtain Image 3, Image M, Image 2, ... Image S. The processing for other categories is similar to Category 1 and is not repeated here. Based on the selected image samples, the first neural network model outputs image features corresponding to the image samples to form a test sample set.

[0040] In step S03, disease classification is performed on a test sample set of multiple disease types using a second neural network model, which may be, for example, a recurrent neural network (RNN) model.

[0041] For each test sample set of multiple disease categories, the second neural network model performs disease classification based on the test sample set of image features extracted from the multiple original images, i.e., improves the accuracy of disease classification based on the test sample set output from the first neural network model. Referring to Figure 4, for example, the first neural network model selects S images that are most likely to be suspected of erosion, and uses the image features of suspected Category 1 (e.g., erosion) extracted from each of the S images as a test sample set. The test sample set is input to the second neural network model, and the second neural network model can confirm whether the subject is indeed suffering from Category 1 disease (e.g., erosive disease), and similarly for other disease categories.

[0042] In step S04, the classification results of the plurality of disease types are combined to obtain a case diagnosis result.

[0043] After the above disease type prediction and disease type identification steps, a large number of raw images collected during patient examinations can be processed to obtain multiple disease type identification results, which are then superimposed to obtain a case diagnosis result. In a specific embodiment, the case diagnosis result is that the patient's lesions include one or more of nine disease type categories. For example, for the above nine disease types, if the identification results for two disease types, hemorrhage and polyp, indicate the presence of a lesion, and the identification results for the other disease types indicate the absence of a lesion, the case diagnosis result is the superimposition of all disease types, that is, the patient has lesions of the two disease types, hemorrhage and polyp.

[0044] Hereinafter, the second neural network model in the endoscopic image identification method according to the embodiment of the present invention will be described in detail with reference to FIG.

[0045] The second neural network model is a recurrent neural network model (RNN). The recurrent neural network model is a recurrent neural network that receives sequence data as input. As shown in the figure, the second neural network model includes, for example, at least one first fully connected layer, at least one bidirectional long short-term memory (LSTM) layer, an attention mechanism, at least one second fully connected layer, and at least one normalized index layer (e.g., a softmax layer).

[0046] A test sample set for a single disease category obtained from the disease category prediction of the first neural network model is used as input for a second neural network model, and the test sample set includes multiple image features obtained from multiple original images.

[0047] The first fully-connected layer includes a plurality of fully-connected units, each of which performs a dimension reduction process on a corresponding image feature, i.e., the plurality of fully-connected units respectively perform a dimension reduction process on a plurality of high-dimensional image features to obtain a plurality of low-dimensional image features.

[0048] The bidirectional long short-term memory layer includes a plurality of forward long short-term memory units and a plurality of backward long short-term memory units for predicting hidden states according to forward and backward directions for a plurality of image features, respectively, where each of the plurality of forward long short-term memory units performs forward prediction on the corresponding image features, and each of the plurality of backward long short-term memory units performs backward prediction on the corresponding image features.

[0049] The inventors have focused on the fact that when doctors make diagnoses based on gastrointestinal images (especially images of the gastrointestinal tract taken continuously), they refer to not only images taken at previous times but also images taken at later times, and combine the images taken at previous and subsequent times to make a diagnosis. In conventional capsule endoscope image processing methods, recurrent neural network models use a one-way long-short-term memory layer, which can only predict the output at the next time from the input at the previous time, and therefore cannot accurately identify disease types based on collected images. Unlike conventional recurrent neural network models, the recurrent neural network model of the present invention employs a bidirectional long-short-term memory layer and identifies disease types by combining image features from previous and subsequent times.

[0050] TIFF0007656097000002.tif114157

[0051] TIFF0007656097000003.tif23157 TIFF0007656097000004.tif13157

[0052] Therefore, the bidirectional long short-term memory layer can capture multiple hidden states corresponding to multiple image features.

[0053] The attention mechanism of the second neural network model is used to weight the hidden states of multiple image features into a final feature.

[0054] TIFF0007656097000005.tif37157

[0055] The hidden states of multiple image features are weighted and combined to obtain the final feature T, as shown in the following equation: TIFF0007656097000006.tif1224

[0056] Furthermore, the second fully connected layer integrates and classifies the final features T extracted in the previous layer. The normalized index layer is used to map the output of the previous layer (i.e., the second fully connected layer) to a probability value in the interval (0, 1), to obtain the probability that each final feature T is classified into a different disease category, i.e., the probability of being suspected as a disease category, and then obtain and output a case diagnosis result based on the probability of being suspected as a disease category.

[0057] The second neural network model identifies disease types based on a test sample set of image features from multiple raw images, and verifies whether the multiple raw images most likely to be suspected of that disease type actually contain a lesion.

[0058] Furthermore, one embodiment of the present invention provides an electronic device including a memory and a processor, wherein the memory stores a computer program executable on the processor, and when the processor executes the program, the steps of the above-mentioned endoscopic image identification method based on deep learning are realized.

[0059] Furthermore, one embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, realizes the steps of the above-described endoscopic image identification method based on deep learning.

[0060] As described above, the deep learning-based endoscopic image identification method, electronic device, and storage medium of the present invention predict the disease type for a single original image, then select and weight and combine multiple images based on the disease type prediction result to improve the accuracy of disease type identification, and obtain a case diagnosis result by overlaying the identification results of multiple disease type categories.

[0061] For convenience of explanation, the above-mentioned device will be described by dividing it into various modules according to their functions. Of course, when implementing the present invention, the functions of each module can be realized by the same or multiple pieces of software and / or hardware.

[0062] The above-described device embodiments are merely examples. Modules described as separate components may or may not be physically separated, and components shown as modules may or may not be physical modules, i.e., they may be located in one place or distributed across multiple network modules. To achieve the objectives of the solutions of the present embodiment, some or all of the modules may be selected according to actual needs. Those skilled in the art can understand and implement the present invention without any creative efforts.

[0063] This specification is described according to embodiments, but each embodiment does not include only independent technical solutions, and this description in the specification is for clarity only, and those skilled in the art should understand the specification as a whole, and it should be understood that the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

[0064] The series of detailed descriptions listed above are only specific descriptions for possible embodiments of the present invention, and are not intended to limit the protection scope of the present invention; any equivalent embodiments or modifications that do not deviate from the technical idea of ​​the present invention should also be included in the protection scope of the present invention.

Claims

1. A method for identifying an endoscope image for a capsule endoscopy system, comprising: The capsule endoscope system includes a host computer that collects a plurality of original images using a capsule endoscope and executes an image identification method for the endoscope; The endoscopic image identification method includes: performing disease prediction for each of the plurality of original images for a plurality of disease categories using a first neural network model by the host computer; constructing test sample sets for the plurality of disease categories, each of which includes a predetermined number of image features of the original images, based on the disease type prediction results of the plurality of original images by the host computer; performing disease classification for each of the test sample sets of the plurality of disease categories using a second neural network model by the host computer; and acquiring a case diagnosis result by superimposing the discrimination results of the plurality of disease types by the host computer, The second neural network model includes a step of weighting and combining a plurality of image features in the test sample set to obtain the disease type identification result. An endoscopic image identification method.

2. The endoscopic image identification method of claim 1, wherein the first neural network model is a convolutional neural network model, which inputs a single image of the plurality of original images and outputs image features and classification probabilities of the plurality of disease categories.

3. 3. The endoscopic image identification method according to claim 2, wherein the second neural network model is a recurrent neural network model, and the recurrent neural network model inputs a plurality of image features in the test sample set and outputs a disease type identification result corresponding to the test sample set.

4. The second neural network model is A first fully connected layer that performs a dimensionality reduction process on each of a plurality of image features in the test sample set; A bidirectional long short-term memory layer that predicts hidden states according to forward and backward directions for multiple image features that have been subjected to dimensionality reduction processing; and an attention mechanism that weights and combines the hidden states of the plurality of image features to form a final feature. The endoscopic image identification method according to claim 1 , wherein the second neural network model obtains a disease type identification result based on the final features.

5. The endoscopic image identification method according to claim 4 , wherein the first fully connected layer includes a plurality of fully connected units each performing a dimensionality reduction process on a corresponding image feature.

6. The endoscopic image identification method of claim 4, wherein the bidirectional long-short-term memory layer includes a plurality of forward long-short-term memory units and a plurality of backward long-short-term memory units, each of the plurality of forward long-short-term memory units performing forward prediction on a corresponding image feature, and each of the plurality of backward long-short-term memory units performing backward prediction on a corresponding image feature.

7. The endoscopic image identification method of claim 4, wherein the weighting and combining step includes a step of weighting and summing the hidden states of the multiple image features, and the weighting coefficients of the multiple image features indicate their influence on disease identification of corresponding disease categories.

8.

9. The endoscopic image identification method of claim 2, wherein the step of constructing test sample sets for the plurality of disease categories includes a step of selecting, for each different disease category among the plurality of disease categories, image features of a predetermined number of original images having the highest classification probability from the plurality of original images to form a test sample set.

10. The endoscopic image identification method according to claim 9, wherein the predetermined number is an arbitrary integer within a range of 2 to 128.

11. The endoscopic image identification method according to claim 1 , wherein the plurality of original images are collected and obtained by one of a fiber optic endoscope, a self-propelled capsule endoscope, and a passive capsule endoscope.

12. 1. An electronic device comprising a memory and a processor, the memory storing a computer program executable by the processor, When the processor executes the program, the steps of an endoscopic image identification method based on deep learning are realized, the method comprising: performing disease prediction for a plurality of disease categories for each of a plurality of original images using the first neural network model; constructing test sample sets for the plurality of disease categories, each of which includes a predetermined number of image features of the original images, based on the disease type prediction results of the plurality of original images; performing disease classification for each of the test sample sets of the plurality of disease categories using a second neural network model; A step of superimposing the disease type identification results of the plurality of disease types to obtain a case diagnosis result, The electronic device, characterized in that the second neural network model weights and combines a plurality of image features in the test sample set to obtain the disease type identification result.

13. A computer-readable storage medium having a computer program stored thereon, When the computer program is executed by a processor, the computer program realizes steps of an endoscopic image identification method based on deep learning, the method comprising: performing disease prediction for a plurality of disease categories for each of a plurality of original images using the first neural network model; constructing test sample sets for the plurality of disease categories, each of which includes a predetermined number of image features of the original images, based on the disease type prediction results of the plurality of original images; performing disease classification for each of the test sample sets of the plurality of disease categories using a second neural network model; A step of superimposing the plurality of disease type identification results to obtain a case diagnosis result, The second neural network model weights and combines a plurality of image features in the test sample set to obtain the disease type identification result.

Images