X-ray COVID-19 recognition and detection method based on fine-grained image recognition and target detection algorithm

Abstract

The present invention discloses a method for identifying and detecting COVID-19 in X-rays based on fine-grained image recognition and target detection algorithms, comprising performing fine-grained image recognition on the X-ray to be detected to obtain a classification result. If the classification result is the first type of COVID-19, yolov4 is used to perform target detection on the X-ray to detect the site infected with the COVID-19 virus and obtain the infected area. The present invention first performs fine-grained classification on the X-ray to quickly screen out X-rays that are not COVID-19. On this basis, target detection is used to further determine the COVID-19 virus infection area, greatly improving the efficiency of identification, detection, and operation. For COVID-19 category images, the infection status of the patient can be further analyzed, such as the size and location of the infected area, and the infection status can be visualized. The present invention can identify and detect all Chest X-RAYs and can assist doctors in diagnosing COVID-19.

Description

Technical Field

[0001] The present invention relates to the technical field of cross-application of fine-grained image recognition and target detection, and in particular to a method for identifying and detecting COVID-19 in X-rays based on fine-grained image recognition and target detection algorithms. Background Art

[0002] During the critical period of the outbreak, faced with a large number of susceptible people to be tested and confirmed patients, it usually takes doctors from more than ten minutes to half an hour to read a lung CT scan. This reading speed is difficult to meet the quantity demanded by the peak of the epidemic, which in turn affects the time it takes to obtain test results.

[0003] Currently, the most mature application area of AI+CT imaging in China is the identification of lung nodules. AI can effectively identify nodules that are easily missed, such as solid nodules and ground-glass nodules smaller than 6 mm. However, there is currently no application for multi-disease detection and diagnosis of the lungs. Most existing technologies use image segmentation technology to perform segmentation and analysis on lung images. Although this method produces a more beautiful visual interface, the image segmentation processing speed is relatively slow, and it is unable to quickly screen out images that are not caused by COVID-19. Summary of the Invention

[0004] The purpose of this invention is to quickly screen non-COVID-19 images with minimal time overhead and to perform more detailed analysis and visualization of COVID-19 infection status in COVID-19 images. To address the challenges of analyzing multiple pneumonia types in COVID-19 medical imaging scenarios, a method for detecting COVID-19 in X-rays based on fine-grained image recognition and target detection algorithms is proposed.

[0005] The present invention adopts the following technical solutions to solve the above problems:

[0006] The X-ray COVID-19 detection method based on fine-grained image recognition and target detection algorithms uses a detection system to automatically perform detection, including the following steps:

[0007] Step 1: Predetermine three categories in the detection system's dataset: 1. COVID-19, 2. Normal lungs, and 3. Viral pneumonia. Obtain the likelihood of the three categories for the X-rays being tested. Use the PMG algorithm to perform fine-grained image recognition on the X-rays input to the detection system to obtain the likelihood of each category.

[0008] Step 2: Obtain the category of the X-ray film to be detected: For the probability of each category of the X-ray film to be detected obtained in step 1, use the min-max normalization algorithm to perform normalization processing, select the category with the greatest probability as the classification result, and obtain and output the recognition result, which is one of the three categories mentioned above;

[0009] Step 3: Perform triage processing on the classification results obtained in step 2: If the recognition result is Class II normal lungs or Class III viral pneumonia, the classification result is directly returned as the diagnosis result; if the recognition result is Class I COVID-19, the X-ray film continues to be processed for target detection;

[0010] Step 4: Obtain the COVID-19 X-ray test results: Use the YOLOv4 target detection algorithm to detect the area infected with the new coronavirus in the X-ray, obtain the location of the area, and visually mark the area to obtain the test results of the marked COVID-19 infected area, that is, the diagnosis result.

[0011] Specifically, the specific process of step 1 is: the input X-ray film to be tested is first normalized, and then the array converted by normalization is input into the Progressive-Multi-Granularity network. After calculation by the network, the elu activation function is used to obtain the respective probabilities of the three categories of "1. New coronavirus pneumonia 2. Normal lungs 3. Viral pneumonia".

[0012] Specifically, the specific process of step 2 is: standardize the three data of the three categories of X-ray film obtained in step 1, map the data to the interval [0,1], and use the normalized values as the confidence of each category, that is, the relative possibility of belonging to the category, and select the category with the largest confidence as the recognition result output.

[0013] Specifically, the specific process of step 3 is: inputting the output result obtained in step 2 into a selection classifier for diversion processing selection, specifically:

[0014] In step 3.1, if the input selection classifier is "2. Normal lung" or "3. Viral pneumonia", the information flow enters the "result return" module, which directly returns the classification result to the user;

[0015] In step 3.2, if the input content of the selection classifier is the "1. COVID-19" category, the information flow enters the "Virus Area Detection Module", which detects the COVID-19 virus-infected area.

[0016] Specifically, the specific process of step 4 is:

[0017] Get the image of the "1. COVID-19" category from the information flow in step 3.2. First, perform data normalization processing on the "Virus Area Detection Module". Then input the processed data into the neural network of yolov4 to obtain the area infected by the new coronavirus and the location of the area. Visually mark the area and obtain the detection result of the COVID-19 category, that is, the diagnosis result.

[0018] The yolov4 neural network is trained under the condition of data enhancement, with CSPDarkNet53 as the backbone, combined with the feature pyramid SPP, using CIOU loss function, learning rate cosine annealing decay and Mish activation function, and processed by the network.

[0019] Compared with the prior art, the present invention has the following technical effects:

[0020] The present invention first performs fine-grained classification on X-rays, allowing rapid screening of non-COVID-19 X-rays. Based on this, target detection is then used to further identify the COVID-19 infected area, significantly improving the efficiency of identification, detection, and operation. For COVID-19 images, the patient's infection status, such as the size and location of the infected area, can be further analyzed and visualized. This invention can identify and detect all chest X-rays, assisting doctors in diagnosing COVID-19.

[0021] By reducing the analysis of non-essential information, the present invention can effectively utilize the information of fine-grained image recognition results, thereby providing users with more accurate information more quickly and improving the recognition effect. On this basis, target detection is used to accurately locate the target object, thereby avoiding useless information from affecting the detection effect and greatly improving the operating efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 Flowchart of the method of the present invention.

[0023] Figure 2 This is the result diagram obtained by fine-grained image recognition of the present invention, where 2a is COVID-19, 2b is normal lungs, and 2c is viral pneumonia.

[0024] Figure 3 The results of the COVID-19 image analysis are marked on the map. DETAILED DESCRIPTION

[0025] The embodiments of the present invention are described in detail below, and examples of the embodiments are shown in the accompanying drawings. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be interpreted as limiting the present invention.

[0026] like Figure 1As shown, the original image is first normalized, and then the processed result is input into the fine-grained image recognition PMG (Progressive-Multi-Granularity-Training) network for fine-grained image recognition classification. The classification results have three specific categories, namely "COVID-19", "Normal Lungs", and "Viral Pneumonia". Then, by "selecting a classifier", the three categories are divided into two categories, namely "COVID-19 category" and "Non-COVID-19 category", and enter two different processing modules respectively. If it is "Non-COVID-19 category", it enters the "Result Return Module" and returns the analysis results to the user visually; if it is "COVID-19 category", it first enters the "Virus Area Detection Module" to detect the virus-infected area target, and then enters the "Result Return Module" to return the analysis results to the user visually. This method can effectively solve the problem of slow medical image segmentation speed, make full use of effective information without increasing time complexity, and eliminate the interference of invalid redundant information.

[0027] The present invention provides a method for detecting COVID-19 in X-rays based on fine-grained image recognition and target detection algorithms, comprising the following steps:

[0028] In step 1, the input X-ray film to be tested is first normalized. Then, the array converted by normalization is input into the Progressive-Multi-Granularity network. After calculation, the network obtains the probability of the three categories of "1. COVID-19 2. Normal lungs 3. Viral pneumonia".

[0029] Step 2: Normalize the three data obtained in step 1 and map them to the interval [0, 1]. Use the normalized values as the confidence of each category, that is, the relative possibility of belonging to that category. Select the category with the highest confidence and output it as the recognition result.

[0030] like Figure 2 As shown in Figure 1, the three images are processed in steps 1 and 2, respectively, and the results of fine-grained image recognition and classification are obtained. The results of fine-grained image recognition are marked in the upper right corner of the image. These two steps complete the fine-grained classification of the input image.

[0031] Step 3: Input the output result obtained in step 2 into a selection classifier for diversion processing and selection. The specific process is as follows:

[0032] In step 3.1, if the input content of the selection classifier is the "2. Normal lung" category or the "3. Viral pneumonia" category, the information flow enters the "result return" module, that is, the classification result is directly returned to the user.

[0033] In step 3.2, if the input content of the selection classifier is the "1. COVID-19" category, the information flow enters the "Virus Area Detection Module", which detects the COVID-19 virus-infected area.

[0034] Step 4. Obtain images of the "1. COVID-19" category from the information flow in step 3.2. First, perform data normalization processing on the "Virus Area Detection Module". Then, input the processed data into the neural network of yolov4. Through network processing, the area infected with the new coronavirus is obtained, and the location of the area is obtained. The area is visually annotated to obtain the detection result of the COVID-19 category, that is, the diagnosis result.

[0035] like Figure 3 As shown in the figure, this is the "Analysis Result Visualization" effect diagram obtained after the COVID-19 image is processed in steps 3 and 4. The rectangular area represents the COVID-19 infected area, and the value on the rectangular box represents the credibility of identifying the area as a COVID-19 infected area.

[0036] The above embodiments are only for illustrating the technical idea of the present invention and cannot be used to limit the protection scope of the present invention. Any changes made on the basis of the technical solution in accordance with the technical idea proposed by the present invention shall fall within the protection scope of the present invention.

Claims

1. X-ray COVID-19 recognition and detection method based on fine-grained image recognition and target detection algorithm, characterized by: The detection system is used to automatically perform the detection, including the following steps: Step 1: Set three categories in the data set of the detection system in advance, namely:

1. New coronary pneumonia, 2. Normal lungs, 3. Viral pneumonia, obtain the probability of the three categories of the X-ray to be tested: use the PMG algorithm to perform fine-grained image recognition on the X-ray to be tested input into the detection system to obtain the probability of each category of the X-ray; Step 2: Obtain the category of the X-ray film to be detected: For the probability of each category of the X-ray film to be detected obtained in step 1, use the min-max normalization algorithm to perform normalization processing, select the category with the greatest probability as the classification result, and obtain and output the recognition result, which is one of the three categories mentioned above; Step 3: Perform triage processing on the classification results obtained in step 2: If the recognition result is Class II normal lungs or Class III viral pneumonia, the classification result is directly returned as the diagnosis result; if the recognition result is Class I COVID-19, the X-ray film continues to be processed for target detection; Step 4: Obtain the COVID-19 X-ray test results: Use the YOLOv4 target detection algorithm to detect the area infected with the new coronavirus in the X-ray, obtain the location of the area, and visually mark the area to obtain the test results of the marked COVID-19 infected area, that is, the diagnosis result.

2. The method for detecting COVID-19 in X-ray films based on fine-grained image recognition and target detection algorithm according to claim 1 is characterized in that: The specific process of step 1 is: The input X-ray film to be tested is first normalized, and then the array converted by normalization is input into the Progressive-Multi-Granularity network. After calculation by the network, the elu activation function is used to obtain the probability of the three categories of "1. COVID-19 2. Normal lungs 3. Viral pneumonia".

3. The method for detecting COVID-19 in X-ray films based on fine-grained image recognition and target detection algorithm according to claim 2 is characterized in that: The specific process of step 2 is: The three data of the three categories of X-ray film obtained in step 1 are standardized and mapped to the interval [0,1]. The values obtained after standardization are used as the confidence of each category, that is, the relative possibility of belonging to the category. The category with the largest confidence is selected as the recognition result output.

4. The method for detecting COVID-19 in X-rays based on fine-grained image recognition and target detection algorithm according to claim 3 is characterized in that: The specific process of step 3 is: The output result obtained in step 2 is input into a selection classifier for diversion processing selection, specifically: In step 3.1, if the input selection classifier is "2. Normal Lung" or "3. Viral Pneumonia," the information flow enters the "Result Return" module, which directly returns the classification result to the user. In step 3.2, if the input selection classifier is "1. COVID-19", the information flow enters the "Virus Area Detection Module", which detects COVID-19 virus-infected areas.

5. The method for detecting COVID-19 in X-rays based on fine-grained image recognition and target detection algorithm according to claim 4 is characterized in that: The specific process of step 4 is: Get the image of the "1. COVID-19" category from the information flow in step 3.

2. First, perform data normalization processing on the "Virus Region Detection Module". Then input the processed data into the YOLOv4 neural network to obtain the area infected by the new coronavirus and the location of the area. Visually mark the area and obtain the detection result of the COVID-19 category, that is, the diagnosis result.

Images