Chest X-ray assisted diagnosis method and system based on deep convolutional neural network
The chest X-ray film is preprocessed and screened through deep convolutional neural network, combined with binary classification and detection models, and the problem of inaccurate lesion area positioning in the existing technology is solved, providing more interpretable diagnostic results. Improves the efficiency and accuracy of doctors' diagnosis.
Patent Information
- Application Number
- CN202011000526.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-09-22
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2040-09-22
AI Technical Summary
The existing AI intelligent auxiliary diagnosis methods cannot accurately give the types of diseases corresponding to the lesion area in chest X-rays, and the predicted results are insufficient, resulting in inaccurate clinical diagnosis.
Using a deep convolutional neural network-based method, chest X-rays are pre-processed and screened, chest positive images are screened through binary classification models, and detection models are used to detect disease types and mark lesion areas to provide more interpretability diagnostic results.
It realizes rapid screening of chest X-rays and precise positioning of lesions, improves the efficiency and accuracy of doctors' diagnosis, and reduces workload.
Smart Images

Figure CN112241961B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of medical technology, and in particular to a chest X-ray film auxiliary diagnosis method and system based on deep convolutional neural network. Background Art
[0002] Chest X-rays are a common tool for examining or diagnosing chest diseases. In tertiary hospitals, the average number of X-rays produced every day is very large. On the one hand, the doctors who read the films will gradually accumulate fatigue during the long reading process, which may lead to misdiagnosis. On the other hand, in small hospitals in towns, although the average number of films taken every day is not large, the lack of experience of the doctors who read the films may also lead to misdiagnosis. This urgently requires an auxiliary diagnostic tool that can help doctors reduce misdiagnosis or provide diagnostic information.
[0003] With the development of deep learning, more and more deep convolutional neural network algorithm technologies have been applied to medical images, and AI intelligent assisted diagnosis systems have emerged.
[0004] Most of the existing AI-assisted diagnosis methods use internationally available data sets (such as ChestXray14) or report data collected from hospitals. After natural language processing, film-level disease labels are obtained from the reports. A multi-label classification network model or multiple single disease classification models are trained. The chest X-rays are then predicted using the class activation mapping (or weighted gradient class activation mapping) method to convert the disease feature information learned by the deep convolutional neural network model into a heat map. The heat map shows where the approximate lesion area of the disease is, thereby further assisting doctors in diagnosis.
[0005] First, the quality of films in international public datasets varies greatly, with some films being overexposed, misaligned, and with poor imaging. In addition, most of the labels for international public datasets or data collected by hospitals are extracted from diagnostic reports, and the extraction method cannot guarantee that the labels are 100% correct. The deep convolutional neural network trained with such data will seriously deviate from the actual doctor's diagnosis results.
[0006] Secondly, the prediction result of the above method is either a 2-point result, or it can only mark the suspicious lesion area but not the corresponding disease type or sign of the suspicious lesion area, or it can only indicate the suspected disease type but cannot give the corresponding lesion area. That is, the existing technology is basically based on the disease classification of the whole piece and does not specifically locate the location of the disease. Even if the class activation mapping technology is used to achieve weakly supervised semantic segmentation capabilities, the lesion area information obtained is often not accurate enough and the false positive rate is very high.
[0007] Since clinicians need to accurately write the specific location of the disease in the diagnosis report, the existing technical methods are inconsistent with the clinical diagnosis situation and it is difficult to provide effective help in assisting doctors in diagnosis. Summary of the invention
[0008] In response to the above technical problems, an embodiment of the present invention provides a chest X-ray film auxiliary diagnosis method and system based on a deep convolutional neural network to solve the technical problem that the AI intelligent auxiliary diagnosis method given by the traditional algorithm cannot give the type of disease corresponding to the lesion area and has poor interpretability.
[0009] In a first aspect, an embodiment of the present invention provides a chest X-ray film-assisted diagnosis method based on a deep convolutional neural network, characterized in that it includes: preprocessing a chest X-ray film to obtain an initial X-ray film image that meets format requirements; screening the initial X-ray film image to detect whether the initial X-ray film image is a chest anteroposterior image; inputting the chest anteroposterior image into a binary classification model of a deep convolutional neural network for positive and negative classification; inputting the chest anteroposterior image with a positive result into a detection model of a deep convolutional neural network to detect the disease type of the chest anteroposterior image and to contour-mark the lesion area in the chest anteroposterior image; and displaying the disease type and the lesion area corresponding to the chest anteroposterior image.
[0010] Optionally, the preprocessing of the chest X-ray film to obtain an initial X-ray film image that meets the format requirements includes: mapping all pixel values of the chest X-ray film to a normal distribution to obtain a window width and a window position; removing noise pixels outside the window width interval, and mapping the removed pixels to an interval range of 0-255 to obtain the initial X-ray film image.
[0011] Optionally, the screening of the initial X-ray film image is specifically: inputting the initial X-ray film image into a chest positive screening model for screening, the chest positive screening model comprising: a Resnet-34 feature extraction network and two fully connected neural networks, the Resnet-34 feature extraction network is used to extract chest features from the initial X-ray film image; the first fully connected neural network is used to determine whether the chest features are chest positive; the second fully connected neural network is used to confirm the photometric interpretation of the chest features.
[0012] Optionally, before inputting the chest AP image into a binary classification model of a deep convolutional neural network, the method further includes: if the photometric interpretation of the chest feature is a grayscale range with increasing pixel values from light to dark, then processing the pixels of the initial X-ray image to obtain a photometric interpretation of a grayscale range with increasing pixel values from dark to light.
[0013] Optionally, the binary classification model of the deep convolutional neural network is used to extract chest features from the initial X-ray image, and classify the extracted chest features into positive and negative.
[0014] Optionally, the detection model of the deep convolutional neural network includes: a feature extraction network, a feature fusion network, a region generation network, a qualifier, a locator and a segmenter, wherein the output of the feature extraction network is the input of the feature fusion network; the output of the feature fusion network is the input of the region generation network; the output of the region generation network is the input of the qualifier, and the qualifier is used to detect the disease type of the chest AP image; the output of the region generation network is the input of the locator, and the locator is used to locate the lesion area; the locator input is the output of the segmenter, and the segmenter is used to mark the contour of the lesion area.
[0015] Optionally, the method also includes: when the classification result in the binary classification model of the deep convolutional neural network is positive, but the disease confidence output by the detection model of the deep convolutional neural network is less than a set threshold, the detection model of the deep convolutional neural network is forced to output the contour of the lesion area and the disease type corresponding to the maximum confidence.
[0016] Optionally, the method further includes: visually displaying the lesion area and the disease type corresponding to the lesion area in a chest disease report.
[0017] Optionally, the training sets of the binary classification model of the deep convolutional neural network and the detection model of the deep convolutional neural network are both from an image archiving and communication system.
[0018] In a second aspect, an embodiment of the present invention provides a chest X-ray auxiliary diagnosis system based on a deep convolutional neural network, comprising: a preprocessing module, which is used to preprocess a chest X-ray to obtain an initial X-ray image that meets the format requirements; a chest disease detection module, which includes: a chest positive screening unit, a positive-negative classification unit and a lesion area positioning and qualitative unit; the chest positive screening unit is used to screen the initial X-ray image to detect whether the initial X-ray image is a chest positive image; the positive-negative classification unit is used to input the chest positive image into a binary classification model of a deep convolutional neural network for positive-negative classification; the lesion area positioning and qualitative unit is used to input the chest positive image with a positive result into the detection model of the deep convolutional neural network to detect the disease type of the chest positive image and to contour the lesion area in the chest positive image; a display module is used to display the disease type corresponding to the chest positive image and the visualized lesion area.
[0019] The chest X-ray auxiliary diagnosis method and system based on deep convolutional neural network provided in the embodiment of the present invention can not only screen the positivity of chest X-rays, but also locate the lesion area, and mark the type of disease or signs in the lesion area, thereby providing doctors with more interpretable reference opinions, thereby improving the efficiency of doctors in issuing reports and reducing their workload. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] One or more embodiments are exemplarily described by pictures in the corresponding drawings, and these exemplified descriptions do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, and unless otherwise stated, the figures in the drawings do not constitute proportional limitations.
[0021] Figure 1 A schematic diagram of a process of a chest X-ray assisted diagnosis method based on a deep convolutional neural network according to an embodiment of the present invention;
[0022] Figure 2 It is a structural framework diagram of a chest anteroposterior screening model provided by an embodiment of the present invention;
[0023] Figure 3 is a schematic diagram of a chest positive screening process based on a deep convolutional neural network provided by an embodiment of the present invention;
[0024] Figure 4 It is a structural framework diagram of a binary classification model of a deep convolutional neural network provided by an embodiment of the present invention;
[0025] Figure 5 It is a structural block diagram of a detection model of a deep convolutional neural network provided by an embodiment of the present invention;
[0026] Figure 6 This is a structural block diagram of a chest X-ray auxiliary diagnosis system based on a deep convolutional neural network provided in an embodiment of the present invention. DETAILED DESCRIPTION
[0027] In order to facilitate the understanding of the present invention, the present invention is described in more detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that when an element is described as "fixed to" another element, it can be directly on another element or there can be one or more centered elements therebetween. When an element is described as "connected" to another element, it can be directly connected to another element or there can be one or more centered elements therebetween. The orientation or positional relationship indicated by the terms "upper", "lower", "inner", "outer", "bottom" etc. used in this specification is based on the orientation or positional relationship shown in the accompanying drawings, only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation of the present invention. In addition, the terms "first", "second", "third", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance.
[0028] Unless otherwise defined, all technical and scientific terms used in this specification have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in this specification and in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used in this specification includes any and all combinations of one or more of the related listed items.
[0029] The present invention proposes an intelligent auxiliary diagnosis method and system that can quickly screen the positive and negative of chest X-ray films and accurately locate the chest lesion area. The main function of this method and system is to first quickly screen out positive films through a binary classification model of a deep convolutional neural network to achieve the effect of quickly excluding negative films, and then accurately locate the lesion area in the positive film and predict which disease it belongs to through another deep convolutional neural network detection model, thereby providing doctors with more effective and explainable reference opinions, which can greatly improve the work efficiency of doctors and reduce their workload.
[0030] The following first describes in detail the specific implementation process of this method and the training process of the binary classification model of the deep convolutional neural network and the detection model of the deep convolutional neural network.
[0031] See also Figure 1 , Figure 1 A chest X-ray assisted diagnosis method based on a deep convolutional neural network is provided in an embodiment of the present invention. Figure 1 As shown, the method comprises the following steps:
[0032] 101. Preprocess the chest X-ray film to obtain an initial X-ray image that meets the format requirements.
[0033] This step converts the chest X-ray into the format required by the following models.
[0034] Specifically, the present invention first maps all pixel values on the X-ray film to a normal distribution, and uses the width of the pixel interval (μ-3σ, μ+3σ) as the window width (σ is the standard deviation, μ is the mean), and the center of the interval as the window position; then, about 0.3% of the noise pixels outside the window width interval are removed, and the pixels after noise removal are mapped to the range of 0 to 255; finally, the X-ray film is scaled to the corresponding size according to the input requirements of different models.
[0035] In the present invention, the input size requirement of the following chest AP screening model for chest AP film screening is: 256×256; the input size requirement of the following binary classification model of the deep convolutional neural network for chest positivity screening is: 256×256.
[0036] 102. Screen the initial X-ray image to detect whether the initial X-ray image is a chest frontal image.
[0037] Screening the X-ray film initial image is specifically: inputting the X-ray film initial image into the chest anteroposterior screening model for screening. In the embodiment of the present invention, the main function of the chest anteroposterior screening model is to determine whether the input X-ray film initial image is a chest anteroposterior film, and whether its photometric interpretation is a grayscale range of increasing pixel values from light to dark (MONOCHROME1) or from dark to light (MONOCHROME2). The structural framework of the model is as follows Figure 2 shown.
[0038] Since the method of the present invention is only applicable to chest AP films, it is necessary to screen whether the chest is in the AP position. In the actual operation process, due to the operating errors of doctors or film-taking staff, the information of the photometric interpretation may be entered incorrectly, resulting in completely opposite imaging. In actual operation, doctors can manually adjust their MONOCHROME1 to MONOCHROME2 when reading the film to save this operating error, and the method of the present invention will not read the photometric interpretation attribute information of the data header information. Therefore, the chest AP screening model adds photometric interpretation to automatically determine whether the information entry of the photometric interpretation is incorrect.
[0039] Specifically, the chest anteroposterior screening model includes: a Resnet-34 feature extraction network and two fully connected neural networks, the Resnet-34 feature extraction network is used to extract chest features from the initial X-ray image; the first fully connected neural network is used to determine whether the chest features are in the anteroposterior position; and the second fully connected neural network is used to confirm the photometric interpretation of the chest features.
[0040] In step 101, image data with a size requirement of 256×256 for chest AP screening is obtained. The processing process of inputting the 256×256 image data into the chest AP screening model is as follows:
[0041] First, the 256×256 image data is copied twice, then overlapped into a 3×256×256 structure and then converted into a 1×3×256×256 structure.
[0042] Then the 1×3×256×256 structure image data is input into the Resnet-34 feature extraction network for network extraction to obtain a 1×1024×8×8 feature map.
[0043] The 1×1024×8×8 feature map is processed by the average pooling layer to obtain a 1×1024×1×1 vector, which is then converted into a 1×1024 structure. It is then input into the first fully connected neural network and the second fully connected neural network with an output channel of 1, respectively. Finally, the two values obtained are processed by the sigmoid activation function layer.
[0044] Finally, we get two probability values with threshold values between 0 and 1, where the probability value corresponding to the first fully connected neural network indicates the probability that the input image data is a chest anteroposterior film. By default, if the probability value is greater than or equal to 0.5, it means that the image data is a chest anteroposterior film, otherwise it is not a chest anteroposterior film. The screening results are as follows: Figure 3 shown.
[0045] The probability value corresponding to the second fully connected neural network represents the probability that the photometric interpretation of the input image data is MONOCHROME 2. By default, if the value is greater than or equal to 0.5, it means that the photometric interpretation of the image data is MONOCHROME 2, otherwise it is MONOCHROME 1.
[0046] When the photometric interpretation is MONOCHROME1, the input image data needs to be converted into: the opposite of each value plus 255 to adjust MONOCHROME1 to MONOCHROME2.
[0047] It should be noted that the chest positive screening model, the following two-classification model of the deep convolutional neural network for chest yin-yang screening, and the following detection model of the deep convolutional neural network for the localization and qualitative analysis of chest lesion areas are all trained with training data that is consistent with the real clinical environment.
[0048] The steps of making the training data are as follows:
[0049] 1. Collect chest X-ray images and corresponding diagnostic reports from the hospital;
[0050] 2. Desensitize and quality control the collected chest X-ray images;
[0051] 3. Analyze images in advance based on the diagnostic report;
[0052] 4. Doctor’s review and annotation of chest X-ray images;
[0053] 5. Algorithm personnel review again.
[0054] In the present invention, chest X-ray images and corresponding diagnostic reports are collected from the hospital by obtaining chest X-ray image data from the hospital's PACS system (Picture Archiving and Communication Systems) or DR, CR equipment through the DICOM protocol, and then collecting the films and their diagnostic reports according to strict search conditions (such as Table 1).
[0055] Table 1
[0056]
[0057]
[0058] Desensitization and quality control of the collected chest X-ray images require the elimination of some sensitive information, such as the patient's name, the name of the testing institution, the name of the doctor, etc.; then, the chest AP films are preliminarily screened out from the collected chest X-ray images, thereby removing a large number of lateral films and non-chest X-ray films; and through manual quality control review, the omitted non-chest AP films and other films of poor quality, such as overexposure, poor imaging, bedside films, pediatric films, and other image data that do not meet the training requirements are further removed.
[0059] When analyzing X-ray images in advance based on diagnostic reports, it is impossible to give all the data to doctors for labeling because the data collected from hospitals is large in volume and most of them are negative (even if "positive" has been selected during the operation). Therefore, the present invention proposes a method for extracting keywords from diagnostic reports when training data. When disease keywords and treated keywords exist at the same time, it is considered that the patient does not have the disease, thereby quickly selecting positive data, and according to specific needs, a certain number of different disease categories are selected for labeling, which can greatly reduce the cost of labeling.
[0060] When doctors mark and review, junior doctors can mark the X-ray images first, outlining the lesion area in the form of polygons. Senior doctors can then review their markings, mainly to check whether the junior doctors have made mistakes or missed marks. If there are missed marks, they will be supplemented, and if there are wrong marks, they will be corrected.
[0061] The algorithm staff will review the data again because they have professional experience in training deep convolutional neural networks. Only rigorous and accurate labeled data can train an accurate network model. Therefore, the algorithm staff will review the data labeled by the doctors. The main content of the review is the rigor of the labeled polygons. For example, some polygons related to pleural effusion are too large and include the background area outside the lungs, so they need to be corrected.
[0062] The training data produced by the above method is close to the real clinical environment and has high data quality.
[0063] 103. The chest frontal image is input into the binary classification model of the deep convolutional neural network for positive and negative classification.
[0064] In the embodiment of the present invention, the main function of the binary classification model of the deep convolutional neural network is to quickly analyze the positive and negative of the chest X-ray. The structural framework of the model is as follows Figure 4 shown.
[0065] In step 101, image data with a size requirement of 256×256 for chest AP screening is obtained. The process of inputting the 256×256 image data into the binary classification model of the deep convolutional neural network for processing is as follows:
[0066] First, the 256×256 image data is copied twice, then overlapped into a 3×256×256 structure and then converted into a 1×3×256×256 structure.
[0067] Then the 1×3×256×256 structure image data is input into the feature extraction network of Densenet-121 for network extraction to obtain a 1×1024×8×8 feature map.
[0068] The 1×1024×8×8 feature map is processed by the average pooling layer to obtain a 1×1024×1×1 vector, which is then converted into a 1×1024 structure. It is then input into a fully connected neural network with a channel number of 1, and finally the obtained value is processed using the sigmoid activation function.
[0069] Finally, a probability value between 0 and 1 is obtained, which indicates the probability that the input image data is a positive film. By default, if the probability value is greater than or equal to 0.5, it indicates that the image data is a positive film, otherwise it is a negative film.
[0070] 104. Input the chest frontal image with a positive result into a detection model of a deep convolutional neural network to detect the disease type of the chest frontal image and mark the contour of the lesion area in the chest frontal image.
[0071] In an embodiment of the present invention, the main function of the detection model of the deep convolutional neural network is to locate the specific lesion area in the positive film, and perform a qualitative analysis of the specific disease type for each located area. The disease types in the chest lesion area include the following 17 types: atelectasis, enlarged cardiac shadow, pleural effusion, infiltration, mass, nodule, pneumonia, pneumothorax, pulmonary consolidation, pulmonary edema, emphysema, pulmonary fibrosis, pleural thickening, diaphragmatic hernia, tuberculosis, rib fracture and aortic calcification.
[0072] The structural framework of the deep convolutional neural network detection model is as follows Figure 5 As shown. It mainly includes: feature extraction network, feature fusion network, region generation network, qualifier, locator and segmenter.
[0073] Among them, the feature extraction network adopts the efficientNet-b2 structure, and the feature fusion network adopts the bidirectional feature pyramid fusion structure. This can more effectively extract more representative features and better integrate the information of shallow and high-level semantic features, providing a basis for subsequent positioning and finalization.
[0074] The region generation network consists of a convolutional layer and two independent convolutional layers. The function of the first convolutional layer is to preliminarily analyze the feature information of the input image data and buffer the direct impact of the back-propagation gradient on the main line during training; two independent convolutional layers, one is used to predict the candidate box of the lesion area, and the other is used to predict the probability value of the candidate box being the lesion area.
[0075] The qualifier performs qualitative analysis on the candidate box output by the regional production network to analyze whether it is diseased. If so, it predicts which of the 17 diseases or signs it belongs to. The locator further optimizes the candidate box output by the regional generation network to output a more accurate positioning box. The role of the segmenter is to segment the lesion area optimized by the locator to find the precise outline of the lesion area.
[0076] In step 101, image data with a size requirement of 640×640 for chest AP screening is obtained. The process of inputting the 640×640 image data into the detection model of the deep convolutional neural network for processing is as follows:
[0077] The 1×3×768×768 structured image is used as input data and passed through the feature extraction network and feature fusion network. The region generation network generates 1000 candidate frames of the lesion region. Then, the 1000 candidate frames of the lesion region and their corresponding feature maps are combined and scaled into feature maps of 1000×256×7×7 structure using the bilinear interpolation method and then input into the qualitative device and the locator. Each candidate frame will be analyzed and the confidence of 18 categories (17 categories of diseases or signs + 1 category of no disease) and the refined frames of the corresponding categories (a total of 18 refined frames) will be output. Then, the category with the largest confidence and the refined frame corresponding to the category will be selected as the output. Finally, the contour of the lesion region will be segmented out by the segmenter (the coordinate information of the contour will eventually be scaled back to the original image size of 768×768).
[0078] After the above-mentioned uneven detection is completed, it is also necessary to organize and output the output information of the chest AP screening model, the binary classification model of the deep convolutional neural network, and the detection model of the deep convolutional neural network. In some embodiments, if the classification result in the binary classification model of the deep convolutional neural network is positive, but the disease confidence output by the detection model of the deep convolutional neural network is less than the set threshold, then in the post-processing stage, the detection model of the deep convolutional neural network is forced to output the disease type corresponding to the maximum confidence and the outline of the annotated lesion area.
[0079] 105. Display the disease type and the visualized lesion area corresponding to the chest AP image.
[0080] This step is to outline the contour of the lesion area in the chest frontal image and mark the disease name or sign corresponding to the contour. At the same time, based on the suspicious probability of the lesion area, a heat map can be made for the doctor's diagnosis reference.
[0081] In some embodiments, the lesion area and the disease type corresponding to the lesion area can also be visually displayed in the chest disease report.
[0082] The chest X-ray auxiliary diagnosis method based on deep convolutional neural network provided in the embodiment of the present invention can not only screen the positivity of the chest X-ray, but also locate the lesion area, and mark the type of disease or signs in the lesion area, thereby providing doctors with more interpretable reference opinions, thereby improving the efficiency of doctors in issuing reports and reducing their workload.
[0083] The embodiment of the present invention also provides a chest X-ray auxiliary diagnosis system 60 based on a deep convolutional neural network, as shown in the figure 6, which includes: a preprocessing module 61, a chest disease detection module 62 and a display module 63.
[0084] The preprocessing module 61 is used to preprocess the chest X-ray film to obtain an initial X-ray film image that meets the format requirements;
[0085] The chest disease detection module 62 includes: a chest positive screening unit 621, a positive-negative classification unit 622 and a lesion area positioning and qualitative unit 623.
[0086] The chest AP screening unit 621 is used to screen the initial X-ray film image to detect whether the initial X-ray film image is a chest AP image; the positive-negative classification unit 622 is used to input the chest AP image into the binary classification model of the deep convolutional neural network for positive-negative classification; the lesion area localization and qualitative unit 623 is used to input the chest AP image with a positive result into the detection model of the deep convolutional neural network to detect the disease type of the chest AP image and to mark the contour of the lesion area in the chest AP image.
[0087] The display module 63 is used to display the disease type and the visualized lesion area corresponding to the chest AP image.
[0088] In some embodiments, the chest X-ray assisted diagnosis system 60 based on deep convolutional neural network also includes: a post-processing module 64, which is used to force the deep convolutional neural network detection model to output the disease type corresponding to the maximum confidence and the outline of the marked lesion area if the classification result in the binary classification model of the deep convolutional neural network is positive, but the disease confidence output by the detection model of the deep convolutional neural network is less than the set threshold.
[0089] The specific application of the preprocessing module 61, the chest positive screening unit 621, the positive-negative classification unit 622, the lesion area positioning and qualitative unit 623 and the post-processing module 64 is the same as the chest X-ray film auxiliary diagnosis method based on deep convolutional neural network in the above-mentioned embodiment, and will not be repeated here.
[0090] The chest X-ray auxiliary diagnosis system based on deep convolutional neural network provided by the embodiment of the present invention can not only screen the positivity of the chest X-ray, but also locate the lesion area, and mark the type of disease or signs in the lesion area, thereby providing doctors with more interpretable reference opinions, thereby improving the efficiency of doctors in issuing reports and reducing their workload.
[0091] Those skilled in the art should further realize that the various steps of the exemplary dual-light image integration method described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the composition and steps of each example according to function. Whether these functions are executed in hardware or software depends on the specific application and design constraints of the technical solution.
[0092] Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention. The computer software may be stored in a computer-readable storage medium, and when the program is executed, it may include the process of the embodiments of the above methods. The storage medium may be a disk, an optical disk, a read-only storage memory, or a random access memory, etc.
[0093] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them. Under the concept of the present invention, the technical features in the above embodiments or different embodiments may also be combined, the steps may be implemented in any order, and there are many other changes in different aspects of the present invention as described above, which are not provided in detail for the sake of simplicity. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they may still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present invention.
Claims
1. A chest X-ray assisted diagnosis method based on deep convolutional neural network, characterized in that: include: Preprocess the chest X-ray film to obtain an initial X-ray image that meets the format requirements; Screening the initial X-ray image to detect whether the initial X-ray image is a chest frontal image; Inputting the chest frontal image into a binary classification model of a deep convolutional neural network for positive and negative classification; Inputting the chest frontal image with a positive result into a detection model of a deep convolutional neural network to detect the disease type of the chest frontal image and to mark the contour of the lesion area in the chest frontal image; Displaying the disease type and the lesion area corresponding to the chest frontal image; The preprocessing of the chest X-ray film to obtain an initial X-ray film image that meets the format requirements includes: Mapping all pixel values of the chest X-ray film to a normal distribution to obtain a window width and a window position; The noise pixels outside the window width interval are removed, and the removed pixels are mapped to the interval range of 0-255 to obtain an initial X-ray image; The screening of the X-ray film initial image is specifically as follows: inputting the X-ray film initial image into a chest anteroposterior screening model for screening, wherein the chest anteroposterior screening model includes: a Resnet-34 feature extraction network and two fully connected neural networks, The Resnet-34 feature extraction network is used to extract chest features from the initial X-ray image; The first fully connected neural network is used to determine whether the chest feature is a normal chest position; A second fully connected neural network is used to confirm the photometric interpretation of the chest features; Before inputting the chest anteroposterior image into a binary classification model of a deep convolutional neural network, the method further includes: If the photometric interpretation of the chest feature is a grayscale range with increasing pixel values from light to dark, then the pixels of the X-ray film initial image are processed to obtain a photometric interpretation of the grayscale range with increasing pixel values from dark to light; The binary classification model of the deep convolutional neural network is used to extract chest features from the initial X-ray image and classify the extracted chest features into positive and negative.
2. The method according to claim 1, characterized in that The detection model of the deep convolutional neural network includes: a feature extraction network, a feature fusion network, a region generation network, a qualifier, a locator and a segmenter. The output of the feature extraction network is the input of the feature fusion network; The output of the feature fusion network is the input of the region generation network; The output of the region generation network is the input of the qualifier, and the qualifier is used to detect the disease type of the chest positive image; The output of the region generation network is the input of the locator, and the locator is used to locate the lesion region; The locator input is the output of the segmenter, and the segmenter is used to mark the contour of the lesion area.
3. The method according to claim 2, characterized in that The method further comprises: When the classification result in the binary classification model of the deep convolutional neural network is positive, but the disease confidence output by the detection model of the deep convolutional neural network is less than the set threshold, The detection model of the deep convolutional neural network is forced to output the contour of the lesion area and the disease type corresponding to the maximum confidence.
4. The method according to claim 3, characterized in that The method further comprises: The lesion area and the disease type corresponding to the lesion area are visually displayed in the chest disease report.
5. The method according to claim 1, characterized in that The training sets of the binary classification model of the deep convolutional neural network and the detection model of the deep convolutional neural network both come from an image archiving and communication system.
6. A chest X-ray assisted diagnosis system based on deep convolutional neural network, characterized in that: include: A preprocessing module, used for preprocessing the chest X-ray film to obtain an initial X-ray film image that meets the format requirements; A chest disease detection module, the chest disease detection module comprising: a chest positive screening unit, a positive and negative classification unit and a lesion area positioning and qualitative unit; The chest positive screening unit is used to screen the initial image of the X-ray film to detect whether the initial image of the X-ray film is a chest positive image; The positive-negative classification unit is used to input the chest frontal image into a binary classification model of a deep convolutional neural network to perform positive-negative classification; The lesion area positioning and qualitative unit is used to input the chest frontal image with a positive result into the detection model of the deep convolutional neural network to detect the disease type of the chest frontal image and to mark the contour of the lesion area in the chest frontal image; A display module, used for displaying the disease type and the visualized lesion area corresponding to the chest frontal image; The preprocessing of the chest X-ray film to obtain an initial X-ray film image that meets the format requirements includes: Mapping all pixel values of the chest X-ray film to a normal distribution to obtain a window width and a window position; The noise pixels outside the window width interval are removed, and the removed pixels are mapped to the interval range of 0-255 to obtain an initial X-ray image; The screening of the X-ray film initial image is specifically as follows: inputting the X-ray film initial image into a chest anteroposterior screening model for screening, wherein the chest anteroposterior screening model includes: a Resnet-34 feature extraction network and two fully connected neural networks, The Resnet-34 feature extraction network is used to extract chest features from the initial X-ray image; The first fully connected neural network is used to determine whether the chest feature is a normal chest position; A second fully connected neural network is used to confirm the photometric interpretation of the chest features; The preprocessing module is also used for: If the photometric interpretation of the chest feature is a grayscale range with increasing pixel values from light to dark, then the pixels of the X-ray film initial image are processed to obtain a photometric interpretation of the grayscale range with increasing pixel values from dark to light; The binary classification model of the deep convolutional neural network is used to extract chest features from the initial X-ray image and classify the extracted chest features into positive and negative.
