Intelligent auxiliary diagnosis method and device for pulmonary tuberculosis and readable storage medium

Abstract

The invention provides an intelligent auxiliary diagnosis method and device for pulmonary tuberculosis and a readable storage medium, and the method comprises the steps: obtaining a lung medical image of an object, the lung medical image comprising a normal position chest radiograph and a lateral position chest radiograph; performing focus recognition on the lung medical image through a pre-trained recognition model to obtain a normal focus recognition result and a lateral focus recognition result; performing three-dimensional reconstruction through an anatomical feature matching algorithm according to the orthotopic chest radiograph and the lateral chest radiograph to obtain a chest three-dimensional anatomical structure; obtaining a focus positioning result in the chest three-dimensional anatomical structure based on an anterior focus recognition result and a lateral focus recognition result; and according to the focus positioning result in the chest three-dimensional anatomical structure and the clinical data of the pulmonary tuberculosis patient, obtaining a pulmonary tuberculosis diagnosis result of the object. The method can improve the focus positioning precision, enhance the model generalization ability, and improve the accuracy of the pulmonary tuberculosis diagnosis result.

Description

Technical Field

[0001] The present application relates to the field of intelligent medical technology, and in particular to an intelligent auxiliary diagnosis method, device and readable storage medium for pulmonary tuberculosis. Background Art

[0002] Pulmonary tuberculosis is caused by infection with Mycobacterium tuberculosis (abbreviated as tuberculosis bacillus or tuberculosis bacteria).

[0003] It is a chronic infectious lung disease caused by tuberculosis. When tuberculosis bacteria infect the lungs, the inflammatory lesions that appear in the lung tissue are called primary lesions.

[0004] Currently, imaging is widely used for tuberculosis examinations, such as chest X-rays or computer tomography (CT), which can help doctors observe lung lesions such as nodules, cavities or inflammation.

[0005] However, existing imaging examination methods for pulmonary tuberculosis still have problems such as low lesion localization accuracy and poor detection effect. Summary of the Invention

[0006] The technical problem to be solved by this application is to provide an intelligent auxiliary diagnosis method, device and readable storage medium for tuberculosis in response to the above-mentioned deficiencies in the existing technology, so as to solve the problems existing in the existing technology.

[0007] In the first aspect, the present application provides an intelligent auxiliary diagnosis method for tuberculosis,

[0008] Methods include:

[0009] S1. Obtaining a lung medical image of the subject, wherein the lung medical image includes an anteroposterior chest X-ray and a lateral chest X-ray;

[0010] S2. Using the pre-trained recognition model, lesion recognition is performed on the lung medical image to obtain frontal lesion recognition results and lateral lesion recognition results;

[0011] S3. Based on the anteroposterior and lateral chest radiographs, a three-dimensional reconstruction is performed using an anatomical feature matching algorithm to obtain the three-dimensional anatomical structure of the chest;

[0012] S4. Obtaining a lesion localization result in the three-dimensional anatomical structure of the chest based on the frontal lesion recognition result and the lateral lesion recognition result;

[0013] S5. Acquire clinical data of the pulmonary tuberculosis patient, and obtain a diagnosis result of pulmonary tuberculosis of the subject based on the lesion localization result in the three-dimensional anatomical structure of the chest and the clinical data of the pulmonary tuberculosis patient.

[0014] In some embodiments, S3 includes:

[0015] S31, extract key anatomical features of the anteroposterior and lateral chest radiographs respectively;

[0016] S32. Based on the key anatomical features of the frontal and lateral chest radiographs, three-dimensional reconstruction is performed using an anatomical feature matching algorithm to obtain the three-dimensional anatomical structure of the chest.

[0017] In some embodiments, S31 includes:

[0018] Automatically preprocess chest X-ray images by clustering pixels in the chest X-ray images and construct a search path probability map;

[0019] Constructing a search path feature set based on the search path probability graph and combining it with a wavelet operator;

[0020] The key anatomical structure feature positioning result of the chest X-ray image is obtained according to the search path feature set.

[0021] In some embodiments, S32 includes:

[0022] The key anatomical features of the frontal and lateral chest radiographs are superimposed, and the frontal and lateral anatomical structure maps are generated from the frontal and lateral key anatomical structure features respectively through an adversarial neural network;

[0023] The generated anteroposterior and lateral anatomical structure maps are optimized using the root mean square error (RMSE) loss function;

[0024] Three-dimensional reconstruction is performed based on the optimized anteroposterior and lateral anatomical structure mapping to obtain the three-dimensional anatomical structure of the chest.

[0025] In some embodiments, S4 includes:

[0026] Based on the results of frontal lesion recognition and lateral lesion recognition, the lesions in the three-dimensional anatomical structure of the chest are marked through an attention mechanism, and the marks are classified into two categories through a fully connected convolutional neural network to obtain the lesion localization results in the three-dimensional anatomical structure of the chest.

[0027] In some embodiments, S5 includes:

[0028] Constructing a pulmonary tuberculosis lesion feature dataset based on the clinical data of the pulmonary tuberculosis patients;

[0029] Based on the pulmonary tuberculosis lesion feature dataset, classify and identify the lesion localization results in the three-dimensional anatomical structure of the chest to obtain a pulmonary tuberculosis diagnosis result for the subject;

[0030] The diagnosis result of pulmonary tuberculosis of the subject includes at least one of the volume, distribution and composition of pulmonary tuberculosis lesions.

[0031] In some embodiments, further comprising:

[0032] A heat map of the lesion area is generated according to the diagnosis result of pulmonary tuberculosis of the subject.

[0033] In a second aspect, the present application provides an intelligent auxiliary diagnosis device for pulmonary tuberculosis, the device comprising:

[0034] an image acquisition module configured to acquire lung medical images of a subject, wherein the lung medical images include an anteroposterior chest X-ray and a lateral chest X-ray;

[0035] A lesion recognition module is configured to perform lesion recognition on lung medical images using a pre-trained recognition model to obtain frontal lesion recognition results and lateral lesion recognition results;

[0036] A three-dimensional reconstruction module is configured to perform three-dimensional reconstruction based on the frontal chest radiograph and the lateral chest radiograph using an anatomical feature matching algorithm to obtain the three-dimensional anatomical structure of the chest;

[0037] a lesion localization module, configured to obtain a lesion localization result in the three-dimensional anatomical structure of the chest based on the frontal lesion recognition result and the lateral lesion recognition result;

[0038] The auxiliary diagnosis module is configured to obtain clinical data of a pulmonary tuberculosis patient and obtain a diagnosis result of pulmonary tuberculosis of the subject based on the lesion localization result in the three-dimensional anatomical structure of the chest and the clinical data of the pulmonary tuberculosis patient.

[0039] In a third aspect, the present application provides an intelligent auxiliary diagnosis device for tuberculosis, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to implement the intelligent auxiliary diagnosis method for tuberculosis described in the first aspect above.

[0040] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the intelligent auxiliary diagnosis method for tuberculosis described in the first aspect above.

[0041] The present application provides an intelligent assisted diagnosis method, device, and readable storage medium for pulmonary tuberculosis, the method comprising: obtaining a lung medical image of a subject, the lung medical image comprising an anteroposterior chest X-ray and a lateral chest X-ray; performing lesion recognition on the lung medical image using a pre-trained recognition model to obtain an anteroposterior lesion recognition result and a lateral lesion recognition result; performing three-dimensional reconstruction based on the anteroposterior chest X-ray and the lateral chest X-ray using an anatomical feature matching algorithm to obtain a three-dimensional anatomical structure of the chest; obtaining a lesion localization result in the three-dimensional anatomical structure of the chest based on the anteroposterior lesion recognition result and the lateral lesion recognition result; obtaining clinical data of a pulmonary tuberculosis patient, and obtaining a pulmonary tuberculosis diagnosis result of the subject based on the lesion localization result in the three-dimensional anatomical structure of the chest and the clinical data of the pulmonary tuberculosis patient. The present application provides an intelligent assisted diagnosis method for pulmonary tuberculosis, which uses a multi-angle image matching algorithm and combines anteroposterior and lateral chest X-rays to achieve three-dimensional spatial mapping of lesions, thereby improving the accuracy of lesion localization. In addition, by fusing multimodal data, the model generalization ability can be enhanced, and the accuracy of the pulmonary tuberculosis diagnosis result can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0043] Figure 1 A flowchart of an intelligent auxiliary diagnosis method for pulmonary tuberculosis provided in an embodiment of the present application;

[0044] Figure 2 A schematic diagram of the structure of an intelligent auxiliary diagnosis device for tuberculosis provided in an embodiment of the present application;

[0045] Figure 3 A schematic diagram of the structure of another intelligent auxiliary diagnosis device for tuberculosis provided in an embodiment of the present application.

[0046] The above drawings illustrate specific embodiments of the present application, which will be described in more detail below. These drawings and the textual description are not intended to limit the scope of the present application in any way, but rather to illustrate the concepts of the present application to those skilled in the art by reference to specific embodiments. DETAILED DESCRIPTION

[0047] In order to enable those skilled in the art to better understand the technical solution of the present application, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0048] It should be understood that the specific embodiments and drawings described herein are only used to explain the present application, rather than to limit the present application.

[0049] It can be understood that, in the absence of conflict, the various embodiments and features in the embodiments of the present application can be combined with each other.

[0050] It will be understood that, for the sake of ease of description, the drawings of this application only show the parts related to this application, while the parts not related to this application are not shown in the drawings.

[0051] It can be understood that each unit and module involved in the embodiments of the present application may correspond to only one physical structure, or may be composed of multiple physical structures, or multiple units and modules may be integrated into one physical structure.

[0052] It can be understood that the terms "first", "second", etc. in the embodiments of the present application are used to distinguish different objects, or to distinguish different processing of the same object, rather than to describe a specific order of objects.

[0053] It is understandable that, in the absence of conflict, the functions and steps marked in the flowcharts and block diagrams of the present application may occur in an order different from that marked in the drawings.

[0054] It is understood that the flowcharts and block diagrams of the present application illustrate the possible architectures, functions, and operations of the systems, devices, equipment, and methods according to the various embodiments of the present application. Each box in the flowchart or block diagram may represent a unit, module, program segment, or code, which contains executable instructions for implementing the specified functions. Moreover, each box or combination of boxes in the block diagram and flowchart may be implemented by a hardware-based system that implements the specified functions, or by a combination of hardware and computer instructions.

[0055] It can be understood that the units and modules involved in the embodiments of the present application can be implemented by software or hardware, for example, the units and modules can be located in a processor.

[0056] Research has shown that existing imaging methods for pulmonary tuberculosis have the following shortcomings:

[0057] (1) Limitations of single-angle imaging: Existing technologies do not integrate information from frontal and lateral chest radiographs, making it impossible to achieve three-dimensional positioning of lesions, resulting in a high missed detection rate.

[0058] (2) Weak data coupling capability: Clinical text and imaging data are not deeply integrated, resulting in the model relying on a single data source and poor generalization.

[0059] (3) High maintenance cost: Multiple software copyright applications lead to code redundancy, increasing the difficulty of system maintenance.

[0060] In view of the above shortcomings, this application provides an intelligent auxiliary diagnosis method for tuberculosis, the main concepts of which include:

[0061] (1) A multi-angle image matching algorithm is proposed, which is combined with the frontal and lateral chest radiographs to achieve three-dimensional spatial mapping of the lesion and improve the positioning accuracy.

[0062] (2) Design a deep reinforcement learning framework to integrate multimodal data (images, clinical texts) and enhance the model's generalization capabilities.

[0063] (3) Reduce code redundancy and improve system maintainability through an integrated algorithm architecture.

[0064] The present application provides an intelligent tuberculosis diagnosis system and method based on multi-angle image matching and deep reinforcement learning, aiming to form an efficient and accurate tuberculosis identification and diagnosis technology algorithm, and by constructing an AI intelligent tuberculosis diagnosis auxiliary system embedded in the tuberculosis diagnosis and treatment process, enable it to make clinical transformation applications in tuberculosis diagnosis and disease management guidance, so that it can be applied to the clinical diagnosis of primary designated medical institutions, improve the tuberculosis prevention and control service system, enhance the tuberculosis prevention and control capabilities of primary designated medical institutions, and comprehensively reduce the missed diagnosis rate and misdiagnosis rate of tuberculosis.

[0065] The following specific embodiments describe in detail the technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0066] The present application provides an intelligent auxiliary diagnosis method for tuberculosis. The working process of the method can be implemented by electronic devices, such as computers, handheld smart terminals, etc. For the convenience of explanation, the embodiments of the present application are described with the method execution subject being a computer.

[0067] Figure 1 A schematic diagram of the intelligent auxiliary diagnosis method for tuberculosis provided in the embodiment of the present application is shown in FIG. Figure 1 As shown, the present application provides an intelligent auxiliary diagnosis method for pulmonary tuberculosis, which includes S1-S5, as follows:

[0068] S1. Obtaining a lung medical image of the subject, wherein the lung medical image includes an anteroposterior chest X-ray and a lateral chest X-ray;

[0069] The AP chest X-ray primarily demonstrates the lungs' overall shape, size, and position, as well as their relationship to surrounding structures (such as the heart, ribs, and diaphragm). It can be used to identify abnormalities such as consolidation, effusions, nodules, cavities, pneumothorax, and rib fractures. The AP chest X-ray is a two-dimensional image, which has limitations in displaying certain overlapping structures. For example, lesions at the apex, base, and mediastinum may be difficult to clearly visualize.

[0070] The lateral chest radiograph can provide complementary information to the frontal chest radiograph, showing more clearly the front-to-back and top-to-bottom relationship of the lungs, as well as their relative position to the mediastinum, diaphragm and other structures. It helps to observe lesions in the apex, base of the lungs, beside the spine, behind the heart, etc., as well as the width of the mediastinum and the position of the trachea. The lateral chest radiograph can make up for the shortcomings of the frontal chest radiograph, provide more comprehensive lung imaging information, and help diagnose lung lesions.

[0071] S2. Using the pre-trained recognition model, lesion recognition is performed on the lung medical image to obtain frontal lesion recognition results and lateral lesion recognition results;

[0072] The recognition model is trained using historical tuberculosis-related images and related labels. The training process of the recognition model is as follows:

[0073] 1. Image acquisition

[0074] Manual Collection: Upload patient X-ray medical images in a variety of file formats, including the DICOM (Digital Imaging and Communications in Medicine) format for medical devices, and automatically capture patient data and image information from medical imaging files. If patient information is incomplete, the system provides a function for supplementary entry of patient data and synchronizes this information to the disease prevention platform.

[0075] Automatic collection: Automatically collect the patient's basic information and X-ray medical image files, support the automatic extraction of patient information and image information in medical image files, and supplementary entry of patient information.

[0076] 2. Image preprocessing

[0077] DICOM information extraction: According to the DICOM protocol standards, service information in DICOM images, such as image information, patient information, etc., is read through standard interfaces.

[0078] Image format conversion: Through the medical image conversion function, DICOM format medical images in the image library are converted into PNG format images to hide the service information in the DICOM image files.

[0079] 3. Lesion feature extraction

[0080] Automatic labeling: Pre-analyze the image based on the lesion feature library and extract the corresponding lesion feature areas and their feature types to provide a reference for the manual labeling process.

[0081] Manual Labeling: Expert users can use the labeling tool to mark characteristic lesion areas in the image and label the lesion type. To ensure the objectivity of the labeling results, image labeling will adopt a "double randomization" rule, that is, the same image will be randomly labeled by different experts, and images marked by the same expert will also be automatically randomly selected.

[0082] Tagging Management: Users can review their previous tagging records at any time. Key data in the tagging record includes: image number, tagging expert, tagging time, tagging status, tagging details, and whether the transaction has been settled. During the tagging process, users can temporarily save and submit tagging results. Users can modify saved tagging results at any time. Submitted tagging results are considered complete. Once the same image has been tagged by a designated expert, an automatic comparison function is triggered.

[0083] Automatic comparison: Using the image area comparison algorithm, the lesion feature areas of the same image that have been marked are automatically compared. If the difference value of the lesion feature areas of the same image is within the specified threshold range, the marking result is considered valid. All marked lesion feature areas will be stored at the same time as training sample data for the AI intelligent algorithm, providing more training and learning basic data to improve the algorithm's recognition accuracy; if the difference value of the lesion feature areas of the same image exceeds the specified threshold, authoritative experts will be automatically invited to verify.

[0084] Manual Comparison: If the difference in lesion feature areas within the same image exceeds a specified threshold, this function will be automatically triggered, and a leading expert will be invited to conduct a review. Different marking results for the same image can be presented separately, compared, and overlapped to facilitate comparative analysis by leading experts. If the expert disagrees with the result, the lesion feature area can be re-marked. The system will store the final result approved by the expert.

[0085] Marked record query: Provides query management functions for marked records, including query based on number, expert query, marking time and other conditions.

[0086] Marking record statistics: By recording the expert's marking, you can use it to calculate the expert's performance. You can count the expert's markings in a certain time period, or generate a marking record ledger for a certain time period to provide data reference for the expert's performance.

[0087] 4. Image management

[0088] DICOM image viewing: Provides DICOM format image viewing function on the front end to assist doctors in identifying lung diseases.

[0089] DICOM image measurement: Provides DICOM format image measurement function on the front end to assist doctors in segmenting areas of interest, calculating volumes, measuring lengths, etc.

[0090] Window width and level adjustment: A display technology used to observe normal tissue or lesions of varying densities, including window width and window level. Because various tissue structures or lesions have different CT values, optimal visualization of a specific tissue structure requires selecting a window width and level appropriate for that specific tissue or lesion. Dynamic adjustment of window width and level allows physicians to adjust these values to achieve optimal visualization of the area or organ of interest.

[0091] Zoom in and out: The system provides the zoom in and out function for DICOM images. Doctors can zoom in and out on the area of interest in the DICOM image.

[0092] 5. Model training

[0093] The recognition model is obtained by training historical tuberculosis-related images and related labels, so as to identify lesions in lung medical images and obtain the frontal lesion recognition results and lateral lesion recognition results.

[0094] S3. Based on the anteroposterior and lateral chest radiographs, a three-dimensional reconstruction is performed using an anatomical feature matching algorithm to obtain the three-dimensional anatomical structure of the chest;

[0095] In this application, multi-angle chest medical image matching is used to lay the foundation for accurate positioning and automatic identification of lesions.

[0096] Specifically, multi-angle chest medical image matching can provide the basis for accurate and efficient identification of lesion areas under the condition that medical image data does not have information depth. Existing recognition models mainly use fully supervised learning methods and are highly dependent on expert annotation information. This fully demonstrates the importance of expert annotation. However, different divisions of the annotation data set and data subsets of different sizes may train recognition models with different performance. In order to obtain the best recognition performance with the least investment of expert resources (such as working hours), it is necessary to explore and design a multi-angle feature intelligent matching mechanism to estimate the three-dimensional anatomical structure from the two-dimensional projection data to provide a spatial resolution basis for lesion localization. On the other hand, after automatic identification of multi-angle lesions, they are intelligently mapped from two-dimensional projection to three-dimensional anatomical space. Studying the method of information depth reconstruction of two-dimensional projection medical image data will lay the foundation for lesion localization and identification.

[0097] In some embodiments, S3 includes:

[0098] S31, extract key anatomical features of the anteroposterior and lateral chest radiographs respectively;

[0099] S32. Based on the key anatomical features of the frontal and lateral chest radiographs, three-dimensional reconstruction is performed using an anatomical feature matching algorithm to obtain the three-dimensional anatomical structure of the chest.

[0100] Specifically, in clinical practice, medical image acquisition often includes anteroposterior chest X-rays and lateral chest X-rays. Previous research has developed a deep learning algorithm based on anteroposterior chest X-rays, which has achieved efficient extraction of imaging features related to tuberculosis. This application is based on the previous algorithm foundation and realizes the development of an algorithm for extracting imaging features related to tuberculosis in lateral chest X-rays. At the same time, by constructing an anatomical feature matching algorithm for anteroposterior chest X-rays and lateral chest X-rays, the anatomical information of the two-dimensional anteroposterior chest X-ray is further expanded into a quasi-three-dimensional anatomical structure, thereby improving the accuracy of lesion localization, exploring the possibility of extracting pathological features of lesions, and improving the effectiveness of artificial intelligence algorithms in assisting the diagnosis of tuberculosis.

[0101] In some embodiments, S31 includes:

[0102] Automatically preprocess chest X-ray images by clustering pixels in the chest X-ray images and construct a search path probability map;

[0103] Constructing a search path feature set based on the search path probability graph and combining it with a wavelet operator;

[0104] The key anatomical structure feature positioning result of the chest X-ray image is obtained according to the search path feature set.

[0105] Specifically, in order to achieve the matching of frontal and lateral chest X-rays, this application develops an intelligent search path method, which automatically preprocesses the source image by clustering pixels in the source image to construct a search path probability map, and constructs a search path feature set based on the probability map combined with the wavelet operator to improve the efficiency of locating key anatomical structures.

[0106] In some embodiments, S32 includes:

[0107] The key anatomical features of the frontal and lateral chest radiographs are superimposed, and the frontal and lateral anatomical structure maps are generated from the frontal and lateral key anatomical structure features respectively through an adversarial neural network;

[0108] The generated anteroposterior and lateral anatomical structure maps are optimized using the root mean square error (RMSE) loss function;

[0109] Three-dimensional reconstruction is performed based on the optimized anteroposterior and lateral anatomical structure mapping to obtain the three-dimensional anatomical structure of the chest.

[0110] Specifically, the anteroposterior and lateral anatomical feature levels are superimposed, and an adversarial neural network is used to generate lateral anteroposterior anatomical structure mappings from the anteroposterior and lateral anatomical structure features respectively. The generated anatomical structure mapping is optimized using the root mean square error (RMSE) loss function, thereby achieving spatial structure matching of anatomical anteroposterior and lateral chest radiographs.

[0111] S4. Obtaining a lesion localization result in the three-dimensional anatomical structure of the chest based on the frontal lesion recognition result and the lateral lesion recognition result;

[0112] In some embodiments, S4 includes:

[0113] Based on the results of frontal lesion recognition and lateral lesion recognition, the lesions in the three-dimensional anatomical structure of the chest are marked through an attention mechanism, and the marks are classified into two categories through a fully connected convolutional neural network to obtain the lesion localization results in the three-dimensional anatomical structure of the chest.

[0114] Specifically, based on the deep learning algorithm of the previous frontal chest X-ray, this application adopts dual-stream parallel lesion feature extraction, directly superimposes feature maps, uses an attention mechanism network to automatically mark the lesions, and uses a fully connected convolutional neural network to perform binary classification (normal / abnormal) on the marks.

[0115] S5. Acquire clinical data of the pulmonary tuberculosis patient, and obtain a diagnosis result of pulmonary tuberculosis of the subject based on the lesion localization result in the three-dimensional anatomical structure of the chest and the clinical data of the pulmonary tuberculosis patient.

[0116] This application is based on a high-performance deep learning algorithm and achieves high-accuracy identification of lesion areas through the joint recognition of multiple sign types.

[0117] A disease may have multiple signs, or multiple signs may exist at the same time, or different signs may appear in different patients. Different signs on the same medical image may reflect the same disease or different diseases. Therefore, it is necessary to study the problem of joint recognition of multiple signs. This application uses the high generalization ability of the automatic feature exploration mechanism of deep learning, combined with the advantages of the human-like attention mechanism achieved by reinforcement learning, to automatically learn from sample data and form a general understanding of the lesion structure, so that it can capture the manifestations of multiple sign types at the same time and improve the recognition accuracy of the algorithm.

[0118] Deep reinforcement learning (DRL) technology leverages recent breakthroughs in training deep neural networks, combining them with reinforcement learning to surpass the performance of top human experts in multiple game scenarios. A small number of studies have applied DRL technology to two relatively simple subfields: medical image registration and anatomical structure detection. Although preliminary and exploratory, these efforts have already achieved significant success. However, similar research in the subfield of lesion detection and identification is scarce. In fact, compared with anatomical structures, lesions exhibit greater variability (poor consistency) in shape, appearance, location, and size, making the problem significantly more complex. Existing DRL methods from other fields cannot be directly applied. Instead, specialized analysis and processing are required for specific problems, and the algorithmic approach must be modified as necessary. Furthermore, coupling multivariate data requires quantifying textual information and prior empirical data, transforming the feature information space that can be associated with lesion characteristics, and achieving feature balancing of multivariate information. This lays the foundation for intelligent assisted diagnosis platforms that meet clinical diagnostic and treatment workflows.

[0119] Based on this, this application adopts a multi-dimensional information coupling algorithm to integrate medical images and clinical data for auxiliary diagnosis of pulmonary tuberculosis, among which clinical examination results play an important role in the accuracy of pulmonary tuberculosis diagnosis. On the one hand, based on the multi-angle medical image matching algorithm, high-precision anatomical feature extraction of lesions is achieved, and these features are coupled with lesion features to realize an artificial intelligence algorithm for joint diagnosis of pulmonary tuberculosis. On the other hand, based on clinical examination results and clinical manifestations, it can effectively assist imaging information in distinguishing chest lesions from lung lesions, improve the accuracy of pulmonary tuberculosis identification, and improve the accuracy of distinguishing lesions in different lung regions.

[0120] In some embodiments, S5 includes:

[0121] Constructing a pulmonary tuberculosis lesion feature dataset based on the clinical data of the pulmonary tuberculosis patients;

[0122] Based on the pulmonary tuberculosis lesion feature dataset, classify and identify the lesion localization results in the three-dimensional anatomical structure of the chest to obtain a pulmonary tuberculosis diagnosis result for the subject;

[0123] The diagnosis result of pulmonary tuberculosis of the subject includes at least one of the volume, distribution and composition of pulmonary tuberculosis lesions.

[0124] Specifically, clinical data from patients with pulmonary tuberculosis were collected and a dataset of pulmonary tuberculosis lesion features was constructed using random forest and decision tree methods. Based on the results of medical image processing, features were classified, such as lung region, location, and lesion nature, to construct a classified dataset. The features of the examination results were then overlaid with the classified dataset. Using machine learning algorithms such as support vector machines (SVMs), a decision tree statistical model was constructed to output refined auxiliary diagnostic information for pulmonary tuberculosis.

[0125] In some embodiments, further comprising:

[0126] A heat map of the lesion area is generated according to the diagnosis result of pulmonary tuberculosis of the subject.

[0127] Specifically, to enable widespread application of intelligent assisted tuberculosis diagnosis methods, this application utilizes embedded technology to embed the algorithm into an AI-powered intelligent tuberculosis diagnosis assistance system, making it readily available to users. After the system completes the tuberculosis diagnosis based on medical images, it automatically alerts the user, who can then view the patient's medical images and AI-powered diagnosis results on the system's front-end page.

[0128] The system also provides a heatmap display for tuberculosis lesions. After the system identifies and diagnoses a lesion, it marks the lesion area in the form of a heatmap. Users can view the heatmap of the lesion area on the patient's medical image. This heatmap display method provides an excellent learning case for grassroots medical staff.

[0129] In some embodiments, the present application also includes building a diagnostic support library. During the intelligent diagnosis process, diagnostic image data and expert-corrected image data will continue to accumulate. To better support the accuracy improvement of the intelligent diagnostic algorithm, the system will establish multiple thematic libraries. The data in each thematic library can be added, modified, deleted, viewed, queried, and other operations according to actual needs. Specifically, the following are included:

[0130] Original image library: The original image library is used to archive and manage the images collected by the system.

[0131] Tagged image library: The tagged image library is used to archive and manage images marked by experts.

[0132] Lesion Feature Library: The Lesion Feature Library is used to archive and manage lesion area features extracted from labeled images. Administrators can also set and manage lesion feature types.

[0133] Diagnostic result library: The diagnostic result library is used to archive and manage the diagnostic result images after the system AI intelligent diagnosis and identification.

[0134] Patient database: The patient database is used to archive and manage the basic information and previous diagnostic information of patients who require intelligent imaging diagnosis.

[0135] Expert Database: This database is used to archive and manage experts in the medical imaging field. Expert information primarily includes ID number, name, ID number, contact information (mobile phone number), degree, educational background, highest institution of study, professional research field, professional title, position, institution, department, academic background, research achievements, entry date, expert category, and bank account information.

[0136] System configuration management: including user management, role management, organizational structure management, and permission management. User management is to implement the functions of adding users, modifying user information, locking users, unlocking users, deleting users, and user review. Role management is to implement the functions of adding roles, deleting roles, assigning role permissions, and assigning user roles. Organizational structure management is to implement the functions of adding, modifying, deleting, and querying organizational structures. Permission management is to implement permission management and role permission assignment by editing and setting the user directory tree; the permissions of different user roles can be set according to the actual use of the platform, mainly including adding permission groups, deleting permission groups, role permission configuration, adding roles, and deleting roles, so that personalized operation interfaces and function menus can be dynamically generated when different users log in.

[0137] Log audit management: The system automatically records user login and logout operations, and automatically saves key event operations.

[0138] This application provides an intelligent assisted diagnosis method for pulmonary tuberculosis. This method uses a multi-angle image matching algorithm and combines chest radiographs with lateral and frontal views to achieve three-dimensional spatial mapping of lesions, thereby improving lesion localization accuracy. Furthermore, by integrating multimodal data, the model's generalization capabilities can be enhanced, improving the accuracy of pulmonary tuberculosis diagnosis results.

[0139] It should be understood that, although the various steps in the flowcharts of the above embodiments are shown in sequence as indicated by the arrows, these steps are not necessarily performed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they may be performed in other orders. Moreover, at least a portion of the steps in the figure may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily performed at the same time, but may be performed at different times, and their execution order is not necessarily sequential, but may be performed in turn or alternately with other steps or at least a portion of sub-steps or stages of other steps.

[0140] Figure 2 A schematic diagram of the intelligent auxiliary diagnosis device for tuberculosis provided in an embodiment of the present application is shown in FIG. Figure 2 As shown, the present application provides an intelligent auxiliary diagnosis device for pulmonary tuberculosis, the device comprising:

[0141] An image acquisition module 11 is configured to acquire lung medical images of a subject, wherein the lung medical images include an anteroposterior chest X-ray and a lateral chest X-ray;

[0142] a lesion recognition module 12, which is configured to perform lesion recognition on lung medical images using a pre-trained recognition model to obtain frontal lesion recognition results and lateral lesion recognition results;

[0143] A three-dimensional reconstruction module 13 is configured to perform three-dimensional reconstruction based on the frontal chest radiograph and the lateral chest radiograph using an anatomical feature matching algorithm to obtain a three-dimensional anatomical structure of the chest;

[0144] A lesion localization module 14 is configured to obtain a lesion localization result in the three-dimensional anatomical structure of the chest based on the frontal lesion recognition result and the lateral lesion recognition result;

[0145] The auxiliary diagnosis module 15 is configured to obtain clinical data of a pulmonary tuberculosis patient, and obtain a diagnosis result of pulmonary tuberculosis of the subject based on the lesion localization result in the three-dimensional anatomical structure of the chest and the clinical data of the pulmonary tuberculosis patient.

[0146] Regarding the limitation of the intelligent auxiliary diagnosis device for pulmonary tuberculosis, reference may be made to the limitation of the intelligent auxiliary diagnosis method for pulmonary tuberculosis in the above embodiments of this application, which will not be repeated in this embodiment.

[0147] Figure 3 Another schematic diagram of the intelligent auxiliary diagnosis device for tuberculosis provided in the embodiment of the present application is shown in FIG. Figure 3 As shown, the device includes a memory 22 and a processor 21, the memory stores a computer program, and the processor is configured to run the computer program to execute the methods in the above embodiments of the present application.

[0148] The memory is connected to the processor, the memory may be a flash memory, a read-only memory or other memory, and the processor may be a central processing unit or a single-chip microcomputer.

[0149] In some embodiments, the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the methods in the above embodiments of the present application are implemented.

[0150] The computer-readable storage medium includes volatile or non-volatile, removable or non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, computer program modules or other data). Computer-readable storage media include, but are not limited to, RAM (Random Access Memory), ROM (Read-Only Memory), EEPROM (Electrically Erasable Programmable read only memory), flash memory or other memory technology, CD-ROM (Compact Disc Read-Only Memory), digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer.

[0151] It is understood that the above embodiments are merely exemplary embodiments for illustrating the principles of the present application, and the present application is not limited thereto. Those skilled in the art may make various modifications and improvements without departing from the spirit and substance of the present application, and such modifications and improvements are also considered to be within the scope of protection of the present application.

Claims

1. An intelligent auxiliary diagnosis method for pulmonary tuberculosis, characterized in that: The method comprises: S1. Obtaining a lung medical image of the subject, wherein the lung medical image includes an anteroposterior chest X-ray and a lateral chest X-ray; S2. Using the pre-trained recognition model, lesion recognition is performed on the lung medical image to obtain frontal lesion recognition results and lateral lesion recognition results; S3. Based on the anteroposterior and lateral chest radiographs, a three-dimensional reconstruction is performed using an anatomical feature matching algorithm to obtain the three-dimensional anatomical structure of the chest; S4. Obtaining a lesion localization result in the three-dimensional anatomical structure of the chest based on the frontal lesion recognition result and the lateral lesion recognition result; S5. Acquire clinical data of the pulmonary tuberculosis patient, and obtain a diagnosis result of pulmonary tuberculosis of the subject based on the lesion localization result in the three-dimensional anatomical structure of the chest and the clinical data of the pulmonary tuberculosis patient.

2. The intelligent auxiliary diagnosis method for pulmonary tuberculosis according to claim 1, characterized in that: S3, including: S31, extract key anatomical features of the anteroposterior and lateral chest radiographs respectively; S32. Based on the key anatomical features of the frontal and lateral chest radiographs, three-dimensional reconstruction is performed using an anatomical feature matching algorithm to obtain the three-dimensional anatomical structure of the chest.

3. The intelligent auxiliary diagnosis method for pulmonary tuberculosis according to claim 2, characterized in that: S31, including: Automatically preprocess chest X-ray images by clustering pixels in the chest X-ray images and construct a search path probability map; Constructing a search path feature set based on the search path probability graph and combining it with a wavelet operator; The key anatomical structure feature positioning result of the chest X-ray image is obtained according to the search path feature set.

4. The intelligent auxiliary diagnosis method for pulmonary tuberculosis according to claim 2, characterized in that: S32, including: The key anatomical features of the frontal and lateral chest radiographs are superimposed, and the frontal and lateral anatomical structure maps are generated from the frontal and lateral key anatomical structure features respectively through an adversarial neural network; The generated anteroposterior and lateral anatomical structure maps are optimized using the root mean square error (RMSE) loss function; Three-dimensional reconstruction is performed based on the optimized anteroposterior and lateral anatomical structure mapping to obtain the three-dimensional anatomical structure of the chest.

5. The intelligent auxiliary diagnosis method for pulmonary tuberculosis according to claim 1, characterized in that: S4, including: Based on the results of frontal lesion recognition and lateral lesion recognition, the lesions in the three-dimensional anatomical structure of the chest are marked through an attention mechanism, and the marks are classified into two categories through a fully connected convolutional neural network to obtain the lesion localization results in the three-dimensional anatomical structure of the chest.

6. The intelligent auxiliary diagnosis method for pulmonary tuberculosis according to claim 1, characterized in that: S5, including: Constructing a pulmonary tuberculosis lesion feature dataset based on the clinical data of the pulmonary tuberculosis patients; Based on the pulmonary tuberculosis lesion feature dataset, classify and identify the lesion localization results in the three-dimensional anatomical structure of the chest to obtain a pulmonary tuberculosis diagnosis result for the subject; The diagnosis result of pulmonary tuberculosis of the subject includes at least one of the volume, distribution and composition of pulmonary tuberculosis lesions.

7. The intelligent auxiliary diagnosis method for pulmonary tuberculosis according to any one of claims 1 to 6, characterized in that: Also includes: A heat map of the lesion area is generated according to the diagnosis result of pulmonary tuberculosis of the subject.

8. An intelligent auxiliary diagnosis device for pulmonary tuberculosis, characterized in that: The device comprises: an image acquisition module configured to acquire lung medical images of a subject, wherein the lung medical images include an anteroposterior chest X-ray and a lateral chest X-ray; A lesion recognition module is configured to perform lesion recognition on lung medical images using a pre-trained recognition model to obtain frontal lesion recognition results and lateral lesion recognition results; A three-dimensional reconstruction module is configured to perform three-dimensional reconstruction based on the frontal chest radiograph and the lateral chest radiograph using an anatomical feature matching algorithm to obtain the three-dimensional anatomical structure of the chest; a lesion localization module, configured to obtain a lesion localization result in the three-dimensional anatomical structure of the chest based on the frontal lesion recognition result and the lateral lesion recognition result; The auxiliary diagnosis module is configured to obtain clinical data of a pulmonary tuberculosis patient and obtain a diagnosis result of pulmonary tuberculosis of the subject based on the lesion localization result in the three-dimensional anatomical structure of the chest and the clinical data of the pulmonary tuberculosis patient.

9. An intelligent auxiliary diagnosis device for pulmonary tuberculosis, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to implement the intelligent auxiliary diagnosis method for pulmonary tuberculosis according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the intelligent auxiliary diagnosis method for tuberculosis according to any one of claims 1 to 7.

Images