Knowledge graph driven clinical microbiological examination interpretation report generation method
Through a knowledge graph-driven approach, the problems of insufficient verification of the rationality of results and uncontrollable generated content in clinical microbiology test reports were solved, the automated verification of drug sensitivity results and the professional accuracy of the interpretation content were achieved, and the credibility of the reports and diagnostic efficiency were improved.
Patent Information
- Application Number
- CN202510782634.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-12
- Publication Date
- 2025-09-23
Smart Images

Figure CN120690366A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of medical information processing and artificial intelligence technology, and more specifically to a method for generating clinical microbiology test interpretation reports based on a knowledge graph. Background Art
[0002] Artificial intelligence (AI) technology is increasingly being integrated into healthcare, showing great potential in image processing, natural language understanding, and knowledge modeling. In clinical microbiology testing, AI is being used to assist with tasks such as colony detection, zone of inhibition identification, interpretation of antimicrobial susceptibility data, and preliminary report generation, becoming a key driver in the evolution of traditional testing processes towards automation and intelligentization.
[0003] At present, the application of artificial intelligence in microbiological testing scenarios is mainly concentrated in several directions. In terms of image recognition, convolutional neural networks are generally used to complete tasks such as colony detection, inhibition zone edge segmentation and morphological classification, which can complete the perception and structured processing of culture dish images with high precision; in terms of time series modeling, recurrent neural network structures such as long short-term memory networks are used to analyze colony growth curves and drug sensitivity change trends; in terms of text generation, pre-trained language models such as BERT, T5, and GPT are initially applied to the generation, summary and structured transcription of clinical reports; in addition, some systems explore the use of rule-based expert systems or knowledge bases for standardized prompts and recommendations of results. These methods have made certain breakthroughs in image perception, data modeling and language expression, but in actual systems used for the generation of clinical microbiological interpretive reports, they still face the following key problems:
[0004] Lack of verification mechanism for the validity of results: The current system struggles to determine the validity of identified bacterial species and antibiotic susceptibility results, and is unable to identify common medical inconsistencies, irrationalities, or conflicts. Clinical review requires manual review, which carries the risk of missed and misjudgment.
[0005] Generated content is uncontrollable and lacks citation: Most natural language generation models are black-box outputs, lacking traceable explanation sources, knowledge support chains, and citation mechanisms. This results in insufficient credibility of the report content, making it difficult for clinicians to accept it.
[0006] Insufficient professional context modeling: Existing models are mostly based on general corpus training and lack understanding of professional context such as clinical terminology, bacterial-drug relationships, and drug resistance mechanisms. This leads to inappropriate wording and weak reasoning logic in the generated content, affecting the quality of medical explanations.
[0007] In response to the problems of existing artificial intelligence technology in the generation of microbiology test interpretation reports, such as task fragmentation, lack of knowledge verification, and uncontrollable semantic generation, the present invention provides a clinical microbiology test interpretation report generation method driven by knowledge graph. Summary of the Invention
[0008] In view of this, the present invention provides a method for generating a clinical microbiology test interpretation report based on a knowledge graph to solve the problems existing in the background technology.
[0009] In order to achieve the above object, the present invention adopts the following technical solutions:
[0010] A method for generating clinical microbiology test interpretation reports based on knowledge graph drive, comprising:
[0011] Obtain drug sensitivity images, classify them through an image type recognition network, and preprocess the classification results; fuse the preprocessed classification results with the bacterial species identification results and input them into the knowledge graph matching module as knowledge query items;
[0012] The knowledge query item is matched with the pre-trained microbial drug sensitivity expert graph in the knowledge graph matching module. If there is no match, an alarm is issued and the item is marked. If there is a match, the semantic template library mounted in the pre-trained microbial drug sensitivity expert graph is called to generate explanatory words based on the test results and integrate them into a structured test explanation report.
[0013] Preferably, the classifying of the drug sensitivity image by the image type recognition network specifically includes receiving the drug sensitivity image I∈R H×W×C ,The input images include inhibition zone images and colorimetric detection images;
[0014] Inhibition zone image i disk , containing a paper piece and a circular transparent area;
[0015] Colorimetric detection image i color , multiple colorimetric reaction results in microplates;
[0016] Image type recognition network The system classifies the input image:
[0017]
[0018] Preferably, the pretreatment specifically includes:
[0019] If T=disk, the system calls the inhibition zone measurement model Perform circular area segmentation and diameter measurement, the output format is:
[0020] R AST={(a i ,d i ,p i ,s i )},d i =Diameter(Zone i ),s i ∈{S,I,R};
[0021] where a i is the i-th antibiotic, d i is the diameter of the i-th inhibition zone, p i is the infection site of the i-th patient, s i is the sensitivity classification determined by the i-th system, belonging to one of the three categories: sensitive S, intermediate I or resistant R; Diameter refers to the size of the inhibition zone area. i ) represents the diameter of the i-th inhibition zone, Zone represents the inhibition zone area in the image, and Diameter quantifies the inhibitory effect of antibiotics by calculating the diameter of the circular area;
[0022] If T = color, the system calls the colorimetric image analysis model Perform hole location and color level comparison, the output format is:
[0023] R AST ={(a j ,MIC j ,p j ,s j )},s j ∈{S,I,R};
[0024] Among them, MIC j is the minimum inhibitory concentration of the jth antibiotic.
[0025] Preferably, the fusion of the pre-processed classification results and the bacterial species identification results specifically includes: combining the bacterial species identification results C of the bacterial species detection report that must be issued before completing the drug sensitivity test in the microbial testing process species , the fungus-drug-part combination (C species ,a i ,p i ) or (c species ,a j ,p j ) is input into the knowledge graph matching module as a knowledge query item.
[0026] Preferably, the matching of the knowledge query item with the pre-trained microbial drug sensitivity expert graph in the knowledge graph matching module specifically includes: Retrieve knowledge entries that match the current knowledge query:
[0027] Match(c=C species ,a=a k ,s=s k ,p=p k ,d=d k ,MIC=MIC k );
[0028] If there is a consistent six-tuple, the test result is judged to be reasonable; if it does not match, an alarm is issued and the abnormal report number is output; the corresponding knowledge entry reference number is Ref k , where Ref k The reference number of the standard knowledge item is used to indicate the reference source or recommended guideline and serve as the basis for subsequent report interpretation. Manual review and marking are also recommended.
[0029] To match the current knowledge query, call the semantic template library mounted in the pre-trained microbial drug sensitivity expert map:
[0030] T c,a,p =Template(c,a,p);
[0031] Among them, T c,a,p Represents a predefined semantic template for a given bacterial species, antibiotic, and patient infection site, used to generate explanatory suggestions; Template represents a given bacterial species, antibiotic, and infection site;
[0032] And based on the test results k ∈{S,I,R},d k ,MIC k Autocomplete generates explanatory sentences:
[0033] T output =Fill(T c,a,p ,s k ,d k ,MIC k );
[0034] Among them, T output For explanatory statements.
[0035] Preferably, the structured inspection interpretation report specifically includes:
[0036] R={I,C species ,R AST ,{T output},Alert};
[0037] Among them, Alert is an abnormal mark reminder.
[0038] Through the above technical solutions, it can be seen that compared with the existing technology, the present invention provides a method for generating clinical microbiology test interpretation reports based on knowledge graph drive. By integrating image recognition and structured knowledge expression capabilities, an integrated system is constructed that can complete drug sensitivity result identification, drug sensitivity result rationality judgment, interpretation content generation and standard report output. By introducing a microbial drug sensitivity expert knowledge graph to replace traditional templates or static rules, the present invention realizes the automated verification, logical consistency reasoning and clinical rule comparison of the bacteria-drug-site combination test results, significantly improving the scientific nature and credibility of the interpretation content; by establishing a knowledge-driven semantic generation mechanism, it can automatically retrieve and embed matching interpretation suggestions based on the recognition results. BRIEF DESCRIPTION OF THE DRAWINGS
[0039] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.
[0040] Figure 1 This is the overall flow chart provided by the present invention. DETAILED DESCRIPTION
[0041] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.
[0042] The embodiment of the present invention discloses a method for generating a clinical microbiology test interpretation report based on a knowledge graph drive, such as Figure 1 As shown, including:
[0043] Obtain drug susceptibility images, classify them through an image type recognition network, and preprocess the classification results; fuse the preprocessed classification results with the bacterial species identification results and input them into the knowledge graph matching module as knowledge query items;
[0044] The knowledge query item is matched with the pre-trained microbial drug sensitivity expert graph in the knowledge graph matching module. If there is no match, an alarm is issued and the information is marked. If there is a match, the semantic template library mounted in the pre-trained microbial drug sensitivity expert graph is called to generate explanatory words based on the test results and integrate them into a structured test interpretation report.
[0045] In a specific embodiment, classifying drug sensitivity images by an image type recognition network specifically includes receiving a drug sensitivity image I∈R H× W×C, the input image includes the inhibition zone image and the colorimetric detection image;
[0046] Inhibition zone image I disk , containing a paper piece and a circular transparent area;
[0047] Colorimetric detection image I color , multiple colorimetric reaction results in microplates;
[0048] Image type recognition network The system classifies the input image:
[0049]
[0050] In a specific embodiment, the preprocessing specifically includes:
[0051] If T=disk, the system calls the inhibition zone measurement model Perform circular area segmentation and diameter measurement, the output format is:
[0052] R AST ={(a i ,d i ,p i ,s i )},d i =Diameter(Zone i ),s i ∈{S,I,R};
[0053] where a i is the i-th antibiotic, d i is the diameter of the i-th inhibition zone, p i is the infection site of the i-th patient, s i is the sensitivity classification determined by the i-th system, belonging to one of the three categories: sensitive S, intermediate I or resistant R; Diameter refers to the size of the inhibition zone. i ) represents the diameter of the i-th zone of inhibition, derived through image processing and region segmentation. Zone represents the zone of inhibition within the image, while Diameter quantifies the antibiotic's inhibitory effect by calculating the diameter of the circular region. This allows the system to accurately measure and quantify the effects of antibiotics based on images, providing precise drug susceptibility data for clinical use.
[0054] If T = color, the system calls the colorimetric image analysis model Perform hole location and color level comparison, the output format is:
[0055] R AST ={(a j ,MIC j ,p j ,s j )},s j ∈{S,I,R};
[0056] Among them, MIC j is the minimum inhibitory concentration of the jth antibiotic.
[0057] Image type recognition network (ConvNeXt architecture):
[0058] The image type recognition network determines the input image type (inhibition zone image or colorimetric image). Based on the ConvNeXt architecture, this network efficiently extracts and classifies local image features. The network structure comprises multiple convolutional layers, pooling layers, and fully connected layers, using residual connections to enhance the network's ability to learn deep features. The final output is an image type prediction, which determines the subsequent processing path.
[0059] Input: II∈R H×W×C , where H is the height of the image, W is the width of the image, and C is the number of channels of the image. For color images, C = 3 (RGB images), and for grayscale images, C = 1.
[0060] Network structure: Based on the ConvNeXt architecture, it includes convolutional layers, activation functions, pooling layers, and multiple fully connected layers. The core feature of this network is that it combines the advantages of convolution operations and the Transformer structure, improving feature extraction through improved convolutional layers and reducing the number of model parameters.
[0061] Output: T∈{disk,color}, which represents the classification result of the image type, disk represents the inhibition zone image, and color represents the colorimetric image.
[0062] Inhibition zone measurement model (U-Net architecture):
[0063] The inhibition zone measurement model is used to segment the inhibition zone from an image and calculate its diameter. This model, based on a U-Net architecture, employs an encoder-decoder structure and features are transferred via skip connections, ensuring sufficient learning of image details. The final model output is the diameter of the inhibition zone.
[0064] Input: I disk ∈R H×W×C The input is the inhibition zone image, H and W are the height and width of the image respectively, and C is the number of channels. The model extracts features from the image and performs accurate segmentation through the U-Net architecture.
[0065] Network structure: U-Net architecture, including encoder (convolutional layer and pooling layer) and decoder (deconvolution layer), which transmits local features through skip connections to ensure that detailed information is not lost.
[0066] Output: d i =Diameter(Zone i ), represents the diameter of the ith inhibition zone, in millimeters. Zone i is the inhibition zone area obtained by image segmentation, d i is the calculated diameter.
[0067] Colorimetric image analysis model (ResNet architecture):
[0068] The colorimetric image analysis model is used to analyze colorimetric reaction results from the broth microdilution method. Based on the ResNet architecture, this model extracts color features through convolutional layers, uses regression methods to predict minimum inhibitory concentrations (MICs), and performs antimicrobial susceptibility classification. By comparing the color intensity of each antibiotic with a standard colorimetric chart, the model calculates the color intensity and determines the antimicrobial susceptibility classification.
[0069] Input: I color ∈R H×W×C , the input is a colorimetric image, H and W are the height and width of the image respectively, and C is the number of channels of the image, usually 3 (RGB image).
[0070] Network Structure: ResNet architecture, using a residual learning mechanism, effectively alleviates the vanishing gradient problem in deep network training. The convolutional layer extracts the color features of the image, performs regression processing on the feature map, and ultimately outputs the antibacterial concentration.
[0071] Output: R AST ={(a j ,MIC j ,s j )},s j ∈{S,I,R}, where a j is the name of the antibiotic, MIC j is the minimum inhibitory concentration obtained by colorimetric analysis, s j It is a drug sensitivity classification determined by the system based on the MIC value, which is divided into three categories: sensitive (S), intermediate (I) and resistant (R).
[0072] These neural network models provide the present invention with powerful image recognition and analysis capabilities, can automatically process different types of drug sensitivity detection images, and provide accurate data support for subsequent drug sensitivity judgment and report generation.
[0073] In a specific embodiment, the fusion of the pre-processed classification results and the bacterial species identification results specifically includes: combining the bacterial species identification results C of the bacterial species detection report that must be issued before completing the drug sensitivity test in the microbial testing process species , the fungus-drug-part combination (C species ,a i ,p i ) or (C species ,a j ,p j ) is input into the knowledge graph matching module as a knowledge query item.
[0074] In a specific embodiment, matching the knowledge query item with the pre-trained microbial drug sensitivity expert graph in the knowledge graph matching module specifically includes: Retrieve knowledge entries that match the current knowledge query:
[0075] Match(c=C species ,a=a k ,s=s k ,p=p k ,d=d k ,MIC=MIC k );
[0076] If there is a consistent six-tuple, the test result is judged to be reasonable; if it does not match or is a spectrum taboo item (such as the presence of a known drug resistance mechanism), the system automatically triggers the alarm module to output the following content: abnormal report number; the corresponding knowledge entry reference number is Ref k , where Ref k The reference number of the standard knowledge item is used to indicate the reference source or recommended guideline and serve as the basis for subsequent report interpretation. Manual review and marking are recommended.
[0077] In a specific embodiment, generating explanation words according to the detection results specifically includes:
[0078] To match the current knowledge query, call the semantic template library mounted in the pre-trained microbial drug sensitivity expert map:
[0079] T c,a,p =Template(c,a,p);
[0080] Among them, T c,a,p"Template" represents a predefined semantic template for a given bacterial species, antibiotic, and patient infection site, used to generate explanatory recommendations. Based on the given bacterial species, antibiotic, and infection site, the system retrieves and invokes the corresponding predefined semantic template from the expert knowledge base of antimicrobial susceptibility testing to generate explanatory recommendations for antimicrobial susceptibility test results. These templates typically include an evaluation of the antimicrobial susceptibility results, clinical recommendations, or further treatment suggestions, helping physicians make more accurate decisions.
[0081] The template content is automatically filled in based on the drug susceptibility test results, for example:
[0082] If the strain is sensitive to the antibiotic (s=S), the template will generate “This strain is sensitive to this drug, and continued use is recommended”;
[0083] If the bacterial species is resistant to the antibiotic (s=R), the template will generate “This bacteria is resistant to this drug, and it is recommended to change the drug”;
[0084] If it is intermediately sensitive (s=I), the result is "This bacterium is intermediately sensitive to this drug, and it is recommended to adjust the dose according to pharmacokinetics."
[0085] And based on the test results k ∈{S,I,R},d k ,MIC k Autocomplete generates explanatory sentences:
[0086] T output =Fill(T c,a,p ,s k ,d k ,MIC k );
[0087] Among them, T output For explanatory statements.
[0088] In a specific embodiment, the structured inspection interpretation report includes:
[0089] R={I,C species ,R AST ,{T output},Alert};
[0090] The object will be rendered into a standard format PDF document through the typesetting module, including: image recognition image (automatically marked with colony / circle diameter); culture and drug sensitivity table; textual interpretation suggestions and knowledge reference numbers; abnormal mark reminder Alert (if any).
[0091] The final report can be automatically pushed to the hospital's LIS or HIS system, achieving seamless integration with clinical processes.
[0092] The present invention uses the PyTorch programming framework to build and train the microbial inspection image recognition and interpretation system of the present invention. Image data is automatically collected by an embedded camera module in the incubator and transmitted to a central server via the intranet. Image types include images of inhibition zones and microbroth dilution colorimetric images. The system's image recognition module uses a multi-task neural network model based on the ConvNeXt architecture. The training data comes from a self-built microbial drug sensitivity image set with a total of more than 80,000 images. The model is trained using the AdamW optimizer, with an initial learning rate set to 3e-4 and a batch size of 16.
[0093] The knowledge graph module is implemented using Neo4j. The graph structure contains over 5,000 standardized triples, covering bacterial species, antibiotic combinations, and sensitivity determination rules, and supports bidirectional reasoning and semantic extension. Interpretation suggestions are generated by a semantic template system using a rule-based slot filling framework, and are manually reviewed and structured by clinical experts.
[0094] The system, deployed on an edge server platform equipped with NVIDIA A100 GPUs, integrates automatic recognition, knowledge verification, semantic generation, and report output modules, supporting closed-loop processing and automatic archiving. Tests on simulated samples confirmed that the system met clinical requirements for image recognition accuracy, consistency in drug susceptibility results, and accuracy in generated interpretations.
[0095] Compared with the existing automatic generation technology of microbiological test reports, the knowledge graph-driven clinical microbiological test interpretation report generation method provided by the present invention has the following significant advantages: (1) It adopts automatic image type recognition and dual-channel image processing structure, which can effectively distinguish between inhibition zone images and colorimetric images, realize accurate recognition adapted to different detection methods, and improve the adaptability and scalability of the system to multi-source images; (2) By integrating the knowledge graph-driven result verification module, the rationality verification of the combination of bacterial species and antibiotics is realized, and potential inconsistent or erroneous results can be automatically identified, thereby enhancing the system's intelligent decision-making ability and clinical safety; (3) A controllable template filling mechanism is used to realize the structured output of explanatory semantics, so that the generated content is professional, accurate, and well-organized, and has traceability, overcoming the problem of uncontrollable and unauditable output of traditional language models; (4) The system as a whole constructs a closed-loop mechanism from image acquisition, result recognition, knowledge judgment to report generation, which can realize high automation of the microbiological test process, reduce manual intervention, and improve diagnostic efficiency and consistency.
[0096] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Reference can be made to the common and similar parts between the various embodiments. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method description.
[0097] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not limited to the embodiments shown herein but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A method for generating clinical microbiology test interpretation reports based on knowledge graph drive, characterized in that: include: Acquire drug sensitivity images, classify the drug sensitivity images through an image type recognition network, and preprocess the classification results; The pre-processed classification results are integrated with the bacterial species identification results and input into the knowledge graph matching module as knowledge query items; The knowledge query item is matched with the pre-trained microbial drug sensitivity expert graph in the knowledge graph matching module. If there is no match, an alarm is issued and the item is marked. If there is a match, the semantic template library mounted in the pre-trained microbial drug sensitivity expert graph is called to generate explanatory words based on the test results and integrate them into a structured test explanation report.
2. A method for generating a clinical microbiology test interpretation report based on knowledge graph drive according to claim 1, characterized in that: The classifying of the drug sensitivity image by the image type recognition network specifically includes receiving a drug sensitivity image I∈R H×W×C ,The input images include inhibition zone images and colorimetric detection images; Inhibition zone image i disk , containing a paper piece and a circular transparent area; Colorimetric detection image I color , multiple colorimetric reaction results in microplates; Image type recognition network The system classifies the input image:
3. A method for generating a clinical microbiology test interpretation report based on knowledge graph drive according to claim 2, characterized in that: The pretreatment specifically includes: If T=disk, the system calls the inhibition zone measurement model Perform circular area segmentation and diameter measurement, the output format is: R AST ={(a i ,d i ,p i ,s i )},d i =Diameter(Zone i ),s i ∈{S,I,R}; where a i is the i-th antibiotic, d i is the diameter of the i-th inhibition zone, p i is the infection site of the i-th patient, s i is the sensitivity classification determined by the i-th system, belonging to one of the three categories: sensitive S, intermediate I or resistant R; Diameter is the size of the inhibition zone area, Diameter (Zone i ) represents the diameter of the i-th inhibition zone; If T = color, the system calls the colorimetric image analysis model Perform hole location and color level comparison, the output format is: R AST ={(a j ,MIC j ,p i ,s j )},s j ∈{S,I,R}; Among them, a j is the jth antibiotic, MIC j is the minimum inhibitory concentration of the jth antibiotic, p j is the infection site of the jth patient, s j The sensitivity classification determined by the jth system belongs to one of the three categories: sensitive S, intermediate I or resistant R.
4. A method for generating a clinical microbiology test interpretation report based on knowledge graph drive according to claim 3, characterized in that: The fusion of the pre-processed classification results and the bacterial species identification results specifically includes: combining the bacterial species identification results C of the bacterial species detection report completed before the drug sensitivity test in the microbial testing process species , the fungus-drug-part combination (C species ,a i ,p i ) or (C species ,a j ,p j ) is input into the knowledge graph matching module as a knowledge query item.
5. A method for generating a clinical microbiology test interpretation report based on knowledge graph drive according to claim 4, characterized in that: The matching of the knowledge query item with the pre-trained microbial drug sensitivity expert graph in the knowledge graph matching module specifically includes: Retrieve knowledge items that match the current knowledge query term, Ref k Reference number for the corresponding knowledge item: Match(c=C species ,a=a k ,s=s k ,p=p k ,d=d k ,MIC=MIC k ); If there is a consistent six-tuple, the test result is judged to be reasonable; if it does not match, an alarm is issued and an abnormality report number is output; at the same time, manual review and marking are recommended.
6. A method for generating a clinical microbiology test interpretation report based on knowledge graph drive according to claim 5, characterized in that: Generating explanatory words according to the detection results specifically includes: To match the current knowledge query, call the semantic template library mounted in the pre-trained microbial drug sensitivity expert map: T c,a,p =Template(c,a,p); Among them, T c,a,p Represents a predefined semantic template for a given bacterial species, antibiotic, and patient infection site, used to generate explanatory suggestions; Template represents a given bacterial species, antibiotic, and infection site; And based on the test results k ∈{S,I,R},d k ,MIC k Autocomplete generates explanatory sentences: T output =Fill(T c,a,p ,s k ,d k ,MIC k ); Among them, T output For explanatory statements.
7. The method for generating a clinical microbiology test interpretation report based on a knowledge graph drive according to claim 6, wherein the structured test interpretation report is integrated into: R={I,C species ,R AST ,{T output },Alert}; Among them, Alert is an abnormal mark reminder.