A medical experiment report intelligent checking method and system and a storage medium
By constructing a multimodal physiological dataset and a professional knowledge base, and combining a visual expert model and a language evaluation model, the problem of collaborative text-image review in medical experimental reports was solved, achieving efficient and accurate review results and an interpretable evaluation mechanism, thereby improving the teaching feedback effect.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CHENGDU TME SOFTWARE
- Filing Date
- 2026-01-23
- Publication Date
- 2026-05-12
AI Technical Summary
Existing technologies cannot efficiently and accurately conduct collaborative review of text and images in medical laboratory reports. In particular, medical functional laboratory reports suffer from insufficient domain expertise, a black box reasoning mechanism, and poor end-to-end fusion, resulting in low review efficiency and inconsistent evaluations.
We construct a multimodal physiological dataset and a professional knowledge base, train a visual expert model through supervised fine-tuning to transform images into professional semantic information, and align a language evaluation model through reinforcement learning to achieve deep fusion of images and text and generation of grading results.
It achieves comprehensive and accurate multimodal marking, enhances the authority and professionalism of marking results, ensures the controllability and consistency of the evaluation mechanism, greatly improves marking efficiency, generates specific guidance feedback, and meets teaching needs.
Smart Images
Figure CN121601136B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of medical information technology, and in particular to a method, system and storage medium for intelligent review of medical laboratory reports. Background Technology
[0002] As a core course bridging basic physiology and clinical practice, medical physiology laboratory science's lab reports are characterized by high specialization, high variability, and numerous influencing factors. Reports typically include textual descriptions (such as experimental objectives, methods, and results analysis) and key image data (such as arterial blood pressure waveforms, electrocardiograms, respiratory rate curves, and neural trunk action potential waveforms). These images are crucial for evaluating the scientific validity of experimental results. However, traditional manual grading methods suffer from inefficiency, inconsistent standards, and inaccurate feedback. Medical physiology laboratory science courses often employ a "small-group experiment + centralized report submission" model, with each report containing 3-5 specialized images and over 2000 words of textual analysis. Instructors must verify the consistency between the image data and the textual description for each report, averaging about 10 minutes per report. Furthermore, manual feedback is often general and lacks specific guidance for addressing particular errors (such as incorrect diastolic blood pressure readings on a waveform).
[0003] To improve grading efficiency, existing automatic grading technology has undergone three generations of development:
[0004] 1. Rule-based and template-based systems: These systems match report content using predefined rules, resulting in poor flexibility and an inability to adapt to the diversity and complexity of medical laboratory reports.
[0005] 2. Machine learning feature extraction-based systems: These systems use traditional machine learning algorithms (such as SVM and decision trees) to extract features from text for classification, but their semantic understanding is shallow and they have difficulty handling technical terms and image content.
[0006] 3. Systems based on deep learning natural language processing (NLP) models: While using neural networks (such as BERT and GPT series) for deep text analysis can improve semantic understanding, they are essentially "blind models" and cannot process key image information embedded in reports, resulting in incomplete reviews.
[0007] In recent years, multimodal large models (such as GPT-4V) have demonstrated the ability to process mixed text and image content, but their direct application in reviewing medical laboratory reports still faces three major technical bottlenecks:
[0008] 1. Insufficient domain expertise: General models lack physiological knowledge support, their understanding of specialized images (such as electrocardiogram waveforms) is unreliable, and they are prone to producing "illusion" outputs.
[0009] 2. The reasoning mechanism is black box: the model decision-making process is unexplainable, the grading results are difficult to control and optimize, and cannot meet the requirements of accuracy and consistency in teaching evaluation.
[0010] 3. Poor end-to-end fusion effect: The model has difficulty completing the complex reasoning chain from image perception to professional evaluation on its own, resulting in weak ability to collaboratively analyze text and image information.
[0011] The aforementioned limitations are further amplified in medical functional laboratory reports. For example, the same experimental phenomenon may lead to different conclusions depending on the dependent variable, and differences in teachers' subjective judgments exacerbate the inconsistency in grading standards. In addition, laboratory courses typically adopt a "small-group experiment + centralized report submission" model, resulting in a concentrated workload for grading, and current technology cannot achieve efficient and accurate multimodal grading.
[0012] Therefore, there is an urgent need for an intelligent solution that can integrate medical expertise, unify evaluation standards, and enable collaborative review of text and images, in order to address the core pain points of current medical laboratory report review. Summary of the Invention
[0013] The purpose of this invention is to overcome the shortcomings of the prior art and provide a method, system and storage medium for intelligent review of medical laboratory reports.
[0014] The objective of this invention is achieved through the following technical solution: The first aspect of this invention provides: an intelligent review method for medical laboratory reports, comprising the following steps:
[0015] Offline phase: Construct a multimodal physiological dataset and a professional knowledge base; then, train a visual expert model through supervised fine-tuning to convert pixel information into domain semantic information; finally, align the language evaluation model through reinforcement learning.
[0016] In the online phase: the report to be reviewed is deconstructed in a multimodal manner, separating the text content and the image set; each image is input into a trained visual expert model for visual information conversion, thereby obtaining a professional text description; the professional text description is inserted into the original report text for information fusion to obtain an enhanced plain text report; the enhanced report is input into a trained language evaluation model, which calls the professional knowledge base for knowledge verification and reasoning to generate the review results.
[0017] Preferably, the multimodal physiological dataset and professional knowledge base are constructed through the following steps:
[0018] Professional knowledge fragments were extracted from historical experimental reports, references, and physiology textbooks annotated by experts, and encoded into high-dimensional vectors using a text embedding model to construct a vector knowledge base based on FAISS. As a knowledge base:
[0019] ,in A collection of professional knowledge snippets; s i For the target professional knowledge segment; A vector representation of the target knowledge segment; Encoder For text embedding models;
[0020] Use the FAISS index for similarity searches: , where q is the query vector; sim For similarity measurement;
[0021] Images are extracted from segments of professional knowledge and correlated with image evaluations in expert annotations to form an image pairing dataset that pairs images with professional descriptions. .
[0022] Preferably, the visual expert model is trained through the following steps:
[0023] Choose a large visual model and use an image pairing dataset. Supervised fine-tuning of a large visual model enables it to learn to use physiological terminology to diagnostically describe images, thus yielding a visual expert model. The objective function for supervised fine-tuning is to minimize the negative log-likelihood:
[0024] ,in L To describe the text length; w t For the t-th word element; These are model parameters; Indicates that in a given input image I and the generated prefix sequence Under the given conditions, the model generates the t-th word. w t The conditional probability; d This represents the professional descriptive text corresponding to the input image.
[0025] Preferably, the image pairing dataset This includes arterial blood pressure waveforms, electrocardiograms, electromyograms, anatomical diagrams, respiratory rate curves, nerve trunk action potential waveforms, and their professional descriptions.
[0026] Preferably, the language evaluation model is aligned through the following steps:
[0027] With large language models M LLM Based on this, a comment generator is constructed by integrating a reward function with multi-dimensional rewards. ,in , , and These are the weighting coefficients for each reward component; Based on generated comments y and human comments y ref semantic similarity, using BERTScosre calculate:
[0028] ;
[0029] Based on generated comments y With vector knowledge base The calculation of factual consistency may employ a strategy based on retrieval-enhanced verification; or it may be based on generated comments. y With vector knowledge base Fact Consistency Calculation The process includes the following steps: generating comments. y Or from generating comments y The extracted key assertion encoding is used as the query vector q to retrieve the vector knowledge base. The most relevant top-karaoke knowledge fragments and their similarity are used to calculate rewards:
[0030] , where 1(⋅) is an indicator function that takes the value 1 when the condition inside the parentheses is true, and 0 otherwise; It is a relevance judgment condition, indicating whether the professional knowledge fragments corresponding to the retrieved knowledge vector are consistent with or related to the generated comments in fact;
[0031] To penalize the model for giving different evaluations to the same type of error, a consistency measure of evaluations among samples within the same batch is calculated.
[0032] The score given by the reward model trained based on the preference ranking of human teachers for the model's output, where This indicates the score given by the reward model to the generated comments;
[0033] Optimizing large language models using reinforcement learning algorithms M LLM To maximize expected reward, a language evaluation model is obtained. :
[0034] ,in It is the optimization objective function in reinforcement learning, representing the current policy parameters. The expected total reward obtained from generating comments at that time; The expectation operator represents the average of all possible comments generated by sampling from a large language model; Represents the trainable parameters of a large language model; This indicates that the reward function performs a comprehensive score on the generated comments.
[0035] Preferably, the online phase specifically includes the following steps:
[0036] Analyze the report to be reviewed and extract the text content. and all embedded images ;
[0037] Each image Input the trained visual expert model To obtain a professional text description d j : ;
[0038] Specialized text description d j As a new text paragraph, it is inserted into the original report text where the image was originally located, generating an enhanced plain text report. :
[0039] ,in It is a text fusion function used to combine specialized text descriptions from each image. d j As a separate paragraph, it is inserted into the original report text where the image was originally located, forming an enhanced plain text report containing complete semantics;
[0040] Enhanced plain text report Input the trained language evaluation model To understand the context of the report and access the vector knowledge base Knowledge verification and reasoning are used to produce approval results. O output : and output a comprehensive score. Scoring of each item Summary comments And annotations and suggestions at specific locations in the original text.
[0041] A second aspect of the present invention provides: an intelligent review system for medical laboratory reports, used to implement any of the above-mentioned intelligent review methods for medical laboratory reports, comprising:
[0042] The knowledge base construction module is used to extract professional knowledge from physiological data, build and maintain a FAISS-based vector knowledge base;
[0043] The Visual Experts module contains supervised fine-tuning visual expert models for converting medical images into professional text descriptions.
[0044] The multimodal fusion module is used to parse the report to be reviewed, fuse image descriptions with the original text, and generate an enhanced plain text report;
[0045] The language evaluation module includes a language evaluation model aligned with reinforcement learning, used to generate grading results based on an enhanced plain text report;
[0046] The grading output module is used to output scores, comments, and location feedback.
[0047] Preferably, the system isolates data storage, AI computing, and API service areas through firewalls; load balancing, queues, databases, and caching services are all deployed in clusters to avoid single points of failure; stateless services are extended through virtual machines, and compute-intensive services are dynamically added with GPU nodes through Kubernetes clusters; and hardware is selected according to workload characteristics, with GPU nodes used for AI model inference and SSDs used for IO-intensive storage needs.
[0048] A third aspect of the present invention provides: a computer-readable storage medium storing computer-executable instructions, wherein when the computer-executable instructions are loaded and executed by a processor, the above-described intelligent review method for medical experimental reports is implemented.
[0049] The beneficial effects of this invention are:
[0050] 1) Achieve comprehensive and accurate multimodal review, overcoming modal isolation problems: By using a visual expert model, key images in the report (such as blood pressure waveforms, electrocardiograms, etc.) are transformed into precise professional text descriptions, achieving deep semantic integration of text descriptions and visual evidence. It can collaboratively analyze the content of images and text, verifying the consistency of "image data - text description - conclusion derivation," solving the core deficiency of traditional text review models that cannot perceive and understand image content, making the review input information complete and the judgment basis more comprehensive.
[0051] 2) Significantly enhances the authority and professionalism of the grading results, addressing the issue of missing domain knowledge: By constructing a vector knowledge base based on authoritative textbooks and expert knowledge, and by performing specialized fine-tuning of the visual and language models in the field of physiology, the AI's grading judgments are grounded in a structured physiological knowledge system. This effectively curbs the "illusion" problem that general-purpose large models are prone to in professional fields, ensuring the accuracy and reliability of the grading conclusions, and its output possesses authority approaching that of domain experts.
[0052] 3) Ensuring a controllable, consistent, and interpretable evaluation mechanism to meet teaching evaluation requirements: By designing a reward function that integrates multiple dimensions such as accuracy, knowledge consistency, evaluation consistency, and human preferences, and by using reinforcement learning to inject the review preferences and scoring standards of human experts into the language model, the model's output is not only correct but also meets the specific requirements of the teaching scenario. This mechanism ensures a high degree of consistency in scoring standards across different reports and batches, and can generate specific and localizable feedback, clearly pointing out the errors (e.g., "The diastolic pressure of a certain blood pressure waveform is incorrectly judged"), making the review process more transparent and interpretable.
[0053] 4) Significantly improves marking efficiency and practicality, becoming a reliable assistant for teachers: This invention provides end-to-end automated marking capabilities, freeing teachers from heavy, repetitive tasks and increasing marking efficiency by several orders of magnitude. Simultaneously, the generated comments are specific and highly instructive, providing students with targeted suggestions for improvement, thereby significantly enhancing teaching feedback and allowing teachers to focus on higher-value teaching guidance and interaction.
[0054] 5) The system is controllable, secure, customizable, and highly practical: The system supports local or private cloud deployment, with all data circulating within the internal network, effectively ensuring the data security of sensitive lab reports. Its modular design allows the system to be customized and optimized according to the specific teaching needs of different institutions, such as adjusting grading weights, updating the professional knowledge base, or adapting to new image types, demonstrating excellent scalability and adaptability. Attached Figure Description
[0055] Figure 1 This is a flowchart of the method of the present invention;
[0056] Figure 2 This is a flowchart of the offline phase.
[0057] Figure 3 Flowchart for the online phase;
[0058] Figure 4 This is a diagram of the system deployment architecture. Detailed Implementation
[0059] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0060] See Figures 1-4 The first aspect of this invention provides: a method for intelligent grading of medical laboratory reports, comprising the following steps:
[0061] Offline phase: Construct a multimodal physiological dataset and a professional knowledge base; then, train a visual expert model through supervised fine-tuning to convert pixel information into domain semantic information; finally, align the language evaluation model through reinforcement learning.
[0062] In the online phase: the report to be reviewed is deconstructed in a multimodal manner, separating the text content and the image set; each image is input into a trained visual expert model for visual information conversion, thereby obtaining a professional text description; the professional text description is inserted into the original report text for information fusion to obtain an enhanced plain text report; the enhanced report is input into a trained language evaluation model, which calls the professional knowledge base for knowledge verification and reasoning to generate the review results.
[0063] In this embodiment, the present invention does not simply throw text and images into a general multimodal model. Instead, it uses a structured process to first convert the image modality into a text description rich in professional semantics, and then hand it over to a powerful language model that has been fine-tuned in the domain and aligned with reinforcement learning for deep comprehensive evaluation, thereby achieving accurate and reliable multimodal information fusion.
[0064] In some embodiments, the multimodal physiological dataset and professional knowledge base are constructed through the following steps:
[0065] Professional knowledge fragments were extracted from historical experimental reports, references, and physiology textbooks annotated by experts, and encoded into high-dimensional vectors using a text embedding model to construct a vector knowledge base based on FAISS. As a knowledge base:
[0066] ,in A collection of professional knowledge snippets; s i For the target professional knowledge segment; A vector representation of the target knowledge segment; Encoder For text embedding models;
[0067] Use the FAISS index for similarity searches: , where q is the query vector; sim Similarity measures (such as inner product or L2 distance);
[0068] Images are extracted from segments of professional knowledge and correlated with image evaluations in expert annotations to form an image pairing dataset that pairs images with professional descriptions. .
[0069] In this embodiment, historical experimental reports (including text, images, ratings, and comments) annotated by experts, references, and authoritative textbooks are collected. Information extraction techniques are used to extract physiological knowledge fragments from the textbooks, and these fragments are encoded into high-dimensional vectors using text embedding models (such as BERT) to construct a FAISS-based vector knowledge base. Images from the reports (such as electrocardiograms, electromyograms, and anatomical diagrams) are extracted and associated with image evaluations in the teachers' annotations, forming a high-quality image-professional description image pairing dataset.
[0070] In some embodiments, the visual expert model is trained through the following steps:
[0071] Choose a large visual model and use an image pairing dataset. Supervised fine-tuning of a large visual model enables it to learn to use physiological terminology to diagnostically describe images, thus yielding a visual expert model. The objective function for supervised fine-tuning is to minimize the negative log-likelihood:
[0072] ,in L To describe the text length; w t For the t-th word element; These are model parameters; Indicates that in a given input image I and the generated prefix sequence Under the given conditions, the model generates the t-th word. w t The conditional probability; d This represents the professional descriptive text corresponding to the input image.
[0073] In this embodiment, a high-performance large-scale visual model (such as a model based on qwen-vl) is selected. Through supervised fine-tuning training, the model learns to provide diagnostic descriptions of images using physiological terminology (e.g., inputting an electrocardiogram segment, outputting "heart rate 72 bpm, PR interval 0.16 s, normal QRS complex morphology and duration, no abnormal ST segment elevation"). The fine-tuned model is called a "visual expert," and its function is to convert pixel information into domain semantic information.
[0074] In some embodiments, the image pairing dataset This includes arterial blood pressure waveforms, electrocardiograms, electromyograms, anatomical diagrams, respiratory rate curves, nerve trunk action potential waveforms, and their professional descriptions.
[0075] In some embodiments, the language evaluation model is aligned through the following steps:
[0076] With large language modelsM LLM Based on this, a comment generator is constructed by integrating a reward function with multi-dimensional rewards. ,in , , and These are the weighting coefficients for each reward component; Based on generated comments y and human comments y ref semantic similarity, using BERTScosre calculate:
[0077] ;
[0078] Based on generated comments y With vector knowledge base The calculation of factual consistency may employ a strategy based on retrieval-enhanced verification; or it may be based on generated comments. y With vector knowledge base Fact Consistency Calculation The process includes the following steps: generating comments. y Or from generating comments y The extracted key assertion encoding is used as the query vector q to retrieve the vector knowledge base. The most relevant top-karaoke knowledge fragments and their similarity are used to calculate rewards:
[0079] , where 1(⋅) is an indicator function that takes the value 1 when the condition inside the parentheses is true, and 0 otherwise; It is a relevance judgment condition, indicating whether the professional knowledge fragments corresponding to the retrieved knowledge vector are consistent with or related to the generated comments in fact;
[0080] To penalize the model for giving different evaluations to the same type of error, a consistency measure of evaluations among samples within the same batch is calculated.
[0081] The score given by the reward model trained based on the preference ranking of human teachers for the model's output, where This indicates the score given by the reward model to the generated comments;
[0082] Optimizing large language models using reinforcement learning algorithms M LLM To maximize expected reward, a language evaluation model is obtained. :
[0083] ,in It is the optimization objective function in reinforcement learning, representing the current policy parameters. The expected total reward obtained from generating comments at that time; The expectation operator represents the average of all possible comments generated by sampling from a large language model; Represents the trainable parameters of a large language model; This indicates that the reward function performs a comprehensive score on the generated comments.
[0084] The language evaluation model is aligned through the following steps:
[0085] With large language models M LLM Based on this, a comment generator is constructed by integrating a reward function with multi-dimensional rewards. ,in , , and These are the weighting coefficients for each reward component; Based on generated comments y and human comments y ref semantic similarity, using BERTScosre calculate:
[0086] ;
[0087] Based on generated comments y With vector knowledge base The calculation of factual consistency can be performed using a strategy based on Retrieval Enhanced Validation (RAV); or based on generated comments. y With vector knowledge base Fact Consistency Calculation The process includes the following steps: generating comments. y Or from generating comments y The extracted key assertion encoding is used as the query vector q to retrieve the vector knowledge base. The most relevant top-karaoke knowledge fragments and their similarity are used to calculate rewards:
[0088] , where 1(⋅) is an indicator function, which takes the value 1 when the condition in the parentheses is true, and 0 otherwise; This is a relevance judgment condition, indicating whether the professional knowledge fragment corresponding to the retrieved knowledge vector is consistent with or related to the generated comment in fact (if the professional knowledge fragment supports or confirms the statement in the generated comment, the condition is true; if the professional knowledge fragment contradicts or is irrelevant to the statement in the generated comment, the condition is false).
[0089] To penalize the model for giving different evaluations to the same type of error, a consistency measure of evaluations among samples within the same batch is calculated.
[0090] The score given by the reward model, which is trained based on the preference ranking of human teachers for the model's output, is where... This indicates the score given by the reward model to the generated comments;
[0091] Optimize large language models using reinforcement learning algorithms (such as PPO). M LLM To maximize expected reward, a language evaluation model is obtained. :
[0092] ,in It is the optimization objective function in reinforcement learning, representing the current policy (i.e., the large language model). M LLM ) parameters are The expected total reward obtained from generating comments at that time; The expectation operator represents the average of all possible comments generated by sampling from a large language model; This represents the trainable parameters of a large language model (i.e., the parameters of the policy network, which in reinforcement learning represent the "policy" that needs to be optimized). This indicates that the reward function performs a comprehensive score on the generated comments.
[0093] In this embodiment, The variance inverse can be calculated using embedding similarity; the resulting language evaluation model She is a "language evaluation expert" who "understands physiology, knows how to grade papers, and whose grading style is close to that of a human teacher".
[0094] In some embodiments, the online phase specifically includes the following steps:
[0095] Analyze the report to be reviewed and extract the text content. and all embedded images ;
[0096] Each image Input the trained visual expert model To obtain a professional text description d j : ;
[0097] Specialized text description d jAs a new text paragraph, it is inserted into the original report text where the image was originally located, generating an enhanced plain text report. :
[0098] ,in It is a text fusion function used to combine specialized text descriptions from each image. d j As a separate paragraph, it is inserted into the original report text where the image was originally located, forming an enhanced plain text report containing complete semantics;
[0099] Enhanced plain text report Input the trained language evaluation model To understand the context of the report and access the vector knowledge base Knowledge verification and reasoning are used to produce approval results. O output : and output a comprehensive score. Scoring of each item Summary comments And annotations and suggestions at specific locations in the original text (corresponding to a certain paragraph or image).
[0100] In this embodiment, the first key information conversion from visual modality to text modality was completed through multimodal deconstruction and visual information transformation, laying the foundation for subsequent deep fusion. The enhanced plain text report generated through multimodal information fusion and context reconstruction includes the student's original textual description and AI-generated precise professional descriptions of the images, achieving deep semantic fusion of text and visual information. Language evaluation model Based on its deep understanding of the entire report context (including student text and image descriptions), and by accessing the FAISS vector knowledge base to enhance its knowledge verification and reasoning capabilities, a complete grading result is ultimately generated.
[0101] The technical advantage of this invention lies in its division of labor and collaboration between "visual experts" and "language evaluation experts," which decomposes the complex multimodal understanding problem into two more easily solved sub-problems. Furthermore, by cleverly achieving accurate and reliable fusion of cross-modal information at the highest semantic level through the key step of converting images into professional text, a super expert (language model) ultimately makes a comprehensive decision. This is more feasible, professional, and controllable than directly training a multimodal model end-to-end.
[0102] A second aspect of the present invention provides: an intelligent review system for medical laboratory reports, used to implement any of the above-mentioned intelligent review methods for medical laboratory reports, comprising:
[0103] The knowledge base construction module is used to extract professional knowledge from physiological data, build and maintain a FAISS-based vector knowledge base;
[0104] The Visual Experts module contains supervised fine-tuning visual expert models for converting medical images into professional text descriptions.
[0105] The multimodal fusion module is used to parse the report to be reviewed, fuse image descriptions with the original text, and generate an enhanced plain text report;
[0106] The language evaluation module includes a language evaluation model aligned with reinforcement learning, used to generate grading results based on an enhanced plain text report;
[0107] The grading output module is used to output scores, comments, and location feedback.
[0108] In some embodiments, the system isolates data storage, AI computing, and API service areas through firewalls; load balancing, queue, database, and caching services are all deployed in clusters to avoid single points of failure; stateless services are extended through virtual machines, and compute-intensive services dynamically add GPU nodes through Kubernetes clusters; and hardware is selected according to workload characteristics, with GPU nodes used for AI model inference and SSDs used for IO-intensive storage needs.
[0109] In this embodiment, the system is deployed in a local data center or private cloud, divided into multiple clearly defined zones, ensuring security and performance, as shown in the attached diagram. Figure 4 As shown, the system adopts a layered, highly available, and securely isolated distributed architecture. Internet users access the system via firewalls and NAT policies, and secure access is achieved through a load-balanced cluster (Nginx / HAProxy, master-slave mode) in the DMZ zone. Internally, the system is divided into trusted network zones, including a web service layer, an application and computing layer, a data storage layer, and a shared service layer.
[0110] Among them, the API gateway cluster (Kong / Spring Cloud Gateway) is deployed on virtual machines, supporting horizontal scaling of stateless services; the task queue and scheduling cluster (RabbitMQ + Celery) runs in high availability mode to ensure stable flow of grading tasks; the GPU computing node cluster is based on NVIDIA A100 / A10 servers and relies on Kubernetes to dynamically schedule Pods to achieve elastic scaling of AI model inference.
[0111] The data storage layer comprises a database cluster (MySQL / PostgreSQL, master-slave replication), a cache cluster (RedisSentinel or Cluster mode), and an object storage cluster (MinIO Ceph, multi-node deployment), supporting structured data, session caching, and large file storage, respectively. The shared service layer provides a vector knowledge base server (Paiss on CPU-OptimizedVM) and a model file server (NFS / S3 compatible storage) for knowledge retrieval and model version management.
[0112] The system employs firewalls for area isolation, and critical services are deployed in clusters to avoid single points of failure. GPU nodes are dedicated to AI inference, and SSD storage meets high I / O requirements. The monitoring and logging system (Prometheus / Grafana / ELK) and jump servers are managed uniformly to ensure end-to-end observability and security. The overall architecture balances security, scalability, and high performance, supporting an intelligent review process for medical laboratory reports.
[0113] The system as a whole supports the collaborative operation of the "offline stage" and "online stage" described in this invention:
[0114] In the offline phase, a semantic vector library for the physiological domain is built by the vector knowledge base server (Paiss), and the training and reinforcement learning alignment of the visual expert model (image → professional text) and the language evaluation model are completed on the GPU computing cluster. The model files are uniformly stored on the model server (NFS / S3).
[0115] During the online phase, users submit lab reports containing text and medical images through the API gateway cluster; the system schedules these reports via a task queue (RabbitMQ + Celery) and executes them in parallel on GPU nodes managed by Kubernetes.
[0116] (1) Professional description of images generated by visual expert models;
[0117] (2) Integrating original text and image descriptions to form enhanced input;
[0118] (3) The language evaluation model is combined with the vector knowledge base for domain alignment, and the output is interpretable and controllable grading results.
[0119] The entire system isolates API services, AI computing, and data storage areas through a firewall; databases, caches, and object storage are all deployed using clusters and SSDs to avoid single points of failure and meet high I / O requirements; stateless services scale horizontally, and compute-intensive tasks are elastically scheduled with GPU resources to ensure the grading process is efficient, accurate, and secure.
[0120] The system has the following features: hierarchical partitioning: the network is divided into different areas through a firewall, which strictly restricts cross-area access and greatly improves security.
[0121] High availability: Critical services (load balancer, queue, database, cache) are all deployed in a cluster, eliminating single points of failure.
[0122] Elastic scaling: Stateless services (API gateway) can be easily scaled by adding virtual machines. Compute-intensive services (review units) can be scaled by adding more GPU nodes to the Kubernetes cluster. Storage: Object storage and databases support horizontal scaling.
[0123] Performance matching: Select hardware based on workload characteristics (GPU for AI computing, large memory for caching and databases, and high-speed SSD for I / O intensive applications).
[0124] A third aspect of the present invention provides: a computer-readable storage medium storing computer-executable instructions, wherein when the computer-executable instructions are loaded and executed by a processor, the above-described intelligent review method for medical experimental reports is implemented.
[0125] The above description is merely a preferred embodiment of the present invention. It should be understood that the present invention is not limited to the forms disclosed herein and should not be construed as excluding other embodiments. It can be used in various other combinations, modifications, and environments, and can be altered within the scope of the concept described herein through the above teachings or related technologies or knowledge. Modifications and variations made by those skilled in the art that do not depart from the spirit and scope of the present invention should be within the protection scope of the appended claims.
Claims
1. A method for intelligently reviewing medical laboratory reports, characterized by: Includes the following steps: Offline phase: Construct a multimodal physiological dataset and a professional knowledge base; then, train a visual expert model through supervised fine-tuning to convert pixel information into domain semantic information; finally, align the language evaluation model through reinforcement learning. In the online phase: the report to be reviewed is deconstructed in a multimodal manner, separating the text content and the image set; each image is input into a trained visual expert model for visual information conversion, thereby obtaining a professional text description; the professional text description is inserted into the original report text for information fusion to obtain an enhanced plain text report; the enhanced report is input into a trained language evaluation model, which calls the professional knowledge base for knowledge verification and reasoning to generate the review results; The aforementioned multimodal physiological dataset and professional knowledge base were constructed through the following steps: Professional knowledge fragments were extracted from historical experimental reports, references, and physiology textbooks annotated by experts, and encoded into high-dimensional vectors using a text embedding model to construct a vector knowledge base based on FAISS. As a knowledge base: ,in A collection of professional knowledge snippets; s i For the target professional knowledge segment; A vector representation of the target knowledge segment; Encoder For text embedding models; Use the FAISS index for similarity searches: , where q is the query vector; sim For similarity measurement; Images are extracted from segments of professional knowledge and correlated with image evaluations in expert annotations to form an image pairing dataset that pairs images with professional descriptions. ; The aforementioned visual expert model is trained through the following steps: Choose a large visual model and use an image pairing dataset. Supervised fine-tuning of a large visual model enables it to learn to use physiological terminology to diagnostically describe images, thus yielding a visual expert model. The objective function for supervised fine-tuning is to minimize the negative log-likelihood: ,in L To describe the text length; w t For the t-th word element; These are model parameters; Indicates that in a given input image I and the generated prefix sequence Under the given conditions, the model generates the t-th word. w t The conditional probability; d This represents the professional descriptive text corresponding to the input image; The language evaluation model is aligned through the following steps: With large language models M LLM Based on this, a comment generator is constructed by integrating a reward function with multi-dimensional rewards. ,in , , and These are the weighting coefficients for each reward component; Based on generated comments y and human comments y ref semantic similarity, using BERTScosre calculate: ； Based on generated comments y With vector knowledge base The calculation of factual consistency may employ a strategy based on retrieval-enhanced verification; or it may be based on generated comments. y The calculation of factual consistency with the vector knowledge base includes the following steps: generating comments. y Or from generating comments y The extracted key assertion encoding is used as the query vector q to retrieve the vector knowledge base. The top-k most relevant knowledge fragments and their similarity are selected, and a reward is calculated based on this: , where 1(⋅) is an indicator function that takes the value 1 when the condition inside the parentheses is true, and 0 otherwise; It is a relevance judgment condition, indicating whether the professional knowledge fragments corresponding to the retrieved knowledge vector are consistent with or related to the generated comments in fact; To penalize the model for giving different evaluations to the same type of error, a consistency measure of evaluations among samples within the same batch is calculated. The score given by the reward model trained based on the preference ranking of human teachers for the model's output, where This indicates the score given by the reward model to the generated comments; Optimizing large language models using reinforcement learning algorithms M LLM To maximize expected reward, a language evaluation model is obtained. : ,in It is the optimization objective function in reinforcement learning, representing the current policy parameters. The expected total reward obtained from generating comments at that time; The expectation operator represents the average of all possible comments generated by sampling from a large language model; Represents the trainable parameters of a large language model; This indicates that the reward function performs a comprehensive score on the generated comments.
2. The intelligent review method for medical laboratory reports according to claim 1, characterized in that: The image pairing dataset This includes arterial blood pressure waveforms, electrocardiograms, electromyograms, anatomical diagrams, respiratory rate curves, nerve trunk action potential waveforms, and their professional descriptions.
3. The intelligent review method for medical laboratory reports according to claim 1, characterized in that: The online phase specifically includes the following steps: Analyze the report to be reviewed and extract the text content. and all embedded images ; Each image Input the trained visual expert model To obtain a professional text description d j : ; Specialized text description d j As a new text paragraph, it is inserted into the original report text where the image was originally located, generating an enhanced plain text report. : ,in It is a text fusion function used to combine specialized text descriptions from each image. d j As a separate paragraph, it is inserted into the original report text where the image was originally located, forming an enhanced plain text report containing complete semantics; Enhanced plain text report Input the trained language evaluation model To understand the context of the report and access the vector knowledge base Knowledge verification and reasoning are used to produce approval results. O output : and output a comprehensive score. Scoring of each item Summary comments And annotations and suggestions at specific locations in the original text.
4. A medical laboratory report intelligent review system, characterized in that: The method for intelligent review of medical laboratory reports as described in any one of claims 1-3 includes: The knowledge base construction module is used to extract professional knowledge from physiological data, build and maintain a FAISS-based vector knowledge base; The Visual Experts module contains supervised fine-tuning visual expert models for converting medical images into professional text descriptions. The multimodal fusion module is used to parse the report to be reviewed, fuse image descriptions with the original text, and generate an enhanced plain text report; The language evaluation module includes a language evaluation model aligned with reinforcement learning, used to generate grading results based on an enhanced plain text report; The grading output module is used to output scores, comments, and location feedback.
5. The intelligent review system for medical laboratory reports according to claim 4, characterized in that: The system isolates data storage, AI computing, and API service areas through firewalls; load balancing, queues, databases, and caching services are all deployed in clusters to avoid single points of failure; stateless services are extended through virtual machines, and compute-intensive services dynamically add GPU nodes through Kubernetes clusters; and hardware is selected according to workload characteristics, with GPU nodes used for AI model inference and SSDs used for IO-intensive storage needs.
6. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which, when loaded and executed by a processor, implement the intelligent review method for medical experimental reports as described in any one of claims 1-3.