Medical report interpretation method and device based on multi-modal dynamic knowledge fusion

Abstract

The invention discloses a medical report interpretation method and device based on multi-modal dynamic knowledge fusion, and the method comprises the steps: recognizing a key region of a target medical report, and extracting a target physiological value index and a target text description index; inputting the baseline feature data into a physiological index interval prediction model to obtain a physiological index reference interval; calculating the physiological value index deviation degree according to the target physiological value index and the physiological index reference interval, and determining the abnormal degree of the physiological value index; key descriptors are extracted, and disease entities, standard text description and disease severity are inquired in the multi-dimensional medical knowledge graph; calculating the semantic similarity between the target text description index and the standard text description, and determining the abnormal degree of the text description index in combination with the disease severity; and determining a comprehensive anomaly score according to the physiological value index anomaly degree and the text description index anomaly degree. The method improves the accuracy and comprehensiveness of medical report interpretation, and can be widely applied to the technical field of artificial intelligence.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method and apparatus for interpreting medical reports based on multimodal dynamic knowledge fusion. Background Technology

[0002] With the development of medical informatization, the trend of digitizing various examination and test reports is becoming increasingly apparent. However, these reports typically contain a large amount of complex medical terminology, indicator data, and multimodal layouts (such as tables and images), making it difficult for ordinary users to quickly understand their meaning. Existing technologies have attempted to use OCR and rule matching to extract report content, but the following problems still exist: 1) Insufficient format parsing: Traditional OCR technology has a low recognition rate for complex layouts (such as nested tables and multi-level headings), which can easily lead to information loss or misjudgment.

[0003] 2) Limited semantic understanding: Relying on keyword matching or simple rules, it is impossible to accurately interpret the deeper meaning behind medical terms. For example, "high blood sugar" may correspond to multiple pathological causes, and current technology cannot make a comprehensive judgment based on the context.

[0004] 3) Lack of personalized recommendations: Failed to provide personalized health analysis and recommendations based on user history data and medical knowledge base.

[0005] 4) Incomplete anomaly detection: Only numerical indicators are considered, while textual descriptions are ignored (such as "edge burrs" in image reports).

[0006] The above problems urgently need to be addressed. Summary of the Invention

[0007] The purpose of this invention is to at least partially solve one of the technical problems existing in the prior art.

[0008] Therefore, one objective of this invention is to provide a medical report interpretation method based on multimodal dynamic knowledge fusion, which improves the accuracy and comprehensiveness of medical report interpretation.

[0009] Another objective of this invention is to provide a medical report interpretation device based on multimodal dynamic knowledge fusion.

[0010] To achieve the above-mentioned technical objectives, the technical solutions adopted in the embodiments of the present invention include: On one hand, embodiments of the present invention provide a method for interpreting medical reports based on multimodal dynamic knowledge fusion, including the following steps: Identify several key areas of the target medical report, and extract target physiological numerical indicators and target textual description indicators from the key areas; The baseline feature data of the target user is input into a pre-trained physiological index interval prediction model to obtain the physiological index reference interval of the target user. The deviation of the physiological numerical index is calculated based on the target physiological numerical index and the reference range of the physiological index, and the degree of abnormality of the physiological numerical index is determined based on the deviation of the physiological numerical index. Key descriptive terms are extracted based on the target text description metrics, and the corresponding disease entities, standard text descriptions of the disease entities, and disease severity are queried in a pre-constructed multi-dimensional medical knowledge graph based on the key descriptive terms. Calculate the semantic similarity between the target text description index and the standard text description, and determine the degree of abnormality of the text description index based on the semantic similarity and the severity of the disease; A comprehensive abnormality score is determined based on the degree of abnormality of the physiological numerical indicators and the degree of abnormality of the text description indicators, and corresponding early warning information is generated based on the comprehensive abnormality score.

[0011] Furthermore, in one embodiment of the present invention, the step of identifying several key regions of the target medical report and extracting target physiological numerical indicators and target textual descriptive indicators from the key regions specifically includes: The target medical report is structurally semantically segmented based on a dynamic page layout algorithm and a medical title keyword regular expression library to obtain multiple page layout regions; Image features are extracted and text features are encoded for each of the aforementioned page areas to obtain image visual features and text semantic features; Based on the attention mechanism, the visual features of the image and the semantic features of the text are fused, and positional encoding is embedded to obtain multimodal fusion features; The multimodal fusion features are input into a pre-trained key difference recognition model to obtain the key region output by the model; The target physiological numerical indicators and the target textual description indicators are extracted from the key region using OCR technology.

[0012] Furthermore, in one embodiment of the present invention, the physiological index interval prediction model is obtained through the following steps: Baseline feature samples and physiological numerical index samples of multiple sample users are obtained based on a pre-set medical dataset; Based on the baseline feature samples and the physiological numerical index samples, an initial model for predicting physiological index intervals is trained using the XGBoost algorithm. The seasonal impact coefficients for different seasons are determined using a seasonal impact factor database, and the regional difference coefficients for different regions are determined using a regional characteristic database. The initial model for predicting the physiological index intervals is adjusted based on the seasonality coefficient and the regional difference coefficient to obtain the physiological index interval prediction model.

[0013] Furthermore, in one embodiment of the present invention, the step of calculating the deviation of the physiological numerical index based on the target physiological numerical index and the reference interval of the physiological index, and determining the degree of abnormality of the physiological numerical index based on the deviation of the physiological numerical index, specifically includes: Calculate the mean and standard deviation of the reference interval for the physiological indicators; Calculate the difference between the target physiological numerical index and the mean, and determine the deviation of the physiological numerical index based on the ratio of the difference to the standard deviation; Physiological value grading thresholds are determined based on a preset medical rule base, and then corrected using seasonality and regional difference coefficients. The degree of abnormality of the physiological numerical index is determined based on the deviation of the physiological numerical index and the corrected physiological numerical grading threshold.

[0014] Furthermore, in one embodiment of the present invention, the step of extracting key descriptive terms based on the target text description index, and querying the corresponding disease entity, standard text description, and disease severity of the disease entity in a pre-constructed multi-dimensional medical knowledge graph based on the key descriptive terms, specifically includes: The key descriptive words are obtained by semantic parsing the target text description; Using the key descriptive terms as indexes, the corresponding disease entities are queried in the multi-dimensional medical knowledge graph; The disease entity is matched with the corresponding standard text description and the severity of the disease.

[0015] Furthermore, in one embodiment of the present invention, the step of calculating the semantic similarity between the target text description index and the standard text description, and determining the degree of abnormality of the text description index based on the semantic similarity and the severity of the disease, specifically includes: Based on the BERT model, the first text embedding vector of the target text description index and the second text embedding vector of the standard text description are extracted. The semantic similarity is determined based on the cosine values ​​of the first text embedding vector and the second text embedding vector; The semantic similarity and the severity of the disease are weighted and summed according to preset weight parameters to obtain the abnormality level of the text description index.

[0016] Furthermore, in one embodiment of the present invention, the step of determining a comprehensive abnormality score based on the degree of abnormality of the physiological numerical indicator and the degree of abnormality of the text description indicator, and generating corresponding early warning information based on the comprehensive abnormality score, specifically includes: By reasoning the causal relationship between the target physiological numerical indicators and the target textual description indicators through the multi-dimensional medical knowledge graph, multiple causal indicator pairs are obtained. Calculate the correlation between the target physiological numerical index / target textual description index and other physiological numerical index / other textual description index of the causal index pair; The correlation anomaly value of the causal indicator pair is calculated based on the degree of anomaly of other physiological numerical indicators / other textual descriptive indicators and the correlation degree. The abnormality degree of the physiological numerical index, the abnormality degree of the text description index, and the correlation anomaly value of the causal index pair are weighted and summed according to the preset weight parameters to obtain the comprehensive anomaly score of the causal index pair. The severity of the causal indicator pair is determined based on the comprehensive anomaly score, and the warning information is generated based on the severity of the indicator.

[0017] On the other hand, embodiments of the present invention provide a medical report interpretation device based on multimodal dynamic knowledge fusion, comprising: The indicator extraction module is used to identify several key areas of the target medical report and extract target physiological numerical indicators and target text description indicators from the key areas. The reference interval determination module is used to input the baseline feature data of the target user into a pre-trained physiological index interval prediction model to obtain the physiological index reference interval of the target user. The physiological numerical index evaluation module is used to calculate the deviation of the physiological numerical index based on the target physiological numerical index and the reference range of the physiological index, and to determine the degree of abnormality of the physiological numerical index based on the deviation of the physiological numerical index. The key descriptor query module is used to extract key descriptors based on the target text description indicators, and query the corresponding disease entities, standard text descriptions of the disease entities, and disease severity in a pre-constructed multi-dimensional medical knowledge graph based on the key descriptors. The text description indicator evaluation module is used to calculate the semantic similarity between the target text description indicator and the standard text description, and to determine the degree of abnormality of the text description indicator based on the semantic similarity and the severity of the disease. The early warning module is used to determine a comprehensive abnormality score based on the degree of abnormality of the physiological numerical indicators and the degree of abnormality of the text description indicators, and to generate corresponding early warning information based on the comprehensive abnormality score.

[0018] On the other hand, embodiments of the present invention provide an electronic device, which includes a memory, a processor, a computer program stored in the memory and executable on the processor, and a data bus for enabling communication between the processor and the memory. When the computer program is executed by the processor, it implements the medical report interpretation method based on multimodal dynamic knowledge fusion as described above.

[0019] On the other hand, embodiments of the present invention also provide a storage medium, which is a computer-readable storage medium for computer-readable storage. The storage medium stores one or more computer programs, which can be executed by one or more processors to implement the medical report interpretation method based on multimodal dynamic knowledge fusion as described above.

[0020] On the other hand, embodiments of the present invention also provide a computer program product, including a computer program that, when executed by a processor, implements the medical report interpretation method based on multimodal dynamic knowledge fusion as described above.

[0021] The advantages and beneficial effects of the present invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention: This invention identifies several key regions of a target medical report and extracts target physiological numerical indicators and target textual description indicators from these regions. Baseline feature data of the target user is input into a pre-trained physiological indicator interval prediction model to obtain a reference interval for the target user's physiological indicators. The deviation of the physiological numerical indicators is calculated based on the target physiological numerical indicators and the reference interval, and the degree of abnormality of the physiological numerical indicators is determined based on the deviation. Key descriptive words are extracted from the target textual description indicators, and the corresponding disease entities, standard textual descriptions of the disease entities, and disease severity are queried in a pre-constructed multi-dimensional medical knowledge graph based on these key descriptive words. The semantic similarity between the target textual description indicators and the standard textual descriptions is calculated, and the degree of abnormality of the textual description indicators is determined based on the semantic similarity and disease severity. A comprehensive abnormality score is determined based on the degree of abnormality of the physiological numerical indicators and the degree of abnormality of the textual description indicators, and corresponding warning information is generated based on the comprehensive abnormality score. This invention extracts target physiological numerical indicators and target textual description indicators based on identified key regions. It determines the corresponding physiological indicator reference interval based on the user's baseline feature data to calculate the degree of abnormality of the physiological numerical indicators. It determines the corresponding disease entity, standard text description, and disease severity based on the key descriptive words of the target textual description indicators, thereby calculating the degree of abnormality of the textual description indicators. Finally, it determines a comprehensive abnormality score based on the degree of abnormality of the physiological numerical indicators and the degree of abnormality of the textual description indicators and generates corresponding early warning information, thereby improving the accuracy and comprehensiveness of medical report interpretation. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the embodiments of the present invention are described below. It should be understood that the drawings described below are only for the convenience of clearly describing some embodiments of the technical solutions of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 A flowchart illustrating the steps of a medical report interpretation method based on multimodal dynamic knowledge fusion provided in an embodiment of the present invention; Figure 2 A flowchart of step S101 provided in an embodiment of the present invention; Figure 3 A flowchart illustrating the steps of generating a physiological index interval prediction model provided in an embodiment of the present invention; Figure 4 A flowchart of step S103 provided in an embodiment of the present invention; Figure 5 A flowchart of step S104 provided in an embodiment of the present invention; Figure 6 A flowchart of step S105 provided in an embodiment of the present invention; Figure 7 A flowchart of step S106 provided in an embodiment of the present invention; Figure 8 A schematic diagram of the structure of a medical report interpretation device based on multimodal dynamic knowledge fusion provided in an embodiment of the present invention; Figure 9 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present invention; Figure 10 This is a schematic diagram of the structure of the storage medium provided in an embodiment of the present invention. Detailed Implementation

[0024] The embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application. It should be noted that although functional modules are divided in the system schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the system schematic diagram or the order in the flowchart. The step numbers in the following embodiments are only set for ease of explanation and do not limit the order between steps. The execution order of each step in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0025] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0026] The meanings of some of the terms used in this article are as follows: OCR (Optical Character Recognition): A technology that converts text content in scanned images or photographs into editable text.

[0027] Large Language Model (LLM): A large-scale neural network model trained on massive amounts of data, possessing powerful natural language understanding and generation capabilities.

[0028] Medical Knowledge Graph: A knowledge base that uses nodes and edges to represent entities and their relationships in the medical field, used to enhance semantic understanding.

[0029] Multimodal data: including data in various forms such as text, tables, and images.

[0030] Health risk analysis: The process of inferring potential health problems in users and providing recommendations based on physical examination or test data.

[0031] Dynamic reference range adaptation: Adjust the normal range of indicators based on the patient's historical test data to avoid misjudgment by a "one-size-fits-all" approach.

[0032] Severity grading of abnormalities: Classify test results (e.g., "minor abnormality", "moderate abnormality", "critical") and trigger corresponding early warning mechanisms.

[0033] The medical report interpretation method based on multimodal dynamic knowledge fusion provided in this application can be applied to a terminal, a server, or software running on a terminal or server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, or in-vehicle terminal, but is not limited to these. The server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server can also be a node server in a blockchain network. The software can be an application implementing a driver status monitoring method based on machine vision, but is not limited to the above forms.

[0034] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0035] It should be noted that in all specific embodiments of this application, when processing data related to user identity or characteristics, such as user information, user behavior data, user historical data, and user location information, user permission or consent is obtained first. Furthermore, the collection, use, and processing of this data comply with relevant laws, regulations, and standards of the relevant countries and regions. In addition, when embodiments of this application require access to sensitive personal information of users, separate permission or consent from the user is obtained through pop-ups or redirects to confirmation pages. Only after obtaining the user's separate permission or consent is the necessary user-related data for the proper functioning of the embodiments of this application obtained.

[0036] This invention aims to address the shortcomings of existing technologies through precise data extraction, deep semantic analysis, and personalized health analysis. The main objectives include: 1) Improve report parsing accuracy: Enhance the ability to recognize reports with complex formats through improved OCR algorithms and multimodal parsing technology; 2) Enhance semantic understanding capabilities: Utilize medical knowledge graphs and large models to achieve deep semantic analysis of medical terms and indicators; 3) Provide personalized health advice: Combine users' historical data and dynamic health risk assessment models to generate customized health analysis reports for users; 4) Achieve real-time interpretation: Through efficient algorithms and edge computing optimization, achieve a response time of seconds.

[0037] like Figure 1 The diagram shows a flowchart of a medical report interpretation method based on multimodal dynamic knowledge fusion provided by an embodiment of the present invention. (Refer to...) Figure 1 This invention provides a method for interpreting medical reports based on multimodal dynamic knowledge fusion, specifically including the following steps: S101. Identify several key areas of the target medical report and extract target physiological numerical indicators and target textual description indicators from the key areas. S102. Input the baseline feature data of the target user into the pre-trained physiological index interval prediction model to obtain the physiological index reference interval of the target user. S103. Calculate the deviation of physiological numerical indicators based on the target physiological numerical indicators and the reference range of physiological indicators, and determine the degree of abnormality of physiological numerical indicators based on the deviation of physiological numerical indicators. S104. Extract key descriptive terms based on the target text description indicators, and query the corresponding disease entities, standard text descriptions of the disease entities, and disease severity in the pre-constructed multi-dimensional medical knowledge graph based on the key descriptive terms. S105. Calculate the semantic similarity between the target text description index and the standard text description, and determine the degree of abnormality of the text description index based on the semantic similarity and the severity of the disease. S106. Determine the comprehensive abnormality score based on the degree of abnormality of physiological numerical indicators and textual description indicators, and generate corresponding early warning information based on the comprehensive abnormality score.

[0038] This invention extracts target physiological numerical indicators and target textual description indicators based on identified key regions. It determines the corresponding physiological indicator reference interval based on the user's baseline feature data to calculate the degree of abnormality of the physiological numerical indicators. It determines the corresponding disease entity, standard text description, and disease severity based on the key descriptive words of the target textual description indicators, thereby calculating the degree of abnormality of the textual description indicators. Finally, it determines a comprehensive abnormality score based on the degree of abnormality of the physiological numerical indicators and the degree of abnormality of the textual description indicators and generates corresponding early warning information, thereby improving the accuracy and comprehensiveness of medical report interpretation.

[0039] like Figure 2 The diagram shown is a flowchart of step S101 provided in an embodiment of the present invention. (Refer to...) Figure 2 As an optional implementation, several key areas of the target medical report are identified, and target physiological numerical indicators and target textual descriptive indicators are extracted from these key areas, specifically including: S1011. Based on the dynamic page segmentation algorithm and the medical title keyword regular expression library, the target medical report is structurally semantically segmented to obtain multiple page areas; S1012. Extract image features and encode text features for each page area to obtain image visual features and text semantic features; S1013. Based on the attention mechanism, feature fusion is performed on image visual features and text semantic features, and position encoding is embedded to obtain multimodal fusion features; S1014. Input the multimodal fusion features into the pre-trained key difference recognition model to obtain the key regions output by the model; S1015. Extract target physiological numerical indicators and target text description indicators from key areas using OCR technology.

[0040] Specifically, this embodiment of the invention employs a Transformer multimodal fusion model (ResNet-50+BERT+attention mechanism), combined with a dynamic layout segmentation algorithm and a medical title keyword regularization library, to locate key areas in the report (test items, reference ranges, image description text), thereby improving the recognition accuracy of complex layouts.

[0041] Image feature extraction layer: ResNet-50 is used as the backbone network to extract global image features. and local features ; Text feature encoding layer: Employs a BERT pre-trained model to encode text features. ; Feature fusion layer: Design an attention mechanism to... , and Weighted fusion is performed, and the fusion formula is as follows:

[0042] Where α, β, and γ are learnable weight parameters that are optimized through backpropagation; Positional encoding embedding: Using sinusoidal positional encoding, the text position information P is encoded as:

[0043]

[0044] Where pos is the position and i is the dimension. For model dimensions.

[0045] Multimodal fusion features are input into a pre-trained key difference recognition model to obtain the key regions output by the model. Target physiological numerical indicators and target textual description indicators are then extracted from the key regions using OCR technology.

[0046] The embodiments of this invention employ a layout analysis and dynamic error correction mechanism, using target segmentation algorithms and title keyword regular expression matching technology to accurately locate key areas (such as "inspection items" and "reference range"), thereby reducing interference from invalid information.

[0047] In some optional embodiments, a context-aware large language model is used, combined with self-supervised learning and instruction fine-tuning, to construct a pre-trained corpus from massive medical literature and clinical guidelines, thereby improving the ability to deeply understand medical terminology; dynamically retrieve triples in the medical knowledge graph (such as "high blood sugar → diabetes → recommend endocrinology department") to constrain the results generated by the large language model, ensuring the professionalism and consistency of the interpretation content; a standardized medical terminology lexicon is constructed, and the problem of polysemy is solved through synonym replacement and association techniques, and the parsing results are optimized by combining expert feedback fine-tuning mechanism; a temporal knowledge graph engine is used to construct the trajectory of indicator changes based on the patient's historical data and associate it with the current outlier.

[0048] This invention introduces an abnormality severity grading rule base to jointly analyze numerical indicators (such as blood glucose levels) and text descriptions (such as "pulmonary nodule shadows" in imaging reports), and realizes dynamic determination of abnormality level and triggers real-time warning based on a dynamic reference interval system.

[0049] like Figure 3The diagram shown is a flowchart illustrating the steps of generating a physiological index interval prediction model according to an embodiment of the present invention. (Refer to...) Figure 3 As an optional implementation method, the physiological index interval prediction model is obtained through the following steps: S201. Obtain baseline feature samples and physiological numerical index samples of multiple sample users based on the preset medical dataset; S202. Based on baseline feature samples and physiological numerical index samples, an initial model for predicting physiological index intervals is trained using the XGBoost algorithm. S203. Determine the seasonal impact coefficients for different seasons through the seasonal impact factor database, and determine the regional difference coefficients for different regions through the regional characteristic database. S204. Adjust the initial model for predicting physiological indicators based on the seasonality influence coefficient and the regional difference coefficient to obtain the physiological indicator interval prediction model.

[0050] Specifically, the physiological index interval prediction model in this embodiment of the invention is a dynamic reference interval adjustment model based on machine learning, as follows: 1) Feature engineering: Extracting patient characteristics such as age, gender, medical history, and lifestyle habits; 2) Model architecture: The XGBoost algorithm is used to construct a reference interval prediction model that integrates multiple features; 3) Training data: A large-scale medical dataset is used, containing the distribution of physiological indicators of different populations; 4) Prediction formula:

[0051] in, is the dynamically adjusted reference interval; X is the feature vector; w1-w4 are the feature weights; b is the bias term.

[0052] 5) Seasonal adjustment mechanism Establish a database of seasonal influencing factors to record the patterns of seasonal changes in different indicators; Introducing time series analysis to predict the seasonal fluctuations of indicators; Adjust the formula:

[0053] in, The reference interval takes seasonal factors into account; t is the current time point; T is the seasonal cycle; α is the seasonality coefficient.

[0054] 6) Adjustment based on regional differences Construct a regional characteristic database to record the physiological differences among populations in different regions; Introducing a Geographic Information System (GIS) enables spatial analysis of regional characteristics; Adjust the formula:

[0055] in, This is the reference interval after considering regional factors; D is the regional difference coefficient; β is the regional influence weight.

[0056] like Figure 4 The diagram shown is a flowchart of step S103 provided in an embodiment of the present invention. (Refer to...) Figure 4 As an optional implementation, the deviation of the physiological numerical index is calculated based on the target physiological numerical index and the reference range of the physiological index, and the degree of abnormality of the physiological numerical index is determined based on the deviation of the physiological numerical index, which specifically includes: S1031. Calculate the mean and standard deviation of the reference interval for physiological indicators; S1032. Calculate the difference between the target physiological numerical index and the mean, and determine the deviation of the physiological numerical index based on the ratio of the difference to the standard deviation. S1033. Determine the physiological value grading threshold based on the preset medical rule base, and correct the physiological value grading threshold through the seasonal influence coefficient and the regional difference coefficient. S1034. Determine the degree of abnormality of physiological numerical indicators based on the deviation of physiological numerical indicators and the corrected physiological numerical grading threshold.

[0057] First, obtain the physiological indicator reference range from the aforementioned steps. and target physiological numerical indicators Calculate the standardized deviation:

[0058] in, The mean of the reference interval. The standard deviation is denoted as .

[0059] Based on deviation Hierarchical classification with medical rule bases, for example: normal: ≤ 1.0; Minor abnormality: 1.0 ≤ 2.0; Moderately abnormal: 2.0 ≤ 3.0; Critical anomaly: 3.0 ≤ 4.0; Critical value: >4.0.

[0060] In addition, the seasonal factor α and the regional difference coefficient β, introduced in the aforementioned steps, are used to correct the grading threshold:

[0061] The degree of abnormality of physiological numerical indicators can be determined based on the corrected grading threshold and the deviation of physiological numerical indicators.

[0062] like Figure 5 The diagram shown is a flowchart of step S104 provided in an embodiment of the present invention. (Refer to...) Figure 5 As an optional implementation, key descriptive terms are extracted based on the target text description indicators, and the corresponding disease entities, standard text descriptions of the disease entities, and disease severity are queried in a pre-constructed multi-dimensional medical knowledge graph based on the key descriptive terms. Specifically, this includes: S1041. Perform semantic parsing on the target text description to obtain key descriptive words; S1042. Use key descriptive terms as indexes to query the corresponding disease entities in the multi-dimensional medical knowledge graph; S1043. Match the corresponding standard text description and disease severity based on the disease entity.

[0063] Specifically, the semantic parsing module is used to extract key descriptive terms from the image report. For example, "lung nodule shadow" → extract ["lung", "nodule", "shadow"]; query the pre-built multi-dimensional medical knowledge graph to retrieve the disease entity corresponding to the keyword and obtain the disease severity attribute, for example, "lung nodule shadow" → ["benign nodule (mild)", "malignant tumor (severe)"].

[0064] The multi-dimensional medical knowledge graph system of this invention includes: 1) Knowledge Graph Construction Entity types: symptoms, diseases, treatment plans, contraindications, examination items, etc. Types of relationships: causal relationship, therapeutic relationship, taboo relationship, examination relationship, etc.; Attribute types: severity, probability of occurrence, treatment cycle, contraindication level, etc.

[0065] 2) Time-series knowledge graph Time dimension: Recording the time-series characteristics of disease development; State transition: describes the state changes at different stages of a disease; Predictive Model: Disease progression prediction based on time-series knowledge graph.

[0066] 3) Cross-modal knowledge graph Text-image association: Establishing a mapping relationship between medical image features and text descriptions; Multimodal feature extraction: using deep learning models to extract image and text features; Feature alignment: Achieving spatial alignment of features from different modalities.

[0067] 4) Knowledge Graph Applications Anomaly detection: Utilizing knowledge graphs for multi-dimensional anomaly analysis; Recommended treatment plan: Optimal treatment plan based on knowledge graph reasoning; Risk warning: Multi-factor risk prediction using knowledge graphs.

[0068] like Figure 6 The diagram shown is a flowchart of step S105 provided in an embodiment of the present invention. (Refer to...) Figure 6 As an optional implementation, the semantic similarity between the target text description index and the standard text description is calculated, and the degree of abnormality of the text description index is determined based on the semantic similarity and the severity of the disease. Specifically, this includes: S1051. Extract the first text embedding vector of the target text description index and the second text embedding vector of the standard text description based on the BERT model; S1052. Determine the semantic similarity based on the cosine value of the first text embedding vector and the second text embedding vector; S1053. Based on the preset weight parameters, the semantic similarity and disease severity are weighted and summed to obtain the abnormality level of the text description index.

[0069] Specifically, the BERT model is used to calculate the similarity between the report description and the standard description in the knowledge graph:

[0070] in, Embed a vector for the report text. Standard description of embedding vectors Based on a comprehensive judgment of similarity and knowledge graph attributes:

[0071] in, , As weight, This represents the severity score in the knowledge graph.

[0072] like Figure 7 The diagram shown is a flowchart of step S106 provided in an embodiment of the present invention. (Refer to...) Figure 7 As an optional implementation, a comprehensive abnormality score is determined based on the degree of abnormality of physiological numerical indicators and textual descriptive indicators, and corresponding early warning information is generated based on the comprehensive abnormality score, specifically including: S1061. By reasoning the causal relationship between the target physiological numerical indicators and the target text description indicators through a multi-dimensional medical knowledge graph, multiple causal indicator pairs are obtained. S1062. Calculate the correlation between the target physiological numerical indicator / target textual description indicator and other physiological numerical indicators / other textual description indicators for the causal indicator pair; S1063. Calculate the correlation anomaly values ​​of causal indicator pairs based on the degree of anomaly and correlation of other physiological numerical indicators / other textual descriptive indicators; S1064. Based on the preset weight parameters, the abnormality degree of the physiological numerical indicators, the abnormality degree of the text description indicators, and the associated outliers of the causal indicator pair are weighted and summed to obtain the comprehensive abnormality score of the causal indicator pair. S1065. Determine the severity of the causal indicator pair based on the comprehensive anomaly score, and generate early warning information based on the severity of the indicator.

[0073] Specifically, correlation analysis is performed on all abnormal indicators (numerical and textual) in the current report. Spearman correlation analysis is used to calculate the correlation between numerical indicators, and knowledge graph is used to infer the causal relationship between textual descriptions and numerical indicators.

[0074] Construct the correlation matrix M of abnormal indicators:

[0075] in, The correlation is calculated based on Spearman correlation analysis. This refers to the semantic relevance based on knowledge graphs.

[0076] The calculation method is as follows: Find the shortest path length L between indicators i and j in the knowledge graph; calculate λ is the attenuation coefficient.

[0077] For each anomaly index i, calculate the overall anomaly score:

[0078] in, Numerical outlier scores (based on) (normalization) Text-based anomaly scores (based on) (normalization) The weighted influence of the related indicators is represented by w1, w2, and w3, which are learnable weights that satisfy w1 + w2 + w3 = 1.

[0079] based on Final grading: normal: <0.3; Minor abnormalities: 0.3 ≤ <0.5; Moderate abnormality: 0.5 ≤ <0.7; Severe abnormality: 0.7 ≤ <0.9; Critical value: ≥ 0.9.

[0080] when When the value is ≥ 0.7, a real-time early warning mechanism is triggered: Pushing alerts to doctors; The report is marked as "requires urgent handling"; Automatically generate medical treatment suggestions (based on knowledge graph reasoning).

[0081] In some optional embodiments, feedback correction for the dynamic reference range is also included: When a compound abnormal pattern is detected, the aforementioned physiological index interval prediction model (XGBoost model) is called back. Based on historical data and current anomaly patterns, the feature weights of the XGBoost model are fine-tuned, and the updated formula is as follows:

[0082] Where η is the learning rate and L is the loss function.

[0083] In some optional embodiments, the present invention incorporates patient historical data for time-series analysis: Extract the abnormal score sequence from the patient's past N tests: ; Calculate the abnormal trend index:

[0084] When Trend> At that time, the current anomaly level will be raised by one level; By combining time-series knowledge graphs, we can predict future abnormal development trajectories.

[0085] In some optional embodiments, a dynamic health risk assessment report is generated by integrating user historical data, current test results, and external factors (such as age, gender, and lifestyle habits); personalized health suggestions, such as diet, exercise, and further examination suggestions, are provided based on user profiles (such as age and medical history) using dynamic prompt word generation technology; the interpretation results are displayed through a visual interface, including line graphs, heat maps, etc.; real-time updates are supported to ensure a response time of seconds; users can intuitively view the trend of changes in their health status and personalized suggestions, and obtain dynamically adjusted interpretation content.

[0086] In some optional embodiments, the process of multi-indicator correlation analysis is as follows: Step 1: Data Preprocessing The test index is standardized as follows: x' = (x-μ) / σ; Outlier removal: The 3σ principle is used.

[0087] Step 2: Correlation Calculation The modified Spearman correlation coefficient was used for calculation:

[0088] Symbol meaning: Spearman correlation coefficient, with a value range of [-1, 1]; : Perfectly positive correlation (if one variable increases, the other variable also increases); : Perfectly negative correlation (when one variable increases, the other variable decreases); No correlation. The difference between the rankings of the i-th sample and the two variables; n: Number of samples.

[0089] Correlation calculation steps: 1) Rank the two variables separately: Sort the values ​​of each variable from smallest to largest and assign them a ranking value (if there are the same value, take the average ranking).

[0090] 2) Calculate the square of the ranking difference: For each sample, calculate the difference in its ranking between the two variables. And find the square.

[0091] 3) Summation: Calculate the summation of all samples. sum; 4) Substitute into the formula: Substitute the summation result into the formula to calculate. .

[0092] Step 3: Determining Composite Anomalies Set the threshold θ = 0.7; When |ρ|>θ, it is determined to be a strong correlation; Joint analysis of strongly correlated indicator combinations: 1) Calculate the overall anomaly score: S = w1·s1 + w2·s2, where w1 and w2 are weights, and s1 and s2 are individual anomaly scores; 2) When S>0.8, a composite anomaly warning is triggered.

[0093] The present invention will be further described below with reference to a specific embodiment.

[0094] Scenario description: The patient's CT imaging report showed "a nodule with a diameter of about 8 mm and spiculated margins in the upper lobe of the right lung", while the blood test showed "the tumor marker CEA was elevated (8.5 ng / mL, reference range 0-5)".

[0095] Step 1: Classification of Numerical Indicator Anomalies Calculate CEA deviation: (Assuming) =2.5, =1.5); Preliminary assessment: Severe abnormality; Considering the patient is a 60-year-old male with a history of smoking, the dynamic reference interval system is used: (Upper limit relaxed) Recalculate: ; Correction judgment: Moderate abnormality ( ).

[0096] Step 2: Text-descriptive anomaly classification Extract keywords: ["right upper lobe", "8mm", "nodular shadow", "marginal spiculation sign"]; Querying knowledge graphs: "Marginal spiculation sign" → a characteristic of malignant tumors (severity 0.85); "8mm nodule" → Further examination required (severity 0.60); Semantic similarity calculation: Sim("marginal spiculation sign", "malignant tumor imaging features") = 0.92; Overall judgment: (Serious abnormality).

[0097] Step 3: Conjoint Analysis Knowledge graph query: "Elevated CEA" ← Causal relationship → "Lung cancer"; "Marginal spiculation sign" ← Strong association → "Lung cancer"; Calculate the correlation: (Path length L=1); Overall abnormal score:

[0098] Step 4: Severity assessment and early warning It was determined to be "seriously abnormal"; Trigger the real-time early warning mechanism; The generated suggestion is: "It is recommended to see a thoracic surgeon or oncologist as soon as possible for an enhanced CT or PET-CT scan to rule out the possibility of malignancy."

[0099] Step 5: Time series analysis (assuming historical data is available) The patient's CEA level was 5.2 ng / mL 3 months ago and 3.8 ng / mL 6 months ago; Abnormal trend: (Assume the threshold is 2.0); Level upgrade: Severe anomaly → Critical value; Final assessment: Critical condition, immediate medical attention required.

[0100] The method flow of the embodiments of the present invention has been described above. It can be understood that the embodiments of the present invention extract target physiological numerical indicators and target textual description indicators based on identified key regions, determine corresponding physiological indicator reference intervals based on user baseline feature data to calculate the degree of abnormality of physiological numerical indicators, determine corresponding disease entities, standard text descriptions, and disease severity based on key descriptive words of target textual description indicators, thereby calculating the degree of abnormality of textual description indicators, and then determine a comprehensive abnormality score based on the degree of abnormality of physiological numerical indicators and textual description indicators, generating corresponding early warning information, thus improving the accuracy and comprehensiveness of medical report interpretation.

[0101] Compared with the prior art, the embodiments of the present invention have the following advantages: 1) Improve the accuracy of report parsing: solve the problem of information loss caused by complex layout and improve the text recognition rate.

[0102] 2) Enhance semantic understanding capabilities: Break through the limitations of traditional keyword matching and support context-aware interpretation of medical terms.

[0103] 3) Improve user experience: Provide real-time interpretation results and personalized health advice to reduce the user's understanding cost.

[0104] 4) Expanding application scenarios: Applicable to hospital examination and test reports, physical examination reports, and other medical document interpretation needs.

[0105] 5) Optimize anomaly detection and health assessment: By linking multi-level anomaly detection with historical data, the accuracy of detection and the reliability of health assessment are significantly improved.

[0106] 6) Improve the accuracy and comprehensiveness of anomaly detection: Break through the limitations of traditional methods that only focus on numerical indicators, and achieve deep integration and analysis of numerical data and text; avoid "one-size-fits-all" misjudgments through dynamic reference intervals, reducing the false positive rate by about 30%; improve detection sensitivity by recognizing composite anomaly patterns, and increase the accuracy of early warning by 25%; the time series analysis mechanism enables the accuracy of disease progression prediction to reach more than 85%.

[0107] like Figure 8 The diagram shown is a structural schematic of a medical report interpretation device based on multimodal dynamic knowledge fusion provided in an embodiment of the present invention. (Refer to...) Figure 8 This invention provides a medical report interpretation device based on multimodal dynamic knowledge fusion, comprising: The indicator extraction module is used to identify several key areas of the target medical report and extract target physiological numerical indicators and target text description indicators from the key areas. The reference interval determination module is used to input the baseline feature data of the target user into a pre-trained physiological index interval prediction model to obtain the reference interval of the target user's physiological index. The physiological numerical index assessment module is used to calculate the deviation of physiological numerical index based on the target physiological numerical index and the reference range of physiological index, and to determine the degree of abnormality of physiological numerical index based on the deviation of physiological numerical index. The key descriptor query module is used to extract key descriptors based on the target text description indicators, and to query the corresponding disease entities, standard text descriptions of the disease entities, and disease severity in a pre-built multi-dimensional medical knowledge graph based on the key descriptors. The text description indicator evaluation module is used to calculate the semantic similarity between the target text description indicator and the standard text description, and to determine the degree of abnormality of the text description indicator based on the semantic similarity and the severity of the disease. The early warning module is used to determine a comprehensive abnormality score based on the degree of abnormality of physiological numerical indicators and textual descriptive indicators, and to generate corresponding early warning information based on the comprehensive abnormality score.

[0108] It is understood that the content of the above method embodiments is applicable to the present device embodiments. The specific functions implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0109] This invention also provides an electronic device, comprising: a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for communication between the processor and the memory. When the program is executed by the processor, it implements the aforementioned medical report interpretation method based on multimodal dynamic knowledge fusion. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0110] like Figure 9 The diagram shown is a hardware structure schematic of an electronic device provided in an embodiment of the present invention. (Refer to...) Figure 9 This invention provides an electronic device, comprising: The processor 901 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present invention. The memory 902 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 902 can store the operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 902 and is called and executed by the processor 901 to execute the medical report interpretation method based on multimodal dynamic knowledge fusion of the embodiments of this invention. The input / output interface 903 is used to implement information input and output; The communication interface 904 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.). Bus 905 transmits information between various components of the device (e.g., processor 901, memory 902, input / output interface 903, and communication interface 904); The processor 901, memory 902, input / output interface 903, and communication interface 904 are connected to each other within the device via bus 905.

[0111] It is understood that the content of the above method embodiments is applicable to this device embodiment. The specific functions implemented by this device embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0112] like Figure 10 The diagram shown is a structural schematic of the storage medium provided in an embodiment of the present invention. (Refer to...) Figure 10 The present invention also provides a storage medium, which is a computer-readable storage medium for computer-readable storage. The storage medium stores one or more programs 1001, which can be executed by one or more processors to implement the above-described medical report interpretation method based on multimodal dynamic knowledge fusion.

[0113] It is understood that the content of the above method embodiments is applicable to this storage medium embodiment. The specific functions implemented in this storage medium embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0114] This invention also discloses a computer program product, including a computer program that, when executed by a processor, implements the above-described method for interpreting medical reports based on multimodal dynamic knowledge fusion.

[0115] It is understood that the content of the above method embodiments is applicable to the embodiments of this program product. The specific functions implemented by the embodiments of this program product are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0116] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0117] The embodiments described in this invention are for the purpose of more clearly illustrating the technical solutions of the embodiments of this invention, and do not constitute a limitation on the technical solutions provided by the embodiments of this invention. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this invention are also applicable to similar technical problems.

[0118] The terms "first," "second," "third," "fourth," etc. (if present) in the specification and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0119] In some alternative embodiments, the functions / operations mentioned in the block diagrams may not occur in the order shown in the operation diagrams. For example, depending on the functions / operations involved, two consecutively shown blocks may actually be executed substantially simultaneously, or the aforementioned blocks may sometimes be executed in reverse order. Furthermore, the embodiments presented and described in the flowcharts of this invention are provided by way of example to provide a more comprehensive understanding of the technology. The disclosed methods are not limited to the operations and logic flows presented herein. Alternative embodiments are contemplated in which the order of various operations is changed and sub-operations described as part of a larger operation are executed independently.

[0120] Furthermore, although the invention has been described in the context of functional modules, it should be understood that, unless otherwise stated, one or more of the aforementioned functions and / or features may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in a separate physical device or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the invention. Rather, given the properties, functions, and internal relationships of the various functional modules in the apparatus disclosed herein, the actual implementation of the module will be understood within the scope of conventional skill of an engineer. Therefore, those skilled in the art can implement the invention as set forth in the claims using ordinary techniques without excessive experimentation. It is also understood that the specific concepts disclosed are merely illustrative and not intended to limit the scope of the invention, which is determined by the full scope of the appended claims and their equivalents.

[0121] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0122] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-including system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0123] More specific examples (a non-exhaustive list) of computer-readable media include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which the aforementioned program can be printed, because the aforementioned program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0124] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0125] In the foregoing description of this specification, references to terms such as "one embodiment," "another embodiment," or "some embodiments" indicate that a specific feature, structure, material, or characteristic described in connection with an embodiment or example is included in at least one embodiment or example of the present invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0126] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

[0127] The above is a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the above embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of this application.

Claims

1. A method for interpreting medical reports based on multimodal dynamic knowledge fusion, characterized in that, Includes the following steps: Identify several key areas of the target medical report, and extract target physiological numerical indicators and target textual description indicators from the key areas; The baseline feature data of the target user is input into a pre-trained physiological index interval prediction model to obtain the physiological index reference interval of the target user. The deviation of the physiological numerical index is calculated based on the target physiological numerical index and the reference range of the physiological index, and the degree of abnormality of the physiological numerical index is determined based on the deviation of the physiological numerical index. Key descriptive terms are extracted based on the target text description metrics, and the corresponding disease entities, standard text descriptions of the disease entities, and disease severity are queried in a pre-constructed multi-dimensional medical knowledge graph based on the key descriptive terms. Calculate the semantic similarity between the target text description index and the standard text description, and determine the degree of abnormality of the text description index based on the semantic similarity and the severity of the disease; A comprehensive abnormality score is determined based on the degree of abnormality of the physiological numerical indicators and the degree of abnormality of the text description indicators, and corresponding early warning information is generated based on the comprehensive abnormality score.

2. The medical report interpretation method based on multimodal dynamic knowledge fusion according to claim 1, characterized in that, The process involves identifying several key areas of the target medical report and extracting target physiological numerical indicators and target textual descriptive indicators from these key areas. Specifically, this includes: The target medical report is structurally semantically segmented based on a dynamic page layout algorithm and a medical title keyword regular expression library to obtain multiple page layout regions; Image features are extracted and text features are encoded for each of the aforementioned page areas to obtain image visual features and text semantic features; Based on the attention mechanism, the visual features of the image and the semantic features of the text are fused, and positional encoding is embedded to obtain multimodal fusion features; The multimodal fusion features are input into a pre-trained key difference recognition model to obtain the key region output by the model; The target physiological numerical indicators and the target textual description indicators are extracted from the key region using OCR technology.

3. The medical report interpretation method based on multimodal dynamic knowledge fusion according to claim 1, characterized in that, The physiological index interval prediction model is obtained through the following steps: Baseline feature samples and physiological numerical index samples of multiple sample users are obtained based on a pre-set medical dataset; Based on the baseline feature samples and the physiological numerical index samples, an initial model for predicting physiological index intervals is trained using the XGBoost algorithm. The seasonal impact coefficients for different seasons are determined using a seasonal impact factor database, and the regional difference coefficients for different regions are determined using a regional characteristic database. The initial model for predicting the physiological index intervals is adjusted based on the seasonality coefficient and the regional difference coefficient to obtain the physiological index interval prediction model.

4. The medical report interpretation method based on multimodal dynamic knowledge fusion according to claim 1, characterized in that, The step of calculating the deviation of the physiological numerical index based on the target physiological numerical index and the reference range of the physiological index, and determining the degree of abnormality of the physiological numerical index based on the deviation of the physiological numerical index, specifically includes: Calculate the mean and standard deviation of the reference interval for the physiological indicators; Calculate the difference between the target physiological numerical index and the mean, and determine the deviation of the physiological numerical index based on the ratio of the difference to the standard deviation; Physiological value grading thresholds are determined based on a preset medical rule base, and then corrected using seasonality and regional difference coefficients. The degree of abnormality of the physiological numerical index is determined based on the deviation of the physiological numerical index and the corrected physiological numerical grading threshold.

5. The medical report interpretation method based on multimodal dynamic knowledge fusion according to claim 1, characterized in that, The step of extracting key descriptive terms based on the target text description indicators, and querying the corresponding disease entities, standard text descriptions, and disease severity in a pre-constructed multi-dimensional medical knowledge graph based on the key descriptive terms, specifically includes: The key descriptive words are obtained by semantic parsing the target text description; Using the key descriptive terms as indexes, the corresponding disease entities are queried in the multi-dimensional medical knowledge graph; The disease entity is matched with the corresponding standard text description and the severity of the disease.

6. The medical report interpretation method based on multimodal dynamic knowledge fusion according to claim 1, characterized in that, The step of calculating the semantic similarity between the target text description index and the standard text description, and determining the degree of abnormality of the text description index based on the semantic similarity and the severity of the disease, specifically includes: Based on the BERT model, the first text embedding vector of the target text description index and the second text embedding vector of the standard text description are extracted. The semantic similarity is determined based on the cosine values ​​of the first text embedding vector and the second text embedding vector; The semantic similarity and the severity of the disease are weighted and summed according to preset weight parameters to obtain the abnormality level of the text description index.

7. A method for interpreting medical reports based on multimodal dynamic knowledge fusion according to any one of claims 1 to 6, characterized in that, The step of determining a comprehensive abnormality score based on the degree of abnormality of the physiological numerical indicators and the degree of abnormality of the text description indicators, and generating corresponding early warning information based on the comprehensive abnormality score, specifically includes: By reasoning the causal relationship between the target physiological numerical indicators and the target textual description indicators through the multi-dimensional medical knowledge graph, multiple causal indicator pairs are obtained. Calculate the correlation between the target physiological numerical index / target textual description index and other physiological numerical index / other textual description index of the causal index pair; The correlation anomaly value of the causal indicator pair is calculated based on the degree of anomaly of other physiological numerical indicators / other textual descriptive indicators and the correlation degree. The abnormality degree of the physiological numerical index, the abnormality degree of the text description index, and the correlation anomaly value of the causal index pair are weighted and summed according to the preset weight parameters to obtain the comprehensive anomaly score of the causal index pair. The severity of the causal indicator pair is determined based on the comprehensive anomaly score, and the warning information is generated based on the severity of the indicator.

8. A medical report interpretation device based on multimodal dynamic knowledge fusion, characterized in that, include: The indicator extraction module is used to identify several key areas of the target medical report and extract target physiological numerical indicators and target text description indicators from the key areas. The reference interval determination module is used to input the baseline feature data of the target user into a pre-trained physiological index interval prediction model to obtain the physiological index reference interval of the target user. The physiological numerical index evaluation module is used to calculate the deviation of the physiological numerical index based on the target physiological numerical index and the reference range of the physiological index, and to determine the degree of abnormality of the physiological numerical index based on the deviation of the physiological numerical index. The key descriptor query module is used to extract key descriptors based on the target text description indicators, and query the corresponding disease entities, standard text descriptions of the disease entities, and disease severity in a pre-constructed multi-dimensional medical knowledge graph based on the key descriptors. The text description indicator evaluation module is used to calculate the semantic similarity between the target text description indicator and the standard text description, and to determine the degree of abnormality of the text description indicator based on the semantic similarity and the severity of the disease. The early warning module is used to determine a comprehensive abnormality score based on the degree of abnormality of the physiological numerical indicators and the degree of abnormality of the text description indicators, and to generate corresponding early warning information based on the comprehensive abnormality score.

9. An electronic device, characterized in that, The electronic device includes a memory, a processor, a computer program stored in the memory and executable on the processor, and a data bus for enabling communication between the processor and the memory. When the computer program is executed by the processor, it implements the medical report interpretation method based on multimodal dynamic knowledge fusion as described in any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the medical report interpretation method based on multimodal dynamic knowledge fusion as described in any one of claims 1 to 7.

Images