Method and system for interpreting medical examination sheet, electronic equipment and storage medium

Abstract

The invention discloses a method and system for interpreting a medical examination sheet, electronic equipment and a storage medium, and relates to the field of data processing. The method comprises the following steps: correcting an uploaded medical examination sheet picture through a preset discrimination model to obtain a target picture at a preset angle, performing optical character recognition on the target picture to extract target information, and constructing structured data based on the target information; filtering the structured data through regular configuration and enumeration mapping to obtain intermediate data; generating a query vector according to the intermediate data, and performing similarity retrieval in a preset vector database according to the query vector to obtain most relevant medical knowledge; and combining the retrieved medical knowledge with the intermediate data to obtain final data, and generating suggestions based on the final data through a preset large language model. By implementing the technical scheme provided by the invention, efficient and accurate interpretation of the medical examination sheet is realized.

Description

Technical Field

[0001] This application relates to the technical field of data processing, and particularly relates to a method, system, electronic device, and storage medium for interpreting medical examination sheets. Background Art

[0002] The interpretation of medical examination sheets is an important part of the medical and health field, and is of great significance for improving the quality and efficiency of medical services. Traditional medical examination sheets usually exist in paper form and need to be manually read and recorded by doctors. This process is not only time-consuming and laborious, but also prone to misreading or omitting important information due to human factors. In recent years, with the development of information technology, more and more research has been dedicated to developing automated medical examination sheet interpretation systems, aiming to achieve efficient and accurate interpretation of medical examination sheets through computer vision and natural language processing technologies.

[0003] However, existing technical means still have certain defects. Especially when dealing with complex and diverse medical examination sheets, it is often difficult to ensure high-precision recognition rates and reliability. For example, OCR technology performs poorly when faced with handwritten or low-quality printed text, and also has limitations when dealing with pictures with deflection angles.

[0004] Therefore, how to further improve the robustness and accuracy of medical examination sheet interpretation systems has become a key problem to be solved urgently. Summary of the Invention

[0005] This application provides a method, system, electronic device, and storage medium for interpreting medical examination sheets, which realizes efficient and accurate interpretation of medical examination sheets and improves the working efficiency of doctors.

[0006] In the first aspect of this application, a method for interpreting medical examination sheets is provided, which is applied to an examination sheet detection platform. The method includes: Correcting the uploaded medical examination sheet picture through a preset discrimination model to obtain a target picture at a preset angle, performing optical character recognition on the target picture to extract target information, and constructing structured data based on the target information; Filtering the structured data through regular configuration and enumeration mapping to obtain intermediate data; Generating a query vector according to the intermediate data, and performing similarity retrieval in a preset vector database according to the query vector to obtain the most relevant medical knowledge; Combining the retrieved medical knowledge with the intermediate data to obtain final data, and generating suggestions based on the final data through a preset large language model.

[0007] Optionally, the correcting the uploaded medical examination sheet picture through a preset discrimination model to obtain a target picture at a preset angle includes: Preprocess the medical examination form image and extract the target features of the preprocessed medical examination form image, where the target features include image edge lines, table structure information, and text structure information; Calculate the deflection angle of the medical examination form image relative to the horizontal direction according to the target features, and perform geometric transformation on the medical examination form image according to the deflection angle to obtain the target image.

[0008] Optionally, the performing optical character recognition on the target image to extract target information and constructing structured data based on the target information includes: Use optical character recognition technology to perform text recognition on the target image to extract text information, where the text information includes examination items, examination results, and reference ranges; Parse and classify the text information to identify abnormal items and corresponding values; According to a preset structured data template, fill the text information in a specified format and fields to construct structured data, where the structured data includes patient basic information and abnormal results.

[0009] Optionally, the filtering the structured data through regular configuration and enumeration mapping to obtain intermediate data includes: Use regular expressions to perform format verification on the numerical information in the structured data, and match the examination items in the structured data with preset medical terms through enumeration mapping. The regular expressions include verification rules for numerical ranges, decimal places, and units; Reconfirm the abnormal items, and classify and mark the abnormal items according to preset rules; Generate intermediate data according to the results of the regular configuration and enumeration mapping. The intermediate data includes the standardized examination item names, corresponding values, and the marks and classifications of abnormal items.

[0010] Optionally, the generating a query vector according to the intermediate data and performing similarity retrieval in a preset vector database according to the query vector to obtain the most relevant medical knowledge includes: Use the RAG technology to take the examination items, abnormal item values, and abnormal item classifications in the intermediate data as knowledge points, and generate corresponding query vectors through a preset language model; Apply the K-nearest neighbor algorithm to calculate the similarity between the query vector and the knowledge vectors in the preset vector database, and determine the most similar preset number of candidate knowledge entries; Use a preset re-ranking model to re-rank the preset number of candidate knowledge entries according to the relevance between the query vector and the preset number of candidate knowledge entries, and use the knowledge entry ranked first after re-ranking as the most relevant medical knowledge.

[0011] Optionally, the method further includes constructing the preset vector database, specifically including: Construct a graph structure from the entities and relationships in the medical knowledge entries, and use a graph neural network to encode the graph structure to generate an embedding vector for each node; Use a mapping network to map the embedding vector to the vector space of the language model to obtain a knowledge vector, and store the knowledge vector in an Elasticsearch database.

[0012] Optionally, the combining the retrieved medical knowledge with the intermediate data to obtain the final data and generating a suggestion based on the final data through a preset large language model includes: Fuse the retrieved medical knowledge with the intermediate data to form final data including inspection items, abnormal item values, abnormal item classifications, and the medical knowledge; Use a preset large language model to process the final data to generate personalized suggestions for the user. The personalized suggestions include explanations of the abnormal items, and present the generated suggestions to the user in the form of natural language.

[0013] In a second aspect of the present application, a system for interpreting a medical examination form is provided, including an identification module, a filtering module, a retrieval module, and a suggestion module, where: The identification module is configured to correct the uploaded medical examination form picture through a preset discrimination model to obtain a target picture at a preset angle, perform optical character recognition on the target picture to extract target information, and construct structured data based on the target information; The filtering module is configured to filter the structured data through regular configuration and enumeration mapping to obtain intermediate data; The retrieval module is configured to generate a query vector according to the intermediate data, and perform similarity retrieval in a preset vector database according to the query vector to obtain the most relevant medical knowledge; The suggestion module is configured to combine the retrieved medical knowledge with the intermediate data to obtain final data, and generate a suggestion based on the final data through a preset large language model.

[0014] In a third aspect of the present application, an electronic device is provided, including a processor, a memory, a user interface, and a network interface. The memory is used to store instructions. Both the user interface and the network interface are used to communicate with other devices. The processor is used to execute the instructions stored in the memory so that the electronic device executes the method described in any one of the above.

[0015] In a fourth aspect of the present application, a computer-readable storage medium is provided. The computer-readable storage medium stores instructions that, when executed, execute the method described in any one of the above.

[0016] In summary, one or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages: 1. By using a preset discrimination model to correct the uploaded medical examination form picture to a preset angle (such as the horizontal direction), the readability of the picture and the accuracy of subsequent processing are improved. This method can effectively solve the problem of picture deflection caused by factors such as shooting angle and equipment, ensuring the accuracy and efficiency of optical character recognition (OCR); during the correction process, target features such as image edge lines, table structure information, and text structure information are extracted, which helps to more accurately calculate the deflection angle of the picture and perform corresponding geometric transformations, further improving the effect of picture correction; 2. Perform OCR processing on the corrected target picture to extract text information, including examination items, examination results, reference ranges, etc., providing a basis for subsequent data processing and analysis; filter the structured data through regular configuration and enumeration mapping to obtain intermediate data. Regular configuration can perform format verification on numerical information to ensure the accuracy and consistency of the data; enumeration mapping matches the examination items with preset medical terms, standardizing the names and descriptions of the examination items, improving the reliability and usability of the data; 3. Generate a query vector based on the intermediate data. This vector can capture the key features of the intermediate data, providing a basis for subsequent similarity retrieval. Perform similarity retrieval in a preset vector database to obtain the most relevant medical knowledge. This method can quickly and accurately find the medical knowledge related to the abnormal items of the examination form, providing strong support for subsequent recommendation generation. Use a preset re-ranking model to re-rank the candidate knowledge entries to ensure that the most relevant medical knowledge is returned first, further improving the accuracy and relevance of the retrieval results.

[0017] 4. Combine the retrieved medical knowledge with the intermediate data to obtain the final data. This fusion method can make full use of the intermediate data and medical knowledge, providing more comprehensive and accurate information for subsequent recommendation generation. By presetting a large language model to generate recommendations based on the final data, accurate and targeted medical recommendations can be generated. The large language model can understand the semantic information of the final data and generate natural and fluent recommendation texts, improving the quality and readability of the recommendations. The generated recommendations include explanations of abnormal items, possible disease diagnoses, recommended further examination items, and treatment plans, etc., which can provide personalized medical recommendations according to the specific situation of the patient, improving the quality of medical services and patient satisfaction.

[0018] 5. Construct the entities and relationships in the medical knowledge entries into a graph structure, use a graph neural network to encode the graph structure, generate the embedding vectors of each node, and map the embedding vectors to the vector space of the language model to obtain knowledge vectors, which are stored in the Elasticsearch database. This method can effectively manage and store a large amount of medical knowledge, providing convenience for the expansion and maintenance of the system. Regularly update and maintain the medical knowledge base to ensure that it contains the latest medical research results and clinical practice experiences, improving the accuracy and reliability of the system, and enhancing the scalability and maintainability of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 is a schematic flowchart of the method for interpreting a medical examination form disclosed in an embodiment of the present application; Figure 2 is a schematic architecture diagram of the examination form detection platform disclosed in an embodiment of the present application; Figure 3 is a schematic module diagram of the system for interpreting a medical examination form disclosed in an embodiment of the present application; Figure 4 is a schematic structural diagram of an electronic device disclosed in an embodiment of the present application.

[0020] Description of the reference numerals: 301, identification module; 302, filtering module; 303, retrieval module; 304, recommendation module; 401, processor; 402, communication bus; 403, user interface; 404, network interface; 405, memory. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0021] In order to enable those skilled in the art to better understand the technical solutions in this specification, the following will clearly and completely describe the technical solutions in the embodiments of this specification with reference to the accompanying drawings in the embodiments of this specification. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments.

[0022] In the description of the embodiments of the present application, words such as "for example" or "for illustration" are used to give examples, illustrations or explanations. Any embodiment or design solution described as "for example" or "for illustration" in the embodiments of the present application should not be construed as being more preferred or having more advantages than other embodiments or design solutions. Rather, the use of words such as "for example" or "for illustration" is intended to present relevant concepts in a specific manner.

[0023] In the description of the embodiments of the present application, the term "plurality" means two or more. For example, a plurality of systems means two or more systems, and a plurality of screen terminals means two or more screen terminals. In addition, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly indicating the technical features indicated. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more of such features. The terms "include", "comprise", "have" and their variants all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0024] This embodiment discloses a method for interpreting a medical examination form, which is applied to an examination form detection platform. Figure 1 It is a schematic flow chart of the method for interpreting a medical examination form disclosed in the embodiments of the present application. As Figure 1 shown, the method includes the following steps: S101. Correct the uploaded medical examination form picture through a preset discrimination model to obtain a target picture at a preset angle, perform optical character recognition on the target picture to extract target information, and construct structured data based on the target information; S102. Filter the structured data through regular configuration and enumeration mapping to obtain intermediate data; S103. Generate a query vector according to the intermediate data, and perform similarity retrieval in a preset vector database according to the query vector to obtain the most relevant medical knowledge; S104. Combine the retrieved medical knowledge with the intermediate data to obtain final data, and generate suggestions based on the final data through a preset large language model.

[0025] The uploaded medical examination form image is corrected by a preset discrimination model to reach a preset angle (such as the horizontal direction). This step can effectively solve the problem of image deflection caused by factors such as shooting angle and equipment, improving the readability of the image and the accuracy of subsequent processing. The specific implementation methods include preprocessing the image, extracting target features such as image edge lines, table structure information, and text structure information, calculating the deflection angle of the image based on these features, and performing corresponding geometric transformations. Optical Character Recognition (OCR) is performed on the corrected target image to extract text information, including examination items, examination results, and reference ranges, etc. These text information are the basis for constructing structured data and provide important basis for subsequent data processing and analysis. OCR technology can convert the text information in the image into editable and processable text data, improving the availability and processing efficiency of the data. The numerical information in the structured data is format-checked through regular expressions to ensure the accuracy and consistency of the data. Regular expressions can check the numerical range, decimal places, and units, etc., effectively identifying and filtering out data that does not conform to the specifications, improving the quality and reliability of the data. The examination items in the structured data are matched with preset medical terms to standardize the names and descriptions of the examination items. Enumeration mapping can unify different examination item names into standard medical terms, facilitating subsequent data processing and analysis. For example, mapping "ALT" to "Alanine Aminotransferase" and "CREA" to "Creatinine", etc. Intermediate data is generated according to the results of regular configuration and enumeration mapping. The intermediate data includes the standardized examination item names, corresponding numerical values, and the marking and classification of abnormal items. These data provide the basis for subsequent query vector generation and medical knowledge retrieval. Query vectors are generated based on the intermediate data. Query vectors can capture the key features of the intermediate data and provide the basis for subsequent similarity retrieval. The specific implementation methods include using the RAG technology, taking the examination items, abnormal item numerical values, and abnormal item classifications in the intermediate data as knowledge points, and generating corresponding query vectors through a preset language model. Similarity retrieval is performed in a preset vector database to obtain the most relevant medical knowledge. A large number of medical knowledge entries are stored in the preset vector database, and each entry corresponds to a knowledge vector. By using the K-Nearest Neighbor algorithm to calculate the similarity between the query vector and the knowledge vector, a preset number of candidate knowledge entries with the highest similarity are determined. A preset re-ranking model re-ranks the candidate knowledge entries according to the relevance between the query vector and the candidate knowledge entries, and takes the knowledge entry ranked first after re-ranking as the most relevant medical knowledge. The retrieved medical knowledge is combined with the intermediate data to obtain the final data. The final data includes information such as examination items, abnormal item numerical values, abnormal item classifications, and relevant medical knowledge, providing a comprehensive and accurate basis for subsequent recommendation generation. Recommendations are generated based on the final data through a preset large language model.The pre-set large language model can understand the semantic information of the final data and generate accurate and targeted medical advice. The advice content includes explanations of abnormal items, possible disease diagnoses, recommended further examination items, treatment plans, etc., providing personalized medical advice for patients and improving the quality of medical services and patient satisfaction.

[0026] Figure 2 It is a schematic architecture diagram of the checklist detection platform disclosed in the embodiments of the present application, in combination with Figure 2 to illustrate the embodiments of the present application.

[0027] Optionally, the correcting the uploaded medical checklist image through a pre-set discrimination model to obtain a target image at a pre-set angle includes: Preprocessing the medical checklist image and extracting target features of the preprocessed medical checklist image, where the target features include image edge lines, table structure information, and text structure information; Calculating the deflection angle of the medical checklist image relative to the horizontal direction according to the target features, and performing geometric transformation on the medical checklist image according to the deflection angle to obtain the target image.

[0028] Denoise the uploaded medical examination form images to reduce noise interference in the images and improve image quality. Median filtering, Gaussian filtering and other methods can be used to smooth the images while retaining important details. Convert the color images to grayscale images. Grayscale processing can reduce the complexity of image processing while retaining the main features of the images, which is beneficial to subsequent operations such as edge detection and angle detection. Use edge detection algorithms (such as Canny edge detection) to identify the straight edges in the pictures. Edge detection can help locate important structures in the pictures, such as the borders and table lines of the examination forms, which are very important for judging the deflection angle of the pictures. Through the edge detection algorithm, the straight edges in the pictures can be extracted, and these edge lines are important bases for judging the deflection angle of the pictures. The examination form usually contains a table structure. Extracting the border and line information of the table helps to more accurately judge the deflection of the picture. The text information in the examination form also has certain structural features. Extracting the distribution and arrangement information of the text can assist in judging the deflection angle of the picture. Apply a straight line detection algorithm (such as the Hough transform) to the pictures after grayscale and edge detection processing to detect the straight lines in the pictures. By analyzing the detected straight lines, calculate the deflection angle of the pictures. The Hough transform can map the straight lines in the image to the parameter space, making it more convenient to detect and calculate the angle of the straight lines. Use the self-trained multi-modal model MiniCpm2.6, combined with the detected straight line angle information, to make a more accurate judgment and correction of the deflection angle of the pictures. The model can adjust and optimize the detected angle according to the content and structural features in the pictures to improve the accuracy of angle correction. According to the detected and calculated deflection angle, perform geometric transformation on the pictures to restore them to a horizontal state. Geometric transformation can be achieved by operations such as image rotation and translation. When performing geometric transformation, it is necessary to pay attention to maintaining the integrity and clarity of the pictures to avoid image distortion and blurring introduced by the transformation. After the pictures are corrected, perform quality detection to ensure that the image quality meets the requirements. If quality problems are found in the corrected pictures, such as blurring and distortion, further optimization processing can be carried out. For example, use super-resolution algorithms to enlarge and clarify the pictures, or use image repair algorithms to repair the defects in the pictures. Return the corrected pictures to the users or other modules in the system for use. Users can view the corrected pictures through the application or web interface and perform further operations such as printing, sharing or storing. At the same time, the system can record the users' usage feedback and data to evaluate the effect of picture correction and continuously optimize the model and algorithms.

[0029] Preprocessing the uploaded medical examination form images can effectively remove noise and interference information in the images, improving the quality and clarity of the images. The preprocessing steps include operations such as image enhancement, denoising, and binarization. These operations can enhance the contrast of the images, highlighting important information in the images and providing a better basis for subsequent feature extraction and image correction. Extract the target features of the preprocessed medical examination form images, including image edge lines, table structure information, and text structure information. These features are the key basis for calculating the deflection angle of the images. Accurate feature extraction can improve the accuracy of the deflection angle calculation. For example, through edge detection algorithms, the edge lines in the images can be accurately extracted, and these edge lines can be used as important references for judging the deflection angle of the images. At the same time, extracting table structure information and text structure information can further assist in judging the deflection of the images and improving the effect of image correction. According to the extracted target features, calculate the deflection angle of the medical examination form images relative to the horizontal direction. Accurate deflection angle calculation is the key to achieving image correction, ensuring that the corrected images reach the preset angle and facilitating subsequent optical character recognition and data processing. According to the calculated deflection angle, perform geometric transformation on the medical examination form images to obtain target images at the preset angle. Geometric transformation includes operations such as rotation, translation, and scaling. Through these operations, the deflected images can be adjusted to the horizontal direction to meet the preset angle requirements. By correcting the medical examination form images through the above steps to reach the preset angle, the accuracy and efficiency of optical character recognition (OCR) can be effectively improved. The corrected images are clearer and more standardized, reducing character recognition errors caused by image deflection.

[0030] Optionally, the optical character recognition of the target images to extract target information and constructing structured data based on the target information includes: Using optical character recognition technology to perform text recognition on the target images to extract text information, where the text information includes examination items, examination results, and reference ranges; Parse and classify the text information to identify abnormal items and corresponding values; According to a preset structured data template, fill in the text information in the specified format and fields to construct structured data, where the structured data includes patient basic information and abnormal results.

[0031] Use optical character recognition (OCR) technology to perform text recognition on the corrected target image and extract the text information therein. OCR technology can convert the text in the image into editable and processable text data, providing a basis for subsequent data processing and analysis. The extracted text information includes inspection items, inspection results, reference ranges, etc. These information are key contents in the medical inspection form and are of great significance for interpreting the inspection form and generating suggestions. For example, the inspection items may include blood routine, urine routine, liver function, kidney function, etc.; the inspection results are specific values or descriptions; the reference range is the standard range for judging whether the inspection results are normal. Analyze the extracted text information to identify the key information. For example, separate the information such as inspection items, inspection results, and reference ranges for subsequent processing and analysis. The analysis process may involve operations such as text segmentation, cleaning, and formatting to ensure the accuracy of the extracted information. Classify the analyzed text information to identify abnormal items and their corresponding values. For example, by comparing the inspection results with the reference range, determine which inspection item results exceed the normal range, thereby identifying abnormal items. At the same time, record the specific values of the abnormal items to provide a basis for subsequent suggestion generation. According to the preset structured data template, fill in the analyzed and classified text information according to the specified format and fields to construct structured data. The structured data template defines the organization form and field content of the data to ensure the standardization and consistency of the data. The constructed structured data includes patient basic information and abnormal results, etc. Patient basic information may include name, age, gender, inspection date, etc.; abnormal results include the name, value, and reference range of the abnormal item. These information provide a basis for subsequent data filtering, knowledge retrieval, and suggestion generation.

[0032] Using optical character recognition (OCR) technology to perform text recognition on the corrected target image can accurately extract text information in the image, including inspection items, inspection results, reference ranges, etc. OCR technology can convert the text information in the image into editable and processable text data, improving the usability and processing efficiency of the data. Parsing and classifying the extracted text information can identify abnormal items and corresponding values. By preset rules and algorithms, analyzing and processing the text information can accurately determine which inspection item results exceed the normal range, thereby identifying abnormal items. According to the preset structured data template, filling the extracted text information into the specified format and fields to construct structured data. The structured data template defines the format and fields of the data, including patient basic information and abnormal results, etc., and can organize the extracted text information into a standardized and unified data structure. The constructed structured data can be conveniently integrated and stored, providing convenience for subsequent data processing and analysis. The structured data can be stored in a database for operations such as querying, statistics, and analysis. During the process of constructing structured data, the extracted text information can be verified and validated to ensure the accuracy and reliability of the data. By preset rules and algorithms, checking the extracted text information can discover and correct errors and inconsistencies in the data. The constructed structured data can ensure the consistency and integrity of the data, providing a reliable data foundation for subsequent data processing and analysis. The structured data template defines the format and fields of the data, ensuring the integrity and consistency of the data. By constructing structured data, it can provide doctors with fast and accurate diagnostic support.

[0033] Optionally, the filtering of the structured data through regular configuration and enumeration mapping to obtain intermediate data includes: Using a regular expression to perform format verification on the numerical information in the structured data, and through enumeration mapping, matching the inspection items in the structured data with preset medical terms. The regular expression includes verification rules for numerical ranges, decimal places, and units; Reconfirming the abnormal items and classifying and marking the abnormal items according to preset rules; Generating intermediate data according to the results of the regular configuration and enumeration mapping. The intermediate data includes the standardized inspection item names, corresponding values, and markings and classifications of abnormal items.

[0034] Setting a numerical range rule through a regular expression to ensure that the numerical information in the structured data is within a reasonable range. For example, for the white blood cell count (WBC) in a blood routine examination, the rule is set as [4.0 - 10.0]×10 9 / L to ensure that abnormal items can be detected. Set the decimal - digit rule through regular expressions to ensure that the numerical information meets the requirements. For example, for blood - glucose values, set the rule to ensure that one decimal place is retained. Set the unit rule through regular expressions to ensure that the unit of the numerical information is correct. For example, for blood - pressure values, set the rule to ensure that the unit is mmHg. Match the inspection items in the structured data with the preset medical terms to standardize the names of the inspection items. For example, map "ALT" to "alanine aminotransferase" and "CREA" to "creatinine". Standardize the descriptions of the inspection items to ensure the consistency and accuracy of the description information. For example, unify the description of "blood urea nitrogen" as "blood urea nitrogen level". Re - confirm the abnormal items in the structured data according to the preset rules. For example, by comparing the inspection results and the reference range, confirm whether it is an abnormal item. Classify and label the confirmed abnormal items for subsequent processing. For example, classify the abnormal items as "high" or "low" and label the corresponding values. Generate intermediate data according to the results of regular configuration and enumeration mapping. The intermediate data includes the standardized names of the inspection items, the corresponding values, and the labels and classifications of the abnormal items. Integrate the standardized data into the intermediate data to provide a basis for subsequent query - vector generation and medical - knowledge retrieval.

[0035] Using regular expressions to perform format verification on the numerical information in the structured data can ensure the accuracy and consistency of the data. By performing enumeration mapping to match the inspection items in the structured data with the preset medical terms, the names and descriptions of the inspection items can be standardized. This helps to improve the consistency and comparability of the data, facilitating subsequent data processing and analysis. Re - confirming the abnormal items and classifying and labeling them according to the preset rules can improve the reliability and usability of the data. Through the preset rules, it is possible to accurately determine the inspection items whose results exceed the normal range, thereby identifying the abnormal items and classifying and labeling them. Generating intermediate data according to the results of regular configuration and enumeration mapping can provide a reliable data basis for subsequent data processing and analysis. The intermediate data includes the standardized names of the inspection items, the corresponding values, and the labels and classifications of the abnormal items, which provide a basis for subsequent query - vector generation and medical - knowledge retrieval.

[0036] Optionally, generating a query vector according to the intermediate data and performing similarity retrieval in a preset vector database according to the query vector to obtain the most relevant medical knowledge includes: Using the RAG technology, taking the inspection items, abnormal - item values, and abnormal - item classifications in the intermediate data as knowledge points, and generating corresponding query vectors through a preset language model; The K-nearest neighbor algorithm is used to calculate the similarity between the query vector and the knowledge vectors in the preset vector database, and the preset number of candidate knowledge entries that are the most similar are determined; A preset re-ranking model is used to re-rank the preset number of candidate knowledge entries according to the relevance between the query vector and the preset number of candidate knowledge entries, and the knowledge entry ranked first after re-ranking is used as the most relevant medical knowledge.

[0037] The RAG technology is a technology in the field of artificial intelligence, with the full name of Retrieval-Augmented Generation. It combines two mechanisms, retrieval and generation. By retrieving external knowledge bases, it enhances the generation ability of language models. Specifically, the RAG technology first retrieves document fragments related to the input question from a large knowledge base, and then inputs these fragments as context information into the language generation model, so as to generate more accurate and informative answers. This method can effectively utilize external knowledge and improve the model's understanding and answering ability for complex questions, especially suitable for tasks that require a wide range of knowledge backgrounds, such as question-answering systems, text generation, etc. The inspection items, abnormal item values, and abnormal item classifications in the intermediate data are used as knowledge points. These knowledge points are the key information for generating query vectors and can accurately reflect the core content of the medical inspection form. For example, the inspection item is "white blood cell count", and the abnormal item value is "12.5×10 9 / L”, the abnormal item is classified as “high”. Generate corresponding query vectors through a pre - set language model (such as Qwen2 - 7B). The pre - set language model can convert knowledge points into high - dimensional vector representations, capturing the semantic information and key features of the knowledge points. For example, the query vector can be represented as [0.1, 0.2, 0.3,...], where the value of each dimension reflects the feature strength of the knowledge point in that dimension. A large number of knowledge vectors of medical knowledge entries are stored in the pre - set vector database. These knowledge vectors are generated through the same embedding technology (such as bge - m3), which can capture the semantic information and key features of medical knowledge. Use the K - Nearest Neighbor algorithm to calculate the similarity between the query vector and the knowledge vectors. The K - Nearest Neighbor algorithm determines the most similar pre - set number of candidate knowledge entries by calculating the distance (such as Euclidean distance or cosine similarity) between the query vector and the knowledge vectors. For example, set the value of K to 5, calculate the distance between the query vector and the knowledge vectors, and select the 5 knowledge entries with the smallest distance as candidate knowledge entries. Use a pre - set re - ranking model (such as bge - reranker - v2 - m3) to re - rank the candidate knowledge entries. The re - ranking model performs a more refined ranking of the candidate knowledge entries according to the relevance between the query vector and the candidate knowledge entries, ensuring that the most relevant medical knowledge is ranked first. The re - ranking model re - ranks the candidate knowledge entries by evaluating the relevance between the query vector and the candidate knowledge entries. The relevance evaluation can be based on indicators such as the semantic similarity and keyword matching degree between the query vector and the candidate knowledge entries. For example, if the knowledge vector of a certain candidate knowledge entry is highly similar to the query vector semantically, this candidate knowledge entry will be ranked in a more forward position. After re - ranking, the knowledge entry ranked first is considered to be the medical knowledge most relevant to the query vector. This knowledge entry will be used for subsequent recommendation generation to provide accurate and targeted medical advice for users.

[0038] Through the RAG technology and the K - Nearest Neighbor algorithm, it is possible to accurately retrieve the medical knowledge most relevant to the intermediate data from the pre - set vector database, improving the accuracy and relevance of the retrieval. By using the re - ranking model to re - rank the candidate knowledge entries, ensuring that the most relevant medical knowledge is ranked first, it provides strong support for subsequent recommendation generation and enhances the effectiveness and pertinence of the recommendations. Through the pre - set language model and the vector database, it is possible to quickly generate query vectors and calculate similarities, improving the processing efficiency and meeting the real - time requirements.

[0039] Optionally, the method further includes constructing the pre - set vector database, specifically including: Construct the entities and relationships in the medical knowledge entries into a graph structure, and use a graph neural network to encode the graph structure to generate the embedding vector of each node; Use a mapping network to map the embedding vectors to the vector space of the language model to obtain knowledge vectors, and store the knowledge vectors in an Elasticsearch database.

[0040] Identify entities from medical knowledge entries, such as diseases, symptoms, examination indicators, etc. These entities are the nodes for constructing the graph structure. Identify the relationships between entities, such as "causes", "manifested as", "used for diagnosis", etc. These relationships are the edges for constructing the graph structure. Construct the identified entities and relationships into a graph structure. For example, the node "white blood cell count" is connected to the node "infection" by the edge "may be related to". Select a suitable graph neural network model, such as GraphSAGE. Use the graph neural network to encode the graph structure to generate the embedding vector for each node. These vectors can capture the semantic information and structural information of the nodes. For example, the embedding vector of the node "white blood cell count" may be [0.1, 0.2, 0.3,...]. Select a suitable mapping network, such as a multi-layer perceptron (MLP). Map the embedding vectors generated by the graph neural network to the vector space of the language model through the mapping network. This step ensures that the embedding vectors are compatible with the input of the language model. For example, the embedding vector [0.1, 0.2, 0.3,...] is mapped to [0.4, 0.5, 0.6,...]. The mapped vector is the knowledge vector. These vectors can capture the semantic information of medical knowledge and are compatible with the vector space of the language model. Select Elasticsearch as the vector database. Store the generated knowledge vectors in the Elasticsearch database. Each knowledge entry corresponds to a knowledge vector, and these vectors can be used for subsequent similarity retrieval. Configure the Elasticsearch database to enable it to efficiently store and retrieve vector data. For example, set appropriate indexing and sharding strategies to improve retrieval performance.

[0041] By encoding the entities and relationships in medical knowledge entries through a graph neural network, the generated embedding vectors can capture the semantic information and structural information of the nodes, improving the accuracy of knowledge representation. By mapping the embedding vectors to the vector space of the language model through a mapping network, it ensures the compatibility of the knowledge vectors with the input of the language model, facilitating subsequent similarity calculation and knowledge retrieval. Storing the knowledge vectors in the Elasticsearch database, leveraging its efficient vector search ability, can quickly retrieve the knowledge entries most similar to the query vector, improving the retrieval efficiency. The Elasticsearch database can handle large-scale vector data, support efficient storage and retrieval, and meet the large-scale data processing requirements of the medical knowledge base.

[0042] Optionally, combining the retrieved medical knowledge with the intermediate data to obtain the final data and generating suggestions based on the final data through a preset large language model includes: Fusing the retrieved medical knowledge with the intermediate data to form final data including inspection items, abnormal item values, abnormal item classifications, and the medical knowledge; Using the preset large language model to process the final data to generate personalized suggestions for the user, where the personalized suggestions include explanations of abnormal items and presenting the generated suggestions to the user in the form of natural language.

[0043] Fusing the retrieved medical knowledge with the intermediate data to form the final data. The final data includes inspection items, abnormal item values, abnormal item classifications, and relevant medical knowledge. This fusion method can make full use of the intermediate data and medical knowledge, providing more comprehensive and accurate information for subsequent suggestion generation. Specific example: Assume the intermediate data contains the following information: Inspection item: White blood cell count (WBC); Abnormal item value: 12.5×10 9 / L; Abnormal item classification: High; The retrieved medical knowledge entry is: "An elevated white blood cell count may be related to conditions such as infection, inflammation, etc."; The fused final data is: "The white blood cell count is elevated at 12.5×10 9 / L, classified as high, and may be related to conditions such as infection, inflammation, etc." Input the fused final data into the preset large language model. The preset large language model can be a model fine-tuned in the medical field, such as ChatGPT, Wenxin Yiyan, etc. The large language model generates personalized suggestions for the user based on the input final data. The suggestion content includes explanations of abnormal items, possible disease diagnoses, suggested further inspection items, and treatment plans, etc. Present the generated suggestions to the user in the form of natural language for the user to understand and refer to. For example, the generated suggestion can be: "Your white blood cell count is elevated at 12.5×10 9 / L, which may be related to infection or inflammation. It is recommended that you further consult a doctor for a detailed examination and diagnosis to determine the specific cause and take corresponding treatment measures."

[0044] By fusing the retrieved medical knowledge with the intermediate data, the generated final data contains more comprehensive information, providing a basis for the large language model to generate accurate and targeted suggestions. Presenting the suggestions in the form of natural language improves the readability and understandability of the suggestions, enabling the user to better understand and refer to the suggestions. The generated suggestions are personalized suggestions for the user, which can provide customized medical suggestions according to the user's specific situation, improving the quality of medical services and user satisfaction.

[0045] This embodiment also discloses a system for interpreting medical examination reports.Figure 3 is a schematic diagram of the modules of the medical examination form interpretation system disclosed in the embodiments of the present application. As Figure 3 shown, the system includes an identification module 301, a filtering module 302, a retrieval module 303, and a suggestion module 304, where: The identification module 301 is configured to correct the uploaded medical examination form picture through a preset discrimination model to obtain a target picture at a preset angle, perform optical character recognition on the target picture to extract target information, and construct structured data based on the target information; The filtering module 302 is configured to filter the structured data through regular configuration and enumeration mapping to obtain intermediate data; The retrieval module 303 is configured to generate a query vector according to the intermediate data, and perform similarity retrieval in a preset vector database according to the query vector to obtain the most relevant medical knowledge; The suggestion module 304 is configured to combine the retrieved medical knowledge with the intermediate data to obtain final data, and generate suggestions based on the final data through a preset large language model.

[0046] Optionally, the identification module 301 is configured to: Preprocess the medical examination form picture, and extract the target features of the preprocessed medical examination form picture, where the target features include image edge lines, table structure information, and text structure information; Calculate the deflection angle of the medical examination form picture relative to the horizontal direction according to the target features, and perform geometric transformation on the medical examination form picture according to the deflection angle to obtain the target picture.

[0047] Optionally, the identification module 301 is configured to: Use optical character recognition technology to perform text recognition on the target picture to extract text information, where the text information includes examination items, examination results, and reference ranges; Parse and classify the text information to identify abnormal items and corresponding values; According to a preset structured data template, fill the text information in a specified format and fields to construct structured data, where the structured data includes patient basic information and abnormal results.

[0048] Optionally, the filtering module 302 is configured to: Use regular expressions to perform format verification on the numerical information in the structured data, and match the examination items in the structured data with preset medical terms through enumeration mapping. The regular expressions include verification rules for numerical ranges, decimal places, and units; Re-confirm the abnormal items, and classify and mark the abnormal items according to the preset rules; Generate intermediate data according to the results of the regular configuration and enumeration mapping. The intermediate data includes the standardized name of the inspection item, the corresponding value, and the mark and classification of the abnormal item.

[0049] Optionally, the retrieval module 303 is configured to: Use the RAG technology to take the inspection items, abnormal item values, and abnormal item classifications in the intermediate data as knowledge points, and generate corresponding query vectors through a preset language model; Use the K-nearest neighbor algorithm to calculate the similarity between the query vector and the knowledge vectors in the preset vector database, and determine the preset number of the most similar candidate knowledge entries; Adopt a preset re-ranking model to re-rank the preset number of candidate knowledge entries according to the relevance between the query vector and the preset number of candidate knowledge entries, and take the knowledge entry ranked first after re-ranking as the most relevant medical knowledge.

[0050] Optionally, the system further includes a construction module, and the construction module is configured to: Construct the entities and relationships in the medical knowledge entries into a graph structure, and use a graph neural network to encode the graph structure to generate embedding vectors for each node; Use a mapping network to map the embedding vectors to the vector space of the language model to obtain knowledge vectors, and store the knowledge vectors in the Elasticsearch database.

[0051] Optionally, the recommendation module 304 is configured to: Fuse the retrieved medical knowledge with the intermediate data to form final data including inspection items, abnormal item values, abnormal item classifications, and the medical knowledge; Use a preset large language model to process the final data to generate personalized recommendations for the user. The personalized recommendations include explanations of the abnormal items, and present the generated recommendations to the user in the form of natural language.

[0052] It should be noted that when the device provided in the above embodiment realizes its functions, only the above-mentioned division of each functional module is used for illustration. In actual applications, the above functions can be allocated to different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0053] This embodiment also discloses an electronic device. Referring to Figure 4 , the electronic device may include: at least one processor 401, at least one communication bus 402, a user interface 403, a network interface 404, and at least one memory 405.

[0054] Among them, the communication bus 402 is used to realize the connection and communication between these components.

[0055] Among them, the user interface 403 may include a display screen (Display) and a camera (Camera). Optionally, the user interface 403 may further include a standard wired interface and a wireless interface.

[0056] Among them, the network interface 404 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface).

[0057] Among them, the processor 401 may include one or more processing cores. The processor 401 connects various parts within the entire server through various interfaces and lines, and by running or executing instructions, programs, code sets, or instruction sets stored in the memory 405, as well as calling data stored in the memory 405, it executes various functions of the server and processes data. Optionally, the processor 401 may be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), or programmable logic array (PLA). The processor 401 may integrate one or several combinations of a central processing unit (CPU), a graphics processing unit (GPU), and a modem, etc. Among them, the CPU mainly processes the operating system, user interface, and application programs, etc.; the GPU is responsible for the rendering and drawing of the content to be displayed on the display screen; the modem is used to process wireless communication. It can be understood that the above modem may not be integrated into the processor 401 and may be implemented separately through a single chip.

[0058] Among them, the memory 405 may include a Random Access Memory (RAM), or may include a Read-Only Memory. Optionally, the memory 405 includes a non-transitory computer-readable storage medium. The memory 405 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 405 may include a program storage area and a data storage area. Among them, the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above method embodiments, etc.; the data storage area may store data involved in the above method embodiments. Optionally, the memory 405 may also be at least one storage device located far from the aforementioned processor 401. As Figure 4 shown, in the memory 405 as a computer storage medium, an operating system, a network communication module, a user interface module, and an application program for the method of interpreting medical examination forms may be included.

[0059] In Figure 4 the electronic device shown, the user interface 403 is mainly used to provide an input interface for the user and obtain data input by the user; while the processor 401 can be used to call the application program for the method of interpreting medical examination forms stored in the memory 405. When executed by one or more processors 401, the electronic device is caused to execute one or more of the methods in the above embodiments.

[0060] It should be noted that for the foregoing method embodiments, for the sake of simple description, they are all expressed as a series of action combinations. However, those skilled in the art should know that this application is not limited by the described action sequence, because according to this application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0061] In the above embodiments, the descriptions of the various embodiments have their own emphases. For the parts not detailed in a certain embodiment, reference may be made to the relevant descriptions of other embodiments.

[0062] In several embodiments provided by the present application, it should be understood that the disclosed device can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection between each other can be through some service interfaces. The indirect coupling or communication connection of the device or unit can be in electrical or other forms.

[0063] The units described as separate components may or may not be physically separated. The components shown as units may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0064] In addition, in each embodiment of the present application, each functional unit can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

[0065] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory 405 and includes several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods in each embodiment of the present application. And the aforementioned memory 405 includes: various media such as USB flash drives, mobile hard disks, magnetic disks, or optical discs that can store program codes.

[0066] The above are only exemplary embodiments of the present disclosure and should not be used to limit the scope of the present disclosure. That is, any equivalent changes and modifications made according to the teachings of the present disclosure still fall within the scope covered by the present disclosure. Those skilled in the art will easily think of other implementation schemes of the present disclosure after considering the disclosure of the specification. The present application aims to cover any variations, uses, or adaptive changes of the present disclosure. These variations, uses, or adaptive changes follow the general principles of the present disclosure and include common general knowledge or conventional technical means in the technical field not recorded in the present disclosure. The specification and embodiments are only regarded as exemplary, and the scope and spirit of the present disclosure are defined by the claims.

Claims

1. A method for interpreting a medical examination form, characterized in that: Applied to a checklist detection platform, the method comprises: Correcting the uploaded medical examination form image through a preset discriminant model to obtain a target image at a preset angle, performing optical character recognition on the target image to extract target information, and constructing structured data based on the target information; Filtering the structured data through regular configuration and enumeration mapping to obtain intermediate data; generating a query vector according to the intermediate data, and performing a similarity search in a preset vector database according to the query vector to obtain the most relevant medical knowledge; The retrieved medical knowledge is combined with the intermediate data to obtain final data, and recommendations are generated based on the final data by using a preset large language model.

2. The method for interpreting a medical examination form according to claim 1, characterized in that: The method of correcting the uploaded medical examination form image by using a preset discrimination model to obtain a target image at a preset angle includes: Preprocessing the medical examination form image, and extracting target features of the preprocessed medical examination form image, wherein the target features include image edge lines, table structure information, and text structure information; The deflection angle of the medical examination single image relative to the horizontal direction is calculated according to the target feature, and the medical examination single image is geometrically transformed according to the deflection angle to obtain the target image.

3. The method for interpreting a medical examination form according to claim 1, characterized in that: The performing optical character recognition on the target image to extract target information, and constructing structured data based on the target information includes: Using optical character recognition technology to perform text recognition on the target image to extract text information, the text information includes inspection items, inspection results and reference range; Parsing and classifying the text information to identify abnormal items and corresponding values; According to a preset structured data template, the text information is filled in according to a specified format and fields to construct structured data, wherein the structured data includes basic patient information and abnormal results.

4. The method for interpreting a medical examination form according to claim 1, characterized in that: The filtering of the structured data by regular configuration and enumeration mapping to obtain intermediate data includes: Performing format verification on the numerical information in the structured data using a regular expression, and matching the examination items in the structured data with preset medical terms through enumeration mapping, wherein the regular expression includes verification rules for numerical range, decimal places and units; Reconfirm abnormal items, and classify and mark them according to preset rules; Intermediate data is generated according to the results of the regular configuration and enumeration mapping, and the intermediate data includes standardized inspection item names, corresponding values, and labels and classifications of abnormal items.

5. The method for interpreting a medical examination form according to claim 1, characterized in that: Generating a query vector according to the intermediate data, and performing similarity search in a preset vector database according to the query vector to obtain the most relevant medical knowledge includes: Using RAG technology, the inspection items, abnormal item values ​​and abnormal item classifications in the intermediate data are used as knowledge points, and corresponding query vectors are generated through a preset language model; Using a K-nearest neighbor algorithm to calculate the similarity between the query vector and the knowledge vector in the preset vector database, and determining a preset number of most similar candidate knowledge items; A preset reordering model is used to reorder the preset number of candidate knowledge items according to the correlation between the query vector and the preset number of candidate knowledge items, and the knowledge item ranked first after reordering is used as the most relevant medical knowledge.

6. The method for interpreting a medical examination form according to claim 5, characterized in that: The method further includes constructing the preset vector database, specifically including: The entities and relationships in the medical knowledge items are constructed into a graph structure, and the graph structure is encoded using a graph neural network to generate an embedding vector for each node; A mapping network is used to map the embedding vector to a vector space of a language model to obtain a knowledge vector, and the knowledge vector is stored in an Elasticsearch database.

7. The method for interpreting a medical examination form according to claim 1, characterized in that: The step of combining the retrieved medical knowledge with the intermediate data to obtain final data, and generating suggestions based on the final data by using a preset large language model includes: The retrieved medical knowledge is integrated with the intermediate data to form final data including examination items, abnormal item values, abnormal item classifications and the medical knowledge; The final data is processed using a preset large language model to generate personalized suggestions for the user, the personalized suggestions including explanations of abnormal items, and the generated suggestions are presented to the user in the form of natural language.

8. A system for interpreting medical examination orders, characterized in that: It includes recognition module, filtering module, retrieval module and suggestion module, among which: A recognition module configured to correct the uploaded medical examination form image through a preset discrimination model to obtain a target image at a preset angle, perform optical character recognition on the target image to extract target information, and construct structured data based on the target information; A filtering module, configured to filter the structured data through regular configuration and enumeration mapping to obtain intermediate data; A retrieval module configured to generate a query vector according to the intermediate data, and perform a similarity search in a preset vector database according to the query vector to obtain the most relevant medical knowledge; A suggestion module is configured to combine the retrieved medical knowledge with the intermediate data to obtain final data, and generate suggestions based on the final data by using a preset large language model.

9. An electronic device, characterized in that: It includes a processor, a memory, a user interface and a network interface, the memory is used to store instructions, the user interface and the network interface are both used to communicate with other devices, and the processor is used to execute the instructions stored in the memory so that the electronic device executes the method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are executed, the method according to any one of claims 1 to 7 is performed.

Images