Report analysis method, device, equipment, system and program product

Abstract

The embodiment of the invention provides a report analysis method. The method comprises the following steps: acquiring a medical detection report; performing image recognition on the medical detection report to obtain report information; the report information is input into a large language model for content analysis so as to obtain key information, and the large language model is obtained based on training of a medical data set configured with training reports of different formats; and performing structured arrangement based on the report content indicated by the key information to obtain a target report. Therefore, the intelligent report analysis process is realized, the unstructured medical detection report is subjected to structured conversion, and large language models suitable for different formats are specifically configured in the conversion process, so that the analysis process can cope with complex report analysis, and the analysis efficiency is improved. And an adaptive text processing process is carried out according to the characteristic of a long text of the medical detection report, so that the condition of disordered analysis content is avoided, and the accuracy of report analysis is improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a report parsing method, device, equipment, system and program product. Background Art

[0002] With the development of medical informatization, electronic health records have become the standard of modern medical care. As an important part of electronic health records, physical examination reports contain information about an individual's health status and potential diseases. How to intelligently analyze physical examination reports has become a difficult problem.

[0003] Generally, the analysis process of physical examination report forms relies on manual reading and understanding, which is inefficient and prone to errors, thus affecting the accuracy of the report form analysis. Summary of the invention

[0004] In view of this, the present application provides a report analysis method to improve the accuracy of report analysis.

[0005] According to a first aspect of an embodiment of the present application, a method for parsing a report form is provided, which can be applied to a system or program including a report form parsing function in a terminal device, and specifically includes:

[0006] Performing image recognition on the medical test report to obtain report information, wherein the report information includes contents of multiple report pages;

[0007] Inputting the report information into a large language model for content analysis to obtain key information, wherein the large language model is trained based on a medical data set, and the medical data set is configured based on training reports in different formats;

[0008] Merging the contents of the report sheets indicated by the key information into the report page to obtain merged information;

[0009] The target report sheet is obtained by structuring the combined information.

[0010] Optionally, in some possible implementations, performing image recognition on the medical test report to obtain report information includes:

[0011] Performing target keyword recognition on the target page of each report in the medical test report;

[0012] If the target keyword is identified, a corresponding report is obtained;

[0013] Perform image recognition on the report to obtain the report information.

[0014] Optionally, in some possible implementations, performing image recognition on the report to obtain the report information includes:

[0015] Performing paging processing on the report to obtain paging text;

[0016] The paged text is input into an optical character recognition model for image recognition to obtain the report information configured based on a markup language format.

[0017] Optionally, in some possible implementations, the training process of the large language model includes:

[0018] Acquiring the medical data set;

[0019] Performing format marking on the training report sheet indicated in the medical data set to obtain a marked report sheet;

[0020] The large language model is fine-tuned multiple times based on the marked reports to train the large language model.

[0021] Optionally, in some possible implementations, formatting the training report sheet indicated in the medical data set to obtain a marked report sheet includes:

[0022] Classifying the training reports indicated in the medical data set according to the source objects to obtain classified reports;

[0023] Determine the table style information of the source object corresponding to the classification report;

[0024] The training report form is formatted based on the table style information to obtain the marked report form.

[0025] Optionally, in some possible implementations, inputting the report information into a large language model for content parsing to obtain key information includes:

[0026] Inputting the report information into a large language model for content analysis to obtain object information, inspection information, test information and conclusion information;

[0027] The object information, the inspection information, the test information and the conclusion information are combined to obtain the key information.

[0028] Optionally, in some possible implementations, the report form content based on the key information indication is merged in the report page to obtain merged information, including:

[0029] Determine the paging result based on the report content indicated by the key information;

[0030] Get the preset module configuration information;

[0031] The paging results are merged in the report page according to the preset module configuration information to obtain merged information.

[0032] Optionally, in some possible implementations, merging the paging results in the report page according to the preset module configuration information to obtain merged information includes:

[0033] Filling the report page with the paging result according to the preset module configuration information to obtain filling information;

[0034] Matching the cross-page components in the filling information to obtain matching information;

[0035] Content merging is performed based on the matching information to obtain the merged information.

[0036] Optionally, in some possible implementations, the combined information is structured to obtain a target report sheet, including:

[0037] Performing data cleaning on the merged information to obtain cleaned information;

[0038] Acquire display structure information corresponding to an interactive system, wherein the interactive system includes an interactive interface, and the interactive interface is used to display the target report sheet;

[0039] The cleaning information is structured and organized according to the display structure information to obtain the target report sheet.

[0040] Optionally, in some possible implementations, the method further includes:

[0041] Get feedback on target report cards;

[0042] Determining a source object indicated by the feedback information;

[0043] Acquire the report data corresponding to the source object, and format the report data;

[0044] Fine-tune the large language model based on the labeled report data.

[0045] According to a second aspect of an embodiment of the present application, a report parsing device is provided, comprising:

[0046] A recognition unit, used for performing image recognition on the medical test report to obtain report information, wherein the report information includes contents of multiple report pages;

[0047] A parsing unit, used for inputting the report information into a large language model for content parsing to obtain key information, wherein the large language model is trained based on a medical data set, and the medical data set is configured based on training reports in different formats;

[0048] The parsing unit is further used to merge the contents of the report sheet indicated by the key information into the report page to obtain merged information;

[0049] The parsing unit is further used to obtain a target report by structuring the merged information.

[0050] Optionally, in some possible implementations, the recognition unit is used to perform target keyword recognition on the target page of each report in the medical test report when performing image recognition on the medical test report to obtain report information; if the target keyword is recognized, the corresponding report is obtained; and image recognition is performed on the report to obtain the report information.

[0051] Optionally, in some possible implementations, the recognition unit is used to perform paging processing on the report to obtain paginated text when performing image recognition on the report to obtain the report information; and input the paginated text into the optical character recognition model for image recognition to obtain the report information configured based on the markup language format.

[0052] Optionally, in some possible implementations, the parsing unit is used to obtain the medical data set; format the training report indicated in the medical data set to obtain a marked report; and fine-tune the large language model multiple times based on the marked report to train the large language model.

[0053] Optionally, in some possible implementations, the parsing unit is used to classify the training report form indicated in the medical data set according to the source object to obtain a classified report form when formatting the training report form indicated in the medical data set to obtain a labeled report form; determine the table style information of the source object corresponding to the classified report form; and format-mark the training report form based on the table style information to obtain the labeled report form.

[0054] Optionally, in some possible implementations, the parsing unit is used to input the report information into a large language model for content parsing to obtain object information, inspection information, verification information and conclusion information when the report information is input into a large language model for content parsing to obtain key information; and combine the object information, the inspection information, the verification information and the conclusion information to obtain the key information.

[0055] Optionally, in some possible implementations, the parsing unit is used to determine the paging result based on the report content indicated by the key information when merging the content of the report based on the key information indicated in the report page to obtain merged information; obtain preset module configuration information; and merge the paging result in the report page according to the preset module configuration information to obtain merged information.

[0056] Optionally, in some possible implementations, the parsing unit is used to fill the paginated results in the report page with content according to the preset module configuration information to obtain fill information when the paginated results are merged in the report page according to the preset module configuration information to obtain merge information; match the cross-page components in the fill information to obtain matching information; and merge the content based on the matching information to obtain the merged information.

[0057] Optionally, in some possible implementations, the parsing unit is used to perform data cleansing on the merged information to obtain cleaned information when the merged information is structured to obtain the target report; obtain display structure information corresponding to the interactive system, the interactive system includes an interactive interface, and the interactive interface is used to display the target report; and structure the cleaned information according to the display structure information to obtain the target report.

[0058] Optionally, in some possible implementations, the parsing unit is used to obtain feedback information for a target report form; determine a source object indicated by the feedback information; obtain report form data corresponding to the source object, and format the report form data; and fine-tune the large language model based on the marked report form data.

[0059] According to a third aspect of the embodiments of the present application, there is provided a report parsing device, including an input and output component and a processor;

[0060] The input and output components are used to obtain medical test reports;

[0061] The processor is used to parse the medical test report obtained by the input-output component by executing the report parsing method as described in the first aspect or any implementation of the first aspect.

[0062] According to a fourth aspect of the embodiments of the present application, a report parsing system is provided, including an interactive client and a server;

[0063] The interactive client is used to obtain the medical test report and send the medical test report to the server, and to display the analysis result of the report output by the server;

[0064] The server is used to parse the medical test report obtained by the interactive client by executing the report parsing method described in the first aspect or any implementation method of the first aspect.

[0065] According to a fifth aspect of an embodiment of the present application, a computer program product is provided, comprising: a computer program, which, when executed by a processor, implements the report parsing method described in the first aspect or any implementation manner of the first aspect.

[0066] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0067] By obtaining a medical test report, performing image recognition on the medical test report to obtain the report information, and inputting the report information into a large language model for content analysis to obtain key information, the large language model is trained based on a medical data set, and the medical data set is configured based on training reports in different formats; and then the report content indicated by the key information is structured to obtain the target report. Thus, an intelligent report parsing process is realized. Since the unstructured medical test report is structured, the conversion process specifically configures a large language model suitable for different formats, so that the parsing process can cope with complex report parsing, and an adaptive text processing process is performed based on the characteristics of the long text of the medical test report, avoiding the occurrence of confusion in the parsing content and improving the accuracy of the report parsing. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0069] Figure 1 A diagram of the network architecture that runs the report ticket parsing system;

[0070] Figure 2 A flow chart of report parsing provided in an embodiment of the present application;

[0071] Figure 3 A flowchart of a report parsing method provided in an embodiment of the present application;

[0072] Figure 4 A scenario flow chart of a report parsing method provided in an embodiment of the present application;

[0073] Figure 5 A scenario diagram of a report parsing method provided in an embodiment of the present application;

[0074] Figure 6 A structural schematic diagram of a report parsing device provided in an embodiment of the present application;

[0075] Figure 7 A schematic diagram of the structure of a terminal device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0076] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0077] It should be understood that the report parsing method provided in the present application can be applied to a system or program in a terminal device that includes a report parsing function, such as a medical application. Specifically, the report parsing system can be run on a computer such as a computer. Figure 1 In the network architecture shown in Figure 1 As shown in FIG. 1 , it is a network architecture diagram of the report analysis system. As can be seen from the figure, the report analysis system can provide a report analysis process with multiple information sources, that is, obtain the medical test report through the acquisition operation on the terminal side, and then send it to the server for report analysis; it can be understood that Figure 1 A variety of terminal devices are shown in FIG. 1 . The terminal devices may be computer devices. In actual scenarios, more or fewer types of terminal devices may participate in the report analysis process. The specific number and type depend on the actual scenario and are not limited here. In addition, Figure 1 One server is shown in the figure, but in actual scenarios, multiple servers may be involved, especially in multi-disciplinary scenarios, and the specific number of servers depends on the actual scenario.

[0078] In this embodiment, the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The terminal can be a smart phone, tablet computer, laptop computer, desktop computer, smart speaker, smart watch, etc., but is not limited to this. The terminal and the server can be directly or indirectly connected through wired or wireless communication, and the terminal and the server can be connected to form a blockchain network, which is not limited in this application.

[0079] It can be understood that the above-mentioned report analysis system can run on a personal mobile terminal, for example: as an application such as smart medical care, it can also run on a server, and it can also run on a third-party device to provide report analysis to obtain the report analysis processing results of the information source; the specific report analysis system can be run in the above-mentioned device in the form of a program, or it can be run as a system component in the above-mentioned device, and it can also be used as a cloud service program. The specific operation mode depends on the actual scenario and is not limited here.

[0080] With the development of medical informatization, electronic health records have become the standard of modern medical care. As an important part of electronic health records, physical examination reports contain information about an individual's health status and potential diseases. How to intelligently analyze physical examination reports has become a difficult problem.

[0081] Generally, the analysis process of physical examination report forms relies on manual reading and understanding, which is inefficient and prone to errors, thus affecting the accuracy of the report form analysis.

[0082] In order to solve the above problems, this application proposes a report analysis method, which is applied to Figure 2 In the process framework of report sheet analysis shown in Figure 2 As shown, it is a process architecture diagram of report parsing provided in an embodiment of the present application, which shows that a medical test report is obtained through a terminal, thereby triggering the server to automatically extract key information in the medical report through a large language model, and perform a structured representation after merging the content, thereby realizing an efficient and accurate medical report parsing process.

[0083] It can be understood that the report parsing method provided in the present application can be written as a program to serve as a processing logic in a hardware system, or as a report parsing device to implement the above processing logic in an integrated or external manner. As an implementation method, the report parsing device obtains a medical test report; then performs image recognition on the medical test report to obtain report information; and inputs the report information into a large language model for content parsing to obtain key information, the large language model is trained based on a medical data set, and the medical data set is configured based on training reports in different formats; and then the report content indicated by the key information is structured to obtain a target report. Thus, an intelligent report parsing process is realized. Since the unstructured medical test report is structured, the conversion process specifically configures a large language model suitable for different formats, so that the parsing process can cope with complex report parsing, and an adaptive text processing process is performed for the characteristics of the long text of the medical test report, avoiding the occurrence of confusion in the parsed content and improving the accuracy of the report parsing.

[0084] Combined with the above process architecture, the following will introduce the report analysis method in this application. Please refer to Figure 3 , Figure 3 A flowchart of a report parsing method provided in an embodiment of the present application, the embodiment of the present application at least comprises the following steps:

[0085] 301. Perform image recognition on the medical test report to obtain report information, which includes content of multiple report pages.

[0086] In this embodiment, the medical examination report is a report obtained through medical examination means, which may specifically be a physical examination report, a diagnosis and treatment report, a health assessment report, etc. This embodiment is described using a physical examination report as an example and is not limited thereto.

[0087] It is understandable that since the medical test report is presented in the form of long text, it is unstructured and the medical knowledge contained therein cannot be directly and accurately obtained. Therefore, this embodiment proposes an efficient and accurate physical examination report parsing method to overcome the shortcomings of the existing technology and realize the automatic extraction and structured representation of key information in the physical examination report.

[0088] Specifically, the parsing and structuring of physical examination report forms refers to converting the unstructured physical examination report text into a computer-readable structured data format for data analysis, storage and sharing. Considering that batch PDF data (medical test reports) as the most original input will be mixed with a lot of irrelevant information, non-physical examination report form data can be removed from the mixed data, that is, a rejection operation is performed. First, the target keyword recognition is performed on the target page of each report in the medical test report; if the target keyword is recognized, the corresponding report is obtained; then the report is image recognized to obtain the report form information. For example, the text of the first page (target page) of the PDF is recognized. If the text contains the name of a specific physical examination institution or target keywords such as "physical examination report", it can be considered to be a physical examination report form that meets the requirements.

[0089] Furthermore, since the report information is a long text content, it contains a lot of text content and will include the content of multiple report pages. That is, a physical examination report PDF file generally has several pages to dozens of pages. Since the length of the OCR large model input at one time is limited, the PDF file is split by page.

[0090] In one possible scenario, an optical character recognition (OCR) process may be performed after paging, that is, the report is paginated to obtain paged text; the paged text is then input into an optical character recognition model for image recognition to obtain report information configured based on a markup language format.

[0091] It can be seen that the medical examination report file can be parsed after the desensitization treatment. That is, the parsing adopts the OCR large model technology. Among them, the OCR large model refers to the large deep learning model used in the OCR field. These models can efficiently extract and recognize text information from images by combining advanced deep learning technologies such as convolutional neural network (CNN), recurrent neural network (RNN) and Transformer.

[0092] It is understandable that the markup language format configuration takes into account that the input of the OCR large model is in the form of images, so the paginated PDF files also need to be converted into common image formats, such as PNG, JPG, etc. Therefore, after being parsed by the OCR large model, the information of each page is completely read out in markdown format (markup language format), which well preserves the positional relationship between text and tables. Among them, markdown is a lightweight markup language that can format ordinary text through simple syntax, thereby improving the efficiency and readability of document writing.

[0093] Therefore, combined with the above processing process for medical examination reports, it can be applied to Figure 4In the process scenario shown, Figure 4 A scenario flow chart of a report parsing method provided in an embodiment of the present application; the figure shows that the report parsing process includes a rejection stage, an OCR stage, a large model parsing stage and a merging stage, wherein the rejection stage and the OCR stage have been explained in the above embodiments and will not be repeated here; the large model parsing stage and the merging stage are explained in conjunction with the embodiments below.

[0094] 302. Input the report information into a large language model for content analysis to obtain key information. The large language model is trained based on a medical data set, and the medical data set is configured based on training reports in different formats.

[0095] In this embodiment, the large language model is a parsing model that can cover various formats of medical examination reports. During the training phase, a pre-trained generative large model can be selected as a base for fine-tuning so that it can understand and parse specific terms and concepts in the medical field. The fine-tuning process includes training on medical professional datasets and optimizing model parameters to adapt to specific parsing tasks.

[0096] Specifically, for the training process of the large language model, you can first obtain a medical data set; then format the training report indicated in the medical data set to obtain a marked report; and then fine-tune the large language model multiple times based on the marked report to train the large language model. The fine-tuned model is recorded as reportLLM. Since the fine-tuning data comes from hundreds of physical examination reports in different formats, reportLLM has the ability to recognize formats in the medical field. In addition, reportLLM can also support the input of OCR large models and OCR small models, and support text and markdown output.

[0097] It can be seen that this embodiment fine-tunes the model in combination with the knowledge in the medical field to obtain the generative large model reportLLM, which enhances the model's ability to understand professional terms and improves the accuracy and recall of parsing; and reportLLM supports markdown format and OCR plain text format input and output, making the parsing of physical examination reports more flexible.

[0098] It is understandable that compared with the format configuration of general reports, reports in the medical field are characterized by strong professional content and a high degree of correlation between format and content. In addition, there is a correlation between different content parts in the format. For example, the abnormal values ​​in the test items need to be strongly correlated with the content in the medical advice. That is, while the large language model is adapting to reports in different formats, it is also learning the correlation between the content indicated by the format. That is, the format learning process and the medical expertise learning process support each other.

[0099] Furthermore, in order to obtain a model that meets the accuracy and recall requirements, multiple trainings can be performed. In addition, training can be performed using medical data sets from different sources, so that the large language model can support medical examination reports from hundreds of different institutions. The model parsing effect can meet the diversity and complexity of medical examination reports. It can be seen that reportLLM covers a large number of medical institution medical examination report types and has high coverage.

[0100] Optionally, for the format marking process, since reports from the same source object have similar table styles, table style learning can also be performed at this time; that is, first, the training report forms indicated in the medical data set are classified according to the source objects to obtain classified report forms; then, the table style information of the source object corresponding to the classified report form is determined, such as the location configuration of the examination items, the display form of abnormal marks, etc.; and then, the training report forms are formatted based on the table style information to obtain marked report forms, so that the large language model can learn the table styles of different medical institutions and perform the data traceability process.

[0101] After training the large language model, the report can be analyzed by reasoning. In the reasoning stage, the OCR information of each page is input into reportLLM, and the trained model is used to parse the OCR text to automatically identify and extract key information.

[0102] Among them, the key information is configured based on the characteristics of the medical test report, that is, the report information is input into the large language model for content analysis to obtain object information, examination information, test information and conclusion information; then the object information, examination information, test information and conclusion information are combined to obtain the key information.

[0103] In a possible scenario, key information includes patient information (object information), examination (examination information), test (test information) and conclusion (conclusion information). Patient information is the basic information of the patient, including gender, age, marital status, physical examination date, etc. Examination information includes examination department, examination items, examination results, reference values, abnormal prompts, etc. Test information includes test department, test items, test results, reference values, abnormal prompts, etc. Conclusions include abnormal item summary and physician advice, etc. This process utilizes the powerful language understanding and generation capabilities of generative large models, which can effectively process complex medical texts. Thus, reportLLM realizes the conversion from unstructured text to structured data, which is convenient for subsequent data analysis and decision support.

[0104] 303. The report sheet contents indicated by the key information are merged in the report page to obtain merged information.

[0105] In this embodiment, the parsing information result of the physical examination report is finally sorted out for a complete physical examination report. After the pagination results are obtained in the previous step, they need to be merged to obtain complete information.

[0106] Specifically, for the merging process, the paging results can be first determined based on the report content indicated by the key information; then the preset module configuration information can be obtained; and then the paging results can be merged in the report page according to the preset module configuration information to obtain the merged information. The preset module configuration information is the required report content, such as patient information, examination, test and conclusion; it can also be other personalized content configurations, such as targeted extraction of blood pressure and blood lipid information for a specific age group.

[0107] Furthermore, since some information may be displayed across pages during the merging process, a set of merging logic is designed to ensure that no information is lost. That is, first, the paging results are filled in the report page according to the preset module configuration information to obtain the filled information; then the cross-page components in the filled information are matched to obtain the matching information; and then the content is merged based on the matching information to obtain the merged information. Among them, the cross-page components are based on the content involving cross-pages. Since the merging process is based on different modules, a physical examination report finally obtains four modules: patient information, examination, inspection, and conclusion. In this process, if there is a cross-page, it is necessary to merge according to the type and content of the data; the matching information in the merging process, that is, the matching process for the same content, for table data, if the table data has no header, then the corresponding header at the end of the previous page is retrieved as its header; if it is plain text data, it belongs to the summary part of the examination / inspection, and the corresponding examination / inspection category at the end of the previous page is retrieved; if it is plain text data, it belongs to the suggestion item of the conclusion, and the abnormal item corresponding to the conclusion at the end of the previous page is retrieved; in the merging process, only one record is retained for duplicate items, thereby improving the simplicity of the report information.

[0108] 304. The target report sheet is obtained by structuring the merged information.

[0109] In this embodiment, the target report is a structured physical examination report, through which the relevant diagnosis and treatment information can be quickly identified, that is, the extracted information is structured and presented to the user or the medical information system in an easy-to-understand form. Structured data can be conveniently further analyzed and applied, such as the user's next medical advice and physical condition assessment.

[0110] Optionally, in order to improve the quality of the data, the extracted data can be converted and cleaned after the merger is completed to remove irrelevant information to ensure the accuracy and consistency of the data, so as to better perform operations such as medical advice and physical condition assessment. For this process, the merged information is first cleaned to obtain cleaned information; then the display structure information corresponding to the interactive system is obtained, and the interactive system includes an interactive interface, which is used to display the target report form; and then the cleaned information is structured according to the display structure information to obtain the target report form.

[0111] In a possible scenario, for the process of making medical recommendations and physical condition assessments based on target report cards, such as Figure 5 As shown, Figure 5 A scenario diagram of a report parsing method provided for an embodiment of the present application; the figure shows that the target report can participate in the multi-party interaction among the patient, the doctor and the intelligent agent, for example, it can be displayed to the doctor to obtain the doctor's advice, displayed to the intelligent agent to obtain the intelligent agent's advice, or displayed in the process of multi-party interaction as data support, etc. The specific display form depends on the actual scenario.

[0112] Furthermore, after performing subsequent medical advice and physical condition assessment based on the target report, corresponding feedback information, such as dissatisfaction or complaint feedback, can also be received; that is, feedback information for the target report is obtained; then the source object indicated by the feedback information is determined; and the report data corresponding to the source object is obtained, and the report data is formatted; and then the large language model is fine-tuned based on the marked report data. This avoids inaccurate report recognition due to the format of the medical institution and further improves the accuracy of report parsing.

[0113] In combination with the above embodiments, it can be known that by obtaining a medical test report, performing image recognition on the medical test report to obtain report information, and inputting the report information into a large language model for content analysis to obtain key information, the large language model is trained based on a medical data set, and the medical data set is configured based on training reports in different formats; and then the report content indicated by the key information is structured to obtain a target report. Thus, an intelligent report parsing process is realized. Since the unstructured medical test report is structured, the conversion process specifically configures a large language model suitable for different formats, so that the parsing process can cope with complex report parsing, and an adaptive text processing process is performed based on the characteristics of the long text of the medical test report, avoiding the occurrence of confusion in the parsed content and improving the accuracy of the report parsing.

[0114] In order to better implement the above solution of the embodiment of the present application, the following also provides related devices for implementing the above solution. Figure 6 , Figure 6 This is a schematic diagram of the structure of a report parsing device provided in an embodiment of the present application. The parsing device 600 includes:

[0115] The recognition unit 601 is used to perform image recognition on the medical test report to obtain report information, wherein the report information includes the contents of multiple report pages;

[0116] A parsing unit 602, configured to input the report information into a large language model for content parsing to obtain key information, wherein the large language model is trained based on a medical data set, and the medical data set is configured based on training reports in different formats;

[0117] The parsing unit 602 is further configured to merge the contents of the report sheet indicated by the key information into the report page to obtain merged information;

[0118] The parsing unit 602 is further used to obtain a target report by structuring the merged information.

[0119] Optionally, in some possible implementations, the recognition unit is used to perform target keyword recognition on the target page of each report in the medical test report when performing image recognition on the medical test report to obtain report information; if the target keyword is recognized, the corresponding report is obtained; and image recognition is performed on the report to obtain the report information.

[0120] Optionally, in some possible implementations, the recognition unit is used to perform paging processing on the report to obtain paginated text when performing image recognition on the report to obtain the report information; and input the paginated text into the optical character recognition model for image recognition to obtain the report information configured based on the markup language format.

[0121] Optionally, in some possible implementations, the parsing unit 602 is used to obtain the medical data set; format the training report indicated in the medical data set to obtain a marked report; and fine-tune the large language model multiple times based on the marked report to train the large language model.

[0122] Optionally, in some possible implementations, the parsing unit 602 is used to classify the training report form indicated in the medical data set according to the source object to obtain a classified report form when formatting the training report form indicated in the medical data set to obtain a marked report form; determine the table style information of the source object corresponding to the classified report form; and format-mark the training report form based on the table style information to obtain the marked report form.

[0123] Optionally, in some possible implementations, the parsing unit 602 is used to input the report information into a large language model for content parsing to obtain object information, inspection information, verification information and conclusion information when the report information is input into a large language model for content parsing to obtain key information; and combine the object information, the inspection information, the verification information and the conclusion information to obtain the key information.

[0124] Optionally, in some possible implementations, the parsing unit 602 is used to determine the paging result based on the report form content indicated by the key information when merging the content of the report form based on the key information indication in the report page to obtain merged information; obtain preset module configuration information; and merge the paging result in the report page according to the preset module configuration information to obtain merged information.

[0125] Optionally, in some possible implementations, the parsing unit 602 is used to fill the paginated results in the report page with content according to the preset module configuration information to obtain fill information when merging the content of the paginated results in the report page according to the preset module configuration information to obtain merge information; match the cross-page components in the fill information to obtain matching information; and merge the content based on the matching information to obtain the merged information.

[0126] Optionally, in some possible implementations, the parsing unit 602 is used to perform data cleansing on the merged information to obtain cleaned information when the merged information is structured to obtain the target report form; obtain display structure information corresponding to the interactive system, the interactive system includes an interactive interface, and the interactive interface is used to display the target report form; and structure the cleaned information according to the display structure information to obtain the target report form.

[0127] Optionally, in some possible implementations, the parsing unit 602 is used to obtain feedback information for a target report form; determine a source object indicated by the feedback information; obtain report form data corresponding to the source object, and format the report form data; and fine-tune the large language model based on the marked report form data.

[0128] By obtaining a medical test report, performing image recognition on the medical test report to obtain the report information, and inputting the report information into a large language model for content analysis to obtain key information, the large language model is trained based on a medical data set, and the medical data set is configured based on training reports in different formats; and then the report content indicated by the key information is structured to obtain the target report. Thus, an intelligent report parsing process is realized. Since the unstructured medical test report is structured, the conversion process specifically configures a large language model suitable for different formats, so that the parsing process can cope with complex report parsing, and an adaptive text processing process is performed based on the characteristics of the long text of the medical test report, avoiding the occurrence of confusion in the parsing content and improving the accuracy of the report parsing.

[0129] The report parsing device provided in this embodiment belongs to the same application concept as the method provided in the above embodiments of this application, can execute the report parsing method provided in any of the above embodiments of this application, and has the corresponding functional modules and beneficial effects of executing the report parsing method. For technical details not fully described in this embodiment, please refer to the specific processing content of the report parsing method provided in the above embodiments of this application, and will not be repeated here.

[0130] The functions implemented by the above identification unit 601 and analysis unit 602 can be implemented by the same or different processors respectively, and the embodiment of the present application is not limited thereto.

[0131] It should be understood that each unit in the above device can be implemented in the form of a processor calling software. For example, the device includes a processor, the processor is connected to a memory, and instructions are stored in the memory. The processor calls the instructions stored in the memory to implement any of the above methods or realize the functions of each unit of the device, wherein the processor can be a general-purpose processor, such as a CPU or a microprocessor, etc., and the memory can be a memory in the device or a memory outside the device. Alternatively, the unit in the device can be implemented in the form of a hardware circuit, and the functions of some or all units can be realized by designing the hardware circuit. The hardware circuit can be understood as one or more processors; for example, in one implementation, the hardware circuit is an ASIC, and the functions of some or all units above are realized by designing the logical relationship of the components in the circuit; for another example, in another implementation, the hardware circuit can be implemented by PLD, taking FPGA as an example, which can include a large number of logic gate circuits, and the connection relationship between the logic gate circuits is configured by a configuration file, so as to realize the functions of some or all units above. All units of the above device can be implemented in the form of a processor calling software, or in the form of a hardware circuit, or in the form of a processor calling software, and the remaining part can be implemented in the form of a hardware circuit.

[0132] In an embodiment of the present application, a processor is a circuit with the ability to process signals. In one implementation, the processor may be a circuit with the ability to read and run instructions, such as a CPU, a microprocessor, a GPU, or a DSP; in another implementation, the processor may implement certain functions through the logical relationship of a hardware circuit, and the logical relationship of the hardware circuit is fixed or reconfigurable, such as a hardware circuit implemented by an ASIC or PLD, such as an FPGA. In a reconfigurable hardware circuit, the process of the processor loading a configuration document to implement the hardware circuit configuration can be understood as the process of the processor loading instructions to implement the functions of some or all of the above units. In addition, it can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as an NPU, TPU, DPU, etc.

[0133] It can be seen that each unit in the above device can be one or more processors (or processing circuits) configured to implement the above method, such as: CPU, GPU, NPU, TPU, DPU, microprocessor, DSP, ASIC, FPGA, or a combination of at least two of these processor forms.

[0134] In addition, all or part of the units in the above device can be integrated together, or can be implemented independently. In one implementation, these units are integrated together and implemented in the form of a SOC. The SOC may include at least one processor for implementing any of the above methods or implementing the functions of each unit of the device. The type of the at least one processor may be different, for example, including a CPU and an FPGA, a CPU and an artificial intelligence processor, a CPU and a GPU, etc.

[0135] The embodiment of the present application also provides a report parsing device; the report parsing device includes a processor and an input-output component. The input-output component is used to obtain a medical test report;

[0136] The processor is used to parse the medical test report obtained by the input-output component by executing any one of the report parsing methods in the above embodiments.

[0137] The above-mentioned interface circuit can be any interface circuit that can realize the data communication function, for example, it can be a USB interface circuit, a Type-C interface circuit, a serial port circuit, a PCIE circuit, etc.

[0138] Optionally, the embodiment of the present application further provides a system, wherein the interactive client is used to obtain a medical test report and send the medical test report to the server, and to display the parsing result of the report output by the server;

[0139] The server is used to parse the medical test report obtained by the interactive client by executing the report parsing method described in any one of the above embodiments.

[0140] Another embodiment of the present application also provides a report parsing device, see Figure 7 As shown, the device includes:

[0141] Memory 700 and processor 710;

[0142] The memory 700 is connected to the processor 710 and is used to store programs;

[0143] The processor 710 is used to implement the report parsing method disclosed in any of the above embodiments by running the program stored in the memory 700.

[0144] Specifically, the report parsing device may further include: a bus, a communication interface 720 , an input device 730 and an output device 740 .

[0145] The processor 710, the memory 700, the communication interface 720, the input device 730 and the output device 740 are connected to each other via a bus.

[0146] A bus may include a pathway that transfers information between components of a computer system.

[0147] Processor 710 may be a general-purpose processor, such as a general-purpose central processing unit (CPU), a microprocessor, etc., or an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the program of the scheme of the present invention. It may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0148] The processor 710 may include a main processor, and may also include a baseband chip, a modem, and the like.

[0149] The memory 700 stores a program for executing the technical solution of the present invention, and may also store an operating system and other key services. Specifically, the program may include a program code, and the program code includes a computer operation instruction. More specifically, the memory 700 may include a read-only memory (ROM), other types of static storage devices that can store static information and instructions, a random access memory (RAM), other types of dynamic storage devices that can store information and instructions, a disk storage, a flash, and the like.

[0150] The input device 730 may include a device for receiving data and information input by a user, such as a keyboard, a mouse, a camera, a scanner, a light pen, a voice input device, a touch screen, a pedometer, or a gravity sensor.

[0151] Output device 740 may include devices that allow information to be output to a user, such as a display screen, printer, speaker, etc.

[0152] The communication interface 720 may include any transceiver or the like to communicate with other devices or communication networks, such as Ethernet, Radio Access Network (RAN), Wireless Local Area Network (WLAN), etc.

[0153] The processor 710 executes the program stored in the memory 700 and calls other devices, which can be used to implement each step of any report parsing method provided in the above embodiments of the present application.

[0154] In addition to the above-mentioned methods and devices, an embodiment of the present application may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the report parsing method according to various embodiments of the present application described in any of the above embodiments of this specification.

[0155] The computer program product may be written in any combination of one or more programming languages ​​to write program codes for performing the operations of the embodiments of the present application, including object-oriented programming languages, such as Java, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as an independent software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0156] In addition, the embodiment of the present application may also be a storage medium on which a computer program is stored. The computer program is executed by a processor to execute the steps of the report parsing method according to various embodiments of the present application described in any of the above embodiments of this specification, and specifically the following steps may be implemented:

[0157] Perform image recognition on the medical test report to obtain report information, which includes the contents of multiple report pages;

[0158] Input the report information into a large language model for content analysis to obtain key information. The large language model is trained based on a medical data set, and the medical data set is configured based on training reports in different formats.

[0159] The report sheet contents based on the key information indication are merged in the report page to obtain the merged information;

[0160] The merged information is structured to obtain the target report.

[0161] For the aforementioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the order of the actions described, because according to the present application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0162] It should be noted that each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the embodiments can be referred to each other. For the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0163] The steps in the methods of each embodiment of the present application can be adjusted in order, combined and deleted according to actual needs, and the technical features recorded in each embodiment can be replaced or combined.

[0164] The modules and sub-modules in the devices and terminals of the various embodiments of the present application can be combined, divided and deleted according to actual needs.

[0165] In the several embodiments provided in the present application, it should be understood that the disclosed terminals, devices and methods can be implemented in other ways. For example, the terminal embodiments described above are only schematic, for example, the division of modules or submodules is only a logical function division, and there may be other division methods in actual implementation, for example, multiple submodules or modules can be combined or integrated into another module, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or modules, which can be electrical, mechanical or other forms.

[0166] The modules or submodules described as separate components may or may not be physically separated, and the components of the modules or submodules may or may not be physical modules or submodules, that is, they may be located in one place, or they may be distributed on multiple network modules or submodules. Some or all of the modules or submodules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0167] In addition, each functional module or submodule in each embodiment of the present application may be integrated into one processing module, or each module or submodule may exist physically separately, or two or more modules or submodules may be integrated into one module. The above-mentioned integrated modules or submodules may be implemented in the form of hardware or in the form of software functional modules or submodules.

[0168] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0169] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly by hardware, software units executed by a processor, or a combination of the two. The software units may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0170] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0171] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A report analysis method, characterized in that: include: Performing image recognition on the medical test report to obtain report information, wherein the report information includes contents of multiple report pages; Inputting the report information into a large language model for content analysis to obtain key information, wherein the large language model is trained based on a medical data set, and the medical data set is configured based on training reports in different formats; Merging the contents of the report sheets indicated by the key information into the report page to obtain merged information; The target report sheet is obtained by structuring the combined information.

2. The method according to claim 1, characterized in that The performing image recognition on the medical test report to obtain report information includes: Performing target keyword recognition on the target page of each report in the medical test report; If the target keyword is identified, a corresponding report is obtained; Perform image recognition on the report to obtain the report information.

3. The method according to claim 2, characterized in that The performing image recognition on the report to obtain the report information includes: Performing paging processing on the report to obtain paging text; The paged text is input into an optical character recognition model for image recognition to obtain the report information configured based on a markup language format.

4. The method according to claim 1, characterized in that: The training process of the large language model includes: Acquiring the medical data set; Performing format marking on the training report sheet indicated in the medical data set to obtain a marked report sheet; The large language model is fine-tuned multiple times based on the marked reports to train the large language model.

5. The method according to claim 4, characterized in that The step of formatting the training report sheet indicated in the medical data set to obtain a marked report sheet includes: Classifying the training reports indicated in the medical data set according to the source objects to obtain classified reports; Determine the table style information of the source object corresponding to the classification report; The training report form is formatted based on the table style information to obtain the marked report form.

6. The method according to claim 1, characterized in that The report information is input into the large language model for content analysis to obtain key information, including: Inputting the report information into a large language model for content analysis to obtain object information, inspection information, test information and conclusion information; The object information, the inspection information, the test information and the conclusion information are combined to obtain the key information.

7. The method according to claim 1, characterized in that The report sheet contents based on the key information indication are merged in the report page to obtain merged information, including: Determine the paging result based on the report content indicated by the key information; Get the preset module configuration information; The paging results are merged in the report page according to the preset module configuration information to obtain merged information.

8. The method according to claim 7, characterized in that The step of merging the paging results in the report page according to the preset module configuration information to obtain merged information includes: Filling the report page with the paging result according to the preset module configuration information to obtain filling information; Matching the cross-page components in the filling information to obtain matching information; Content merging is performed based on the matching information to obtain the merged information.

9. The method according to claim 1, characterized in that: The combined information is structured to obtain a target report sheet, including: Performing data cleaning on the merged information to obtain cleaned information; Acquire display structure information corresponding to an interactive system, wherein the interactive system includes an interactive interface, and the interactive interface is used to display the target report sheet; The cleaning information is structured and organized according to the display structure information to obtain the target report sheet.

10. The method according to any one of claims 1 to 9, characterized in that: The method further comprises: Get feedback on target report cards; Determining a source object indicated by the feedback information; Acquire the report data corresponding to the source object, and format the report data; Fine-tune the large language model based on the labeled report data.

11. A report analysis device, characterized in that: include: A recognition unit, used for performing image recognition on the medical test report to obtain report information, wherein the report information includes contents of multiple report pages; A parsing unit, used for inputting the report information into a large language model for content parsing to obtain key information, wherein the large language model is trained based on a medical data set, and the medical data set is configured based on training reports in different formats; The parsing unit is further used to merge the contents of the report sheet indicated by the key information into the report page to obtain merged information; The parsing unit is further used to obtain a target report by structuring the merged information.

12. A report parsing device, comprising an input and output component and a processor; The input and output components are used to obtain medical test reports; The processor is used to parse the medical test report obtained by the input and output component by executing the report parsing method as described in any one of claims 1 to 10.

13. A report sheet parsing system, comprising an interactive client and a server; The interactive client is used to obtain the medical test report and send the medical test report to the server, and to display the analysis result of the report output by the server; The server is used to parse the medical test report obtained by the interactive client by executing the report parsing method described in any one of claims 1 to 10.

14. A computer program product, characterized in that include: A computer program, which, when executed by a processor, implements the report parsing method according to any one of claims 1 to 10.

Images