Method and device for carrying out structured integration on pathological reports

Abstract

The invention provides a method and device for performing structured integration on a pathological report, and the method comprises the following steps: obtaining multi-modal medical data, and standardizing each medical term in the medical data to obtain standardized data; labeling the standardized data in a layered labeling manner to obtain labeled data, and performing entity and entity relationship extraction on the labeled data by using a pre-trained multi-dimensional relationship extraction network to obtain entity relationship data; and performing knowledge graph construction on the entity relationship data in entity dimension, time dimension, space dimension and evidence dimension to obtain a structured pathological report. According to the scheme, through automatic structured processing of the pathological data, key information can be rapidly extracted from massive pathological reports, a high-quality data basis is provided for subsequent data analysis and application, and the accuracy and efficiency of medical research and clinical decision making are improved.

Description

Technical Field

[0001] The present application relates to the field of data processing, and in particular to a method and device for structured integration of pathology reports. Background Art

[0002] In today's era, medical information technology is booming at an unprecedented speed, and has had a profound impact on all aspects of the medical field. The importance of pathology reports, as a crucial basis in disease diagnosis, is self-evident. It carries the doctor's professional judgment on the patient's pathological condition, and is related to many key issues such as the formulation of subsequent treatment plans and the prediction of disease development trends.

[0003] However, traditional pathology reports have always been presented in simple text form. This type of text-based pathology report has exposed many serious problems in practical applications. First of all, it is extremely difficult to extract information from pathology reports. Pathology reports contain a large amount of complex and critical information, such as the patient's basic physical condition indicators, specific characteristics of the lesion site, cell morphology and other details, but this information is distributed in the text in a disorderly manner. Accurately extracting specific key information from it is like looking for a needle in a haystack, which requires a lot of time and effort, and it is often difficult to ensure the accuracy of the extraction.

[0004] Secondly, the difficulty in integrating and analyzing data is also a major problem. Due to the lack of standardized structure in the text form, different pathology reports have great differences in content organization and terminology usage, which leads to many obstacles when trying to aggregate the data of many pathology reports for comprehensive analysis. For example, some reports use more casual terms to describe the degree of lesions, while others use different reference standards, making it difficult to conduct unified and effective analysis after data integration.

[0005] In today's big data era, the value of data is becoming more and more prominent, and the medical field is in urgent need of making full use of massive medical data to provide strong support for important tasks such as disease research, clinical decision support, and medical quality assessment. However, the shortcomings of traditional pathology reports make it difficult to use these valuable medical data efficiently. Summary of the invention

[0006] The embodiment of the present application provides a method for structured integration of pathology reports. Through automatic structured processing of pathology data, key information can be quickly extracted from massive pathology reports, providing a high-quality data foundation for subsequent data analysis and application, and improving the accuracy and efficiency of medical research and clinical decision-making.

[0007] In a first aspect, an embodiment of the present application provides a method for structurally integrating a pathology report, the method comprising: Acquire multimodal medical data, standardize each medical term in the medical data to obtain standardized data, and perform disambiguation based on contextual semantic relationships of each medical term during the standardization process; The standardized data is labeled in a hierarchical labeling manner to obtain labeled data, and a pre-trained multi-dimensional relationship extraction network is used to extract entities and entity relationships from the labeled data to obtain entity relationship data, wherein the diagnostic conclusion layer, anatomical structure layer, pathological feature layer, and molecular marker layer of the standardized data are labeled respectively in the hierarchical labeling process; A knowledge graph is constructed for the entity relationship data in entity dimension, time dimension, space dimension and evidence dimension to obtain a structured pathology report, wherein the entity dimension includes the medical entity information of each patient, the time dimension includes the pathological evolution process of each patient, the space dimension includes the spatial coordinate information of each patient's pathological section in the body, and the evidence dimension includes the credibility weight of each patient's labeled data.

[0008] In a second aspect, an embodiment of the present application provides a device for structurally integrating pathology reports, including: an acquisition module, configured to acquire multimodal medical data, standardize each medical term in the medical data to obtain standardized data, and perform disambiguation based on contextual semantic relationships of each medical term during the standardization process; A labeling module is used to label the standardized data in a hierarchical labeling manner to obtain labeled data, and use a pre-trained multi-dimensional relationship extraction network to extract entities and entity relationships from the labeled data to obtain entity relationship data, wherein the diagnostic conclusion layer, anatomical structure layer, pathological feature layer, and molecular marker layer of the standardized data are labeled respectively during the hierarchical labeling process; The integration module is used to construct a knowledge graph for the entity relationship data in entity dimension, time dimension, space dimension and evidence dimension to obtain a structured pathology report, wherein the entity dimension includes the medical entity information of each patient, the time dimension includes the pathological evolution process of each patient, the space dimension includes the spatial coordinate information of each patient's pathological section in the body, and the evidence dimension includes the credibility weight of each patient's annotated data.

[0009] In a third aspect, an embodiment of the present application provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute a method for structured integration of pathology reports.

[0010] In a fourth aspect, an embodiment of the present application provides a readable storage medium, in which a computer program is stored. The computer program includes a program code for controlling a process to execute a process, and the process includes a method for structured integration of pathology reports.

[0011] The main contributions and innovations of the present invention are as follows: The embodiment of the present application obtains multimodal medical data and performs standardized processing, utilizes contextual semantic relationship disambiguation, and combines hierarchical annotation, multidimensional relationship extraction, and knowledge graph construction technology to obtain structured pathology reports, thereby providing comprehensive and accurate information for medical diagnosis and research. At the same time, the constructed bimodal feature fusion recognition model can effectively identify handwritten fonts and seal coverage areas, thereby improving data extraction accuracy. In addition, based on the standardized method with different mapping tables and weight settings, more accurate terminology standardization is achieved for disease and drug information, and the division of labor and collaboration among the acquisition, annotation, and integration modules make the entire process efficient and orderly, ultimately improving the efficiency, accuracy, and clinical application value of medical data processing.

[0012] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings: Figure 1 is a flow chart of a method for structured integration of pathology reports according to an embodiment of the present application; Figure 2 is a structural block diagram of a device for structurally integrating pathology reports according to an embodiment of the present application; Figure 3 It is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0014] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with one or more embodiments of this specification. Instead, they are merely examples of devices and methods consistent with some aspects of one or more embodiments of this specification as detailed in the appended claims.

[0015] It should be noted that: in other embodiments, the steps of the corresponding method are not necessarily performed in the order shown and described in this specification. In some other embodiments, the steps included in the method may be more or less than those described in this specification. In addition, a single step described in this specification may be decomposed into multiple steps for description in other embodiments; and multiple steps described in this specification may be combined into a single step for description in other embodiments.

[0016] Embodiment 1 The present application embodiment provides a method for structurally integrating pathology reports. By automatically structuring pathology data, key information can be quickly extracted from massive pathology reports, providing a high-quality data basis for subsequent data analysis and application, and improving the accuracy and efficiency of medical research and clinical decision-making. Specifically, reference Figure 1 , the method comprising: Acquire multimodal medical data, standardize each medical term in the medical data to obtain standardized data, and perform disambiguation based on contextual semantic relationships of each medical term during the standardization process; The standardized data is labeled in a hierarchical labeling manner to obtain labeled data, and a pre-trained multi-dimensional relationship extraction network is used to extract entities and entity relationships from the labeled data to obtain entity relationship data, wherein the diagnostic conclusion layer, anatomical structure layer, pathological feature layer, and molecular marker layer of the standardized data are labeled respectively in the hierarchical labeling process; A knowledge graph is constructed for the entity relationship data in entity dimension, time dimension, space dimension and evidence dimension to obtain a structured pathology report, wherein the entity dimension includes the medical entity information of each patient, the time dimension includes the pathological evolution process of each patient, the space dimension includes the spatial coordinate information of each patient's pathological section in the body, and the evidence dimension includes the credibility weight of each patient's labeled data.

[0017] In some embodiments, multimodal medical data is acquired by establishing a medical data interface engine, wherein the multimodal medical data includes DICOM images, pathological slice digital images (WSI), LIS text reports, electronic medical record structured fields, etc.

[0018] Furthermore, this solution also uses the HL7 FHIR standard for cross-system data mapping to solve the compatibility issues of heterogeneous data sources.

[0019] In some specific embodiments, the multimodal data may include some PDF scanned medical data, which are often handwritten by doctors and stamped to ensure authenticity. The stamp will also cover some fonts, so if the text recognition is simply performed on the scanned version of the PDF, the accuracy is low. Therefore, this solution constructs a pre-trained bimodal feature fusion recognition model to recognize the handwritten fonts and stamp coverage areas in the multimodal medical data. The bimodal feature fusion recognition model includes a parallel handwriting recognition network and a semantic reasoning network. The handwriting recognition network uses a large number of handwriting fonts to identify the handwritten fonts and stamp coverage areas. The training is performed to obtain a handwritten font recognition network, and prior knowledge of the pathology report structure is embedded in the semantic reasoning network, so as to identify the seal coverage area through context reasoning, and a dynamic fusion mechanism is set in the bimodal feature fusion recognition model. When handwritten fonts are recognized, the weight of the handwriting recognition network is set to a first weight, and the weight of the semantic reasoning network is set to a second weight through the dynamic fusion mechanism; when the seal coverage area is recognized, the weight of the semantic reasoning network is set to a first weight, and the weight of the handwriting recognition network is set to a second weight through the dynamic fusion mechanism, and the first weight is greater than the second weight.

[0020] Specifically, the handwriting recognition network extracts stroke-level details, such as ink connection points in cursive characters and topological structures of doctors' special abbreviations, through 12 layers of residual convolution, so as to perform accurate handwriting font recognition.

[0021] Specifically, the semantic reasoning network is a Transformer structure, and the prior knowledge introduced in the Transformer structure includes the fixed position of the diagnosis conclusion area, the standard arrangement order of immunohistochemical indicators, etc.

[0022] Exemplarily, the first weight is 70% and the second weight is 30%. That is, when handwriting is detected, the weight of the handwriting recognition network is set to 70% and the weight of the semantic reasoning network is set to 30% through the dynamic fusion mechanism, and when the seal coverage area is detected, the weight of the handwriting recognition network is set to 30% and the weight of the semantic reasoning network is set to 70% through the dynamic fusion mechanism.

[0023] Specifically, the pre-trained bimodal feature fusion recognition model effectively improves the recognition accuracy of handwritten fonts and seal coverage areas in multimodal medical data, enhances the model's adaptability to different recognition scenarios, and provides a strong guarantee for the accurate extraction of medical data and subsequent analysis and application.

[0024] In some embodiments, in order to avoid the PDF scanned version of the pathology report being difficult to read due to its low resolution, the present solution constructs a pre-trained super-resolution adversarial network to repair blurred text.

[0025] Specifically, the generator of the super-resolution adversarial network is a U-Net structure, and a medical text edge enhancement module is added to the decoder to repair the easily confusing features between numbers and English letters; document layout consistency detection is introduced in the discriminator to ensure that the repaired text maintains the paragraph structure and table alignment features of the original report.

[0026] Furthermore, the super-resolution adversarial network performs repair through a resolution adaptive strategy. When the DPI of the input image is lower than a repair threshold, the super-resolution adversarial network performs text repair on the input image.

[0027] In some embodiments, a basic medical terminology mapping table and an extended terminology mapping table are constructed to standardize each medical term in the medical data, wherein the medical terminology mapping table is constructed based on international medical terminology standards, and the extended terminology mapping table is constructed based on the hospital's local diagnosis expression habits.

[0028] Furthermore, a dynamically updated term mapping table is additionally constructed to standardize each medical term in the medical data, wherein new medical terms in the medical field are learned through an active learning mechanism, and a dynamically updated term mapping table is constructed based on the new medical terms.

[0029] Specifically, the medical field is a field of continuous development and innovation. New diseases, new diagnostic technologies and treatment methods are constantly emerging, and with them comes a large number of new terms. The dynamic update layer captures these new terms in real time through an active learning mechanism. The active learning mechanism uses advanced machine learning algorithms and data mining technologies to automatically identify potential new terms from massive data sources such as medical literature and clinical medical records. Once new terms are discovered, they will be submitted to pathologists for review. Pathologists rely on their professional knowledge and rich experience to strictly check the accuracy, rationality and applicability of new terms in the medical field. After confirmation by pathologists, new medical terms can be added to the dynamically updated term mapping table.

[0030] Specifically, the standardization process of medical terminology is to convert the medical expressions representing the same disease in different pathology reports into the same expression in a mapping manner, so as to facilitate subsequent structured organization and thus provide assistance for the diagnosis of the disease.

[0031] In some embodiments, when standardizing each medical term, the character similarity between the medical term and each element in the mapping table is calculated to obtain a similarity calculation result, the contextual semantic relationship of each medical term is captured by a context encoder to obtain a context calculation result, and the similarity calculation result and the context calculation result are integrated to obtain a mapping object for each medical term to complete standardization.

[0032] Specifically, when performing similarity calculation, optimization is performed based on special rules of medical abbreviations, for example, Ca is mapped to cancer rather than the chemical element calcium.

[0033] Specifically, the context encoder is a bidirectional LSTM structure, and the context encoder is used to capture contextual features appearing in medical terms, such as "abnormal" specifically refers to ductal epithelial dysplasia in breast pathology, and refers to intestinal metaplasia in digestive pathology.

[0034] Furthermore, time-weighted matching is performed on each element in the extended term mapping table, that is, the more recently added an element is to the extended term mapping table, the higher the probability that it will be matched as a mapped object.

[0035] Furthermore, when standardizing medical condition information, the weight of the similarity calculation result is set to the third weight, and the weight of the context calculation result is set to the fourth weight; when standardizing drug information, the weight of the similarity calculation result is set to the fourth weight, and the weight of the context calculation result is set to the third weight, and the third weight is greater than the fourth weight.

[0036] Specifically, the third weight in this scheme is 60% and the fourth weight is 40%. Since tumor information has a clear and standardized expression system, and the globally used TNM staging system has a fixed combination of letters and numbers to represent each stage, character similarity is extremely critical in matching, which directly determines whether the staging information can be accurately identified; but drug information is not only numerous, but also may have different meanings and usage methods in different medical scenarios. For the same drug, doctors may express it differently under different diseases and combined medication schemes. For example, "aspirin" may have different combinations and descriptions in different scenarios of cardiovascular disease prevention and antipyretic analgesia, so different weights are set for the context calculation results and the similarity calculation results when standardizing the disease information and drug information.

[0037] In some specific embodiments, since different medical terms may have different meanings in the medical field, for example, "degree of differentiation" can be used to indicate the grading of cancers in different organs, the context information is combined to disambiguate it to determine whether to adopt the WHO 2019 digestive system tumor grading standard or the ISUP prostate cancer grading system.

[0038] In some specific embodiments, the diagnosis conclusion layer includes the nature of the disease, the anatomical structure layer includes the specific body part where the lesion occurs, the pathological feature layer includes the microscopic features of the diseased tissue, and the molecular marker layer represents the biological characteristics of the disease.

[0039] Exemplarily, the information annotated on the diagnosis conclusion layer is malignant tumors, benign lesions, precancerous lesions, etc., the information annotated on the anatomical structure layer is left upper lobe of the lung, right anterior lobe of the liver, etc., the information annotated on the pathological feature layer is glandular duct structure, nuclear division image count, and the information annotated on the molecular marker layer is HER2 (3+), PD-L1 (22C3), etc.

[0040] Furthermore, the multidimensional relationship extraction network is a hybrid architecture of a graph neural network and a conditional random field. In the multidimensional relationship extraction network, basic entity recognition is first performed through conditional random fields, and then the topological relationship between entities is modeled based on the graph neural network.

[0041] In some specific embodiments, the entity dimension includes patients, lesions, and treatment core nodes; the time dimension includes the pathological evolution process from biopsy to surgery and then to recurrence; the spatial dimension includes spatial coordinate information such as a pathological slice being a slice of the liver region; the evidence dimension includes the credibility weight of each patient's annotated data. For example, in the diagnosis of lung cancer, the weight of the histopathological examination results is higher, while the weight of suspicious shadows found in imaging examinations is relatively low. The reliability of the diagnostic conclusion can be intuitively judged through the evidence dimension, helping doctors weigh the diagnostic results and make decisions.

[0042] In some embodiments, if the structured pathology report constructed by this scheme detects "positive vascular cancer thrombus", the system automatically associates it with the adjuvant chemotherapy regimen recommended by the NCCN guidelines.

[0043] In some embodiments, the structured pathology report constructed by the present solution automatically performs contradiction detection, for example, it can detect the contradictory combination of "ER-positive" and "triple-negative breast cancer" diagnoses, thereby assisting doctors in making judgments.

[0044] Embodiment 2 Based on the same idea, refer to Figure 2 , the present application also proposes a device for structurally integrating pathology reports, including: an acquisition module, configured to acquire multimodal medical data, standardize each medical term in the medical data to obtain standardized data, and perform disambiguation based on contextual semantic relationships of each medical term during the standardization process; A labeling module is used to label the standardized data in a hierarchical labeling manner to obtain labeled data, and use a pre-trained multi-dimensional relationship extraction network to extract entities and entity relationships from the labeled data to obtain entity relationship data, wherein the diagnostic conclusion layer, anatomical structure layer, pathological feature layer, and molecular marker layer of the standardized data are labeled respectively during the hierarchical labeling process; The integration module is used to construct a knowledge graph for the entity relationship data in entity dimension, time dimension, space dimension and evidence dimension to obtain a structured pathology report, wherein the entity dimension includes the medical entity information of each patient, the time dimension includes the pathological evolution process of each patient, the space dimension includes the spatial coordinate information of each patient's pathological section in the body, and the evidence dimension includes the credibility weight of each patient's annotated data.

[0045] Embodiment 3 This embodiment also provides an electronic device, referring to Figure 3 , comprises a memory 404 and a processor 402, wherein the memory 404 stores a computer program, and the processor 402 is configured to run the computer program to execute the steps in any of the above method embodiments.

[0046] Specifically, the processor 402 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0047] Among them, the memory 404 may include a large capacity memory 404 for data or instructions. For example, but not limitation, the memory 404 may include a hard disk drive (HDD), a floppy disk drive, a solid state drive (SSD), a flash memory, an optical disk, a magneto-optical disk, a tape, or a universal serial bus (USB) drive, or a combination of two or more of these. In appropriate cases, the memory 404 may include a removable or non-removable (or fixed) medium. In appropriate cases, the memory 404 may be inside or outside the data processing device. In a specific embodiment, the memory 404 is a non-volatile memory. In a specific embodiment, the memory 404 includes a read-only memory (ROM) and a random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (Programmable Read-Only Memory, PROM for short), an erasable PROM (Erasable Programmable Read-Only Memory, EPROM for short), an electrically erasable PROM (Electrically Erasable Programmable Read-Only Memory, EEPROM for short), an electrically alterable ROM (Electrically Alterable Read-Only Memory, EAROM for short) or a flash memory (FLASH) or a combination of two or more of these. In appropriate circumstances, the RAM may be a static random access memory (SRAM) or a dynamic random access memory (DRAM), wherein the DRAM may be a fast page mode dynamic random access memory 404 (FPMDRAM), an extended data output dynamic random access memory (EDODRAM), a synchronous dynamic random access memory (SDRAM), etc.

[0048] The memory 404 may be used to store or cache various data files required for processing and / or communication, as well as possible computer program instructions executed by the processor 402 .

[0049] The processor 402 reads and executes the computer program instructions stored in the memory 404 to implement any one of the methods for structurally integrating pathology reports in the above embodiments.

[0050] Optionally, the electronic device may further include a transmission device 406 and an input / output device 408 , wherein the transmission device 406 is connected to the processor 402 , and the input / output device 408 is connected to the processor 402 .

[0051] The transmission device 406 can be used to receive or send data via a network. Specific examples of the above-mentioned network may include a wired or wireless network provided by a communication provider of the electronic device. In one example, the transmission device includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 406 can be a radio frequency (Radio Frequency, referred to as RF) module, which is used to communicate with the Internet wirelessly.

[0052] The input / output device 408 is used to input or output information. In this embodiment, the input information may be multimodal medical data, etc., and the output information may be a structured pathology report, etc.

[0053] Optionally, in this embodiment, the processor 402 may be configured to perform the following steps through a computer program: Acquire multimodal medical data, standardize each medical term in the medical data to obtain standardized data, and perform disambiguation based on contextual semantic relationships of each medical term during the standardization process; The standardized data is labeled in a hierarchical labeling manner to obtain labeled data, and a pre-trained multi-dimensional relationship extraction network is used to extract entities and entity relationships from the labeled data to obtain entity relationship data, wherein the diagnostic conclusion layer, anatomical structure layer, pathological feature layer, and molecular marker layer of the standardized data are labeled respectively in the hierarchical labeling process; A knowledge graph is constructed for the entity relationship data in entity dimension, time dimension, space dimension and evidence dimension to obtain a structured pathology report, wherein the entity dimension includes the medical entity information of each patient, the time dimension includes the pathological evolution process of each patient, the space dimension includes the spatial coordinate information of each patient's pathological section in the body, and the evidence dimension includes the credibility weight of each patient's labeled data.

[0054] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be described in detail here.

[0055] In general, various embodiments may be implemented in hardware or dedicated circuits, software, logic, or any combination thereof. Some aspects of the invention may be implemented in hardware, while other aspects may be implemented in firmware or software that may be executed by a controller, microprocessor, or other computing device, but the invention is not limited thereto. Although various aspects of the invention may be shown and described as block diagrams, flow charts, or using some other graphical representation, it should be understood that, as non-limiting examples, the boxes, devices, systems, techniques, or methods described herein may be implemented in hardware, software, firmware, dedicated circuits or logic, general-purpose hardware or controllers or other computing devices, or some combination thereof.

[0056] Embodiments of the present invention may be implemented by computer software that is executable by a data processor of a mobile device, such as in a processor entity, or by hardware, or by a combination of software and hardware. Computer software or programs (also referred to as program products) including software routines, applets and / or macros may be stored in any device-readable data storage medium, and they include program instructions for performing specific tasks. A computer program product may include one or more computer executable components configured to perform an embodiment when the program is run. One or more computer executable components may be at least one software code or a portion thereof. In addition, at this point, it should be noted that, for example, Figure 3 Any block of the logic flow in the program may represent program steps, or interconnected logic circuits, blocks and functions, or a combination of program steps and logic circuits, blocks and functions. The software may be stored on physical media such as memory chips or storage blocks implemented within a processor, magnetic media such as hard disks or floppy disks, and optical media such as, for example, DVDs and their data variants, CDs, etc. Physical media are non-transitory media.

[0057] Those skilled in the art should understand that the technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0058] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for structured integration of pathology reports, characterized in that: The following steps are involved: Acquire multimodal medical data, standardize each medical term in the medical data to obtain standardized data, and perform disambiguation based on contextual semantic relationships of each medical term during the standardization process; The standardized data is labeled in a hierarchical labeling manner to obtain labeled data, and a pre-trained multi-dimensional relationship extraction network is used to extract entities and entity relationships from the labeled data to obtain entity relationship data, wherein the diagnostic conclusion layer, anatomical structure layer, pathological feature layer, and molecular marker layer of the standardized data are labeled respectively in the hierarchical labeling process; A knowledge graph is constructed for the entity relationship data in entity dimension, time dimension, space dimension and evidence dimension to obtain a structured pathology report, wherein the entity dimension includes the medical entity information of each patient, the time dimension includes the pathological evolution process of each patient, the space dimension includes the spatial coordinate information of each patient's pathological section in the body, and the evidence dimension includes the credibility weight of each patient's labeled data.

2. A method for structured integration of pathology reports according to claim 1, characterized in that: A pre-trained bimodal feature fusion recognition model is constructed to identify handwriting fonts and seal coverage areas in multimodal medical data. The bimodal feature fusion recognition model includes a parallel handwriting recognition network and a semantic reasoning network. The handwriting recognition network is trained with a large number of handwriting fonts for handwriting recognition. Prior knowledge of pathology report structure is embedded in the semantic reasoning network, so as to identify the seal coverage area through contextual reasoning.

3. A method for structured integration of pathology reports according to claim 2, characterized in that: A dynamic fusion mechanism is set in the dual-modal feature fusion recognition model. When handwritten fonts are recognized, the weight of the handwriting recognition network is set to the first weight, and the weight of the semantic reasoning network is set to the second weight through the dynamic fusion mechanism; when the seal coverage area is recognized, the weight of the semantic reasoning network is set to the first weight, and the weight of the handwriting recognition network is set to the second weight through the dynamic fusion mechanism, and the first weight is greater than the second weight.

4. A method for structured integration of pathology reports according to claim 1, characterized in that: A basic medical terminology mapping table and an extended terminology mapping table are constructed to standardize each medical term in the medical data, wherein the medical terminology mapping table is constructed based on international medical terminology standards, and the extended terminology mapping table is constructed based on the local diagnosis expression habits of the hospital.

5. A method for structured integration of pathology reports according to claim 1, characterized in that: When standardizing each medical term, the character similarity between the medical term and each element in the mapping table is calculated to obtain a similarity calculation result. The contextual semantic relationship of each medical term is captured by a context encoder to obtain a context calculation result. The similarity calculation result and the context calculation result are integrated to obtain the mapping object of each medical term to complete standardization.

6. A method for structured integration of pathology reports according to claim 5, characterized in that: When standardizing medical condition information, the weight of the similarity calculation result is set to the third weight, and the weight of the context calculation result is set to the fourth weight; when standardizing drug information, the weight of the similarity calculation result is set to the fourth weight, and the weight of the context calculation result is set to the third weight, and the third weight is greater than the fourth weight.

7. A method for structured integration of pathology reports according to claim 1, characterized in that: The diagnostic conclusion layer includes the nature of the disease, the anatomical structure layer includes the specific body part where the lesion occurs, the pathological feature layer includes the microscopic features of the lesion tissue, and the molecular marker layer represents the biological characteristics of the disease.

8. A device for structured integration of pathology reports, characterized in that: include: an acquisition module, configured to acquire multimodal medical data, standardize each medical term in the medical data to obtain standardized data, and perform disambiguation based on contextual semantic relationships of each medical term during the standardization process; A labeling module is used to label the standardized data in a hierarchical labeling manner to obtain labeled data, and use a pre-trained multi-dimensional relationship extraction network to extract entities and entity relationships from the labeled data to obtain entity relationship data, wherein the diagnostic conclusion layer, anatomical structure layer, pathological feature layer, and molecular marker layer of the standardized data are labeled respectively during the hierarchical labeling process; The integration module is used to construct a knowledge graph for the entity relationship data in entity dimension, time dimension, space dimension and evidence dimension to obtain a structured pathology report, wherein the entity dimension includes the medical entity information of each patient, the time dimension includes the pathological evolution process of each patient, the space dimension includes the spatial coordinate information of each patient's pathological section in the body, and the evidence dimension includes the credibility weight of each patient's annotated data.

9. An electronic device comprising a memory and a processor, characterized in that: The memory stores a computer program, and the processor is configured to run the computer program to execute the method for structured integration of pathology reports according to any one of claims 1 to 7.

10. A readable storage medium, characterized in that: The readable storage medium stores a computer program, wherein the computer program includes a program code for controlling a process to execute a process, wherein the process includes a method for structured integration of a pathology report according to any one of claims 1-7.

Images