Method and apparatus for recommending medical image report, and device and storage medium

Abstract

Provided in the present application is a method for recommending a medical image report, by means of which method, a historical image report which is relatively similar to the current image report can be retrieved as a reference for a doctor. The method comprises: acquiring patient data and image data corresponding to the current image report, wherein the image data is data generated by an image scanning device after the image scanning device performs image scanning on a scanned subject; according to preset dimensions defined by a graph ontology structure, obtaining an image report information graph of the current image report on the basis of at least the patient data and the image data; and on the basis of the similarities between the image report information graph of the current image report and image report information graphs of historical image reports, determining, from a set of historical image reports, a reference image report for the current image report.

Description

Recommended methods, devices, equipment, and storage media for medical imaging reports

[0001] This application claims priority to Chinese patent application number 202311253469.1, filed on September 25, 2023, entitled “Recommendation method, device, equipment and storage medium for medical imaging reports,” the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the technical field of medical imaging report processing, and in particular to a method, apparatus, computer equipment, storage medium, and computer program product for recommending medical imaging reports. Background Art

[0003] A medical imaging report is a document issued by a doctor after an examination using imaging scanning equipment, containing images, findings, and a diagnosis. With the advancement of diagnostic equipment and improvements in living standards, people are paying more and more attention to their health and are increasingly seeking physical examinations. This is especially true during public health emergencies, which creates a significant demand for imaging exams. This increases the workload and challenges doctors face in terms of analysis and judgment.

[0004] When processing a current imaging report, a doctor may search the database for historical imaging reports similar to the current imaging report as a reference. If the retrieved historical imaging report has low similarity with the current imaging report, it will affect the doctor's report quality and work efficiency.

[0005] Summary of the Invention

[0006] On one hand, the present application provides a method for recommending a medical imaging report, comprising:

[0007] Obtaining the medical consultation data and imaging data corresponding to the current imaging report; the imaging data is data generated by the imaging scanning device after the imaging scanning device scans the scanned object;

[0008] Obtaining an imaging report information graph of the current imaging report based on at least the medical data and the imaging data according to a preset dimension defined in the graph ontology structure;

[0009] Based on the similarity between the image report information map of the current image report and the image report information map of the historical image reports, a reference image report for the current image report is determined in the historical image report set.

[0010] In some embodiments, the imaging report information graph of the current imaging report is obtained based on at least the medical data and the imaging data according to the preset dimensions defined by the graph ontology structure, including:

[0011] Get several fields under the preset dimensions defined by the graph ontology structure;

[0012] Determining the contents corresponding to the plurality of fields in the medical data and the image data to obtain multi-tuple data of the preset dimension;

[0013] Based on the multi-tuple data of the preset dimension, an imaging report information map of the current imaging report is obtained.

[0014] In some embodiments, determining the contents corresponding to the plurality of fields in the medical data and the image data to obtain the multi-tuple data of the preset dimension includes:

[0015] Parsing the medical data and the imaging data to obtain content corresponding to the first category of fields;

[0016] According to the preset knowledge data expansion rules, the content corresponding to the second category of fields is determined based on the content corresponding to the first category of fields;

[0017] The multi-tuple data of the preset dimension is obtained according to the content corresponding to the first category of fields and the content corresponding to the second category of fields.

[0018] In some embodiments, obtaining the tuple data of the preset dimension according to the content corresponding to the first category field and the content corresponding to the second category field includes:

[0019] According to the content corresponding to the first category of fields, the content corresponding to the second category of fields and the content corresponding to the third category of fields, the multi-tuple data of the preset dimension is obtained; the content corresponding to the third category of fields is obtained by processing the scanned image obtained based on the image data.

[0020] In some embodiments, before determining a reference image report for the current image report in a set of historical image reports based on the similarity between the image report information graph of the current image report and the image report information graph of historical image reports, the method further comprises:

[0021] The image report information graph of the current image report and the image report information graph of the historical image report are processed using a graph analysis algorithm to obtain similarity.

[0022] In some embodiments, the graph analysis algorithm is used to process the image report information graph of the current image report and the image report information graph of the historical image report to obtain similarity, including:

[0023] Acquire multiple current image map nodes of an image report information map of a current image report, and multiple historical image map nodes of an image report information map of a historical image report;

[0024] Based on the graph analysis algorithm, determining the common nodes among the multiple current image atlas nodes and the historical image atlas nodes to obtain the number of common nodes, wherein the common nodes are nodes having the same fields and contents as the current image atlas and the historical image atlas;

[0025] The similarity is obtained according to the number of common nodes.

[0026] In some embodiments, obtaining similarity based on the number of shared nodes includes:

[0027] According to the number of common nodes under different preset dimensions, the sub-similarity of different preset dimensions is obtained;

[0028] According to the weight of each preset dimension, the sub-similarity of different preset dimensions is weighted to obtain the similarity.

[0029] In some embodiments, processing the image report information map of the current image report and the image report information map of the historical image reports to obtain similarity includes:

[0030] Determine the preset dimensions and / or fields involved in similarity calculation from the graph ontology structure.

[0031] In some embodiments, the method for recommending a medical imaging report further includes:

[0032] Obtaining user feedback on the recommendation result; the recommendation result includes a reference image report and / or user evaluation for the current image report;

[0033] The graph ontology structure is updated according to the feedback content.

[0034] In some embodiments, updating the graph ontology structure includes:

[0035] The weight of each of the preset dimensions is updated.

[0036] In some embodiments, the preset dimensions defined in the atlas ontology structure include at least: a patient information dimension and an image information dimension.

[0037] In some embodiments, before determining a reference image report for the current image report in a set of historical image reports based on the similarity between the image report information graph of the current image report and the image report information graph of historical image reports, the method further comprises:

[0038] Performing a similarity comparison on the contents of each preset field of the image report information map of the current image report and the image report information map of the historical image reports under the patient information dimension to obtain a sub-similarity for the patient information dimension;

[0039] Performing a similarity comparison on the contents of each preset field of the image report information graph of the current image report and the image report information graph of the historical image report under the image information dimension to obtain a sub-similarity for the image information dimension;

[0040] The similarity is obtained according to the sub-similarity for the patient information dimension and the sub-similarity for the image information dimension.

[0041] In some embodiments, the preset dimension defined by the graph ontology structure also includes a text information dimension, which corresponds to the text content seen in the image, and the entities under the image information dimension are associated with the entities under the text information dimension.

[0042] In some embodiments, the imaging report information graph of the current imaging report obtained based on the medical data and the imaging data according to the preset dimensions defined in the graph ontology structure includes:

[0043] The content corresponding to the fields under the patient information dimension is extracted from the medical data and the image data.

[0044] In some embodiments, the imaging report information graph of the current imaging report obtained based on at least the medical data and the imaging data according to the preset dimensions defined in the graph ontology structure includes:

[0045] Obtaining a scanned image based on the image data constituting the image in the image data; identifying key areas in the scanned image based on a pre-built deep neural network model; and / or

[0046] Extract the content corresponding to the fields under the image information dimension in the image data.

[0047] In some embodiments, the method for recommending a medical imaging report further includes:

[0048] Obtaining entities in the graph ontology structure;

[0049] The entity is normalized.

[0050] In some embodiments, the standardizing the entity includes:

[0051] The entities are standardized according to an open source knowledge base or knowledge graph.

[0052] Another aspect of the present application provides a device for recommending a reference image report, comprising:

[0053] A data acquisition module is used to acquire the medical data and image data corresponding to the current imaging report; the image data is the data generated by the imaging scanning device after the imaging scanning device scans the scanned object;

[0054] A graph acquisition module, configured to obtain an imaging report information graph of the current imaging report based on at least the medical data and the imaging data according to preset dimensions defined in the graph ontology structure;

[0055] A reference report determination module is used to determine a reference image report for the current image report in a set of historical image reports based on the similarity between the image report information map of the current image report and the image report information map of the historical image reports.

[0056] On the other hand, the present application provides a computer device including a memory and a processor, wherein the memory stores a computer program, and the processor executes the method for recommending a medical imaging report in any of the above embodiments.

[0057] On the other hand, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for recommending a medical imaging report in any of the above embodiments.

[0058] On the other hand, the present application provides a computer program product having a computer program stored thereon, which implements the method for recommending a medical imaging report in any of the above embodiments when the computer program is executed by a processor.

[0059] The details of one or more embodiments of the invention are set forth in the accompanying drawings and the description below. Other features, objects, and advantages of the invention will become apparent from the description, drawings, and claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to better describe and illustrate the embodiments of the present application, reference may be made to one or more drawings, but the additional details or examples used to describe the drawings should not be considered as limiting the scope of the invention of the present application, any of the currently described embodiments or preferred methods.

[0061] FIG1 is a diagram illustrating an application environment of a method for recommending medical imaging reports according to an embodiment;

[0062] FIG2 is a schematic diagram of a graph body structure in one embodiment;

[0063] FIG3 is a flow chart of a method for recommending a medical imaging report in another embodiment;

[0064] FIG4 is a structural block diagram of a device for recommending medical imaging reports in one embodiment;

[0065] FIG5 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0066] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0067] References to "embodiments" in this application mean that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described in this application may be combined with other embodiments.

[0068] The medical imaging report recommendation method provided in this application can be executed by a computer device. In some scenarios, a recommendation switch can be provided to doctors and other users. When doctors and other users turn on the recommendation switch, similar medical imaging reports can be recommended to doctors and other users according to the method provided in this application. When doctors and other users turn off the recommendation switch, similar medical imaging reports may not be recommended to doctors and other users.

[0069] The recommended method for medical imaging reporting provided in this application includes the steps shown in Figure 1:

[0070] Step S101: Obtain the medical consultation data and imaging data corresponding to the current imaging report.

[0071] The current imaging report may be the imaging report corresponding to the reference imaging report to be retrieved. The current imaging report may include the diagnostic text content of the imaging findings written by the doctor. In this case, the current imaging report is a completed imaging report. The current imaging report may not include the above-mentioned diagnostic text content of the imaging findings, or the included diagnostic text content of the imaging findings is not completed by the doctor. In this case, the current imaging report is an unfinished imaging report.

[0072] The medical data primarily includes information about the scanned subject, such as age and gender. Medical data can be obtained before the imaging device scans the subject. Image data is data generated by the imaging device after the subject has been scanned. The imaging device can first obtain the patient's medical data, then perform an image scan on the subject, thereby outputting device-generated data. The device-generated data can be in DICOM (Digital Imaging and Communications in Medicine) format and can include both medical data and image data.

[0073] The computer device can parse the above-mentioned medical data and image data from the device-generated data output by the image scanning device.

[0074] Step S102 : obtaining an imaging report information graph of the current imaging report based on at least the medical consultation data and the imaging data according to the preset dimensions defined in the graph ontology structure.

[0075] The graph ontology structure may include, for example, fields under preset dimensions and the contents of the fields.

[0076] In order to recommend historical imaging reports similar to the current imaging report so that doctors can use the historical imaging reports as a reference to complete the diagnostic text content of the imaging findings in the current imaging report or adjust the diagnostic text content of the imaging findings in the current imaging report, this application constructs an imaging report information map for imaging reports (including current imaging reports and historical imaging reports), and retrieves historical imaging reports similar to the current imaging report based on the comparison of the imaging report information maps.

[0077] If the scanned object in the current imaging report is A, and the scanned object in the historical imaging report is B, generally speaking, if the features of scanned objects A and B in dimensions other than the imaging information dimension are similar (such as age, gender, height, weight, etc.), and the features of scanned objects A and B in the imaging information dimension are similar, then the historical imaging report of scanned object B has a certain reference value for the current imaging report. Among them, the relevant information of dimensions other than the imaging information dimension (such as age, gender, height, weight, etc.) is generally in the medical data. The graph ontology structure defined in this application may include, for example, a patient information dimension and an imaging information dimension; wherein the patient information dimension includes multiple entities, each entity includes one or more fields, and the content of the fields of each entity can be derived from the medical data, and the imaging information dimension includes multiple entities, and the content of the fields of each entity can be derived from the imaging data.

[0078] After obtaining the device-generated data in DICOM format, the device-generated data is parsed and the corresponding content is extracted from the medical data and image data.

[0079] In some embodiments, a computer device can search for a patient information tag within device-generated data in a DICOM format, locate the patient information data area, obtain the patient data, and process the patient data using a pre-designed patient information processor to extract content corresponding to fields within the patient information dimension within the graph ontology structure, such as gender and age. It will be understood that the patient information processor herein refers to a software module for analyzing and processing patient data.

[0080] The computer device can search for image information tags in the device-generated data in the DICOM format to obtain image data, and obtain a scanned image based on the image data that constitutes the image in the image data; and identify key areas (such as lesion areas) in the scanned image based on a pre-built deep neural network model. In addition, the computer device can use a pre-designed image information processor to process the image data and extract the content in the image data corresponding to the fields under the image information dimension in the atlas ontology structure. The fields under the image information dimension include dimensions such as the examined organ, examined location, lesion location, and lesion size. It can be understood that the image information processor here refers to a software module that analyzes and processes image data.

[0081] After the computer device extracts the corresponding content from the medical data and imaging data, it fills the content into the nodes of the field according to the graph ontology structure to form the imaging report information graph of the current imaging report.

[0082] Step S103 : determining a reference image report for the current image report in the historical image report set based on the similarity between the image report information graph of the current image report and the image report information graph of the historical image reports.

[0083] The image report information map of the historical image report may be constructed in the same manner as the above-described method for forming the current image report information map.

[0084] After obtaining the imaging report information map of the current imaging report and the imaging report information map of the historical imaging report, the computer device compares the similarity between the imaging report information map of the current imaging report and the imaging report information map of the historical imaging report, and uses the historical imaging report whose similarity is greater than a threshold or ranks in the top N in terms of similarity as the reference imaging report for the current imaging report and recommends it to the doctor.

[0085] It is understandable that the image report of the current image report can be compared with some historical image reports in the historical image report set in terms of information maps, and can also be compared with all historical image reports in the historical image report set in terms of information maps.

[0086] In the above-mentioned medical imaging report recommendation method, an imaging report information map of the current imaging report is generated based on at least two aspects of the medical data and the imaging data, and the imaging report information map of the historical imaging report is also constructed based on at least two aspects of the medical data and the imaging data. A similarity comparison is performed based on the constructed imaging report information map, thereby retrieving historical imaging reports that are relatively similar to the current imaging report in at least two aspects of the medical data and the imaging data, and recommending them to doctors as a reference to improve the quality of doctors' imaging reports and work efficiency.

[0087] In one embodiment, step S102 obtains an imaging report information graph of the current imaging report based on at least medical data and imaging data according to the preset dimensions defined in the atlas ontology structure, including: obtaining several fields under the preset dimensions defined in the atlas ontology structure; determining the contents corresponding to several fields in the medical data and imaging data to obtain multi-tuple data of the preset dimensions; and obtaining the imaging report information graph of the current imaging report based on the multi-tuple data of the preset dimensions.

[0088] Whether historical imaging reports have a certain reference value for current imaging reports, in addition to paying attention to the condition of the lesion itself, you can also pay attention to other conditions of the scanned object besides the lesion, such as age, gender, height, weight, etc.

[0089] If the scanned object in the current imaging report is A and the scanned object in the historical imaging report is B, generally speaking, if the features of the scanned objects A and B in the imaging information dimension are similar, and the features of the scanned objects A and B in other dimensions besides the imaging information dimension are similar (such as age, gender, height, weight, etc.), then the historical imaging report of the scanned object B has a certain reference role for the current imaging report of the scanned object A.

[0090] Therefore, the preset dimensions defined by the atlas ontology structure may include the patient information dimension and the image information dimension. The patient information dimension corresponds to other conditions of the scanned object except the lesion, and is mainly used to determine whether the features of the scanned objects in other dimensions except the image information dimension are similar; the image information dimension corresponds to the condition of the lesion itself, and is mainly used to determine whether the features of the scanned objects in the image information dimension are similar.

[0091] When the atlas ontology structure includes the above-mentioned patient information dimension and imaging information dimension, the constructed imaging report information atlas of the current imaging report and the imaging report information atlas of the historical imaging report include the above-mentioned two dimensions, thereby retrieving historical imaging reports with similar conditions other than the lesion and similar conditions of the lesion itself, which has a certain reference role for the current imaging report.

[0092] In addition, since the preset fields under the patient information dimension, such as age, gender, weight, etc., will not change significantly within a certain period of time, the information of the same scanned object under the patient information dimension is mostly consistent within a certain period of time. Defining the patient information dimension in the graph ontology structure makes it more likely to find the historical imaging report of the same scanned object. The doctor can check the current image based on the historical imaging report of the scanned object, compare whether the situation has improved, and form a corresponding diagnostic conclusion.

[0093] Several fields under the patient information dimension may include gender, age, age group, weight, etc., and several fields under the image information dimension may include examination organ, examination location, lesion location, lesion size, lesion morphology, etc.

[0094] For example, a computer device can process the patient data using a patient information processor, extracting the contents of several fields under the patient information dimension. These fields can be represented in a tuple format of <patient, gender, age, ...>, thereby obtaining tuple data in the patient information dimension. The computer device can also process the image data using a deep neural network model and an image information processor, extracting the contents of several fields under the image information dimension. These fields can be represented in a tuple format of <lesion, examination organ, examination location, lesion location, lesion size, ...>, thereby obtaining tuple data in the image information dimension.

[0095] The computer device fills the multi-tuple data of the patient information dimension and the multi-tuple data of the imaging information dimension into the nodes of the fields under the corresponding dimensions according to the graph ontology structure, and obtains the imaging report information graph of the current imaging report.

[0096] In the above embodiment, after extracting the content corresponding to the field from the medical data and imaging data, it is represented in the multi-tuple format of the preset dimension, so that the corresponding content can be accurately filled into the node of the field under the corresponding dimension, thereby improving the accuracy of the graph construction.

[0097] In one embodiment, the contents corresponding to several fields in the medical data and imaging data are determined to obtain multi-tuple data of a preset dimension, including: parsing the medical data and imaging data to obtain the contents corresponding to the first category of fields; according to the preset knowledge data expansion rules, based on the contents corresponding to the first category of fields, determining the contents corresponding to the second category of fields; based on the contents corresponding to the first category of fields and the contents corresponding to the second category of fields, obtaining multi-tuple data of a preset dimension.

[0098] Each preset dimension includes multiple fields, among which the contents of some fields can be directly obtained by parsing the medical data and imaging data, and these fields can be called first-category fields; the contents of some fields can be obtained by combining the preset knowledge data expansion rules and the contents of the first-category fields, and these fields can be called second-category fields. For example, the "age" field under the patient information dimension, the content of this field can be directly parsed from the medical data, so the "age" field belongs to the first-category field; for another example, the "age group" field under the patient information dimension, the content of this field needs to be combined with the preset knowledge data expansion rules and the content of the "age" field, so the "age group" field belongs to the second-category field. Exemplarily, each second-category field, as well as the preset knowledge data expansion rules and the content of the first-category field used to determine the content of the second-category field, are shown in Table 1.

[0099] Table 1

[0100] Considering that the information obtained from parsing medical data and imaging data is limited, the above embodiment combines preset knowledge data expansion rules to expand the knowledge data of the atlas to avoid missing historical imaging reports for reference.

[0101] In one embodiment, obtaining tuple data of a preset dimension based on the content corresponding to the first type of field and the content corresponding to the second type of field includes:

[0102] According to the content corresponding to the first category of fields, the content corresponding to the second category of fields and the content corresponding to the third category of fields, multi-tuple data of preset dimensions are obtained; the content corresponding to the third category of fields is obtained by processing the scanned image obtained based on the image data.

[0103] Among the fields included in the preset dimensions, there are also some fields that can be obtained by processing the scanned images through a deep neural network model. These fields can be called the third type of fields, such as the "lesion area" field under the image information dimension.

[0104] After obtaining the image data, the computer device obtains a scanned image, inputs the scanned image into the deep neural network model, obtains the content of the third category field, and obtains multi-tuple data of preset dimensions based on the content corresponding to the first category field, the content corresponding to the second category field, and the content of the third category field.

[0105] The above embodiment further expands the knowledge of the atlas based on the content of the third type of field obtained by processing the scanned image, enriches the atlas knowledge, and further avoids missing historical image reports for reference.

[0106] In one embodiment, the method provided by the present application further includes: obtaining entities in the graph ontology structure; and standardizing the entities.

[0107] Taking into account the differences in numerical settings of different imaging scanning devices and the differences in doctors' habit of writing reports, there will be a situation where multiple words have the same meaning, which is reflected in the presence of multiple unrelated entities expressing the same meaning in the atlas. For example, "new coronavirus pneumonia", "new coronavirus pneumonia", "COVID-19", etc. all express the same pneumonia. In addition, due to the limited knowledge of doctors, it is difficult to fully express what is seen in the image. Therefore, if the entities are not standardized and the entity relationships are not completed, the retrieval and analysis will be incomplete during the atlas query and graph analysis process due to the weak entity correlation. Based on this, this embodiment standardizes the entities in the atlas ontology structure.

[0108] Furthermore, the entities are standardized, including: standardizing the entities according to an open source knowledge base or a knowledge graph.

[0109] Among them, open source knowledge bases such as ICD-10 (International Classification of Diseases maintained by the World Health Organization), UMLS (the integrated medical language system of a national medical library), SNOMED-CT (a systematic clinical medical terminology collection maintained by the International Organization for the Development of Medical Terminology), etc., as well as the Chinese symptom database (published by a university on the OpneKG website) and CMeKG (Chinese Medical Knowledge Graph) further expanded on the basis of these open source knowledge bases, obtain knowledge graphs.

[0110] Based on these open source knowledge bases or knowledge graphs, entities can be represented in a standardized manner, and some additional knowledge, such as clinical symptoms, diagnosis and treatment techniques, can be added to complete the missing entity relationship data and increase the association strength between entities.

[0111] In one embodiment, based on the similarity between the image report information map of the current image report and the image report information map of the historical image report in step S103, before determining the reference image report for the current image report in the historical image report set, the method provided by the present application also includes: using a graph analysis algorithm to process the image report information map of the current image report and the image report information map of the historical image report to obtain the similarity.

[0112] The graph analysis algorithm may include algorithms such as simRank and random walk. Through the graph analysis algorithm, the similarity between the image report information graph of the current image report and the image report information graph of the historical image report can be obtained.

[0113] Using a graph analysis algorithm, the image report information graph of the current image report and the image report information graph of the historical image report are processed to obtain similarity, further including: obtaining multiple current image graph nodes of the image report information graph of the current image report and multiple historical image graph nodes of the image report information graph of the historical image report; based on the graph analysis algorithm, determining the common nodes among the multiple current image graph nodes and each historical image graph node to obtain the number of common nodes; and obtaining similarity based on the number of common nodes.

[0114] The image report information map of the current image report includes multiple current image map nodes, each node has corresponding fields and content; the image report information map of the historical image report includes multiple historical image map nodes, each node has corresponding fields and content.

[0115] When the fields and contents corresponding to a current image map node and a historical image map node are the same, it can be determined that the two nodes are the same, and are common nodes of the image report information map of the current image report (hereinafter referred to as the current map) and the image report information map of the historical image report (hereinafter referred to as the historical map).

[0116] The computer device uses a graph analysis algorithm to obtain the common nodes of the current graph and the historical graph, and obtain the number of common nodes; illustratively, the computer device can also obtain the number of nodes contained in the current graph, which is called the first node number, and obtain the number of nodes of the historical graph, which is called the second node number. After adding the first node number and the second node number, the computer device subtracts the common node number to obtain the union node number of the current graph and the historical graph; the computer device can use the common node number as the intersection node number of the current graph and the historical graph, divide the intersection node number by the union node number to obtain the similarity between the current graph and the historical graph.

[0117] After obtaining the similarity between the current image and different historical images, the historical images can be sorted in descending order of similarity. The historical imaging reports of the top K historical images are used as reference imaging reports and recommended to the doctor. The doctor can browse the reference imaging reports and compare them with the current imaging report being processed, using the imaging findings and diagnostic conclusions of the reference imaging reports as a reference, thereby improving the efficiency of imaging report processing.

[0118] In some embodiments, similarity is obtained based on the number of common nodes, including: obtaining sub-similarity of different preset dimensions based on the number of common nodes under different preset dimensions; and obtaining similarity by weighting the sub-similarity of different preset dimensions based on the weight of each preset dimension.

[0119] For example, for a preset dimension D, the similarity between the current graph and the historical graph in the preset dimension D can be called sub-similarity, which can be recorded as sub_similar_score(R1, D, R2), where R1 and R2 represent the current graph and the historical graph respectively.

[0120] According to the formula Get the sub-similarity of the current graph and the historical graph in the preset dimension D; the numerator in the formula represents the number of common nodes between the current graph and the historical graph under the preset dimension D, that is, the number of intersection nodes, and the denominator in the formula represents the number of union nodes between the current graph and the historical graph under the preset dimension D.

[0121] According to the above method, the computer device can obtain the sub-similarity of the current graph and the historical graph in each preset dimension.

[0122] Then, according to the weights of each preset dimension [ω1,ω2…ω n ], perform weighted summation of the sub-similarity of each preset dimension to obtain the similarity between the current graph and the historical graph. The formula is as follows The subscript of the weight indicates the preset dimension corresponding to the weight.

[0123] Furthermore, the method provided in the present application also includes: obtaining user feedback on the recommendation results; the recommendation results include reference image reports and / or user evaluations for the current image report; and based on the feedback content, updating the image information graph ontology structure, such as preset dimensions, fields or weights.

[0124] During the process of doctors querying the current imaging report, writing the current imaging report, and verifying the current imaging report, similar historical imaging reports can be recommended to the doctors, and a feedback mechanism can be added for doctors to score the recommended results. The higher the score, the more relevant the recommended result is to the retrieval requirements, and the lower the score, the less relevant the recommended result is to the user's needs. At the same time, text descriptions can be added.

[0125] For example, when a doctor writes or verifies a current imaging report of "a 0.3 cm nodule in the left lung lobe of a 23-year-old female", the recommended similar historical imaging reports are "age group: 20-25 years old, gender: female, examined organ: lung, lesion location: left lung lobe, lesion size: 0.25-0.3 cm nodule". These historical imaging reports may be more useful than the historical imaging report of "age group: 40-45 years old, gender: male, examined organ: lung, lesion location: right lung lobe, lesion size: 0.1-0.15 cm nodule".

[0126] Therefore, using the feedback mechanism, doctors can score the recommendation results, which can include text content of the evaluation, to strengthen the model to output more accurate results. The specific operation can be divided into the following steps: obtaining the doctor's feedback on the recommendation results, which can include ratings and text content, filtering the feedback content according to a pre-defined filtering model, and retaining feedback content with a validity higher than a threshold; wherein the filtering model can be a classification model trained by annotating a certain amount of feedback content. Then, based on the feedback content with a validity higher than the threshold, the weights used for the graph or similarity calculation are updated.

[0127] For example, it can be determined whether the feedback is positive or negative. If it's positive feedback, the weight used in the similarity calculation is increased, thereby improving the strength of the graph's associations between corresponding entities. For example, if the doctor's review text reads, "This result is excellent, especially the age range, which is very consistent," this is positive feedback, and the age range is a key field, so the weight of this field corresponding to the preset dimension is increased. If it's negative feedback, on the one hand, the weight used in the similarity calculation can be reduced, thereby weakening the strength of the graph's associations between corresponding entities. For example, if the doctor's review text reads, "These reports are all concentrated between the ages of 20 and 25, which is too narrow," this is negative feedback, and the age range is a key field, so the weight of this field corresponding to the preset dimension is reduced. On the other hand, based on the above text, it can be seen that the doctor believes the age range is too small, so the range of age values ​​can be increased during the graph construction process.

[0128] In some embodiments, weights can be assigned to fields within each dimension. When comparing the image report information graph of a current imaging report with the image report information graph of historical imaging reports, similarity is calculated based on common nodes and their corresponding weights. Similarly, the weights of specific fields can be modified based on the doctor's evaluation.

[0129] Furthermore, considering that different doctors may have their own search habits or requirements, result recommendations and graph optimization can be adapted to each user, achieving personalized recommendations. This means that operations such as updating weights or expanding the range of patient age groups are performed on the user side without modifying the underlying graph. The backend monitors feedback in real time. When certain positive or negative feedback meets certain conditions, further updates to the entire graph structure can be considered. For example, the range of the "age field" can be narrowed or expanded, or fields without reference value can be removed or new fields can be added.

[0130] In some embodiments, the image report information graph of the current image report and the image report information graph of the historical image report are processed to obtain similarity, including determining preset dimensions and / or fields involved in similarity calculation from the graph ontology structure.

[0131] Before performing similarity calculation, the preset dimensions and / or fields involved in the similarity calculation can be determined based on the doctor's choice, for example, to improve the efficiency of subsequent similarity calculations and historical imaging report recommendations. For example, from a specific atlas ontology structure, the doctor selects preset dimensions and / or fields with strong correlation based on the patient's condition to perform atlas comparison, and filters out preset dimensions and / or fields with weak correlation and low reference value. It can be understood that in other embodiments, the preset dimensions and / or fields involved in the comparison can be automatically determined based on the patient information without manual selection. Based on the determined preset dimensions and / or fields involved in the similarity comparison, the similarity calculation method introduced above is used to perform similarity calculation. In one embodiment, the preset dimensions defined in the atlas ontology structure include at least: patient information dimension, imaging information dimension and text information dimension; the entities under the imaging information dimension are associated with the entities under the text information dimension.

[0132] For example, as shown in Figure 2, the graph ontology structure includes patient information dimensions, imaging information dimensions, and text information dimensions. The patient information dimension includes multiple preset fields, such as "gender," "age," and "age range." The imaging information dimension includes a preset field called "lesion," which includes multiple sub-preset fields, such as "examination organ," "examination location," "lesion location," "lesion size," and "lesion morphology." The text information dimension corresponds to, for example, the textual content of an image written by a doctor. Entities in the text information dimension have the association relationship shown in Figure 2 with entities in the imaging information dimension.

[0133] In some embodiments, the method provided herein further includes modifying the content of each field derived from the image data based on the text content found in the image. For example, for the same field, if the content derived from the image data is inconsistent with the content determined by the text content found in the image written by the doctor, the content determined by the text content found in the image is used as the content of the field.

[0134] In one embodiment, the method provided in the present application also includes: performing a similarity comparison on the contents of each preset field under the patient information dimension of the imaging report information map of the current imaging report and the imaging report information map of the historical imaging report to obtain a sub-similarity for the patient information dimension; performing a similarity comparison on the contents of each preset field under the imaging information dimension of the imaging report information map of the current imaging report and the imaging report information map of the historical imaging report to obtain a sub-similarity for the imaging information dimension; obtaining similarity based on the sub-similarity for the patient information dimension and the sub-similarity for the imaging information dimension.

[0135] The image report information map of the current image report is referred to as the current map; the image report information map of the historical image report is referred to as the historical map.

[0136] When calculating the sub-similarity of the current graph and the historical graph in the patient information dimension, it can be done in the manner described in the above embodiment. Specifically, the number of intersection nodes and the number of union nodes of the current graph and the historical graph in the patient information dimension can be calculated, and the number of intersection nodes can be divided by the number of union nodes to obtain the sub-similarity of the patient information dimension.

[0137] When calculating the sub-similarity of the current graph and the historical graph in the image information dimension, it can be done in the manner described in the above embodiment. Specifically, the number of intersection nodes and the number of union nodes of the current graph and the historical graph in the image information dimension can be calculated, and the number of intersection nodes can be divided by the number of union nodes to obtain the sub-similarity of the image information dimension.

[0138] After obtaining the sub-similarity of the patient information dimension and the sub-similarity of the image information dimension, the two sub-similarity can be weighted and summed according to the weight of the patient information dimension and the weight of the image information dimension to obtain the similarity between the current map and the historical map.

[0139] Furthermore, when the current imaging report includes diagnostic text related to imaging findings, the constructed imaging report information graph also includes content corresponding to several pre-set fields within the text information dimension. The computer device can utilize a pre-designed text information processor to perform entity recognition on the diagnostic text related to imaging findings. According to the graph ontology structure, the identified entities are populated into the nodes of the corresponding fields, thereby obtaining an imaging report information graph containing the text information dimension. It is understood that the text information processor herein refers to the software module that analyzes and processes the diagnostic text related to imaging findings.

[0140] When the current graph also includes a text information dimension, the sub-similarity between the current graph and the historical graph in the text information dimension can also be calculated. The calculation process can be performed in the manner described in the above embodiment, that is, the image report information graph of the current image report and the image report information graph of the historical image report are compared for similarity in each preset field under the text information dimension to obtain the sub-similarity for the text information dimension. Specifically, the number of intersection nodes and the number of union nodes of the current graph and the historical graph in the text information dimension can be calculated, and the sub-similarity in the text information dimension can be obtained by dividing the number of intersection nodes by the number of union nodes.

[0141] After obtaining the sub-similarity of the text information dimension, the sub-similarity of the patient information dimension, and the sub-similarity of the image information dimension, the three sub-similarity can be weighted and summed according to the weight of the text information dimension, the weight of the patient information dimension, and the weight of the image information dimension to obtain the similarity between the current map and the historical map.

[0142] To better understand the above method, the following describes an application example of the recommended method for medical imaging reports in this application. In this application example, medical data and imaging data are extracted from a device generated in DICOM format. An imaging report information graph is constructed based on the medical data, imaging data, and the diagnostic text written by the doctor regarding the imaging findings. In addition, this application example also combines customizable preset knowledge data expansion rules and an open source knowledge base to expand the knowledge of the imaging report information graph. Based on the constructed imaging report information graph, a graph analysis algorithm is used to retrieve and output similar imaging reports.

[0143] This application example includes the steps shown in Figure 3:

[0144] Step S301, defining the graph ontology structure.

[0145] Step S302: Obtain the medical consultation data, imaging data, and diagnostic text of the imaging findings written by the doctor from a plurality of imaging reports.

[0146] In step S303, different processors are used to process the medical data, imaging data, and imaging diagnostic text respectively, and multi-tuple data of each preset dimension is obtained according to the defined graph ontology structure, and stored in the graph database to generate an imaging report information graph.

[0147] It is understood that the image report information map can be a virtual structure that reflects the organization and logic of the data. The content of the image report information map can be stored in the same way as data stored in a general database, such as a MySQL database.

[0148] Step S304: combining preset knowledge data expansion rules and open source knowledge base, perform data standardization and knowledge expansion on the imaging report information map.

[0149] Through the processing of step S304, the information dimension of the image report can be enriched and the quality of the atlas can be improved.

[0150] Step S305: Obtain information of the target imaging report, use a graph analysis algorithm, search for similar reports in the imaging report information graph from multiple dimensions, and output them.

[0151] This application example can be divided into three parts: the first part is mainly to generate the imaging report information map, the second part is mainly to expand the knowledge of the imaging report information map, and the third part is mainly to recommend similar imaging reports.

[0152] Part 1: Generate image report information map:

[0153] (1) Definition of graph ontology structure:

[0154] This application example constructs a graph ontology structure based on actual business needs and data granularity, as shown in Figure 2. The ontology structure can be divided into three pre-defined dimensions: patient information, imaging information, and text information. Fields in the patient information dimension include gender, age, and age range; fields in the imaging information dimension include examination organs, examination locations, lesion location, and lesion size; and fields in the text information dimension include examination organs, examination locations, lesion location, and lesion size. Entities in the text information dimension are associated with entities in the imaging information dimension.

[0155] (2) Data analysis and image report information map generation:

[0156] Obtain relevant data, such as device-generated data in DICOM format and diagnostic text written by doctors based on imaging findings.

[0157] For device-generated data in DICOM format, the data is parsed according to the DICOM protocol. Within the device-generated data, the patient information tag is searched for, and the patient information data area is located to obtain the patient data. A pre-designed patient information processor is used to process the patient data, extracting fields such as the patient's gender and age. These fields are then represented in the <patient, gender, age, ...> tuple format to obtain tuple data for the patient information dimension.

[0158] Data is parsed according to the DICOM protocol, and image information tags are searched within the device-generated data to obtain the image data. A deep neural network model is constructed and trained using the scanned images to identify key areas (such as lesions) within the scanned images. A pre-designed image information processor is used to process the image data, extracting fields such as the examination organ, examination location, lesion location, and lesion size, and representing them in the multi-tuple format of <lesion, examination organ, examination location, lesion location, lesion size, ...>.

[0159] A pre-designed text information processor processes the doctor's written diagnostic text based on imaging findings. Fields such as the examination organ, examination location, lesion location, and lesion size are extracted from the text and represented in the tuple format of <text information, examination organ, examination location, lesion location, lesion size, ...> to obtain tuple data of the text information dimension. The text information processor can be an entity recognition model, such as a CRF (Conditional Random Fields), a Bi-directional Long Short-Term Memory (BiLSTM) + CRF, or a BERT (Bidirectional Encoder Representations) model. fromTransformer, the bidirectional encoder representation model in Transformer) + CRF and other models are not restricted here. First, some diagnostic texts seen in the images are collected, and the key fields in the diagnostic texts seen in the images are annotated with entities. For example, the "posterior segment of the left upper lung apex" in the text "Multiple fibrous cords and patchy high-density shadows can be seen in the posterior segment of the left upper lung apex, with point-like calcifications distributed within it with clear boundaries" is annotated as "lesion location", "multiple fibrous cords" is annotated as "lesion morphology", "patchy" is annotated as "lesion morphology", "high-density shadows" is annotated as "density", and "point-like calcifications" are annotated as "lesion morphology". The diagnostic texts seen in the images with entity annotations are then used as training data to train the entity recognition model. The trained entity recognition model is used for entity recognition of the diagnostic texts seen in the images, thereby extracting the content corresponding to the fields in the diagnostic texts seen in the images.

[0160] The multi-tuple data of the patient information dimension, imaging information dimension, and text information dimension are mapped to the corresponding nodes of the graph ontology structure, and the association between the nodes is established. The data is stored in the graph database, thereby generating an imaging report information graph.

[0161] Part II: Expanding the knowledge of imaging report information graph:

[0162] For the image report information map generated in the first part, in order to enrich the data dimension and improve the map quality, this application example combines preset knowledge data expansion rules and open source knowledge base to expand the map knowledge.

[0163] Considering the limited knowledge obtained from imaging data, a series of preset knowledge data expansion rules can be designed with reference to expert knowledge to expand the knowledge data in the graph. For example, as shown in Table 1, if the entity relationship "text information A-[lesion location]-posterior segment of the left upper lung apex" exists in the imaging report information graph, according to the "if examination location contain lung then output = lung" rule in "examination organ", it can be obtained that the examination organ is the lung. Therefore, the entity relationship "text information A-[examination organ]-lung" can be added to the imaging report information graph.

[0164] Furthermore, given the differences in numerical settings across imaging scanners and the varying language used by physicians in writing reports, multiple words with the same meaning may occur. This manifests itself as multiple unrelated entities within the imaging report information graph expressing the same meaning. For example, "novel coronavirus pneumonia," "new coronary pneumonia," and "COVID-19" all refer to the same pneumonia. Furthermore, physicians' limited knowledge makes it difficult to fully convey imaging findings. Therefore, without standardizing entities and completing entity relationships, weak entity associations during graph query and analysis can lead to incomplete retrieval and analysis. To alleviate this issue, this application example uses open-source knowledge bases to standardize entities within the graph. Examples of open-source knowledge bases include ICD-10 (International Classification of Diseases maintained by the World Health Organization), UMLS (the Unified Medical Language System of a national medical library), SNOMED-CT (Systematic Clinical Medical Terminology maintained by the International Organization for the Development of Medical Terminology Standards), and further extensions of these knowledge bases, such as the Chinese Symptom Database (published by a university on the OpneKG website) and CMeKG (Chinese Medical Knowledge Graph), resulting in a knowledge graph. Based on these open source knowledge bases or knowledge graphs, not only can the entities in the imaging report graph be represented in a standardized manner, but also some additional knowledge can be added, such as clinical symptoms, diagnosis and treatment techniques, to complete the missing entity relationship data and increase the association strength between entities.

[0165] Part III: Recommended Similar Imaging Reports:

[0166] Obtain the imaging report information graph obtained by expanding the second part for imaging report retrieval and similar imaging report recommendation.

[0167] During the imaging report retrieval process, the doctor enters a search statement, and the pre-trained intent recognition model and entity recognition model are used to obtain the doctor's search intent and keywords. Then, based on the search intent and keywords, as well as the defined graph ontology structure, a graph query statement is generated for graph query, and finally the query result, i.e. the relevant imaging report, is returned. For example, the doctor enters the search statement "children's COVID-19 imaging". The intent recognition model shows that the search intent of the search statement is to query imaging information. The entity recognition model shows that the search statement contains the entities "age group: children", "disease: COVID-19", and "examination organ: lung". According to the knowledge expansion rules, it can be further obtained that the synonymous entities of "disease: COVID-19" are "disease: novel coronavirus pneumonia" and "disease: COVID-19", without the doctor having to manually construct keywords, thereby generating a query statement:

[0168] search

[0169] (report)-[include]-(patient information)-[age]-(age group == children) and

[0170] (report)-[contains]-(text information)-[examination organ]-(examination organ == lungs) and

[0171] (report)-[Contains]-(Text Information)-[Disease]-(Disease in (COVID-19, novel coronavirus))

[0172] return report,report.Image information

[0173] By using this query statement to search in the imaging report information graph, imaging reports and corresponding imaging information related to the search statement can be obtained. The pre-trained intent recognition model can be a text classification model, and classification algorithms such as SVM (Support Vector Machine), LSTM (Long Short Term Memory), and BERT can be used. First, some search statements are obtained and the search statements are annotated with intents. Then, the search statements with intent annotations are used as training data to train the intent classification model to obtain a model that can perform intent recognition on new search statements. The entity recognition model is similar to the entity recognition model trained in the first part. The obtained search statements can be annotated with entities first, and then the search statements with entity annotations are used as training data to train the entity recognition model to obtain a model that can perform entity recognition on new search statements.

[0174] In the process of recommending similar imaging reports, there may be two scenarios: one is to retrieve similar imaging reports from the two dimensions of patient information and imaging information when the imaging scanning device completes the scan. At this time, the dimensions used for the retrieval include the patient information dimension and the imaging information dimension; the other is to retrieve similar imaging reports in real time while the doctor is writing the diagnostic text of the imaging findings. At this time, the dimensions used for the retrieval may include two or more of the patient information dimension, the imaging information dimension and the text information dimension.

[0175] Specifically, for the first scenario, when the imaging scanning device completes the scan, it will generate device-generated data in the DICOM format of the target imaging report. The patient information processor and image information processor in the first part are used to parse the device-generated data in the DICOM format of the target imaging report to obtain multi-tuple data of the patient information dimension and multi-tuple data of the imaging information dimension. Combined with the graph ontology structure, the image report information graph of the target imaging report is obtained; the image report information graph of the target imaging report is compared with the image report information graph of the historical imaging report for similarity, and the historical imaging reports are sorted according to the similarity. The top K historical imaging reports are regarded as historical imaging reports similar to the target imaging report and output for the doctor's reference.

[0176] For the second scenario, in addition to using the patient information processor and image information processor in the first part to parse the device-generated data in DICOM format of the target imaging report to obtain multi-tuple data in the patient information dimension and multi-tuple data in the image information dimension, the text information processor in the first part is also used to parse the diagnostic text of the imaging findings in the target imaging report to obtain multi-tuple data in the text information dimension. Based on the multi-tuple data in the patient information dimension, the multi-tuple data in the image information dimension, and the multi-tuple data in the text information dimension, combined with the graph ontology structure, the imaging report information graph of the target imaging report is obtained. The imaging report information graph of the target imaging report is compared with the imaging report information graph of historical imaging reports for similarity, and the historical imaging reports are sorted by similarity. The top K historical imaging reports are regarded as historical imaging reports similar to the target imaging report and output for the doctor's reference.

[0177] In addition, this application example also provides a feedback mechanism. When doctors query, write or verify the target imaging report, similar historical imaging reports can be recommended to them. A feedback mechanism is also added for doctors to score the recommended results. The higher the score, the more relevant the recommended result is to the retrieval requirements, and the lower the score, the less relevant the recommended result is to the user's needs. At the same time, text descriptions can be added.

[0178] For example, when a doctor writes or verifies a current imaging report of "a 0.3 cm nodule in the left lung lobe of a 23-year-old female", the recommended similar historical imaging reports are "age group: 20-25 years old, gender: female, examined organ: lung, lesion location: left lung lobe, lesion size: 0.25-0.3 cm nodule". These historical imaging reports may be more useful than the historical imaging report of "age group: 40-45 years old, gender: male, examined organ: lung, lesion location: right lung lobe, lesion size: 0.1-0.15 cm nodule".

[0179] Therefore, using the feedback mechanism, doctors can score the recommendation results, which can include textual content of the evaluation, to strengthen the model's output of more accurate results. The specific operation can be divided into the following steps: obtaining the doctor's feedback on the recommendation results, which can include a score and textual content, filtering the feedback content according to a pre-defined filtering model, and retaining feedback content with a validity higher than a threshold; wherein, the filtering model can be a classification model trained by annotating a certain amount of feedback content. Then, based on the feedback content with a validity higher than the threshold, the weights used for the graph or similarity calculation are updated.

[0180] Specifically, it can be determined whether the feedback is positive or negative. If it's positive, the weight used in the similarity calculation is increased, thereby strengthening the graph's association strength between corresponding entities. For example, if the doctor's review text reads, "This result is excellent, especially the age range, which is very consistent," this is positive feedback, and the age range is a key field, so the weight of this field is increased. If it's negative feedback, the weight used in the similarity calculation can be reduced, thereby weakening the graph's association strength between corresponding entities. For example, if the doctor's review text reads, "These reports are all concentrated between the ages of 20 and 25, which is too narrow," this is negative feedback, and the age range is a key field, so the weight of this field is reduced. On the other hand, based on the text, it can be seen that the doctor believes the age range is too narrow, so the range of age values ​​can be increased during graph construction.

[0181] Furthermore, considering that different doctors may have their own search habits or requirements, result recommendations and graph optimization can be adapted to each user, achieving personalized recommendations. This means that operations such as updating weights or expanding the range of patient age groups are performed on the user side, without modifying the underlying complete graph. The backend monitors feedback in real time, and when certain positive or negative feedback meets certain conditions, further updates to the entire graph can be considered, such as narrowing or expanding the range of the "age field," and performing a full update of the complete graph.

[0182] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0183] In one embodiment, as shown in FIG4 , a device for recommending medical imaging reports is provided, including:

[0184] The data acquisition module 401 is used to acquire the medical data and image data corresponding to the current imaging report; the image data is the data generated by the imaging scanning device after the imaging scanning device scans the scanned object;

[0185] A graph acquisition module 402 is configured to obtain an imaging report information graph of the current imaging report based on at least the medical data and the imaging data according to a preset dimension defined in the graph ontology structure;

[0186] The reference report determination module 403 is configured to determine a reference image report for the current image report in a set of historical image reports based on the similarity between the image report information map of the current image report and the image report information map of the historical image reports.

[0187] In one embodiment, the atlas acquisition module 402 is also used to obtain several fields under the preset dimensions defined by the atlas ontology structure; determine the contents corresponding to several of the fields in the medical data and the imaging data to obtain the multi-tuple data of the preset dimensions; and obtain the imaging report information atlas of the current imaging report based on the multi-tuple data of the preset dimensions.

[0188] In one embodiment, the atlas acquisition module 402 is also used to parse the medical data and the imaging data to obtain the content corresponding to the first category of fields; according to the preset knowledge data expansion rules, based on the content corresponding to the first category of fields, determine the content corresponding to the second category of fields; based on the content corresponding to the first category of fields and the content corresponding to the second category of fields, obtain the multi-tuple data of the preset dimension.

[0189] In one embodiment, the atlas acquisition module 402 is also used to obtain the multi-tuple data of the preset dimension based on the content corresponding to the first category of fields, the content corresponding to the second category of fields, and the content corresponding to the third category of fields; the content corresponding to the third category of fields is obtained by processing the scanned image obtained based on the image data.

[0190] In one embodiment, the device further includes a similarity comparison module for processing the image report information graph of the current image report and the image report information graph of the historical image report using the graph analysis algorithm to obtain similarity.

[0191] In one embodiment, the similarity comparison module is also used to obtain multiple current image map nodes of the image report information map of the current image report, and multiple historical image map nodes of the image report information map of the historical image report; based on the graph analysis algorithm, determine the common nodes among the multiple current image map nodes and the multiple historical image map nodes to obtain the number of common nodes; and obtain the similarity based on the number of common nodes.

[0192] In one embodiment, the similarity comparison module is further configured to obtain sub-similarity of different dimensions according to the number of common nodes in different dimensions; and perform weighted processing on the sub-similarity of different dimensions according to the weight of each dimension to obtain similarity.

[0193] In one embodiment, the device further includes an optimization module for obtaining user feedback on the recommendation result; the recommendation result includes a reference image report for the current image report; and the weight is updated according to the feedback content.

[0194] In one embodiment, the preset dimensions defined in the graph ontology structure include at least: patient information dimension, image information dimension and text information dimension; entities under the image information dimension are associated with entities under the text information dimension.

[0195] In one embodiment, the similarity comparison module is also used to perform a similarity comparison on the contents of each preset field of the imaging report information map of the current imaging report and the imaging report information map of the historical imaging report under the patient information dimension, to obtain a sub-similarity for the patient information dimension; perform a similarity comparison on the contents of each preset field of the imaging report information map of the current imaging report and the imaging report information map of the historical imaging report under the imaging information dimension, to obtain a sub-similarity for the imaging information dimension; obtain similarity based on the sub-similarity for the patient information dimension and the sub-similarity for the imaging information dimension.

[0196] In one embodiment, the device further includes an entity standardization processing module, which is used to obtain entities in the graph ontology structure and perform standardization processing on the entities.

[0197] In one embodiment, the entity standardization processing module is further used to standardize the entity based on an open source knowledge base or a knowledge graph.

[0198] The specific definition of the device for recommending medical imaging reports can be found in the definition of the method for recommending medical imaging reports above and will not be repeated here. The various modules in the aforementioned device for recommending medical imaging reports can be implemented in whole or in part through software, hardware, or a combination thereof. The aforementioned modules can be embedded in or independent of a processor in a computer device in hardware form, or can be stored in a memory in a computer device in software form, so that the processor can call and execute the corresponding operations of the aforementioned modules.

[0199] In one embodiment, a computer device is provided, the internal structure of which may be shown in FIG5 . The computer device includes a processor, memory, a communication interface, a display screen, and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal via wired or wireless communication. The wireless communication may be achieved via Wi-Fi, a mobile cellular network, NFC (near-field communication), or other technologies. The computer device also includes an input / output interface, which is a connection circuit for exchanging information between the processor and external devices. The input / output interface is connected to the processor via a bus and is referred to as an I / O interface. When executed by the processor, the computer program implements a method for recommending medical imaging reports. The display screen of the computer device may be a liquid crystal display or an electronic ink display. The input device of the computer device may be a touch screen layer covering the display screen, or may be buttons, a trackball, or a touchpad provided on the computer device housing, or may be an external keyboard, touchpad, or mouse.

[0200] Those skilled in the art will understand that the structure shown in FIG5 is merely a block diagram of a portion of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0201] In one embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0202] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned method embodiments are implemented. The computer-readable storage medium includes, for example, a non-volatile computer-readable storage medium.

[0203] In one embodiment, a computer program product is provided, on which a computer program is stored. The computer program is used by a processor to execute the steps in the above-mentioned various method embodiments.

[0204] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0205] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the above-mentioned computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0206] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0207] The above embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. A method for recommending a medical imaging report, the method comprising: Obtaining the medical consultation data and image data corresponding to the current image report; the image data is the data generated by the image scanning device after the image scanning device performs an image scanning on the scanned object; Obtaining an image report information map of the current image report based on at least the medical consultation data and the image data according to a preset dimension defined in the map ontology structure; Based on the similarity between the image report information map of the current image report and the image report information map of the historical image reports, a reference image report for the current image report is determined in the historical image report set.

2. The method according to claim 1, wherein: The preset dimension defined by the atlas ontology structure, based on at least the medical consultation data and the image data, obtains the image report information atlas of the current image report, including: Get several fields under the preset dimensions defined by the graph ontology structure; Determining the contents corresponding to the plurality of fields in the medical data and the image data to obtain the multi-tuple data of the preset dimension; Based on the multi-group data of the preset dimension, an image report information map of the current image report is obtained.

3. The method according to claim 2, wherein: The step of determining the contents corresponding to the plurality of fields in the medical data and the image data to obtain the multi-tuple data of the preset dimension includes: Parsing the medical data and the imaging data to obtain content corresponding to the first category of fields; According to the preset knowledge data expansion rules, the content corresponding to the second category of fields is determined based on the content corresponding to the first category of fields; According to the content corresponding to the first category of fields and the content corresponding to the second category of fields, the multi-tuple data of the preset dimension is obtained.

4. The method according to claim 3, wherein: The obtaining the multi-tuple data of the preset dimension according to the content corresponding to the first type of field and the content corresponding to the second type of field includes: According to the contents corresponding to the first category of fields, the contents corresponding to the second category of fields and the contents corresponding to the third category of fields, the multi-tuple data of the preset dimension is obtained; the contents corresponding to the third category of fields are obtained by processing the scanned image obtained based on the image data.

5. The method according to any one of claims 1 to 4, wherein: Based on the similarity between the image report information map of the current image report and the image report information map of the historical image reports, before determining a reference image report for the current image report in the historical image report set, the method further includes: Using a graph analysis algorithm, the image report information graph of the current image report and the image report information graph of the historical image report are processed to obtain the similarity.

6. The method according to claim 5, wherein: The image report information map of the current image report and the image report information map of the historical image report are processed by the graph analysis algorithm to obtain similarity, including: Acquire multiple current image map nodes of an image report information map of a current image report, and multiple historical image map nodes of an image report information map of a historical image report; Based on the graph analysis algorithm, determine the common nodes among multiple current image map nodes and each historical image map node to obtain the number of common nodes, wherein the common nodes are nodes having the same fields and contents as the current image map and the historical image map; The similarity is obtained according to the number of common nodes.

7. The method according to claim 6, wherein: According to the number of common nodes, similarity is obtained, including: According to the number of common nodes under different preset dimensions, the sub-similarity of different preset dimensions is obtained; According to the weight of each preset dimension, the sub-similarity of different preset dimensions is weighted to obtain the similarity.

8. The method according to claim 5, wherein the processing of the image report information map of the current image report and the image report information map of the historical image report to obtain the similarity comprises: Determine the preset dimensions and / or fields involved in similarity calculation from the graph ontology structure.

9. The method according to claim 1, further comprising: Obtain user feedback on recommendation results; The recommendation result includes a reference image report and / or a user evaluation for the current image report; The graph ontology structure is updated according to the feedback content.

10. The method according to claim 8, wherein updating the graph ontology structure comprises: The weight of each of the preset dimensions is updated.

11. The method according to claim 1, wherein: The preset dimensions defined in the atlas ontology structure include at least: patient information dimension and image information dimension.

12. The method according to claim 11, wherein: Based on the similarity between the image report information map of the current image report and the image report information map of the historical image reports, before determining a reference image report for the current image report in the historical image report set, the method further includes: Performing a similarity comparison on the contents of each preset field of the image report information map of the current image report and the image report information map of the historical image report under the patient information dimension to obtain a sub-similarity for the patient information dimension; Performing a similarity comparison on the contents of each preset field of the image report information map of the current image report and the image report information map of the historical image report under the image information dimension to obtain a sub-similarity for the image information dimension; The similarity is obtained according to the sub-similarity for the patient information dimension and the sub-similarity for the image information dimension.

13. The method according to claim 11, wherein: The preset dimensions defined in the atlas ontology structure also include a text information dimension, which corresponds to the text content seen in the image, and entities under the image information dimension are associated with entities under the text information dimension.

14. The method according to claim 11, wherein: The image report information map of the current image report obtained based on the medical data and the image data according to the preset dimensions defined in the map ontology structure includes: Content corresponding to the fields under the patient information dimension is extracted from the medical data and the image data.

15. The method according to claim 11, wherein the preset dimension defined by the atlas ontology structure, based on at least the medical consultation data and the imaging data, obtains the imaging report information atlas of the current imaging report, comprising: Obtaining a scanned image based on the image data constituting the image in the image data; identifying key areas in the scanned image based on a pre-built deep neural network model; and / or Extract the content corresponding to the fields under the image information dimension in the image data.

16. The method according to claim 1, further comprising: Obtaining entities in the graph ontology structure; The entity is normalized.

17. The method according to claim 16, wherein the step of performing standardization on the entity comprises: The entities are standardized according to an open source knowledge base or a knowledge graph.

18. A device for recommending a reference image report, the device comprising: A data acquisition module, used to acquire the medical consultation data and image data corresponding to the current image report; the image data is the data generated by the image scanning device after the image scanning device performs image scanning on the scanned object; A graph acquisition module, configured to obtain an image report information graph of the current image report based on at least the medical consultation data and the image data according to a preset dimension defined in the graph ontology structure; A reference report determination module is used to determine a reference image report for the current image report in a historical image report set based on the similarity between the image report information map of the current image report and the image report information map of the historical image reports.

19. A computer device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the method according to any one of claims 1 to 17 when executing the computer program.

20. A computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the method according to any one of claims 1 to 17 when executed by a processor.

Images