Knowledge-driven medical image report structured processing method and device

Abstract

The invention provides a knowledge-driven medical image report structured processing method and device, and the method is based on an image knowledge base constructed through the combination of various modes such as medical knowledge, large language model or rule matching, manual verification and the like, and the image knowledge base comprises complete medical image knowledge. Furthermore, the accuracy of the medical entity in the image report identified based on the medical knowledge in the image knowledge base is higher. And carrying out hierarchical combination on the medical entities with the cascade relationship in the same clause to realize the planning output of the structured information. And finally, for the medical entity information of all clauses in the image report, removing repeated and redundant entity information existing among different clauses to obtain structured information corresponding to the image report, and further improving the accuracy of the structured information in the image report.

Description

Technical Field

[0001] The present invention belongs to the field of computer technology, and in particular relates to a knowledge-driven structured processing method and device for medical imaging reports. Background Art

[0002] Medical imaging reports are an important part of medical texts, recording rich and valuable clinical information, including imaging findings, detailed descriptions, and imaging diagnostic results of various examinations. However, these reports are usually stored in a semi-structured or even unstructured manner, containing a large number of fragmented and non-standardized descriptive terms, which makes it difficult to store and mine the clinical information contained therein.

[0003] The difficulty of structuring medical imaging reports lies in their high information density, the existence of a large number of synonyms, non-standardized description methods, and the complex relationships between various medical entities. At present, the commonly used medical report structuring methods are mainly based on language models or the combination of artificial and language models. Medical information is extracted from massive imaging report data in a data-driven way, and then compared with the existing standard medical knowledge base to obtain the structured information of the imaging report. However, this method requires a large amount of data support for the training of the language model, and the extracted structured information is inaccurate, has omissions, or has a lot of redundancy. Summary of the invention

[0004] In view of this, the purpose of the present invention is to provide a medical image report processing method and device to achieve accurate and efficient extraction of rich information in the image report. The technical solution provided is as follows:

[0005] In a first aspect, the present application provides a knowledge-driven structured processing method for medical imaging reports, comprising:

[0006] The imaging report is broken down into sentences to obtain multiple sentences;

[0007] Based on the keywords corresponding to each medical entity in the pre-created image knowledge base, the medical entity information in each sentence is identified, wherein the medical entity information includes the medical entity and the association between different types of medical entities, the image knowledge base includes the label and corresponding keywords of each medical entity, and the cascade relationship between medical entities of the same type, and the medical entity includes the sign entity, the anatomical location entity and the auxiliary information entity;

[0008] Hierarchical merging is performed for medical entities with cascading relationships in the same sentence;

[0009] For the medical entity information of all sentences in the imaging report, the repeated and redundant entity information between different sentences is removed to obtain the structured information corresponding to the imaging report.

[0010] Optionally, the identifying of the presence of medical entity information in each sentence based on keywords corresponding to each medical entity in a pre-created image knowledge base includes:

[0011] Identify the sign entities in each sentence based on the keywords corresponding to the sign entities in the image knowledge base;

[0012] Identify anatomical location entities in each sentence based on keywords corresponding to the anatomical location entities in the image knowledge base;

[0013] Identify the subsidiary information entities in each sentence based on the keywords corresponding to the subsidiary information entities in the image knowledge base;

[0014] The association between the anatomical position and the sign entity, or the association between the auxiliary information entity and the sign entity and the anatomical position entity is determined based on rule matching or a large language model.

[0015] Optionally, the identifying the sign entities in each sentence based on the keywords corresponding to the sign entities in the image knowledge base includes:

[0016] For any sentence in the image report, based on the keyword matching method, search whether any sentence contains information that successfully matches the keyword of the abnormal sign entity. If it does, search whether any sentence contains the exclusion keyword. If not, determine that the abnormal sign entity exists. If so, determine that the abnormal sign entity does not exist.

[0017] Optionally, searching for information on whether any of the sentences contains a keyword match successfully with an abnormal phenomenon entity based on a keyword matching method includes:

[0018] If the keyword of the abnormal phenomenon entity is a full match keyword, when any of the sentences contains a word that completely matches the full match keyword, it is determined that the sentence contains information that successfully matches the keyword;

[0019] If the keyword of the abnormal phenomenon entity is a combined matching keyword including at least two keyword segments, when any of the sentences simultaneously includes all the keyword segments of the abnormal phenomenon entity, it is determined that the sentence includes information that successfully matches the keyword.

[0020] Optionally, the identifying the anatomical location entities in each sentence based on the keywords corresponding to the anatomical location entities in the image knowledge base includes:

[0021] After the sign entity is identified in any sentence, when it is determined based on the keywords corresponding to the anatomical location entity that the any sentence does not contain the anatomical location entity, identifying whether the first adjacent sentence contains the anatomical location entity, the first adjacent sentence being the previous sentence adjacent to the any sentence;

[0022] If the first adjacent clause contains a first anatomical position entity, determine whether there is a correlation between the first anatomical position entity and the sign entity. If there is a correlation, continue to identify whether the second adjacent clause contains the anatomical position entity until there is no correlation between the identified anatomical position entity and the sign entity, then stop traversing the clauses and obtain the anatomical position entity associated with the sign entity. The second adjacent clause is the previous clause adjacent to the clause where the first anatomical position is located.

[0023] Optionally, determining whether there is a correlation between the first anatomical position entity and the sign entity includes:

[0024] Obtaining the complete set of anatomical positions corresponding to the sign entity from the image knowledge base, determining whether the first anatomical position entity is included in the complete set of anatomical positions, and if so, determining that the first anatomical position is associated with the sign entity, and if not, determining that the first anatomical position entity is not associated with the sign entity;

[0025] or,

[0026] The imaging report and the identified sign entity and anatomical location entity are input into a large language model to obtain a judgment result of whether there is a correlation between the sign entity and the anatomical location entity.

[0027] Optionally, identifying the subsidiary information entities in each sentence based on keywords corresponding to the subsidiary information entities in the image knowledge base includes:

[0028] For any sentence in the imaging report, after identifying the sign entity and the associated anatomical location entity contained in the any sentence, when it is determined that the any sentence does not contain information matching the keyword corresponding to the subsidiary information entity, identifying whether the first adjacent sentence contains the subsidiary information entity based on the keyword corresponding to the subsidiary information entity, the first adjacent sentence being the previous sentence adjacent to the sentence where the sign entity is located;

[0029] If the first adjacent sentence contains the first subsidiary information entity, determine whether there is a correlation between the first subsidiary information entity and the sign entity and the associated anatomical position entity. If there is a correlation, continue to identify whether the second adjacent sentence contains the subsidiary information entity until there is no correlation between the identified subsidiary information entity and the sign entity and the anatomical position entity, stop sentence traversal, and obtain the subsidiary information entity associated with the sign entity and the anatomical position entity. The second adjacent sentence is the previous sentence adjacent to the sentence where the first subsidiary information entity is located.

[0030] Optionally, determining whether there is a correlation between the first subsidiary information entity and the sign entity and the associated anatomical location entity includes:

[0031] Associating the identified subsidiary information entity to the nearest neighboring sign entity and anatomical position of the clause where the subsidiary information entity is located; if the sign entity and anatomical position of the first adjacent clause contain at least one medical entity that is the same as the sign entity and anatomical position of the nearest neighboring clause, then continuing to transfer the association of the first subsidiary information entity to the previous clause until the sign entity and anatomical position entity corresponding to the previous clause are all different from the sign entity and anatomical position entity of the nearest neighboring clause, and determining that the subsidiary information entity is not associated with the sign entity and the anatomical position entity;

[0032] or,

[0033] The imaging report and the identified sign entity, anatomical location entity and auxiliary information entity are input into a large language model to obtain a judgment result of whether the auxiliary information entity is associated with the sign entity and the anatomical location entity.

[0034] Optionally, the step of hierarchically merging medical entities in a cascade relationship in the same sentence includes:

[0035] For the sign entities or anatomical position entities in the same sentence, based on the entity nodes with cascade relationship defined in the image knowledge base, redundant upper-level entity nodes are removed and the lowest-level entity nodes are retained.

[0036] Optionally, for the medical entity information of all sentences in the imaging report, the repeated and redundant entity information between different sentences is removed to obtain the structured information corresponding to the imaging report, including:

[0037] After traversing all the sentences in the image report, the structured information of all the sentences is merged, and the repeated and redundant entity information between the sentences is removed to obtain the structured information corresponding to the image report.

[0038] Optionally, the process of creating the image knowledge base includes:

[0039] Create an initial imaging knowledge base based on the medical entity information related to medical imaging in the pre-existing medical knowledge;

[0040] Read the content of the current imaging report, and filter out the information in the imaging report that matches the medical entity information based on the medical entity information in the imaging knowledge base;

[0041] When the medical entity information in the current image report is not fully covered by the image knowledge base, the information related to the medical entity is screened from the remaining report information after filtering the current image report by manual verification, and the screened information is defined according to the medical definition rules of the image knowledge base and then updated to the image knowledge base;

[0042] When there is an unprocessed image report, the content of the next image report is read as the current image report until the medical entity information in the current image report is covered by the image knowledge base, and the image knowledge base is created.

[0043] In a second aspect, the present application also provides an electronic device comprising: a processor and a memory; the memory is used to store program code; the processor is used to run the program code, so that the electronic device implements the knowledge-driven structured processing method of medical imaging reports as described in any one of the first aspects.

[0044] In a third aspect, the present application further provides a computer-readable storage medium having instructions stored thereon, which, when executed on an electronic device, enables the electronic device to execute the knowledge-driven structured processing method for medical imaging reports as described in any one of the first aspects.

[0045] Compared with the prior art, the knowledge-driven structured processing method for medical imaging reports provided by the present invention contains complete medical imaging knowledge by constructing an imaging knowledge base based on a combination of medical knowledge, large language models or rule matching, manual verification, and other methods. Furthermore, the accuracy of medical entities in imaging reports identified based on the imaging knowledge base is higher. Moreover, the imaging knowledge base normalizes medical entities with the same meaning into a unified label to achieve standardized output. Moreover, associations between different medical entities are established in the imaging knowledge base, and the associations between medical entities in imaging reports are identified based on the imaging knowledge base, thereby outputting structured information of medical entities with associations, further improving the accuracy of structured information in imaging reports. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0047] Figure 1 is a flowchart of a process of creating an image knowledge base provided by an embodiment of the present application;

[0048] Figure 2 is a flow chart of a knowledge-driven structured processing method for medical imaging reports provided in an embodiment of the present application;

[0049] Figure 3 is a flowchart of another knowledge-driven structured processing method for medical imaging reports provided in an embodiment of the present application;

[0050] Figure 4 It is a flowchart of a knowledge-driven sentence structuring process of a medical imaging report provided in an embodiment of the present application. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0052] The inventors of this application have studied the current medical imaging report structuring methods, which are mainly based on language models or technologies combining manual and language models, and found that such technologies have the following limitations:

[0053] First, such technologies need to obtain high-quality medical structured information in a data-driven way, which requires a large amount of medical imaging data support, especially the training process of language models. The acquisition and annotation of medical data usually requires professional medical knowledge and clinical experience, which makes the cost of data governance high.

[0054] Secondly, the existing medical knowledge base mainly targets general medical texts, while medical imaging reports may contain a large amount of detailed image description information that is not included in the knowledge base. Structuring medical imaging reports based on the existing knowledge base is prone to omissions of report information.

[0055] In order to solve the above problems, the present application provides a knowledge-driven structured processing method for medical imaging reports, which mainly includes an imaging knowledge base construction process for medical imaging reports and a report structuring process. Among them, the imaging knowledge base construction process is constructed based on a combination of a large language model, rule matching, and manual verification. Then, based on the imaging knowledge base, medical entities with the same meaning are normalized to a unified label by keyword matching to achieve standardized output. In addition, taking the sign entity as the information subject, according to the recognition process of sign entity → anatomical location → ancillary information, the structured information of sign entity + anatomical location, sign entity + anatomical location + ancillary information is extracted in turn.

[0056] Let's combine Figure 1 Introduce the image knowledge base construction process, such as Figure 1 As shown, the process may include the following steps:

[0057] S101, creating an initial imaging knowledge base based on medical entity information related to medical imaging in the pre-set medical knowledge.

[0058] Pre-medical knowledge can include medical imaging textbooks, existing medical knowledge bases, medical imaging data sets and other materials. Medical entity information related to medical imaging is obtained from the above materials, such as sign entities (normal sign entities and abnormal sign entities), anatomical location entities and auxiliary information entities. Based on the above information, an initial imaging knowledge base is established, including the labels of the above medical entities, corresponding keywords and upper and lower entity nodes.

[0059] Among them, the symptom entity may include a normal symptom entity and an abnormal symptom entity.

[0060] Normal Signs entities are descriptions of the overall normality of an anatomical location that appear in the imaging report, such as "No abnormalities were found in the heart," and descriptions of the absence of a typical abnormality, such as "No signs of pneumothorax were found."

[0061] Abnormal sign entities include abnormalities, variations, lesions, diseases, medical devices, postoperative changes, etc. that appear in imaging reports, such as "nodules", "pneumonia", "pacemaker", etc.

[0062] Anatomical location entity, the anatomical location that appears in the imaging report, includes specific organs, parts, etc., such as "left upper lobe of the lung".

[0063] The subsidiary information entity corresponds to some states or attributes attached to a sign entity when it appears at a certain anatomical position, including uncertainty, severity, time progression, local significance, quantitative indicators, etc. For example, in the sentence "possible severe pneumonia in the left lung", the abnormal sign entity "pneumonia" at the anatomical position "left lung" has subsidiary information of severity "severe" and uncertainty "possible".

[0064] Each of the above-mentioned entities has a standardized label and corresponding keywords. Keyword matching further includes full match words, combined match words and exclusion words. For example, the label of the abnormal sign entity is "ground glass nodule", and the corresponding full match words can be "ground glass nodule", "Ground-glass nodule" or "GGN", the combined match words can be "ground glass" + "nodule", and the exclusion word can be "not seen".

[0065] Each sign and anatomical location entity also has corresponding parent entity nodes and child entity nodes, representing the parent concept and child concept corresponding to the entity. For example, the parent entity corresponding to the abnormal sign entity "ground glass nodule" is "nodule", and the child entities are "pure glass nodule" and "mixed ground glass nodule".

[0066] S102, reading the current imaging report content, and filtering out the information in the imaging report that matches the medical entity information based on the medical entity information in the imaging knowledge base.

[0067] The imaging reports of this embodiment include multiple ones. One of the imaging reports is input and the entity tags and keywords defined in the current imaging knowledge base are used to perform keyword matching in the imaging report content, and the successfully matched information is deleted, and the unmatched information is retained, that is, the remaining report information after filtering.

[0068] In the scenario of the first image report processed after the initial image knowledge base is created, the current image knowledge base is the initial image knowledge base.

[0069] In the scenario where the image report is processed after the initial image knowledge base is updated, the current image knowledge base is the updated image knowledge base.

[0070] The medical entity information in the imaging knowledge base can be matched with the information in the imaging report based on keyword matching. For example, the keywords corresponding to all medical entities in the imaging knowledge base are traversed and matched with the information in the imaging report one by one. If the match is successful, the matched information is deleted from the imaging report.

[0071] In a possible implementation, this step can directly input the original imaging report content.

[0072] In other possible implementations, the steps shown in S106 may be performed on the original imaging report, that is, information extraction is performed based on the large language model to obtain image description sentences, and then the steps shown in S102 are performed on the image description sentences. The process of extracting information based on the large language model may be to use the large language model to perform sentence decomposition, deduplication, and de-redundancy on the original imaging report to obtain the corresponding image description sentences.

[0073] S103, determine whether the medical entity information in the imaging report is fully covered by the imaging knowledge base; if not, execute S104 and then execute S105; if fully covered by the imaging knowledge base, execute S105.

[0074] If there are still unknown medical entities in the image report after filtering through the image knowledge base, it indicates that the medical entity information in the current image report is not fully covered by the current image knowledge base, indicating that there may still be valuable medical information in the remaining report information obtained after filtering, and also indicating that the image medical knowledge in the image knowledge base is still incomplete, and S104 can be continued.

[0075] If the medical entity information in the current imaging report has been completely covered by the imaging knowledge base, it indicates that the medical entity information in the imaging report has been completely identified based on the imaging knowledge base and no unknown medical entities exist, and it can be determined whether there is an unprocessed imaging report.

[0076] S104, filtering out information related to the medical entity from the remaining report information after filtering, and defining standard knowledge corresponding to the information and adding it to the initial imaging knowledge base.

[0077] For the remaining report information obtained in S102, the valuable information related to medical entities contained therein is further screened through manual verification, and the corresponding standard knowledge such as medical entities, keywords, upper and lower entities are defined, and these standard knowledge are added to the imaging knowledge base to enrich the medical knowledge of the imaging knowledge base for medical imaging.

[0078] S105, determine whether there is an unprocessed image report; if yes, read the next image report; if not, execute S106.

[0079] The process of creating an image knowledge base requires iteratively executing the processing steps S102 to S104 on multiple image reports. The unprocessed image reports herein refer to image reports that have not been processed by the processes S102 to S104.

[0080] After the valuable information in the image report is updated to the image knowledge base, it is determined whether there is an unprocessed image report. If there is, the next image report is read and the process shown in S102 to S104 is repeated.

[0081] S106, the image knowledge base is created.

[0082] After the initial imaging knowledge base is created, S102 to S104 are continuously executed on multiple imaging reports, and all medical entity information in the imaging reports is successfully matched with the medical knowledge in the imaging knowledge base, indicating that the imaging knowledge base has been created.

[0083] The imaging knowledge base construction process provided in this embodiment first creates an initial imaging knowledge base based on the pre-set medical knowledge, and further identifies and filters out the medical entity information in the imaging report based on the current imaging knowledge base. If the imaging report still contains valuable information, the valuable information is filtered out through manual verification and other methods, and the medical knowledge corresponding to the valuable information is defined and added to the imaging database. It can be seen that the imaging knowledge base is constructed through the pre-set medical knowledge and a large amount of detailed imaging description information in the imaging report, so the imaging knowledge base contains richer medical information. Furthermore, the imaging report is structured based on the imaging knowledge base, which improves the accuracy of the structured information.

[0084] The following will be combined Figure 2 The structured processing process of the knowledge-driven medical imaging report provided by the embodiment of the present application is introduced, such as Figure 2 As shown, the structured processing may include:

[0085] S201, decomposing the imaging report to obtain a plurality of sentences.

[0086] In an exemplary embodiment, the sentence decomposition may be performed using a rule-based method based on punctuation marks. In another exemplary embodiment, the sentence decomposition may also be performed using a large language model. The present application does not impose any particular restrictions on the specific method of sentence decomposition.

[0087] For example, the content of the imaging report is "small nodules and linear shadows in the left lung", and the two sentences are: "small nodules and linear shadows in the left lung" and "linear shadows".

[0088] S202, obtaining the sentence to be processed this time (ie, the current sentence) in order from front to back.

[0089] Still taking "small nodules in the left lung, linear shadows" as an example for explanation, the sentence "small nodules in the left lung" is read first as the sentence to be processed this time, and after processing this sentence, the sentence "linear shadows" is read again as the sentence to be processed this time.

[0090] S203, identifying the indicative entity in the current sentence based on the keyword matching method.

[0091] According to the normal sign entity keywords and abnormal sign entity keywords defined in the image knowledge base, the normal sign entity label or abnormal sign entity label in the sentence is identified by keyword matching.

[0092] In the embodiment of the present application, the keyword matching method may include full matching keywords, combined matching keywords and exclusion keywords.

[0093] For full match keywords, when a single defined keyword appears in a sentence, it indicates that the keyword match is successful. For example, the full match keyword corresponding to the abnormal symptom entity "atelectasis" can be "atelectasis". If "atelectasis" is detected in the sentence, it indicates that the abnormal symptom entity "atelectasis" exists.

[0094] The combination matching keywords include at least two keyword fragments to process fragmented information. Only when all these keywords appear in the sentence, it means that the combination matching keywords are matched successfully. For example, for the abnormal sign entity "pulmonary tuberculosis", its corresponding combination matching keywords can be "lung" and "tuberculosis". When "lung" and "tuberculosis" appear in the sentence at the same time, it means that the abnormal sign entity "pulmonary tuberculosis" exists.

[0095] Exclusion keywords: When such keywords appear in a sentence, it means that the corresponding entity tag match is unsuccessful, and the exclusion keywords have the highest priority. For example, the exclusion keyword corresponding to the abnormal sign entity "nodule" can be "not seen". When "not seen" appears in a sentence, it means that the abnormal sign entity "nodule" does not exist.

[0096] In some embodiments, if the current sentence is identified based on the keyword matching method as containing information that successfully matches the full match keyword or the combination match keyword with the abnormal sign entity, then continue to identify whether the sentence contains the exclusion keyword. If it does not contain the exclusion keyword, it is determined that the sentence includes the abnormal sign entity; if it contains the exclusion keyword, it is determined that the sentence does not include the abnormal sign entity.

[0097] S204, identifying the anatomical location entity in the current sentence based on the keyword matching method.

[0098] When a certain sign entity is matched, the anatomical position label corresponding to the sign entity is matched in the current sentence according to the anatomical position keyword information defined in the image knowledge base. The keyword matching format is the same as the keyword matching of the sign entity, which will not be repeated here. Further, it is determined whether the successfully matched anatomical position entity is associated with the current sign entity. If so, the sign entity and the corresponding anatomical position are regarded as a group of entity objects, and the corresponding ancillary information of the group of entity objects is further identified.

[0099] S205: Identify the subsidiary information entity in the current sentence based on the keyword matching method.

[0100] After matching a set of sign entities and anatomical location entities, the corresponding auxiliary information tags are identified in the current sentence according to the auxiliary information keywords defined in the imaging knowledge base, which may include uncertainty, severity, time progression, relatively significant parts, quantitative indicators, etc.

[0101] For example, in “possible severe pneumonia in the left lung”, the abnormal sign entity is “pneumonia”, the anatomical location entity is “left lung”, and the attached information entities are severity “severe” and uncertainty “possible”.

[0102] S206, determine whether all the sentences in the image report have been traversed; if they have been traversed, execute S207; if they have not been traversed, return to execute S102, that is, continue to obtain the next sentence and execute the subsequent processes shown in S203 to S205.

[0103] After identifying the medical entity contained in the current sentence, it is determined whether all the sentences in the imaging report have been traversed. If all the sentences have been traversed, the step shown in S207 is continued. If not, the next sentence in the imaging report is obtained and the steps shown in S203 to S205 are repeated.

[0104] S207, hierarchically merging the sign entities or anatomical position entities that have a superior-subordinate relationship in the same sentence.

[0105] For the sign entities or anatomical position entities that are matched in the same sentence and have a superior-subordinate relationship, hierarchical merging is performed according to the superior-subordinate entity nodes defined in the image knowledge base, and redundant superior entity nodes are removed, while the lowest entity nodes are retained.

[0106] For example, "ground glass nodule in the left lung" matches two abnormal signs, "nodule" and "ground glass nodule", located in the anatomical location of "left lung" based on the keyword, and "nodule" is the parent node of "ground glass nodule". Therefore, the merged result should be "ground glass nodule" located in the "left lung".

[0107] S208, removing repeated and redundant entity information between all sentences in the imaging report to obtain structured report information.

[0108] After traversing all the sentences in the imaging report, the structured information of all the sentences is merged, and the repeated and redundant entity information between the sentences is removed to obtain the complete structured information at the imaging report level as the final output result.

[0109] For example, an imaging report is broken down into two clauses: Clause 1 "Low-density shadow in the left upper lobe of the lung", and Clause 2 "Low-density lesions in the liver". Among them, the <sign entity: anatomical location entity> in Clause 1 is <Low-density shadow: left upper lobe of the lung>, and the <sign entity: anatomical location entity> in Clause 2 is <Low-density shadow: liver>. After the two clauses are combined, <sign: anatomical location> is <Low-density shadow: left upper lobe of the lung, liver>.

[0110] The following will be combined Figure 3 The structured processing process of medical images driven by knowledge provided in the embodiment of the present application is further introduced. Figure 2 The main difference between the embodiment shown is that the embodiment further refines the identification process of the anatomical location entity and the auxiliary information entity. Figure 3 As shown, the process may include the following steps:

[0111] S301, decomposing the imaging report to obtain a plurality of sentences.

[0112] S302, obtaining the sentence to be processed this time (ie, the current sentence) in order from front to back.

[0113] S303, identifying the indicative entity in the current sentence based on the keyword matching method.

[0114] For the implementation process of S301~S303, please refer to Figure 2 The implementation process of S201 to S203 will not be repeated here.

[0115] S304, identifying whether the current sentence contains an anatomical location entity; if not, executing S305; if yes, executing S306.

[0116] After the sign entity in the current sentence is identified, the anatomical position keyword matching is continued in the current sentence to identify whether the current sentence contains the anatomical position entity. If the current sentence does not contain the anatomical position keyword, the content of the previous sentence adjacent to the current sentence is continued to be obtained to match the anatomical position keyword. If the current sentence contains the anatomical position entity, it is continued to determine whether the anatomical position entity is associated with the sign entity, that is, S306 is executed.

[0117] S305, obtaining the content of the previous sentence adjacent to the current sentence, and performing anatomical position keyword matching.

[0118] If the current sentence does not contain the anatomical location entity, the anatomical location keyword is matched to continue to identify whether the previous sentence contains the anatomical location entity.

[0119] S306, determine whether the anatomical location entity and the sign entity are related; if so, return to S305, that is, take the previous sentence as the new current sentence, and identify whether the previous sentence adjacent to the new current sentence contains the anatomical location entity. If not, continue to execute S307.

[0120] After the anatomical location entity is identified, it is determined whether the anatomical location entity is related to the symptom entity in the current sentence. If there is a correlation, the content of the previous sentence of the previous sentence is obtained to match the anatomical location keyword until there is no correlation between the matched anatomical location and the symptom entity, and the sentence traversal is terminated.

[0121] For example, the imaging report "small nodules and linear shadows in the left lung" is broken down into two clauses: Clause 1 "small nodules in the left lung" and Clause 2 "linear shadows". The anatomical location corresponding to the sign entity "linear shadows" in Clause 2 is missing in this clause. By traversing the clauses, the corresponding anatomical location "left lung" can be found in the previous clause.

[0122] In an exemplary embodiment, it is possible to determine whether the anatomical location entity and the sign entity are associated based on rules. This method requires defining the complete set of anatomical locations where each sign entity may appear in advance in the image knowledge base, and then determining whether the identified anatomical location is included in the complete set of anatomical locations. If the complete set of anatomical locations includes the identified anatomical location, then there is an association relationship; if the complete set of anatomical locations does not include the identified anatomical location, then it is determined that there is no association relationship. For example, the anatomical locations where the abnormal sign entity "pneumonia" may appear are "lung", "left lobe of lung", "right lobe of lung", etc., but it is impossible for it to appear in the anatomical location "liver".

[0123] In another exemplary embodiment, whether there is a correlation between the anatomical location entity and the sign entity can be determined based on a large language model, with the original imaging report and the identified anatomical location entity and sign entity used as model inputs, and a binary judgment result of whether the model inputs are correlated can be obtained.

[0124] S307, identifying whether the current sentence contains an attached information entity. If not, execute S308; if yes, execute S309.

[0125] If the current sentence contains an anatomical location entity and there is no correlation between the anatomical location entity and the symptom entity, continue to identify whether the current sentence contains an attached information entity. The current sentence here refers to the sentence in which the symptom entity is identified. Exemplarily, it can be identified by keyword matching, that is, matching the attached information keyword for the current sentence.

[0126] S308, obtaining the content of the previous sentence adjacent to the current sentence, and performing keyword matching of the attached information.

[0127] For scenarios where the context of the imaging report contains associated ancillary information, the adjacent sentences are traversed to match the ancillary information keywords. For example, if the current sentence does not contain an ancillary information entity, the previous sentence adjacent to the current sentence is further identified to determine whether the ancillary information entity is included. The association of the identified ancillary information with the current sign entity and the anatomical location is further determined.

[0128] S309, determine whether the subsidiary information entity is associated with the sign entity and the anatomical position; if yes, traverse the previous sentence and return to execute S308; if not, execute S310.

[0129] In an exemplary embodiment, it can be determined based on rules whether the subsidiary information entity is associated with the current sign entity and anatomical location entity. In this way, the matched subsidiary information entity is first associated with the sign entity and anatomical location entity of the nearest neighbor before the sentence containing the subsidiary information entity (including the sentence), and if the sign entity and anatomical location entity pair of the previous sentence contains an entity that is the same as one of the sign entity and anatomical location entity of the nearest neighbor, the association is passed forward until the sign entity and anatomical location entity of the previous sentence are different from the sign entity and anatomical location entity of the nearest neighbor, then it is determined that the subsidiary information is not associated with the sign entity and anatomical location entity, and the traversal of the sentence is stopped.

[0130] For example, the imaging report is "multiple cords, patches, and nodules in the left lung, new compared to the previous film", which includes clause 1 "multiple cords in the left lung", clause 2 "patches", clause 3 "nodules", and clause 4 "new compared to the previous film". Clause 4 "new compared to the previous film" includes the subsidiary information "new" about the time progression, and its corresponding abnormal signs "nodules", "patches", "cords" and anatomical position signs "left lung" all appear in the previous clauses, so the abnormal sign entity and anatomical position entity corresponding to the subsidiary information entity can be found by clause traversal.

[0131] In another exemplary embodiment, it is possible to determine whether the ancillary information is associated with the identified sign entity and anatomical location based on the large language model. In this way, the imaging report and the identified sign entity, anatomical location entity and ancillary information entity are used as model data, and the model input is a binary judgment result of whether there is an association.

[0132] S310, determine whether all the sentences in the image report have been traversed, if yes, execute S311; if not, return to execute S302.

[0133] S311, hierarchically merge the sign entities or anatomical position entities that have a superior-subordinate relationship in the same sentence.

[0134] S312, for all sentences in the imaging report, remove repeated and redundant entity information between sentences to obtain structured report information.

[0135] For the implementation process of S310 to S312 in this embodiment, please refer to Figure 2 S206~S208 will not be described in detail here.

[0136] The structured processing process of the medical imaging report provided in this embodiment is based on an imaging knowledge base constructed by combining medical knowledge, large language models or rule matching, manual verification and other methods, which contains complete medical imaging knowledge. The accuracy of the medical entities in the imaging report identified based on the imaging knowledge base is higher. Moreover, the imaging knowledge base normalizes medical entities with the same meaning into a unified label to achieve standardized output. Moreover, the association relationship between different medical entities is established in the imaging knowledge base, and the association relationship between medical entities in the imaging report is identified based on the imaging knowledge base, thereby outputting the structured information of the medical entities with the association relationship, further improving the accuracy of the structured information in the imaging report.

[0137] On the other hand, the above-mentioned structuring process applied to the entire imaging report can also be used to structure a sentence in the imaging report, that is, the input of the above-mentioned structuring method can also be a sentence in the imaging report to obtain the structural information of the sentence.

[0138] See also Figure 4 , which shows a schematic diagram of a structured processing flow of another knowledge-driven medical imaging report provided in an embodiment of the present application, such as Figure 4 As shown, the method may include the following steps:

[0139] S401, obtaining a sentence in an imaging report.

[0140] S402, performing keyword matching of the sign entity on the sentence to identify the sign entity in the sentence.

[0141] S403, performing anatomical position keyword matching on the sentence, and identifying the anatomical position entity in the sentence.

[0142] In one scenario, there may be no anatomical location entity in the sentence, in which case it is determined that the sentence does not have the anatomical location entity, and it may continue to identify whether there is an auxiliary information entity in the sentence.

[0143] S404: perform keyword matching of the subsidiary information on the sentence to identify the subsidiary information entity in the sentence.

[0144] In one scenario, there may be no attached information entity in the currently processed sentence. In this scenario, it is determined that the sentence does not have an attached information entity, and the medical entity recognition process for the sentence ends, and S405 is continued.

[0145] S405: hierarchically merge the sign entities or anatomical position entities that have a superior-subordinate relationship in the sentence to obtain structural information of the sentence.

[0146] The implementation process of S402 to S405 in this embodiment is similar to Figure 2 The implementation process of the relevant steps in is the same and will not be repeated here.

[0147] This embodiment provides a method for structured processing of medical imaging reports, which can perform structured processing on a sentence in an imaging report to achieve sentence-level structured information output.

[0148] The present application provides an electronic device, which includes a processor and a memory, wherein the memory stores a program that can be run on the processor. When the processor runs the program stored in the memory, the method for processing a medical imaging report described in any of the above embodiments is implemented. The electronic device in this article can be a server, a PC, a PAD, a mobile phone, etc.

[0149] The present application also provides a storage medium executable by a computing device, in which a program is stored. When the program is executed by the computing device, the above-mentioned method for processing medical imaging reports is implemented.

[0150] For the above-mentioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that the present invention is not limited by the order of the actions described, because according to the present invention, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0151] It should be noted that the technical features recorded in the various embodiments in this specification can be replaced or combined with each other, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the various embodiments can be referred to each other. For the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0152] The steps in the methods of the embodiments of the present application can be adjusted in order, combined, and deleted according to actual needs.

[0153] The modules and sub-modules in the devices and terminals in the embodiments of the present application can be combined, divided and deleted according to actual needs.

[0154] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

[0155] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A knowledge-driven structured processing method for medical imaging reports. It is characterized in that include: The imaging report is broken down into sentences to obtain multiple sentences; Based on the keywords corresponding to each medical entity in the pre-created image knowledge base, the medical entity information in each sentence is identified, wherein the medical entity information includes the medical entity and the association between different types of medical entities, the image knowledge base includes the label and corresponding keywords of each medical entity, and the cascade relationship between medical entities of the same type, and the medical entity includes the sign entity, the anatomical location entity and the auxiliary information entity; Hierarchical merging is performed for medical entities with cascading relationships in the same sentence; For the medical entity information of all sentences in the imaging report, the repeated and redundant entity information between different sentences is removed to obtain the structured information corresponding to the imaging report.

2. The method according to claim 1, It is characterized in that The identifying of the presence of medical entity information in each sentence based on the keywords corresponding to each medical entity in the pre-created image knowledge base includes: Identify the sign entities in each sentence based on the keywords corresponding to the sign entities in the image knowledge base; Identify anatomical location entities in each sentence based on keywords corresponding to the anatomical location entities in the image knowledge base; Identify the subsidiary information entities in each sentence based on the keywords corresponding to the subsidiary information entities in the image knowledge base; The association between the anatomical position and the sign entity, as well as the association between the auxiliary information entity and the sign entity and the anatomical position entity are determined based on rule matching or a large language model.

3. The method according to claim 2, It is characterized in that The identifying the sign entities in each sentence based on the keywords corresponding to the sign entities in the image knowledge base includes: For any sentence in the imaging report, based on the keyword matching method, search whether any sentence contains information that successfully matches the keyword of the abnormal sign entity. If it does, search whether any sentence contains exclusion keywords. If not, determine that the abnormal sign entity exists. If so, determine that the abnormal sign entity does not exist.

4. The method according to claim 3, It is characterized in that The keyword matching method is used to search for information on whether any of the sentences contains a keyword that successfully matches the abnormal phenomenon entity, including: If the keyword of the abnormal phenomenon entity is a full match keyword, when any of the sentences contains a word that completely matches the full match keyword, it is determined that the sentence contains information that successfully matches the keyword; If the keyword of the abnormal phenomenon entity is a combined matching keyword including at least two keyword segments, when any of the sentences simultaneously includes all the keyword segments of the abnormal phenomenon entity, it is determined that the sentence includes information that successfully matches the keyword.

5. The method according to claim 2, It is characterized in that The identifying the anatomical location entities in each sentence based on the keywords corresponding to the anatomical location entities in the image knowledge base includes: After the sign entity is identified in any sentence, when it is determined based on the keywords corresponding to the anatomical location entity that the any sentence does not contain the anatomical location entity, identifying whether the first adjacent sentence contains the anatomical location entity, the first adjacent sentence being the previous sentence adjacent to the any sentence; If the first adjacent clause contains a first anatomical position entity, determine whether there is a correlation between the first anatomical position entity and the sign entity. If there is a correlation, continue to identify whether the second adjacent clause contains the anatomical position entity until there is no correlation between the identified anatomical position entity and the sign entity, then stop traversing the clauses and obtain the anatomical position entity associated with the sign entity. The second adjacent clause is the previous clause adjacent to the clause where the first anatomical position is located.

6. The method according to claim 5, It is characterized in that Determining whether there is a correlation between the first anatomical position entity and the sign entity includes: Obtaining the complete set of anatomical positions corresponding to the sign entity from the image knowledge base, determining whether the first anatomical position entity is included in the complete set of anatomical positions, and if so, determining that the first anatomical position is associated with the sign entity, and if not, determining that the first anatomical position entity is not associated with the sign entity; or, The imaging report and the identified sign entity and anatomical location entity are input into a large language model to obtain a judgment result of whether there is a correlation between the sign entity and the anatomical location entity.

7. The method according to claim 2, It is characterized in that Identifying the subsidiary information entities in each sentence based on the keywords corresponding to the subsidiary information entities in the image knowledge base includes: For any sentence in the imaging report, after identifying the sign entity and the associated anatomical location entity contained in the any sentence, when it is determined that the any sentence does not contain information matching the keyword corresponding to the subsidiary information entity, identifying whether the first adjacent sentence contains the subsidiary information entity based on the keyword corresponding to the subsidiary information entity, the first adjacent sentence being the previous sentence adjacent to the sentence where the sign entity is located; If the first adjacent sentence contains the first subsidiary information entity, determine whether there is a correlation between the first subsidiary information entity and the sign entity and the associated anatomical position entity. If there is a correlation, continue to identify whether the second adjacent sentence contains the subsidiary information entity until there is no correlation between the identified subsidiary information entity and the sign entity and the anatomical position entity, stop sentence traversal, and obtain the subsidiary information entity associated with the sign entity and the anatomical position entity. The second adjacent sentence is the previous sentence adjacent to the sentence where the first subsidiary information entity is located.

8. The method according to claim 7, It is characterized in that Determining whether there is a correlation between the first subsidiary information entity and the sign entity and the associated anatomical location entity includes: Associating the identified subsidiary information entity to the nearest neighboring sign entity and anatomical position of the clause where the subsidiary information entity is located; if the sign entity and anatomical position of the first adjacent clause contain at least one medical entity that is the same as the sign entity and anatomical position of the nearest neighboring clause, then continuing to transfer the association of the first subsidiary information entity to the previous clause until the sign entity and anatomical position entity corresponding to the previous clause are all different from the sign entity and anatomical position entity of the nearest neighboring clause, and determining that the subsidiary information entity is not associated with the sign entity and the anatomical position entity; or, The imaging report and the identified sign entity, anatomical location entity and auxiliary information entity are input into a large language model to obtain a judgment result of whether the auxiliary information entity is associated with the sign entity and the anatomical location entity.

9. The method according to claim 1, It is characterized in that The step of hierarchically merging medical entities in a cascade relationship in the same sentence includes: For the sign entities or anatomical position entities in the same sentence, based on the entity nodes with cascade relationship defined in the image knowledge base, redundant upper-level entity nodes are removed and the lowest-level entity nodes are retained.

10. The method according to claim 1, It is characterized in that For the medical entity information of all sentences in the imaging report, the repeated and redundant entity information between different sentences is removed to obtain the structured information corresponding to the imaging report, including: After traversing all the sentences in the image report, the structured information of all the sentences is merged, and the repeated and redundant entity information between the sentences is removed to obtain the structured information corresponding to the image report.

11. The method according to claim 1, It is characterized in that The process of creating the image knowledge base includes: Create an initial imaging knowledge base based on the medical entity information related to medical imaging in the pre-existing medical knowledge; Read the content of the current imaging report, and filter out the information in the imaging report that matches the medical entity information based on the medical entity information in the imaging knowledge base; When the medical entity information in the current image report is not fully covered by the image knowledge base, the information related to the medical entity is screened from the remaining report information after filtering the current image report by manual verification, and the screened information is defined according to the medical definition rules of the image knowledge base and then updated to the image knowledge base; When there is an unprocessed image report, the content of the next image report is read as the current image report until the medical entity information in the current image report is covered by the image knowledge base, and the image knowledge base is created.

12. An electronic device, It is characterized in that include: Processor and memory; The memory is used to store program code; The processor is used to run the program code so that the electronic device implements the knowledge-driven structured processing method for medical imaging reports as described in any one of claims 1 to 11.

13. A computer-readable storage medium, It is characterized in that Instructions are stored thereon, and when the instructions are executed on the electronic device, the electronic device executes the knowledge-driven structured processing method for medical imaging reports as described in any one of claims 1 to 11.

Images