A method and system for automatic generation of an image structured report template

Abstract

The application discloses an automatic generation method and system of an image structured report template, and comprises the following steps: step S1, extracting a qualitative entity from a natural language representing an image description of historical image report big data, and performing importance statistics on the qualitative entity to obtain an importance degree of the qualitative entity, and then selecting the qualitative entity as a quantitative entity according to the importance degree; step S2, constructing a structured image data table template based on the quantitative entity, and constructing a structured image description language template based on the quantitative entity and the image data table template. The application constructs a structured report template based on an organ tissue quantitatively described according to expert experience to realize quantitative description and qualitative description of medical images, and takes into account standardized and structured expression. Meanwhile, structuring the medical images in a data form is helpful to statistically manage the generated structured report, thereby improving the degree of automatic retrieval.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image reporting, and in particular to an automatic generation method and system of an image structured report template. BACKGROUND

[0002] Compared with the traditional report, the structured report is structured and standardized in content. The traditional report is usually written according to the personal habits of the doctor, and the content is complex and the words are various, so it is difficult to effectively extract the valuable information in the report, and it is impossible to manage and utilize the information. Not only is a large amount of valuable medical record information wasted, but also the omission and errors in report writing are easy to occur. When the era of big data and artificial intelligence comes, structured medical record information is the most basic data. Therefore, the popularization and application of structured report are imminent.

[0003] The prior art CN202110892281.6 discloses a prostate MR cancer structured report design method, which includes the following steps: logging into a user interface and providing a plurality of data options on the user interface; filling in the PI_RADS score of the prostate and inserting the PACS image in the structured report template, and then uploading to the database; the database receives the uploaded data, classifies and archives the key information; the database saves the received information; inputs the query data request in the user interface and sends it to the database; the database receives the request, matches and filters the data information in the request with the existing information in the database; the database returns all the data to be queried by the user and automatically generates a structured report form.

[0004] Although the above-mentioned prior art can generate a structured report, the generated structured report lacks image description expressed in natural language or still needs the doctor to input the image description, and the degree of automation is low. SUMMARY

[0005] The present application aims to provide an automatic generation method and system of an image structured report template, to solve the technical problem that the generated structured report in the prior art lacks image description expressed in natural language or still needs the doctor to input the image description, and the degree of automation is low.

[0006] To solve the above technical problems, the present application specifically provides the following technical solutions:

[0007] An automatic generation method of an image structured report template, comprising the following steps:

[0008] Step S1, qualitative entities are extracted from natural language of image description in historical image report big data, and importance of the qualitative entities is counted to obtain importance of the qualitative entities, and the qualitative entities are selected as quantitative entities according to the importance, the qualitative entities are organ tissues selected based on personal experience knowledge of doctors to describe pathological conditions in medical images, and the quantitative entities are organ tissues selected from the qualitative entities to describe pathological conditions in medical images according to experience knowledge of experts;

[0009] Step S2, a structured image data table template is constructed based on the quantitative entities, and a structured image description language template is constructed based on the quantitative entities and the image data table template, wherein,

[0010] The image data table template is used as a data table filled with image data of the quantitative entities to realize quantitative description of medical images, the image description language template is used as a language table filled with natural language of the quantitative entities to realize qualitative description of medical images, and the image data is obtained by AI evaluation of an AI-assisted quantitative evaluation system;

[0011] Step S3, a basic information filling template for filling personal information of a patient is extracted from a historical image report template, and an image filling template for filling medical images of the patient is extracted, and the basic information filling template, the image filling template, the image description language template and the image data table template are sequentially spliced to generate an image structured report template, wherein,

[0012] The basic information filling template, the image filling template, the image data table template and the image description language template are respectively mapped and linked with a patient basic information storage unit item, a medical image storage unit item, an image description storage unit item and an image data storage unit item in a patient image report database to realize automatic retrieval of the image structured report under a limited retrieval condition;

[0013] Step S4, a disease category is identified from medical images of a patient, a corresponding image structured report template is selected according to the disease category, and an image structured report of the patient is obtained by automatically filling the image structured report template with image data of the medical images of the patient to realize improvement of the automatic degree of generating a structured report of the patient.

[0014] As a preferred scheme of the present application, the extracting of the qualitative entities from natural language of image description in historical image report big data comprises:

[0015] A group of historical image reports are randomly selected from historical image report big data, and natural language representing organ tissues in the historical image reports are all extracted as qualitative entities.

[0016] quantifying the historical image report and the qualitative entity from the natural language form into the data vector form to obtain a natural language vector of the historical image report and a natural language vector of the qualitative entity, and corresponding labeling the natural language vector of the historical image report and the natural language vector of the qualitative entity as a single qualitative entity sample, wherein the single qualitative entity sample is represented as [the natural language vector of the historical image report, the natural language vector of the qualitative entity];

[0017] randomly dividing all the qualitative entity samples into two sets as a training set and a test set respectively, and using the training set and the test set to train an LSTM-crf model to obtain a qualitative entity extraction model, wherein the sample number of the training set and the sample number of the test set have a B:C ratio relationship, wherein C+B=1, C∈[0,1], B∈[0,1];

[0018] using the qualitative entity extraction model to extract entities in the historical image report big data to obtain all the qualitative entities involved in the historical image report big data.

[0019] As a preferred scheme of the present application, the importance statistics of the qualitative entity obtains the importance of the qualitative entity, and the qualitative entity is selected as a quantitative entity according to the importance, comprising:

[0020] statistically obtaining the total number of historical image reports of the historical image report big data, and statistically obtaining the total number of historical image reports of each qualitative entity, constructing the importance of each qualitative entity based on the ratio of the total number of historical image reports of each qualitative entity to the total number of historical image reports of the historical image report big data, and the calculation formula of the importance is:

[0021]

[0022] wherein, P i representing the importance of the i-th qualitative entity, m i representing the total number of historical image reports of the i-th qualitative entity, n representing the total number of historical image reports of the historical image report big data, and i is a constant of measurement without substantial meaning;

[0023] setting a selected threshold P, wherein,

[0024] when P i >P, the i-th qualitative entity is selected as a quantitative entity;

[0025] when P i ≤P, the i-th qualitative entity is not selected as a quantitative entity;

[0026] Preferably, the importance of the qualitative entity is determined by importance statistics, and the importance of the qualitative entity needs to be determined by excessive removal of the qualitative entity to ensure the uniqueness of each qualitative entity, wherein,

[0027] The similarity of any two qualitative entities is calculated, and one of the two qualitative entities with a similarity exceeding a similarity threshold is randomly removed until the similarity of any two qualitative entities does not exceed the similarity threshold;

[0028] The calculation formula of the similarity is:

[0029]

[0030] In the formula, I jk denotes the similarity value of the jth qualitative entity and the kth qualitative entity, A j , A k denote the natural language vector of the jth qualitative entity and the natural language vector of the kth qualitative entity, respectively, j and k are measurement constants and have no substantial meaning.

[0031] As a preferred scheme of the present application, the structured image data table template is constructed based on the quantitative entity, comprising:

[0032] The quantitative entity is taken as the first column header item of the table, a plurality of index items of the quantitative entity are taken as the first row header item of the table, and a two-dimensional table framework is constructed based on the first row header item and the first column header item as a data table template for quantitatively describing the index characteristics of the quantitative entity, wherein the characteristic item represents a pathological index of the quantitative entity;

[0033] The quantitative entity is taken as the second column header item of the table, a plurality of conclusion items of the quantitative entity are taken as the second row header item of the table, and a two-dimensional table framework is constructed based on the second row header item and the second column header item as a data table template for quantitatively describing the trend characteristics of the quantitative entity, wherein the conclusion item represents a pathological conclusion representing the development trend of the disease condition obtained from the pathological index of the quantitative entity;

[0034] The data table template of the index characteristics and the data table template of the trend characteristics are combined to form the image data table template.

[0035] As a preferred scheme of the present application, the structured image description language template is constructed based on the quantitative entity and the image data table template, comprising:

[0036] Natural language representing image description is extracted from a group of the historical image reports, and quantitative entities and image data are extracted from the natural language representing image description;

[0037] The natural language representing the image description and the quantitative entity and the image data extracted from the natural language representing the image description are quantified from the natural language form to the data vector form to obtain the natural language vector representing the image description, the natural language vector of the quantitative entity and the image data vector, and the natural language vector representing the image description, the natural language vector of the quantitative entity and the image data vector are correspondingly labeled as a single image description sample, wherein the single image description sample is represented as [the natural language vector of the quantitative entity, the image data vector, the natural language vector representing the image description];

[0038] All image description samples are randomly divided into two sets as a training set and a test set, and the training set and the test set are used for a data-to-seq model in a natural language processing technology to obtain an image description natural language generation model, wherein the sample number of the training set and the sample number of the test set have a B:C ratio relationship, wherein C+B=1, C∈[0,1], B∈[0,1];

[0039] The image description natural language generation model is used to generate the image description language template in the image data extracted from the quantitative entity and the image data table template.

[0040] As a preferred scheme of the present application, the image structured report template is generated by sequentially splicing the basic information filling template, the image filling template, the image description language template and the image data table template, comprising:

[0041] The four templates of the basic information filling template, the image filling template, the image description language template and the image data table template are randomly arranged and combined to form a plurality of splicing sequences;

[0042] A splicing sequence is randomly selected as the splicing sequence of the basic information filling template, the image filling template, the image description language template and the image data table template to constitute the image structured report template, and the image structured report template is generated by splicing according to the splicing sequence.

[0043] As a preferred scheme of the present application, the mapping link is used to correspondingly transmit all data items of the basic information filling template, the image filling template, the image data table template and the image description language template to the patient basic information storage unit item, the medical image storage unit item, the image description storage unit item and the image data storage unit item to realize the database storage of the image structured report of the patient.

[0044] As a preferred scheme of the present application, the automatic retrieval of the image structured report under the limited retrieval condition comprises:

[0045] The search condition is converted into a database search statement, and the search is performed on the patient image report database to obtain an image structured report of a patient meeting the search condition, thereby achieving statistical management of the image structured report.

[0046] As a preferred scheme of the present application, the medical image of the patient is identified to obtain a disease category, and a corresponding image structured report template is selected according to the disease category.

[0047] A multi-disease classifier is constructed based on a set of historical image reports, wherein,

[0048] The historical medical image of each historical image report in the set of historical image reports is extracted to obtain a set of historical medical images, and the disease category of each historical medical image is labeled, and the historical medical image and the disease category are labeled as a single disease identification sample, wherein the single disease identification sample is represented as [historical medical image, disease category];

[0049] All disease identification samples are randomly divided into two sets as a training set and a test set, and the training set and the test set are used to train an SVM model to obtain a multi-disease classifier, wherein the number of samples in the training set and the number of samples in the test set have a B:C ratio, wherein C+B=1, C∈[0,1], B∈[0,1];

[0050] The medical image of the patient is input into the multi-disease classifier, and the multi-disease classifier outputs a disease category corresponding to the medical image of the patient, and a corresponding image structured report template is selected according to the disease category.

[0051] As a preferred scheme of the present application, the present application provides a generation system of the automatic generation method of the image structured report template, comprising:

[0052] The entity recognition unit is configured to extract qualitative entities from natural language representing image descriptions in historical image report big data, and to obtain the importance of the qualitative entities by performing importance statistics on the qualitative entities, and to select the qualitative entities as quantitative entities according to the importance.

[0053] The template construction unit is configured to construct a basic information filling template, an image filling template, an image description language template, an image data table template, and an image structured report template.

[0054] The template application unit is configured to identify a disease category from the medical image of the patient, and to select a corresponding image structured report template according to the disease category, and to automatically fill the image structured report template with image data of the medical image of the patient to obtain a patient image structured report.

[0055] A database unit is configured to build an image report database of structured image reports, so as to complete statistical management of the structured image reports of patients.

[0056] Compared with the prior art, the present application has the following beneficial effects:

[0057] The present application extracts the qualitative description of organ tissues obtained by the personal experience of doctors from the historical image report big data in natural language expression into the quantitative description of organ tissues obtained by the expert experience, so as to construct the structured report template based on the quantitative description of organ tissues obtained by the expert experience to realize the quantitative description and the qualitative description of medical images, and to give consideration to the standardized and structured expression. Meanwhile, the structured characterization of medical images in the form of data is helpful to the statistical management of the generated structured report, so as to improve the degree of automatic retrieval. BRIEF DESCRIPTION OF DRAWINGS

[0058] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the following embodiment or prior art description will be briefly introduced. Obviously, the drawings in the following description are only exemplary, and other drawings can be obtained by the provided drawings without creative labor for those skilled in the art.

[0059] Figure 1 The flow chart of the automatic generation method of the image structured report template provided by the embodiment of the present application is shown in the figure;

[0060] Figure 2 The data table template schematic diagram of the index feature of the quantitative description quantitative entity provided by the embodiment of the present application is shown in the figure;

[0061] Figure 3 The data table template schematic diagram of the trend feature of the quantitative description quantitative entity provided by the embodiment of the present application is shown in the figure;

[0062] Figure 4.1 The basic information filling template schematic diagram provided by the embodiment of the present application is shown in the figure;

[0063] Figure 4.2 The image filling template schematic diagram provided by the embodiment of the present application is shown in the figure;

[0064] Figure 4.3 The image description language template schematic diagram provided by the embodiment of the present application is shown in the figure;

[0065] Figure 4.4 The image data table template schematic diagram provided by the embodiment of the present application is shown in the figure;

[0066] Figure 5 The structure block diagram of the generation system provided by the embodiment of the present application is shown in the figure.

[0067] The reference numerals in the drawings represent the following, respectively:

[0068] 1 - entity recognition unit; 2 - template construction unit; 3 - template application unit; 4 - database unit. DETAILED DESCRIPTION

[0069] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0070] As shown in Figure 1 The present application provides a method for automatically generating an image structured report template, comprising the following steps:

[0071] Step S1, extracting qualitative entities from the natural language representing image description in historical image report big data, and performing importance statistics on the qualitative entities to obtain the importance of the qualitative entities, and then selecting the qualitative entities as quantitative entities according to the importance, wherein the qualitative entities are organ tissues selected based on the personal experience knowledge of doctors to describe pathological conditions in medical images, and the quantitative entities are organ tissues selected from the qualitative entities to describe pathological conditions in medical images according to expert experience knowledge;

[0072] In the historical image report, only natural language is used to describe each organ tissue in the medical image. For example, in the CT image report of a COVID-19 patient, only qualitative description of the pathological symptoms of the lung tissue is present: "mixed ground glass shadow is visible in the outer 1 / 3 of the left lung upper lobe, the lesion area is less than 50%", "mixed ground glass shadow is visible in the outer 1 / 3 of the left lung lower lobe, the lesion area is less than 50%" and the like. These pathological information can only be qualitatively expressed in natural language, so that the pathological data cannot be truly grasped, and it is difficult to quantitatively manage in batches, such as: it is difficult to extract the image report of the patient with lung nodules between 5mm-10mm, and only manual search can be performed in all reports. Therefore, the present embodiment provides a method for automatically generating an image structured report template, so that the generated image structured report can quantitatively represent the image data of the organ tissue in the image, and also can qualitatively represent the image data of the organ tissue, realizing the unity of searchability, standardization and readability.

[0073] In the structured table form of the image data, first, determine which data needs to be quantitatively characterized, the embodiment provides a kind of extraction of expert experience in historical image report big data, so that from the image description of the personal experience of doctor in each historical image report, the organ tissue of personal attention of doctor is extracted, and the total number of attention of all organ tissues of personal attention extracted by doctor personal experience is counted, the organ tissue of attention of most doctors is as the organ tissue needing quantitative characterization, and the organ tissue of attention of most doctors, it is necessary to pay attention to the organ tissue in this disease category, it is important parameter for subsequent pathological analysis, and expert experience is a kind of universal experience summary, i.e. the organ tissue of attention of most doctors, it is necessary to pay attention to the organ tissue when generating image, it is beneficial to subsequent analysis pathology, finally, it can be considered that the organ tissue of universal attention needs to be quantitatively characterized to form a kind of expert experience, therefore, in the characterization image description natural language of historical image report big data, qualitative entity is extracted, including:

[0074] In historical image report big data, a group of historical image reports are randomly selected, and the natural language of the organ tissue in the historical image report is extracted as a qualitative entity;

[0075] The historical image report and the qualitative entity are quantified from natural language form to data vector form to obtain the natural language vector of the historical image report and the natural language vector of the qualitative entity, and the natural language vector of the historical image report and the natural language vector of the qualitative entity are correspondingly labeled as a single qualitative entity sample, wherein the single qualitative entity sample is characterized as [the natural language vector of the historical image report, the natural language vector of the qualitative entity];

[0076] All qualitative entity samples are randomly divided into two sets as training set and test set, and the training set and test set are used to train LSTM-crf model to obtain qualitative entity extraction model, wherein the sample number of training set and the sample number of test set have B:C ratio relationship, in the formula, C+B=1, C∈ [0,1], B∈ [0,1];

[0077] The qualitative entity extraction model is used for entity extraction in historical image report big data to obtain all qualitative entities involved in historical image report big data.

[0078] The importance of qualitative entity is statistically obtained, and the qualitative entity is selected as quantitative entity according to the importance, including:

[0079] The total number of historical image reports of the historical image report big data is counted, and the total number of historical image reports of each qualitative entity is counted. The importance of each qualitative entity is constructed based on the ratio of the total number of historical image reports of each qualitative entity to the total number of historical image reports of the historical image report big data. The calculation formula of the importance is:

[0080]

[0081] In the formula, P i The importance of the i-th qualitative entity is represented by m i The total number of historical image reports of the i-th qualitative entity is represented by n, the total number of historical image reports of the historical image report big data is represented by n, and i is a constant without substantial meaning.

[0082] The total number of historical image reports of the historical image report big data n is a constant value, and the total number of historical image reports of the i-th qualitative entity m i The greater the value is, the The smaller the value is, and The greater the value is, which means that the i-th qualitative entity is mentioned by more doctors in the historical image report, and the attention of the doctor is high. Therefore, the importance of the i-th qualitative entity is higher, and the probability of being a quantitative entity is higher. Conversely, the total number of historical image reports of the i-th qualitative entity m i The smaller the value is, the The greater the value is, and The smaller the value is, which means that the i-th qualitative entity is mentioned by fewer doctors in the historical image report, and the attention of the doctor is low. Therefore, the importance of the i-th qualitative entity is lower, and the probability of being a quantitative entity is higher, which conforms to the rules formulated by experts.

[0083] A threshold P is set, wherein,

[0084] When P i >P, the i-th qualitative entity is selected as a quantitative entity;

[0085] When P i ≤P, the i-th qualitative entity is not selected as a quantitative entity;

[0086] The qualitative entities mentioned in the historical image report of the new coronary pneumonia patient include: left lung, right lung, left upper lobe, right upper lobe, left lower lobe, right lower lobe, bronchus, and pleura. In actual calculation, only the importance of left lung, right lung, left upper lobe, right upper lobe, left lower lobe, and right lower lobe is higher than the selected threshold, so that left lung, right lung, left upper lobe, right upper lobe, left lower lobe, and right lower lobe are selected as quantitative entities.

[0087] Preferably, the importance of the qualitative entity obtained by statistically analyzing the importance of the qualitative entity needs to be removed to ensure the uniqueness of each qualitative entity, wherein,

[0088] The similarity of any two qualitative entities is calculated, and one of the two qualitative entities with a similarity exceeding a similarity threshold is randomly removed until the similarity of any two qualitative entities does not exceed the similarity threshold;

[0089] The calculation formula of the similarity is:

[0090]

[0091] In the formula, I jk denotes the similarity value of the jth qualitative entity and the kth qualitative entity, A j , A k denote the natural language vector of the jth qualitative entity and the natural language vector of the kth qualitative entity, respectively, j and k are measurement constants and have no substantial meaning.

[0092] The higher the similarity of the two qualitative entities, the more similar the two qualitative entities, that is, one of them can be randomly used for representation, so that the qualitative entity is retained while the redundant items in the entity sample type are removed, realizing the data simplification of one from many, and finally achieving the retention of sample diversity while removing sample redundancy.

[0093] In step S2, a structured image data table template is constructed based on the quantitative entity, and a structured image description language template is constructed based on the quantitative entity and the image data table template, wherein,

[0094] The image data table template is used as a data table filled with image data representing the quantitative entity to realize the standardized quantitative description of the medical image, and the image description language template is used as a language table filled with natural language representing the qualitative entity to realize the standardized qualitative description of the medical image, and the image data is obtained by AI evaluation of an AI-assisted quantitative evaluation system;

[0095] The structured image data table template is constructed based on the quantitative entity, including:

[0096] As shown in Figure 2 , the quantitative entity is taken as the first column header item of the table, the multiple index items of the quantitative entity are taken as the first row header item of the table, and a two-dimensional table framework is constructed based on the first row header item and the first column header item as a data table template for describing the index characteristics of the quantitative entity, the index characteristics refer to the basic pathological indexes representing the organ tissue, and the characteristic items represent the pathological indexes of the quantitative entity.

[0097] As shown in Figure 3As shown, the quantitative entity is taken as the second column header item of the table, a plurality of conclusion items of the quantitative entity are taken as the second row header items of the table, and a two-dimensional table framework is constructed based on the second row header items and the second column header items as a data table template for describing the trend characteristics of the quantitative entity, the trend characteristics being characteristics representing the pathological trend of the lesion, and the conclusion items representing the pathological conclusions representing the development trend of the disease state obtained from the pathological indicators of the quantitative entity.

[0098] The data table template of the index characteristics and the data table template of the trend characteristics are combined to form the image data table template.

[0099] When the quantitative entity is the left lung, the right lung, the left upper lobe, the right upper lobe, the left lower lobe, and the right lower lobe, the first column header item and the second column header item are the left lung, the right lung, the left upper lobe, the right upper lobe, the left lower lobe, and the right lower lobe, and the first row header item includes lung volume, volume of opacity, etc.; the second row header item includes MLD, SD, LAV, HAV, etc.

[0100] Based on the quantitative entity and the image data table template, a structured image description language template is constructed, including:

[0101] In a set of historical image reports, natural language representing image descriptions is extracted, and quantitative entities and image data are extracted from the natural language representing image descriptions;

[0102] The natural language representing image descriptions, the natural language vectors of the quantitative entities, and the image data vectors are quantified from the natural language form to the data vector form to obtain the natural language vectors representing image descriptions, the natural language vectors of the quantitative entities, and the image data vectors, and the natural language vectors representing image descriptions, the natural language vectors of the quantitative entities, and the image data vectors are correspondingly labeled as a single image description sample, wherein the single image description sample is represented as [the natural language vectors of the quantitative entities, the image data vectors, the natural language vectors representing image descriptions];

[0103] All image description samples are randomly divided into two sets as a training set and a test set, and the training set and the test set are used for a data-to-seq model in natural language processing technology to obtain an image description natural language generation model, wherein the number of samples of the training set and the number of samples of the test set have a B:C ratio relationship, wherein C+B=1, C∈[0,1], and B∈[0,1];

[0104] The image description natural language generation model is used to generate an image description language template in the image data extracted from the quantitative entity and the image data table template.

[0105] The image description natural language generation model obtained by the above training can generate image descriptions represented by natural language according to the quantitative entity and the image data in the image book table. The image descriptions can be automatically generated without manual input by doctors, thereby improving the degree of automation and the readability of the structured report, rather than only having structured image data that is difficult to obtain a readable description.

[0106] As shown in Figures 4.1-4.4 Step S3, the basic information filling template for filling the patient's personal information and the image filling template for filling the patient's medical image are extracted from the historical image report template. The basic information filling template, the image filling template, the image description language template, and the image data table template are sequentially spliced to generate an image structured report template, wherein,

[0107] The basic information filling template, the image filling template, the image data table template, and the image description language template are respectively mapped and linked with the patient basic information storage unit item, the medical image storage unit item, the image description storage unit item, and the image data storage unit item in the patient image report database to realize automatic retrieval of the image structured report under a limited retrieval condition.

[0108] The basic information filling template, the image filling template, the image description language template, and the image data table template are sequentially spliced to generate an image structured report template, comprising:

[0109] The four templates of the basic information filling template, the image filling template, the image description language template, and the image data table template are randomly arranged and combined to form a plurality of splicing sequences.

[0110] A splicing sequence is randomly selected as the splicing sequence of the basic information filling template, the image filling template, the image description language template, and the image data table template to constitute the image structured report template, and the image structured report template is spliced according to the splicing sequence. The splicing sequence can be customized by the user.

[0111] The mapping link is used to transmit all data items of the basic information filling template, the image filling template, the image data table template, and the image description language template to the patient basic information storage unit item, the medical image storage unit item, the image description storage unit item, and the image data storage unit item to realize database storage of the image structured report of the patient.

[0112] The automatic retrieval of the image structured report under the limited retrieval condition comprises:

[0113] The search condition is converted into a database search statement, and the search is performed on the patient image report database to obtain the image structured report of the patient meeting the search condition, so as to realize statistical management of the image structured report, such as screening the image report of all patients with a lung nodule of 5-10 mm in the patient image report database, and counting the patient recovery rate, and the like, so as to easily complete various searches and statistical management.

[0114] In step S4, the medical image of the patient is identified to obtain a disease category, a corresponding image structured report template is selected according to the disease category, and the image structured report of the patient is obtained by automatically filling the image structured report template according to the image data of the medical image of the patient, so as to improve the automation degree of generating the structured report of the patient.

[0115] The medical image of the patient is identified to obtain a disease category, and a corresponding image structured report template is selected according to the disease category, including:

[0116] A multi-disease classifier is constructed based on a set of historical image reports, wherein,

[0117] The historical medical image of each historical image report in the set of historical image reports is extracted to obtain a set of historical medical images, and the disease category of the set of historical medical images is labeled respectively, and the historical medical image and the disease category are labeled as a single disease recognition sample, wherein the single disease recognition sample is represented as [historical medical image, disease category];

[0118] All disease recognition samples are randomly divided into two sets as a training set and a test set, and the training set and the test set are used to train an SVM model to obtain a multi-disease classifier, wherein the sample number of the training set and the sample number of the test set have a B:C ratio relationship, wherein C+B=1, C∈[0,1], B∈[0,1];

[0119] The medical image of the patient is input into the multi-disease classifier, the multi-disease classifier outputs the disease category corresponding to the medical image of the patient, and a corresponding image structured report template is selected according to the disease category.

[0120] The multi-disease classifier can realize automatic identification of the disease category of the medical image of the patient, so as to match the image structured report template of the disease category, and further improve the automation efficiency.

[0121] As shown in Figure 5 Based on the above-mentioned automatic generation method of the image structured report template, the application provides a generation system, which comprises:

[0122] The entity recognition unit 1 is used for extracting qualitative entities from natural language of the represented image description of the historical image report big data, and performing importance statistics on the qualitative entities to obtain importance of the qualitative entities, and selecting the qualitative entities as quantitative entities according to the importance;

[0123] The template construction unit 2 is used for constructing a basic information filling template, an image filling template, an image description language template, an image data table template, and an image structured report template.

[0124] The template application unit 3 is used for identifying a disease category of a medical image of a patient, selecting a corresponding image structured report template according to the disease category, and automatically filling the image structured report template with image data of the medical image of the patient to obtain a patient image structured report.

[0125] The database unit 4 is used for constructing an image report database of the image structured report to complete statistical management of the patient image structured report.

[0126] The present application extracts qualitative description of organ tissues obtained by personal experience of doctors from historical image report big data in natural language expression into quantitative description of organ tissues obtained by expert experience, thereby constructing a structured report template based on the quantitative description of organ tissues obtained by expert experience to realize quantitative description and qualitative description of medical images, and taking into account standardized and structured expression.

[0127] The above examples are only exemplary embodiments of the present application, and are not used to limit the present application, and the protection scope of the present application is defined by the claims. Those skilled in the art can make various modifications or equivalent replacements to the present application within the spirit and protection scope of the present application, and such modifications or equivalent replacements are also regarded as falling within the protection scope of the present application.

Claims

1. A method for automatically generating an image structured reporting template, characterized in that, Comprise the following steps: Step S1, extracting qualitative entities in the natural language of the historical image report big data representing image description, and performing importance statistics on the qualitative entities to obtain the importance of the qualitative entities, and then selecting the qualitative entities as quantitative entities according to the importance, wherein the qualitative entities are organ tissues selected based on the personal experience knowledge of doctors to describe pathological conditions in medical images, and the quantitative entities are organ tissues selected based on expert experience knowledge to describe pathological conditions in medical images; Step S2, constructing a structured image data table template based on the quantitative entities, and constructing a structured image description language template based on the quantitative entities and the image data table template, wherein, The image data table template is used as a data table filled with image data representing the quantitative entities to realize the standardized quantitative description of medical images, and the image description language template is used as a language table filled with natural language representing the quantitative entities to realize the standardized qualitative description of medical images, and the image data is obtained by AI evaluation of an AI-assisted quantitative evaluation system; Step S3, extracting a basic information filling template for filling patient personal information and an image filling template for filling patient medical images in the historical image report template, and sequentially splicing the basic information filling template, the image filling template, the image description language template and the image data table template to generate an image structured report template, wherein, The basic information filling template, the image filling template, the image data table template and the image description language template are respectively mapped and linked with the patient basic information storage unit item, the medical image storage unit item, the image description storage unit item and the image data storage unit item in the patient image report database to realize automatic retrieval of the image structured report under a limited retrieval condition; Step S4, identifying the disease category of the patient's medical image, selecting the corresponding image structured report template according to the disease category, and automatically filling the image data of the patient's medical image into the image structured report template to obtain the patient's image structured report, thereby improving the automation degree of generating the patient's structured report.

2. The method of claim 1, wherein: The qualitative entities are extracted from the natural language representing image description in the historical image report big data, comprising: Randomly selecting a group of historical image reports from the historical image report big data, and extracting the natural language representing organ tissues in the historical image reports as qualitative entities; Quantifying the historical image reports and the qualitative entities from natural language form to data vector form to obtain the natural language vector of the historical image report and the natural language vector of the qualitative entity, and labeling the natural language vector of the historical image report and the natural language vector of the qualitative entity as a single qualitative entity sample, wherein the single qualitative entity sample is represented as [natural language vector of the historical image report, natural language vector of the qualitative entity]; All qualitative entity samples are randomly divided into two sets as a training set and a test set respectively, and the training set and the test set are used to train the LSTM-crf model to obtain a qualitative entity extraction model, wherein the sample number of the training set and the sample number of the test set have a B:C ratio relationship, wherein C+B=1, , ; The qualitative entity extraction model is used to extract entities in the historical image report big data to obtain all the qualitative entities involved in the historical image report big data.

3. The method of claim 2, wherein: The importance statistics of the qualitative entities obtains the importance of the qualitative entities, and a qualitative entity is selected as a quantitative entity according to the importance, including: The total number of historical image reports in the historical image report big data is counted, and the total number of historical image reports in which each qualitative entity appears is counted, and the importance of each qualitative entity is constructed based on the ratio of the total number of historical image reports in which each qualitative entity appears to the total number of historical image reports in the historical image report big data, and the calculation formula of the importance is: ； In the formula, the importance degree of the i-th qualitative entity, the total number of historical image reports representing the i-th qualitative entity, n represents the total number of historical image reports of the historical image report big data, and i is a constant of measurement without substantial meaning. A selection threshold P is set, wherein, When then the i-th qualitative entity is selected as a quantitative entity; When then the i-th qualitative entity is not selected as a quantitative entity; The importance statistics of the qualitative entities obtains the importance of the qualitative entities, and a qualitative entity is selected as a quantitative entity according to the importance, including: The similarity of any two qualitative entities is calculated, and one of the two qualitative entities whose similarity exceeds a similarity threshold is randomly removed until the similarity of any two qualitative entities does not exceed the similarity threshold; The calculation formula of the similarity is: ； In the formula, characterizing a similarity value of the jth qualitative entity and the kth qualitative entity, , respectively characterizing a natural language vector of the jth qualitative entity and a natural language vector of the kth qualitative entity, j, k are constant of measurement, without substantial meaning.

4. The method of claim 1, wherein: The structured image data table template is constructed based on the quantitative entity, including: The quantitative entity is taken as a first column header item of the table, a plurality of index items of the quantitative entity are taken as a first row header item of the table, and a two-dimensional table framework is constructed based on the first row header item and the first column header item as a data table template for quantitatively describing the index characteristics of the quantitative entity, and the characteristic item represents a pathological index of the quantitative entity; The quantitative entity is taken as a second column header item of the table, a plurality of conclusion items of the quantitative entity are taken as a second row header item of the table, and a two-dimensional table framework is constructed based on the second row header item and the second column header item as a data table template for quantitatively describing the trend characteristics of the quantitative entity, and the conclusion item represents a pathological conclusion representing the development trend of the disease represented by the pathological index of the quantitative entity; The data table template of the index characteristics and the data table template of the trend characteristics are combined to constitute the image data table template.

5. The method of claim 4, wherein: The structured image description language template is constructed based on the quantitative entity and the image data table template, including: Natural language representing image description is extracted from a group of the historical image reports, and quantitative entities and image data are extracted from the natural language representing image description; The natural language representing image description, the natural language vector of the quantitative entity, and the image data vector are quantified from the natural language form to the data vector form to obtain a natural language vector representing image description, and the natural language vector of the quantitative entity and the image data vector are correspondingly labeled as a single image description sample, wherein the single image description sample is represented as [the natural language vector of the quantitative entity, the image data vector, the natural language vector representing image description]; All image description samples are randomly divided into two sets as a training set and a test set respectively, and the training set and the test set are used for a data-to-seq model in a natural language processing technology to obtain an image description natural language generation model, wherein the sample number of the training set and the sample number of the test set have a B:C ratio relationship, wherein C+B=1, , ; The image description natural language generation model is used to generate the image description language template in the image data extracted from the quantitative entity and the image data table template.

6. The method of claim 1, wherein: The filling template of basic information, the filling template of image, the template of image description language and the template of image data table are sequentially spliced to generate the image structured report template, comprising: The four templates of the filling template of basic information, the filling template of image, the template of image description language and the template of image data table are randomly arranged and combined to form multiple splicing sequences; A splicing sequence is randomly selected as the splicing sequence of the filling template of basic information, the filling template of image, the template of image description language and the template of image data table to constitute the image structured report template, and the splicing is performed according to the splicing sequence to generate the image structured report template.

7. The method for automatically generating an image structured report template according to claim 1, characterized in that, The mapping link is used for transmitting all data items of the filling template of basic information, the filling template of image, the template of image data table and the template of image description language to the patient basic information storage unit item, the medical image storage unit item, the image description storage unit item and the image data storage unit item to realize the database storage of the image structured report of the patient.

8. The method of claim 1, wherein, The automatic retrieval of the image structured report under the defined retrieval condition, comprising: The search defined search condition is converted into a database retrieval statement and input into the patient image report database to search to obtain the image structured report of the patient meeting the defined search condition, so as to realize the statistical management of the image structured report.

9. The method of claim 1, wherein, The medical image of the patient is identified to obtain the disease category, and the corresponding image structured report template is selected according to the disease category, comprising: A multi-disease classifier is constructed based on a group of historical image reports, wherein, A group of historical medical images are extracted from the historical medical image of each historical image report in a group of historical image reports, and the disease category of the group of historical medical images is labeled, and the historical medical image and the disease category are labeled as a single disease identification sample, wherein the single disease identification sample is represented as [historical medical image, disease category]; All disease recognition samples are randomly divided into two sets as a training set and a test set respectively, and the training set and the test set are used to train the SVM model to obtain a multi-disease classifier, wherein the sample number of the training set and the sample number of the test set have a B:C ratio relationship, wherein C+B=1, , ; The medical image of the patient is input into the multi-disease classifier, and the multi-disease classifier outputs the disease category corresponding to the medical image of the patient, and the corresponding image structured report template is selected according to the disease category.

10. A generating system of a method of automatic generation of a structured report of images according to any one of claims 1 to 9, characterized in that, Comprising: An entity recognition unit (1) is used for extracting qualitative entities from the natural language of the image description of the historical image report big data, and the importance of the qualitative entities is obtained by importance statistics, and the qualitative entities are selected as quantitative entities according to the importance; A template construction unit (2) is used for constructing the filling template of basic information, the filling template of image, the template of image description language, the template of image data table, and the template of image structured report; A template application unit (3) is used for identifying the disease category of the medical image of the patient, selecting the corresponding image structured report template according to the disease category, and automatically filling the image data of the medical image of the patient into the image structured report template to obtain the image structured report of the patient; A database unit (4) is used for constructing the image report database of the image structured report to complete the statistical management of the image structured report of the patient.

Images