Medical report generation method and device, storage medium and electronic equipment

Abstract

The invention discloses a medical report generation method and device, a storage medium and electronic equipment, and the method comprises the steps: obtaining patient information, inputting the patient information into a report generation model, obtaining diagnosis features, determining historical diagnosis information related to the patient information in a historical diagnosis record as associated information according to the diagnosis features, and generating a medical report according to the associated information. And determining prompt information, inputting the prompt information and the diagnosis features into a report generation model, and generating a medical report. According to the method, a doctor only needs to input the patient information into the report generation model, the medical report generated by the model can be obtained, the doctor does not need to manually write the medical report, and the determination efficiency of the medical report is improved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method, device, storage medium and electronic device for generating a medical report. Background Art

[0002] Usually, doctors need to write medical reports during the diagnosis and treatment of patients. When writing medical reports, doctors need to make a comprehensive diagnosis of patients based on their experience, combined with the patient's condition, medical test results, imaging examination results, etc., and organize language based on the diagnosis results to complete the writing of medical reports.

[0003] It can be seen that the process of writing a medical report is relatively cumbersome, which requires doctors to spend a lot of energy and time, and reduces the efficiency of doctors' diagnosis and treatment. Therefore, this specification provides a method for generating a medical report. Summary of the invention

[0004] The present specification provides a medical report generation method, device, storage medium and electronic device to at least partially solve the above-mentioned problems existing in the prior art.

[0005] This manual adopts the following technical solutions:

[0006] This specification provides a method for generating a medical report, comprising:

[0007] Acquiring patient information, and inputting the patient information into a report generation model to obtain diagnostic features;

[0008] According to the diagnosis feature, determining historical diagnosis information related to the patient information in the historical diagnosis record as associated information;

[0009] Prompt information is determined based on the associated information, and the prompt information and the diagnostic features are input into the report generation model to generate a medical report.

[0010] Optionally, the report generation model includes an image encoding layer and a text encoding layer, and the patient information includes image data and text data;

[0011] The patient information is input into the report generation model to obtain diagnostic features, including:

[0012] Inputting the image data into the image coding layer to obtain image diagnostic features, and inputting the text data into the text coding layer to obtain text diagnostic features;

[0013] A diagnostic feature is determined according to the image diagnostic feature and the text diagnostic feature.

[0014] Optionally, according to the diagnosis feature, historical diagnosis information related to the patient information is determined in the historical diagnosis record as associated information, specifically including:

[0015] Determining the coding features of each piece of historical diagnosis information contained in the historical diagnosis record;

[0016] Determining the similarity between the coding feature of each historical diagnosis information and the diagnosis feature respectively;

[0017] The encoding features of the historical diagnosis information with a similarity greater than or equal to a preset value are used as the associated information.

[0018] Optionally, determining prompt information according to the association information, inputting the prompt information and the diagnostic features into the report generation model, and generating a medical report specifically includes:

[0019] Acquire the preceding information of the medical report to be generated input by the user, and use the preceding information and the associated information as prompt information;

[0020] Inputting the prompt information and the diagnostic features into the report generation model, and predicting the subsequent content of the preceding information through the report generation model to obtain subsequent information of the preceding information;

[0021] A medical report is determined based on the preceding information and the following information.

[0022] Optionally, the subsequent information consists of a plurality of pieces of information to be selected;

[0023] Determine a medical report based on the above information and the following information, including:

[0024] In response to the user's selection operation on each piece of candidate information, determining the candidate information selected by the user;

[0025] A medical report is determined based on the preceding information and the candidate information selected by the user.

[0026] Optionally, the medical report is a structured report;

[0027] Determining prompt information according to the association information, inputting the prompt information and the diagnostic features into the report generation model, and generating a medical report, specifically includes:

[0028] Determine a task prompt according to the user operation, wherein the task prompt is used to characterize the report structured task;

[0029] Using the task prompt and the associated information as prompt information;

[0030] The prompt information and the diagnostic features are input into the report generation model to obtain the medical report.

[0031] Optionally, inputting the prompt information and the diagnostic features into the report generation model to obtain the medical report specifically includes:

[0032] Finalize report templates and initial diagnostic reports;

[0033] Obtaining each diagnostic item in the report template, and determining a diagnostic text corresponding to the diagnostic item based on the initial diagnostic report;

[0034] Based on the diagnosis text, determining the diagnosis conclusion of the diagnosis item;

[0035] A medical report is determined based on the diagnostic conclusion.

[0036] Optionally, based on the diagnosis text, determining the diagnosis conclusion of the diagnosis item specifically includes:

[0037] When it is determined that the diagnosis text corresponding to the diagnosis item indicates that an abnormality exists, determining that the judgment result of the diagnosis item is abnormal;

[0038] The diagnosis text of the diagnosis item and the judgment result of the diagnosis item are used as the diagnosis conclusion of the diagnosis item.

[0039] The present specification provides a computer-readable storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned medical report generation method is implemented.

[0040] The present specification provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned medical report generating method when executing the program.

[0041] At least one of the above technical solutions adopted in this specification can achieve the following beneficial effects:

[0042] In the medical report generation method provided in this specification, patient information is obtained, the patient information is input into a report generation model, and diagnostic features are obtained. According to the diagnostic features, historical diagnostic information related to the patient information is determined in the historical diagnostic records as associated information. According to the associated information, prompt information is determined, and the prompt information and the diagnostic features are input into the report generation model to generate a medical report. In this method, the doctor only needs to input the patient information into the report generation model to obtain the medical report generated by the model, and the doctor no longer needs to manually write the medical report, which improves the efficiency of determining the medical report. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The drawings described herein are used to provide a further understanding of this specification and constitute a part of this specification. The illustrative embodiments and descriptions of this specification are used to explain this specification and do not constitute an improper limitation on this specification. In the drawings:

[0044] Figure 1 A schematic diagram of a process for generating a medical report in this specification;

[0045] Figure 2 A completion task effect diagram provided in an embodiment of this specification;

[0046] Figure 3 A diagram of a generation process of a structured report provided in this embodiment;

[0047] Figure 4 A method corresponding to Figure 3 Structured report renderings;

[0048] Figure 5 A method corresponding to Figure 3 Structured report renderings;

[0049] Figure 6 A diagram of a medical report generation process provided in an embodiment of this specification;

[0050] Figure 7 A structural diagram of a report generation model provided in an embodiment of this specification;

[0051] Figure 8 A structural diagram of a report generation model provided in an embodiment of this specification;

[0052] Fig. 9 A schematic diagram of a medical report generating device provided in this specification;

[0053] Fig.10 The corresponding Figure 1 Schematic diagram of electronic equipment. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solutions and advantages of this specification more clear, the technical solutions of this specification will be clearly and completely described below in combination with the specific embodiments of this specification and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0055] The technical solutions provided by the embodiments of this specification are described in detail below in conjunction with the accompanying drawings.

[0056] Figure 1 The following is a flow chart of a method for generating a medical report in this specification, which specifically includes the following steps:

[0057] S100: Acquire patient information, input the patient information into a report generation model, and obtain diagnostic features.

[0058] All steps in the medical report generation method provided in this specification can be implemented by any electronic device with computing functions, such as a terminal, a server, etc. For ease of description, the medical report generation method provided in this specification is described below with the server as the execution subject.

[0059] In this specification, the patient medical report is generated by the report generation model. When the doctor needs to determine the patient's medical report, the patient information obtained during the diagnosis and treatment process is input into the report generation model, and the server obtains the patient's diagnostic characteristics through the report generation model.

[0060] This specification does not limit the specific content of patient information. For example, patient information can be text data such as basic patient information, medical history, and chief complaint, or image data of any type of medical image such as CT images, MRI images, and DR images, or voice data for generating medical reports input by doctors during the diagnosis process, structured table data, etc. This specification does not limit the data type of patient information, and it can contain only one type of data or multiple types of data at the same time.

[0061] The report generation model may include one or more coding layers for feature coding, and different coding layers are used to encode features of different types of patient information. This specification does not limit the specific network structure of the coding layer, and the network structures of different coding layers may be the same or different.

[0062] The specific coding layer included in the report generation model can be set according to the actual application scenario. By configuring different coding layers for the report generation model, the report generation model can be applied to different application scenarios.

[0063] For example, in one application scenario, the patient information includes text data, and the report generation model can be configured to include a text encoding layer, and the server inputs the text data into the text encoding layer to obtain text diagnostic features, and uses the text diagnostic features as diagnostic features. In another application scenario, the patient information includes image data, and the report generation model can be configured to include an image encoding layer, and the server inputs the image data into the image encoding layer to obtain image diagnostic features, and uses the image diagnostic features as diagnostic features.

[0064] S102: According to the diagnosis feature, determine historical diagnosis information related to the patient information in the historical diagnosis record as associated information.

[0065] The historical diagnosis records in the hospital database contain the historical diagnosis information of each patient saved in the historical diagnosis process. When a medical report needs to be generated for the current patient, the server can search the historical diagnosis records to determine the historical diagnosis information related to the patient information as the associated information.

[0066] A historical diagnosis information includes the patient information and the corresponding medical report in a historical diagnosis process. The historical diagnosis information can be the medical report, examination results, medical images, etc. in the patient's historical diagnosis process. Therefore, the historical diagnosis information can be composed of information of various data types such as text data and image data. The server inputs the historical diagnosis information into the coding layer of the corresponding data type in the report generation model in advance to obtain the coding features of the historical diagnosis information.

[0067] After the server obtains the diagnostic features of the current patient information, it determines the similarity between the coding features of each historical diagnostic information and the diagnostic features determined in step S100, and uses the coding features of the historical diagnostic information with a similarity greater than or equal to a preset value as the associated information.

[0068] The associated information represents the retrieved historical patient information similar to the patient information, or medical cases related to the patient information, so as to serve as prompt information in the process of generating a medical report, thereby guiding the report generation model to generate a more accurate medical report.

[0069] S104: Determine prompt information according to the associated information, input the prompt information and the diagnostic features into the report generation model, and generate a medical report.

[0070] The report generation model in this specification includes a decoding layer, which can be configured as a large language model of any structure, and is used to generate a medical report based on diagnostic features and associated information.

[0071] The server determines the prompt information based on the associated information, inputs the prompt information and the diagnostic features into the report generation model, and obtains the medical report output by the decoding layer of the report generation model.

[0072] The medical report generation method in this specification can generate reports based on patient information of various data types, and is applicable to more diverse medical diagnosis scenarios. In addition, this method uses the associated information related to the patient information to guide the generation of the report to improve the accuracy of the medical report, and can determine a medical report that is more consistent with the disease or abnormality represented by the patient information, thereby improving the doctor's diagnostic efficiency and ensuring the accuracy of the generated medical report.

[0073] In the above step S100, when the patient information contains data of multiple data types, diagnostic features of each type of data are obtained through different coding layers.

[0074] In one or more embodiments of the present specification, the patient information includes image data and text data, and the report generation model may correspondingly include an image encoding layer and a text encoding layer.

[0075] The server inputs the image data into the image coding layer to obtain the image diagnostic features, and inputs the text data into the text coding layer to obtain the text diagnostic features. Then, the server determines the diagnostic features corresponding to the patient information based on the image diagnostic features and the text diagnostic features.

[0076] Specifically, the server may fuse the image diagnostic features and the text diagnostic features through a neural network to determine the diagnostic features. Alternatively, the image diagnostic features and the text diagnostic features may be spliced ​​to determine the diagnostic features.

[0077] Because the diagnostic features in this embodiment are determined based on image diagnostic features and text diagnostic features, the diagnostic features in this embodiment simultaneously contain the semantics of image-type patient data and text-type patient data, and more completely represent the semantics of patient information through the semantic dimensions of multiple data types, thereby enabling the report generation model to determine a more accurate medical report based on the diagnostic features.

[0078] The report generation model in this specification can be used for auto-completion tasks.

[0079] In the automatic completion task, the prompt information in the above step S104 may also include previous information input by the user. The previous information may be input by text or voice by the user, and is used to guide the report generation model to predict the subsequent information of the previous information, and output a medical report that matches the previous information, thereby improving the accuracy of the medical report. The user in this specification refers to medical personnel, medical researchers, medical data analysts and other personnel who have the need to obtain medical reports.

[0080] Specifically, the server obtains the pre-text information of the medical report to be generated input by the user, and uses the pre-text information and the associated information determined in step S102 as prompt information, and inputs the prompt information and the diagnostic features into the decoding layer of the report generation model, so that the decoding layer of the report generation model predicts the subsequent content of the pre-text information to obtain the post-text information of the pre-text information. Finally, the server obtains the medical report according to the pre-text information input by the user and the post-text information predicted by the report generation model.

[0081] In this embodiment, the previous information input by the user is a small part of the content in the medical report. The subsequent information of the previous information is predicted by the automatic completion task, which reduces the user's input amount, improves the accuracy of the medical report, and ensures the efficiency of medical report generation.

[0082] In order to further increase the accuracy of the medical report obtained by the report generation model in the automatic completion task, in one or more embodiments of the present specification, the subsequent information generated by the report generation model consists of multiple pieces of information to be selected.

[0083] The report generation model predicts the previous information and obtains multiple pieces of candidate information. The server displays each piece of candidate information to the user. In response to the user's selection operation on each piece of candidate information, the server determines the candidate information selected by the user. Then, the server determines the medical report based on the candidate information selected by the user and the previous information. This specification does not limit the specific number of candidate information generated, which can be set according to actual needs.

[0084] Figure 2 This is a completion task effect diagram provided in the embodiments of this specification, such as Figure 2 As shown in the figure, when the user inputs the preceding information "right upper lobe", the server automatically predicts the following information of the preceding information and obtains five pieces of candidate information, namely "spotted high-density shadow", "strip-like high-density shadow", "strip-like increased density shadow", "multiple solid nodule shadows", and "multiple nodules". The user can select one of the candidate information according to the actual diagnosis of the patient.

[0085] The automatic completion task in this specification is a dynamic process. The server obtains the previous information that the user continues to input based on the previous information that has been input in real time, determines the updated previous information, and regenerates the candidate information based on the updated previous information. Therefore, as the previous information input by the user changes, the server automatically updates the prediction of the subsequent information to generate a medical report that better meets the user's intention.

[0086] For example, if the user Figure 2 Continue to input the previous information based on the input, and get the previous information "right upper lobe of the lung", then the server will automatically Figure 2The five pieces of information to be selected are updated to two: "cord-like high-density shadow" and "flaky high-density shadow", which improves the accuracy of the generated medical report.

[0087] In one or more embodiments of the present specification, in order to meet the diverse needs of medical report generation, the server may, based on the user's task prompts and through a report generation model, perform various types of medical report generation tasks, such as image finding generation tasks, report generation tasks, and report structuring tasks.

[0088] When the server performs the task of generating at least one type of medical report, it needs to obtain the task prompt from the user. The server can obtain the task prompt in a variety of ways, such as determining the task prompt in response to the user's operation on the graphical interface, determining according to the user's text or voice input, etc.

[0089] When the server determines a task prompt in response to a user's operation on a graphical interface, the graphical interface may include interface elements corresponding to an image finding generation task, a report generation task, and a report structuring task. In response to a user's selection of a certain interface element, the server determines a preset keyword of a medical report generation task corresponding to the interface element as a task prompt. When the server determines a task prompt based on a user's voice or text input, the server directly uses the user's text or voice input as a task prompt.

[0090] After determining the task prompt, the server uses the task prompt and associated information as prompt information, and inputs the prompt information and diagnostic features into a report generation model, so that the report generation model generates a medical report that matches the medical report generation task corresponding to the task prompt.

[0091] In one or more embodiments of the present specification, in order to obtain a more accurate medical report, the server may also obtain input information matching the medical report generation task corresponding to the task prompt.

[0092] Different types of medical report generation tasks require different input information.

[0093] In the image findings generation task, the task prompt is used to characterize the image findings generation task, and the input information can be the previous information of the image findings to be generated. The server uses the determined task prompt, the previous information of the image findings to be generated, and the associated information as prompt information, and inputs the prompt information and the diagnostic features into the report generation model to generate the subsequent information of the previous information of the image findings to be generated, and obtain the image findings. Then, the server displays the generated image findings to the user as a medical report. That is, in this embodiment, the server determines the medical report in the form of the above-mentioned automatic completion task.

[0094] In the report generation task, the task prompt is used to characterize the report generation task. The input information can be what is seen in the image or the previous information of the initial diagnosis report. The initial diagnosis report is an unstructured medical report.

[0095] When the input information is what is seen in the image, the server will determine the task prompt, what is seen in the image, and the related information as prompt information. Because in this embodiment, what is seen in the image is used as part of the prompt information, the report generation model can generate an initial diagnosis report based on what is seen in the image. When the input information is the preceding information of the initial diagnosis report, the server predicts the following information of the preceding information of the unstructured medical report in the form of an automatic completion task to obtain an initial diagnosis report. After the server determines the initial diagnosis report, it displays the initial diagnosis report to the user as a medical report.

[0096] The medical report obtained by executing the report structuring task is a structured medical report classified according to human body parts or diseases. In order to obtain a structured medical report, the input information corresponding to the task includes a report template.

[0097] The report template contains multiple preset diagnostic items. This manual does not limit the classification criteria of the diagnostic items, and the classification criteria can be determined according to symptoms, diseases, body parts, etc. For example, a diagnostic item classified by symptoms can include lung shadows, lung nodules, lung consolidation, lung masses, etc.

[0098] The report template may be a report template selected by the user from a plurality of report templates pre-stored in the server, or may be a report template customized by the user.

[0099] In the report structuring task, the task prompt is used to characterize the report structuring task, and the input information can be the initial diagnosis report or the previous information in the structured medical report.

[0100] When the input information is the preceding information in a structured medical report, the server predicts the following information of the preceding information in the structured medical report in the form of an automatic completion task to obtain a medical report.

[0101] When the input information of the report structuring task is the initial diagnosis report.

[0102] First, the server determines a report template and an initial diagnosis report, obtains each diagnosis item in the report template, and determines the diagnosis text corresponding to each diagnosis item based on the initial diagnosis report. The diagnosis text is a specific diagnostic description of the diagnosis item.

[0103] Then, the server determines the diagnosis conclusion of each diagnostic item based on the diagnosis text of each diagnostic item and obtains a medical report.

[0104] Specifically, the server adds the diagnosis conclusion of each diagnosis item to the position corresponding to each diagnosis item in the report template. Finally, the medical report is determined according to the diagnosis conclusion of each diagnosis item added in the report template.

[0105] In this embodiment, there are multiple report formats of the structured medical report in this specification to adapt to the diverse medical report generation requirements.

[0106] In an example of a report format, the server can use the diagnosis text of each diagnosis item as the diagnosis conclusion of each diagnosis item. The medical report in this example uses the diagnosis text as the diagnosis conclusion, and the content is detailed, which is convenient for doctors to conduct more detailed verification of each diagnosis item according to the medical report.

[0107] Figure 3 is a diagram of a generation process of a structured report provided in this embodiment, Figure 3 On the left is an unstructured initial diagnostic report, and on the right is a structured medical report. Figure 3 In the illustrated embodiment, the diagnosis conclusion is the diagnosis text.

[0108] like Figure 3 As shown in the right figure, the report template corresponding to the structured report contains a total of 18 diagnostic items, namely: lung shadows, lung nodules, lung masses, lung calcifications, translucent shadows, lung consolidation, pneumonia, tuberculosis, interstitial changes, atelectasis, pulmonary edema, pulmonary lymphadenopathy, bullae / air cysts, emphysema, mediastinal masses, mediastinal calcifications, mediastinal lymphadenopathy, and cardiac and vascular calcifications.

[0109] by Figure 3 Taking the diagnostic item "lung shadow" in the report as an example, in the initial diagnostic report, the server determines that the diagnostic text corresponding to "lung shadow" is "mixed density nodules are seen in the right middle lobe of the lung, with cavities and fluid levels inside, linear shadows around, and traction of the adjacent pleura; tiny irregular nodules are scattered in both lungs, small calcifications are seen in the right upper lobe of the lung; scattered flakes and linear shadows are seen in both lungs", then the diagnostic text corresponding to "lung shadow" is added as the diagnostic conclusion of the diagnostic item to the position corresponding to "lung shadow" in the report template.

[0110] For a diagnostic item, when the server does not determine the diagnostic text corresponding to the diagnostic item in the initial diagnostic report, the diagnostic conclusion of the diagnostic item is set to a default text, which can be set to any content such as "none", "0", etc. Figure 3 It can be seen that there are multiple diagnostic items with a diagnosis conclusion of "none". These diagnostic items are diagnostic items for which no corresponding diagnostic texts have been determined in the initial diagnostic report. Figure 3 The default text for the examples in is None.

[0111] In an example of a report format, a structured report consists of a diagnostic text and a judgment result of a diagnostic item that characterizes an abnormality. Therefore, after determining the diagnostic text of the diagnostic item, the server determines whether the diagnostic text corresponding to the diagnostic item characterizes the presence of an abnormality. The server uses the diagnostic text and the judgment result corresponding to the diagnostic item as the diagnostic conclusion of the diagnostic item. And based on the diagnostic conclusions of each diagnostic item that characterizes the presence of an abnormality, a structured medical report is determined. The medical report in this example only contains the diagnostic text that characterizes the abnormality and the judgment result corresponding to the diagnostic text. The medical report is relatively concise, which allows doctors to quickly notice abnormal diagnostic items and focus on checking the abnormal diagnostic items based on the diagnostic text.

[0112] Figure 4 A method corresponding to Figure 3 The structured report effect diagram, and Figure 3 By comparison, it can be seen that Figure 4 The diagnostic conclusions of only those diagnostic items that indicate the presence of abnormalities are included. Figure 4 In the illustrated embodiment, the diagnosis conclusion consists of a diagnosis text and a judgment result.

[0113] In an example of a report format, the structured report consists of the diagnosis conclusions of each diagnosis item, but does not include the diagnosis text of each diagnosis item. The content of the medical report in this example is more concise and more structured, which is conducive to the unification of the format of the medical report.

[0114] After determining the diagnostic text of the diagnostic item, the server determines whether the diagnostic text corresponding to the diagnostic item indicates an abnormality. The server uses the determination result as the diagnostic conclusion of the diagnostic item. A structured medical report is determined based on the diagnostic conclusions of each diagnostic item.

[0115] Figure 5 A method corresponding to Figure 3 The structured report effect diagram, and Figure 3 compared to, Figure 5 It only contains the judgment results of each diagnostic item. Figure 5 In the illustrated embodiment, the diagnosis conclusion is the judgment result.

[0116] In a report generation task, the user can select one of the above-mentioned image generation task, report generation task and report structuring task to execute, or can select multiple tasks to execute at the same time.

[0117] The input information in this specification can be input by the user or can be the calculation result obtained by the server itself.

[0118] If only one of the image finding generation task, report generation task and report structuring task is executed during a report generation process, the input information matching the medical report generation task needs to be input by the user.

[0119] If at least two of the image generation tasks, report generation tasks, and report structuring tasks are executed in sequence during a report generation process, the input information corresponding to the first executed report generation task needs to be input by the user, and the input information corresponding to other subsequently executed report generation tasks can be the server's calculation result of the last executed report generation task. In this embodiment, the server determines the input information corresponding to the first executed report generation task in response to the user input, and for the subsequently executed report generation task, the server obtains its own output result of the last executed report generation task as the input information of the subsequently executed report generation task.

[0120] Then, the server uses the input information, task prompt and associated information as prompt information, and inputs the prompt information and diagnostic features into a report generation model to obtain a medical report.

[0121] For example, if the patient information contains image data corresponding to a medical image, and the user chooses to execute the above three tasks at the same time, the execution order of the three tasks is, image finding generation task, report generation task and report structuring task. The server executes these three tasks in sequence to obtain a structured medical report.

[0122] In this embodiment, when the server executes the image finding generation task, the image finding can be determined in the form of a completion task. After obtaining the image finding, the server uses the image finding obtained in executing the image finding generation task as the input information of the report generation task, uses the input information as part of the prompt information of the report generation task, inputs the prompt information and diagnostic features of the report generation task into the report generation model, and obtains an initial diagnosis report. Finally, the server uses the initial diagnosis report obtained by executing the report generation task as the input information of the report structuring task, uses the input information as part of the prompt information of the report structuring task, inputs the prompt information and diagnostic features of the report structuring task into the report generation model, and obtains a structured medical report.

[0123] Therefore, in this embodiment, the input information in the image generation task needs to be obtained through user input. The input information in the report generation task and the input information in the report structuring task are all the calculation results of the server itself, and do not need to be input by the user again.

[0124] It can be seen that when the user chooses to perform the above three tasks at the same time, the user only needs to input in the first task, that is, the image findings generation task, and the subsequent report generation tasks and report structuring tasks can be automatically performed through the report generation model to obtain a structured medical report. That is, in the process of medical report generation, the user's workload is very small, which greatly improves the efficiency of medical report generation.

[0125] In addition to the image data, text data, voice data, and table data mentioned in step S100, the patient information in this specification may also include the processing results of medical images by machine learning models of any function, such as medical images divided into regions obtained by image segmentation models, medical images with marked abnormal regions obtained by image recognition models, etc.

[0126] The processing results of machine learning models with different functions can correspond to different encoding layers, which are used to encode the features of different processing results and obtain diagnostic features corresponding to different processing results.

[0127] For example, if the patient information includes a medical image obtained through a region segmentation model, the report generation model can be configured to include a segmentation result coding layer, and the server inputs the medical image after region segmentation into the segmentation result coding layer to obtain the segmentation result diagnostic features. If the patient information includes a medical image with abnormal regions marked by an image recognition model, the report generation model can be configured to include a recognition result coding layer, and the server inputs the medical image with abnormal regions marked into the recognition result coding layer to obtain the recognition result diagnostic features.

[0128] The report generation model in this specification can be configured as a combination of one or more of a text coding layer, an image coding layer, a speech coding layer, a table coding layer, a segmentation result coding layer, and a recognition result coding layer. Of course, the types of coding layers that can be included in the report generation model are not limited to the various coding layers mentioned above, and can be expanded according to the data type of patient information involved in actual applications.

[0129] When the patient information contains data of multiple data types, the data of each data type obtains corresponding features through different encoding layers. The server can fuse the features of all types of data to obtain the diagnostic features of the patient information.

[0130] In one or more embodiments of the present specification, when the server fuses different types of features through a neural network, the report generation model in the present specification may also include a fusion layer.

[0131] The server inputs the features of patient information of different data types obtained by each encoding layer into the fusion layer, and performs cross-attention calculation on the features of patient information of different data types through the fusion layer to obtain diagnostic features.

[0132] In one or more embodiments of the present specification, when the report generation model includes an image coding layer, in order to perform more targeted encoding of different types of medical images to obtain more accurate image diagnostic features, the image coding layer may include multiple sub-coding layers, each sub-coding layer being used for feature encoding of different types of medical images.

[0133] The server determines a sub-model corresponding to the image data contained in the patient information according to the medical image type corresponding to the image data contained in the patient information, and inputs the image data into the sub-model to obtain image diagnostic features.

[0134] When patient information contains data of multiple data types, the server needs to fuse the features of all types of data. However, since different types of patient information are encoded separately through different coding layers, different data types correspond to different vector spaces. Because the semantics of features are reflected as distances in vector space, the features of patient information of different data types differ in semantic expression. If the features of patient information of different data types are directly fused, the semantic integration effect between the features is poor.

[0135] Therefore, when the patient data contains both text data and non-text data, in order to more accurately integrate the features of different types of patient information semantically, the report generation model of this specification may also include a migration layer, which is used to map the features corresponding to the non-text data to the vector space of the text data features.

[0136] The migration layer is configured correspondingly to the coding layer for feature encoding of non-text data, that is, the coding layer of non-text data in the report generation model can be configured with a migration layer. Then the report generation model may include an image migration layer corresponding to the image coding layer, a speech migration layer corresponding to the speech coding layer, a segmentation migration layer corresponding to the segmentation result coding layer, a recognition migration layer corresponding to the recognition result coding layer, and so on.

[0137] When the report generation model includes a migration layer, the server continues to input the features output by each encoding layer into the migration layer to obtain the migration features output by the migration layer, and then fuses the migration features with the text diagnosis features.

[0138] For example, a report generation model includes an image coding layer and a text coding layer, and an image migration layer corresponding to the image coding layer. The server inputs the image diagnostic features into the image migration layer to obtain image migration features, and then fuses the image migration features with the text diagnostic features to obtain diagnostic features.

[0139] Figure 6 A diagram of a medical report generation process provided in an embodiment of this specification. Figure 6 The report generation model in contains image encoding layer, text encoding layer, image transfer layer, fusion layer and decoding layer.

[0140] like Figure 6 As shown, the patient information includes image data and text data. The server inputs the image data into the image encoding layer to obtain image diagnostic features, and inputs the text data into the text encoding layer to obtain text diagnostic features. Then, the image diagnostic features are input into the image migration layer to obtain image migration features, and the image migration features and text diagnostic features are input into the fusion layer to integrate the features corresponding to patient data of different data types to obtain diagnostic features.

[0141] The server determines the historical diagnosis information related to the patient information in the historical diagnosis record according to the diagnosis feature as the associated information. Then, a task prompt for at least one medical report generation task among the image finding generation task, the report generation task, and the report structuring task, and input information matching the medical report generation task corresponding to the task prompt are obtained, and the task prompt, the input information, and the associated information are used as prompt information, and the prompt information and the diagnosis feature are input into the decoding layer to obtain the medical report.

[0142] The medical report generation method in this specification automatically generates medical reports through a report generation model, thereby reducing the workload of users in the process of generating medical reports and improving diagnosis efficiency.

[0143] The above content has introduced various embodiments of the report generation model generating medical reports in the application stage. The following will introduce the training method of the report generation model in this specification. The server adopts a layer-by-layer training method to train each neural network layer included in the report generation model. The structure of the report generation model is different, and there are corresponding different training methods.

[0144] In one or more embodiments of the present specification, the report generation model includes an encoding layer corresponding to patient information of at least one data type, and a decoding layer.

[0145] The server first trains the coding layer, and after the coding layer training is completed, fixes the coding layer parameters and trains the decoding layer. When the report generation model contains coding layers corresponding to multiple data types, this specification does not limit the training order of the coding layers corresponding to each data type.

[0146] Figure 7 A structural diagram of a report generation model provided in an embodiment of this specification is shown in FIG. Figure 7 As shown in Figure 1, the report generation model includes an image encoding layer, a text encoding layer, and a decoding layer. Figure 7 Taking the report generation model shown in FIG. 1 as an example, the training method of the report generation model is specifically described.

[0147] The report generation model in this specification also includes an auxiliary coding layer, which is a text encoder. The server trains the image coding layer and the auxiliary coding layer by contrastive learning. The training method of the image coding layer is as follows:

[0148] First, the server obtains each medical image in the historical diagnosis record as each sample image, and determines a text description corresponding to each sample image.

[0149] Secondly, the server inputs each sample image into the image coding layer to be trained, and inputs the text description corresponding to each sample image into the auxiliary coding layer to be trained. Through the training method of comparative learning, the model parameters of the image coding layer to be trained and the model parameters of the auxiliary coding layer to be trained are adjusted.

[0150] The report generation model in this specification also includes a first auxiliary decoding layer. The text encoding layer and the first auxiliary decoding layer are trained together. The training method of the text encoding layer is as follows:

[0151] First, the service obtains the historical diagnosis text in the historical diagnosis record, shields the local text segment in the historical diagnosis text, and obtains the sample text. Secondly, the server inputs the sample text into the text encoding layer to be trained to obtain the text features of the sample text. Then, the text features of the sample text are input into the first auxiliary decoding layer, and the shielded local text segment is predicted to obtain the predicted text segment. Finally, the server adjusts the model parameters of the text encoding layer to be trained and the first auxiliary decoding layer according to the difference between the local text segment and the predicted text segment.

[0152] The training order of the above image coding layer and text coding layer is not specific, they can be trained simultaneously or sequentially. After the training of the image coding layer and text coding layer is completed, the server fixes the model parameters of the image coding layer and text coding layer, and trains the decoding layer in the following way:

[0153] First, the server obtains each historical diagnosis text in the historical diagnosis record, and determines the medical image and patient text data when each historical diagnosis text is obtained. For each historical diagnosis text, the medical image when the historical diagnosis text is obtained is used as a sample image, and the patient text data when the historical diagnosis text is obtained is used as a sample text.

[0154] Secondly, the server inputs the sample image into the trained image encoding layer to obtain image features, and inputs the sample text into the trained text encoding layer to obtain text features. Then, the server fuses the image features and the text features to obtain diagnostic features, and inputs the diagnostic features into the decoding layer to be trained to obtain predicted diagnostic text. According to the difference between the historical diagnostic text and the predicted diagnostic text output by the decoding layer, the model parameters of the decoding layer to be trained are adjusted.

[0155] based on Figure 7 The model structure shown illustrates the training methods of two types of coding layers, namely, text coding layer and image coding layer. The image coding layer is a coding layer for non-text data. When there are coding layers for non-text data such as speech coding layer, table coding layer, segmentation result coding layer, recognition result coding layer, etc. that can be expanded according to the patient data type, each coding layer for non-text data is trained separately. The training method is similar to that of the image coding layer. Please refer to the description of the training process of the image coding layer.

[0156] According to the above discussion, when generating a medical report in the application stage, the server may use the user's text or voice input as a task prompt, and use the task prompt as part of the prompt information for generating the medical report.

[0157] In this way, because it is more flexible according to the text or voice input of the user, the determined task prompts are also diverse. In order to enable the report generation model to accurately identify the medical report generation task corresponding to the task prompt based on the diverse task prompts, and generate a medical report matching the task prompt.

[0158] In one or more embodiments of the present specification, after the decoding layer training is completed, the server fine-tunes the report generation model according to the task prompt sample.

[0159] For each type of medical report generation task, multiple task prompts with different expressions matching the type of medical report generation task are used as task prompt samples for the type of medical report generation task.

[0160] First, the server obtains historical diagnosis texts of each medical report type in the historical diagnosis records, and determines the medical images and patient text data when each historical diagnosis text is obtained.

[0161] The historical diagnosis records include historical diagnosis texts of three types of medical reports corresponding to the above-mentioned three medical report generation tasks, namely, image findings corresponding to the image findings generation task, unstructured initial diagnosis report corresponding to the report generation task, and structured report corresponding to the report structuring task.

[0162] For each historical diagnosis text, the medical image at the time of the historical diagnosis text is obtained as a sample image, and the patient text data at the time of the historical diagnosis text is obtained as a sample text. At the same time, the task prompt sample corresponding to the historical diagnosis text is determined, and the task prompt sample is used to prompt the report generation model to execute the medical report generation task corresponding to the medical report type.

[0163] Secondly, the server inputs the sample image into the trained image coding layer to obtain image features. The server inputs the sample text into the trained text coding layer to obtain text features. The image features and the text features are fused to obtain diagnostic features.

[0164] Then, based on the diagnostic feature, the server determines the historical diagnostic information related to the sample image and sample text in the historical diagnostic record as the associated information. The task prompt sample and the associated information are used as prompt information. The prompt information and the diagnostic feature are input into the trained decoding layer to obtain the predicted diagnostic text output by the decoding layer.

[0165] Finally, the server determines the difference between the predicted diagnosis text and the historical diagnosis text, and fine-tunes the model parameters of the decoding layer with the goal of minimizing the difference.

[0166] According to the discussion in the previous application section, each encoding layer of non-text data in the report generation model can be configured with a corresponding migration layer. The training method of the migration layer corresponding to different non-text data is the same. Now, we will take the image migration layer as an example to explain the training method of the migration layer.

[0167] After the training of the coding layer is completed, the server fixes the model parameters of the coding layer and performs training on the image migration layer.

[0168] Taking the encoding layer including the image encoding layer and the text encoding layer as an example, the training process of the image migration layer is described. The server can train the image migration layer according to various loss functions.

[0169] In one or more embodiments of the present specification, the server trains the image migration layer according to the image-text contrast loss (ITC). The specific method for determining the image-text contrast loss is as follows:

[0170] First, the server obtains each medical image in the historical diagnosis record as each sample image, and determines the text description corresponding to each sample image. Then, the server inputs each sample image into the trained image coding layer, and inputs the text description corresponding to each sample image into the trained auxiliary coding layer, and determines the image-text comparison loss through the comparison learning task.

[0171] In one or more embodiments of the present specification, the server trains the image migration layer according to the image-text matching loss (ITM). The report generation model also includes a prediction layer, and the specific determination method of the image-text matching loss is as follows:

[0172] First, the server obtains each medical image in the historical diagnosis record as each sample image, and determines the text description corresponding to each sample image. Then, the server inputs each sample image into the trained image coding layer, and inputs the text description corresponding to each sample image into the trained auxiliary coding layer, and determines the image-text matching loss through the image-text matching task.

[0173] In one or more embodiments of the present specification, the server trains the image migration layer according to the image-grounded text generation loss (ITG). The report generation model to be trained includes an auxiliary encoding layer and a second auxiliary decoding layer, and the specific determination method of the image-grounded text generation loss is as follows:

[0174] First, the server obtains each medical image in the historical diagnosis record as each sample image, and determines the text description corresponding to each sample image. Then, the server inputs each sample image into the trained image coding layer, and inputs the text description corresponding to each sample image into the trained auxiliary coding layer, and determines the image-text generation loss through the image-text generation task.

[0175] The above three embodiments of the training image migration layer may be used alone or in any combination.

[0176] When the server uses three loss functions, namely, image-text comparison loss, image-text matching loss, and image-text generation loss, to train the image transfer layer, the model structure of the report generation model is as follows: Figure 8 shown.

[0177] Figure 8 This is a structural diagram of a report generation model provided in an embodiment of this specification. The report generation model includes an image coding layer, an auxiliary coding layer, a text coding layer, a first auxiliary decoding layer, an image migration layer, a prediction layer and a decoding layer.

[0178] In one or more embodiments of the present specification, the report generation model may further include a fusion layer for fusing diagnostic features corresponding to various types of patient information. The fusion layer may be trained together with the decoding layer, and the specific training method is as follows:

[0179] First, the server obtains each historical diagnosis text in the historical diagnosis record, and determines the medical image and patient text data when each historical diagnosis text is obtained. For each historical diagnosis text, the medical image when the historical diagnosis text is obtained is used as a sample image, and the patient text data when the historical diagnosis text is obtained is used as a sample text.

[0180] Secondly, the server inputs the sample image into the trained image coding layer to obtain image features. The server inputs the sample text into the trained text coding layer to obtain text features. The server inputs the image features and the text features into the fusion layer to be trained to obtain the diagnostic features output by the fusion layer.

[0181] Then, the server inputs the diagnostic features into the decoding layer to be trained to obtain the predicted diagnostic text. According to the difference between the historical diagnostic text and the predicted diagnostic text, the model parameters of the fusion layer to be trained and the decoding layer to be trained are adjusted.

[0182] The above is the medical report generation method provided in this specification. Based on the same idea, this specification also provides a corresponding medical report generation device, such as Fig. 9 shown.

[0183] Fig. 9 A schematic diagram of a medical report generating device provided in this specification specifically includes:

[0184] The diagnostic feature determination module 200 is used to obtain patient information, input the patient information into the report generation model, and obtain diagnostic features;

[0185] A retrieval module 202, for determining, in the historical diagnosis record, historical diagnosis information related to the patient information according to the diagnosis feature, as associated information;

[0186] The generating module 204 is used to determine prompt information according to the associated information, input the prompt information and the diagnostic features into the report generating model, and generate a medical report.

[0187] Optionally, the report generation model includes an image coding layer and a text coding layer, the patient information includes image data and text data, and the diagnostic feature determination module 200 is specifically used to input the image data into the image coding layer to obtain image diagnostic features, input the text data into the text coding layer to obtain text diagnostic features, and determine the diagnostic features based on the image diagnostic features and the text diagnostic features.

[0188] Optionally, the retrieval module 202 is specifically used to determine the coding features of each historical diagnostic information contained in the historical diagnostic record, respectively determine the similarity between the coding features of each historical diagnostic information and the diagnostic feature, and use the coding features of the historical diagnostic information whose similarity is greater than or equal to a preset value as associated information.

[0189] Optionally, the generation module 204 is specifically used to obtain the previous information of the medical report to be generated input by the user, use the previous information and the related information as prompt information, input the prompt information and the diagnostic features into the report generation model, predict the subsequent content of the previous information through the report generation model, obtain the subsequent information of the previous information, and determine the medical report based on the previous information and the subsequent information.

[0190] Optionally, the subsequent information is composed of multiple pieces of candidate information, and the generation module 204 is specifically used to determine the candidate information selected by the user in response to the user's selection operation on each piece of candidate information, and determine the medical report based on the previous information and the candidate information selected by the user.

[0191] Optionally, the medical report is a structured report, and the generation module 204 is specifically used to determine the task prompt according to the user operation, and the task prompt is used to characterize the report structured task, determine the report template and the initial diagnosis report, use the task prompt and the related information as prompt information, input the prompt information and the diagnostic features into the report generation model, and obtain the medical report.

[0192] Optionally, the generation module 204 is specifically used to determine a report template and an initial diagnosis report, obtain each diagnosis item in the report template, determine the diagnosis text corresponding to the diagnosis item based on the initial diagnosis report, determine the diagnosis conclusion of the diagnosis item based on the diagnosis text, and determine the medical report based on the diagnosis conclusion.

[0193] Optionally, the generation module 204 is specifically used to determine that the judgment result of the diagnostic item is abnormal when it is determined that the diagnostic text corresponding to the diagnostic item indicates the existence of an abnormality, and use the diagnostic text of the diagnostic item and the judgment result of the diagnostic item as the diagnostic conclusion of the diagnostic item.

[0194] This specification also provides a computer-readable storage medium, which stores a computer program, which can be used to execute the above Figure 1 A method for generating a medical report is provided.

[0195] This manual also provides Fig.10 The schematic structure diagram of the electronic device shown in FIG. Fig.10As mentioned above, at the hardware level, the electronic device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory, and may also include other hardware required for the business. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to achieve the above Figure 1 Of course, in addition to the software implementation, this specification does not exclude other implementations, such as logic devices or a combination of software and hardware, etc., that is, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0196] For the improvement of a technology, it can be clearly distinguished whether it is a hardware improvement (for example, improvement of the circuit structure of diodes, transistors, switches, etc.) or a software improvement (improvement of the method flow). However, with the development of technology, many improvements of the method flow today can be regarded as direct improvements of the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that the improvement of a method flow cannot be implemented with a hardware entity module. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system on a PLD by programming themselves, without having to ask chip manufacturers to design and make dedicated integrated circuit chips. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs, and the original code before compilation must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages ​​and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that implements the logic method flow.

[0197] The controller can be implemented in any appropriate manner, for example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (such as software or firmware) that can be executed by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that in addition to implementing the controller in a purely computer-readable program code manner, the controller can be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, this controller can be considered as a hardware component, and the devices included therein for implementing various functions can also be regarded as structures within the hardware component. Or even, the devices for implementing various functions can be regarded as both software modules for implementing the method and structures within the hardware component.

[0198] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0199] For the convenience of description, the above device is described in various units according to their functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0200] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0201] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0202] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0203] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0204] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0205] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0206] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0207] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0208] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems or computer program products. Therefore, this specification may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0209] This specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0210] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0211] The above description is only an embodiment of this specification and is not intended to limit this specification. For those skilled in the art, this specification may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this specification should be included in the scope of the claims of this application.

Claims

1. A method for generating a medical report, characterized in that: include: Acquiring patient information, and inputting the patient information into a report generation model to obtain diagnostic features; According to the diagnosis feature, determining historical diagnosis information related to the patient information in the historical diagnosis record as associated information; Prompt information is determined based on the associated information, and the prompt information and the diagnostic features are input into the report generation model to generate a medical report.

2. The method according to claim 1, characterized in that The report generation model includes an image encoding layer and a text encoding layer, and the patient information includes image data and text data; The patient information is input into the report generation model to obtain diagnostic features, including: Inputting the image data into the image coding layer to obtain image diagnostic features, and inputting the text data into the text coding layer to obtain text diagnostic features; A diagnostic feature is determined according to the image diagnostic feature and the text diagnostic feature.

3. The method according to claim 1, characterized in that According to the diagnosis feature, historical diagnosis information related to the patient information is determined in the historical diagnosis record as associated information, specifically including: Determining the coding features of each piece of historical diagnosis information contained in the historical diagnosis record; Determining the similarity between the coding feature of each historical diagnosis information and the diagnosis feature respectively; The encoding features of the historical diagnosis information with a similarity greater than or equal to a preset value are used as the associated information.

4. The method according to claim 1, characterized in that Determining prompt information according to the association information, inputting the prompt information and the diagnostic features into the report generation model, and generating a medical report, specifically includes: Acquire the preceding information of the medical report to be generated input by the user, and use the preceding information and the associated information as prompt information; Inputting the prompt information and the diagnostic features into the report generation model, and predicting the subsequent content of the preceding information through the report generation model to obtain subsequent information of the preceding information; A medical report is determined based on the preceding information and the following information.

5. The method according to claim 4, characterized in that The subsequent information is composed of a plurality of pieces of information to be selected; Determine a medical report based on the above information and the following information, including: In response to the user's selection operation on each piece of candidate information, determining the candidate information selected by the user; A medical report is determined based on the preceding information and the candidate information selected by the user.

6. The method according to claim 1, characterized in that The medical report is a structured report; Determining prompt information according to the association information, inputting the prompt information and the diagnostic features into the report generation model, and generating a medical report, specifically includes: Determine a task prompt according to the user operation, wherein the task prompt is used to characterize the report structured task; Using the task prompt and the associated information as prompt information; The prompt information and the diagnostic features are input into the report generation model to obtain the medical report.

7. The method according to claim 6, characterized in that Inputting the prompt information and the diagnostic features into the report generation model to obtain the medical report specifically includes: Finalize report templates and initial diagnostic reports; Obtaining each diagnostic item in the report template, and determining a diagnostic text corresponding to the diagnostic item based on the initial diagnostic report; Based on the diagnosis text, determining the diagnosis conclusion of the diagnosis item; A medical report is determined based on the diagnostic conclusion.

8. The method according to claim 7, characterized in that Based on the diagnosis text, determining the diagnosis conclusion of the diagnosis item specifically includes: When it is determined that the diagnosis text corresponding to the diagnosis item indicates that an abnormality exists, determining that the judgment result of the diagnosis item is abnormal; The diagnosis text of the diagnosis item and the judgment result of the diagnosis item are used as the diagnosis conclusion of the diagnosis item.

9. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 7 is implemented.

Images