Medical report generation method and apparatus, storage medium, and electronic device

Abstract

The present application discloses a medical report generation method and apparatus, a storage medium, and an electronic device. The method comprises: obtaining patient information, and inputting the patient information into a report generation model to obtain diagnostic features; on the basis of the diagnostic features, determining, from historical diagnostic records, historical diagnostic information related to the patient information as associated information; and determining prompt information on the basis of the associated information, and inputting the prompt information and the diagnostic features into the report generation model to generate a medical report. In the present method, by simply inputting patient information into a report generation model, a doctor can obtain a medical report generated by the model, without the need for the doctor to manually write a medical report, thereby improving the efficiency of determining medical reports.

Description

Methods, devices, storage media and electronic devices for generating medical reports

[0001] Cross-reference to related applications

[0002] The related applications of this application respectively claim priority to Chinese patent applications filed on August 30, 2024, with application number 202411217057.7 entitled "A method, apparatus, storage medium and electronic device for generating medical reports", and filed on September 27, 2024, with application number 202411374405.1 entitled "A method, apparatus, storage medium and electronic device for generating medical reports", the entire contents of which are incorporated herein by reference. Technical Field

[0003] This application relates to the field of computer technology, and in particular to a method, apparatus, storage medium and electronic device for generating medical reports. Background Technology

[0004] Typically, doctors need to write medical reports during the diagnosis and treatment of patients. Writing a medical report requires doctors to make a comprehensive diagnosis based on their experience, combined with the patient's condition, medical test results, imaging examination results, and other factors. They also need to organize their language according to the diagnostic results to complete the medical report.

[0005] It is evident that the process of writing medical reports is quite cumbersome, requiring a significant amount of effort and time from doctors, thus reducing their diagnostic and treatment efficiency. Therefore, this application provides a method for generating medical reports. Summary of the Invention

[0006] This application provides a method for generating medical reports, including:

[0007] Obtain patient information, input the patient information into the report generation model, and obtain diagnostic features;

[0008] Based on the diagnostic features, historical diagnostic information related to the patient information is identified in the historical diagnostic records and used as associated information;

[0009] Based on the associated information, a prompt message is determined, and the prompt message and the diagnostic features are input into the report generation model to generate a medical report.

[0010] In some embodiments, the report generation model includes an image encoding layer and a text encoding layer, and the patient information includes image data and text data;

[0011] The patient information is input into the report generation model to obtain diagnostic features, specifically including:

[0012] The image data is input into the image encoding layer to obtain image diagnostic features, and the text data is input into the text encoding layer to obtain text diagnostic features;

[0013] Diagnostic features are determined based on the image diagnostic features and the text diagnostic features.

[0014] In some embodiments, based on the diagnostic features, historical diagnostic information related to the patient information is determined from historical diagnostic records as associated information, specifically including:

[0015] Determine the encoding characteristics of each historical diagnostic information contained in the historical diagnostic records;

[0016] The similarity between the encoding features of each historical diagnostic information and the diagnostic features is determined respectively;

[0017] The encoded features of historical diagnostic information with a similarity greater than or equal to a preset value are used as associated information.

[0018] In some embodiments, based on the associated information, a prompt message is determined, and the prompt message and the diagnostic features are input into the report generation model to generate a medical report, specifically including:

[0019] Obtain the context information of the medical report to be generated input by the user, and use the context information and the associated information as prompt information;

[0020] The prompt information and the diagnostic features are input into the report generation model. The report generation model is used to predict the subsequent content of the preceding information to obtain the subsequent information of the preceding information.

[0021] The medical report is determined based on the preceding and following information.

[0022] In some embodiments, the following information consists of a plurality of candidate information;

[0023] Based on the foregoing and subsequent information, a medical report is determined, specifically including:

[0024] In response to the user's selection operation on each candidate information, determine the candidate information selected by the user;

[0025] Based on the aforementioned information and the candidate information selected by the user, a medical report is determined.

[0026] In some embodiments, the medical report is a structured report;

[0027] Based on the associated information, a prompt message is determined. The prompt message and the diagnostic features are then input into the report generation model to generate a medical report, specifically including:

[0028] Based on user actions, a task prompt is determined, which is used to characterize the structured task of the report;

[0029] The task prompt and the associated information are used as the prompt information;

[0030] The prompt information and the diagnostic features are input into the report generation model to obtain the medical report.

[0031] In some embodiments, the prompt information and the diagnostic features are input into the report generation model to obtain the medical report, specifically including:

[0032] Determine the report template and initial diagnostic report;

[0033] Obtain each diagnostic item in the report template, and determine the corresponding diagnostic text for each diagnostic item based on the initial diagnostic report;

[0034] Based on the diagnostic text, determine the diagnostic conclusion for the diagnostic item;

[0035] Based on the diagnostic conclusion, a medical report is determined.

[0036] In some embodiments, determining the diagnostic conclusion of the diagnostic item based on the diagnostic text specifically includes:

[0037] When it is determined that the diagnostic text corresponding to the diagnostic item indicates an anomaly, the judgment result of the diagnostic item is determined to be abnormal.

[0038] The diagnostic text and the judgment result of the diagnostic item shall be used as the diagnostic conclusion of the diagnostic item.

[0039] This application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described medical report generation method.

[0040] This application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the above-described medical report generation method.

[0041] In the medical report generation method provided in this application embodiment, patient information is obtained, input into a report generation model to obtain diagnostic features, and based on the diagnostic features, historical diagnostic information related to the patient information is determined from historical diagnostic records as associated information. Based on the associated information, prompt information is determined, and this prompt information and diagnostic features are input into the report generation model to generate a medical report. In this method, doctors only need to input patient information into the report generation model to obtain the model-generated medical report, eliminating the need for doctors to manually write medical reports and improving the efficiency of medical report generation.

[0042] A method for generating a medical report, comprising:

[0043] Determine and present the initial diagnostic report;

[0044] Obtain input information from the user based on the initial diagnostic report, predict the user's subsequent input based on the input information, and determine the predicted information;

[0045] Based on the input information and the prediction information, the initial diagnostic report is adjusted to determine the medical report.

[0046] In some embodiments, determining and displaying an initial diagnostic report specifically includes:

[0047] Determine a report template that includes pre-defined diagnostic text for each part of the body;

[0048] Based on the report template, an initial diagnostic report is determined and then displayed.

[0049] In some embodiments, determining an initial diagnostic report based on the report template specifically includes:

[0050] Acquire medical images and determine the region of interest within the medical images;

[0051] Among the various human body parts, the human body parts corresponding to the region of interest are identified as the regions of interest;

[0052] Based on the region of interest, determine the initial diagnostic text for the area of ​​interest;

[0053] Based on the initial diagnostic text of the area of ​​interest, an initial diagnostic report is determined and displayed.

[0054] In some embodiments, predicting the user's subsequent input based on the input information and determining the prediction information specifically includes:

[0055] Based on the input information, predict the user's subsequent input and determine the predicted candidate information;

[0056] The candidate information is then displayed to the user.

[0057] In response to the user's selection of each candidate information, the candidate information selected by the user is used as the prediction information.

[0058] In some embodiments, displaying the candidate information to the user specifically includes:

[0059] Determine the matching probability between each candidate information and the input information;

[0060] Based on the magnitude of each matching probability, the candidate information corresponding to each matching probability is sorted to determine the sorted candidate information.

[0061] The sorted candidate information is then displayed to the user.

[0062] In some embodiments, the initial diagnostic report is adjusted based on the input information and the prediction information to determine the medical report, specifically including:

[0063] Based on the input information and the prediction information, the adjustment information is determined;

[0064] In the initial diagnostic report, the text fragment corresponding to the adjustment information is identified;

[0065] The text fragment is adjusted according to the adjustment information.

[0066] In some embodiments, the input information is voice information or text information;

[0067] Based on the input information and the prediction information, adjustment information is determined, specifically including:

[0068] When the input information is voice information, determine the text information corresponding to the voice information, integrate the text information with the prediction information, and determine the adjustment information;

[0069] When the input information is text information, the text information is integrated with the prediction information to determine the adjustment information.

[0070] In some embodiments, the initial diagnostic report is adjusted based on the input information and the prediction information to determine the medical report, specifically including:

[0071] Based on the input information and the prediction information, the adjustment information is determined;

[0072] Among the various human body parts, the human body part corresponding to the adjustment information is determined as the target part;

[0073] Based on the adjustment information, the initial diagnostic text for the target site in the initial diagnostic report is adjusted to determine the medical report.

[0074] In some embodiments, the number of target regions is multiple;

[0075] Based on the adjustment information, the initial diagnostic text for the target site in the initial diagnostic report is adjusted, specifically including:

[0076] The adjustment information specifies the adjustment details for each target part.

[0077] For each target area, adjust the initial diagnostic text for that target area according to the adjustments made to that target area.

[0078] In some embodiments, the initial diagnostic text for the target region is adjusted according to the adjustment content of the target region, specifically including:

[0079] When the adjustment content of the target part contains keywords that represent the semantics of deletion, the initial diagnostic text of the target part is deleted;

[0080] When the adjustment content of the target part does not contain keywords that represent deletion semantics, the adjustment content of the target part is integrated with the initial diagnostic text of the target part.

[0081] In some embodiments, the adjustment content of the target area is integrated with the initial diagnostic text of the target area, specifically including:

[0082] Semantic analysis is performed on the adjustment content of the target area and the initial diagnostic text of the target area;

[0083] When it is determined that the adjustment content of the target part contradicts the semantics of the initial diagnostic text of the target part, the initial diagnostic text of the target part shall be replaced with the adjustment content of the target part.

[0084] When it is determined that the adjustment content of the target part overlaps with the semantics of the initial diagnostic text of the target part, the adjustment content of the target part and the initial diagnostic text of the target part are merged.

[0085] In some embodiments, the initial diagnostic report is used to describe the imaging information of medical images;

[0086] Based on the input information and the prediction information, the initial diagnostic report is adjusted to determine the medical report, specifically including:

[0087] The initial diagnostic report is adjusted based on the input information and the prediction information.

[0088] The adjusted initial diagnostic report is input into the third language model to generate the examination conclusion of the medical images;

[0089] Based on the adjusted initial diagnostic report and the examination conclusions of the medical images, a medical report is determined.

[0090] In some embodiments, after determining the medical report, the method further includes:

[0091] Obtain patient information, which includes at least one of the following: basic information, chief complaint, relevant examination results, and medical history;

[0092] Determine whether there are any discrepancies between the medical report and the patient information;

[0093] If so, then based on the statement in the medical report corresponding to the mismatched content, a prompt message is determined, which is used to prompt the user to check the mismatched content again;

[0094] If not, then the medical report is displayed to the user.

[0095] This application provides a medical report generation device, including:

[0096] The initial diagnostic report determination module determines and displays the initial diagnostic report.

[0097] The input module acquires input information from the user based on the initial diagnostic report, predicts the user's subsequent input based on the input information, and determines the predicted information.

[0098] The medical report determination module adjusts the initial diagnostic report based on the input information and the prediction information to determine the medical report.

[0099] This application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described medical report generation method.

[0100] This application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the above-described medical report generation method.

[0101] In the medical report generation method provided in this application embodiment, an initial diagnostic report is determined and displayed; input information input by the user based on the initial diagnostic report is obtained; subsequent inputs from the user are predicted based on the input information; predicted information is determined; and the initial diagnostic report is adjusted based on the input information and the predicted information to determine the medical report. Through this method, during the process of determining the medical report, based on the user's input for adjusting the initial diagnostic report, predicted information for further user input for adjusting the initial diagnostic report can be automatically generated, and the initial diagnostic report is adjusted based on the input information and the predicted information. Therefore, this method requires only a small amount of user input when determining the medical report, reducing the user's workload during the medical report generation process and improving the efficiency of medical report generation.

[0102] Details of various embodiments of the present invention will be described in the following drawings and description. Other features, problems solved, and technical effects of the present invention will be readily understood by those skilled in the art based on the specification, drawings, and claims. Attached Figure Description

[0103] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0104] Figure 1 is a flowchart illustrating a medical report generation method provided in this application;

[0105] Figure 2 is a schematic diagram of a completion task provided in an embodiment of this application;

[0106] Figure 3 is a flowchart of a structured report generation process provided in an embodiment of this application;

[0107] Figure 4 is a schematic diagram of a structured report corresponding to Figure 3 provided in an embodiment of this application;

[0108] Figure 5 is another schematic diagram of a structured report corresponding to Figure 3 provided in an embodiment of this application;

[0109] Figure 6 is a flowchart of a medical report generation process provided in an embodiment of this application;

[0110] Figure 7 is a schematic diagram of the structure of a report generation model provided in an embodiment of this application;

[0111] Figure 8 is a schematic diagram of another report generation model provided in the embodiments of this application;

[0112] Figure 9 is a schematic diagram of a medical report generation device provided in this application;

[0113] Figure 10 is a schematic structural diagram of the electronic device corresponding to Figure 1 provided in this application;

[0114] Figure 11 is a flowchart illustrating another medical report generation method provided in this application;

[0115] Figure 12 is a schematic diagram of a prediction information generation process provided in an embodiment of this application;

[0116] Figure 13 is a schematic diagram of another prediction information generation process provided in an embodiment of this application;

[0117] Figure 14 is a schematic diagram of an initial diagnostic report provided in an embodiment of this application;

[0118] Figure 15 is a schematic diagram of a medical report generation process provided in an embodiment of this application;

[0119] Figure 16 is a flowchart of a voice-triggered optimization instruction provided in an embodiment of this application;

[0120] Figure 17 is another flowchart for generating medical reports provided in the embodiments of this application;

[0121] Figure 18 is a schematic diagram of another medical report generation device provided in this application;

[0122] Figure 19 is a schematic structural diagram of the electronic device corresponding to Figure 11 provided in this application. Detailed Implementation

[0123] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0124] The technical solutions provided by the various embodiments of this application are described in detail below with reference to the accompanying drawings.

[0125] Figure 1 is a flowchart illustrating a medical report generation method described in this specification, which includes the following steps:

[0126] S1001: Obtain patient information, input the patient information into the report generation model, and obtain diagnostic features.

[0127] All steps in the medical report generation method provided in this manual can be implemented by any electronic device with computing capabilities, such as a terminal or server. For ease of description, the following explanation focuses on a server as the executing entity.

[0128] In this manual, a report generation model is used to generate patient medical reports. When a doctor needs to review a patient's medical report, the patient information obtained during the diagnosis and treatment process is input into the report generation model. The server then uses this model to obtain the patient's diagnostic characteristics.

[0129] This manual does not limit the specific content of patient information. For example, patient information can be text data such as basic patient information, medical history, and chief complaint; it can also be image data from any type of medical imaging, such as CT images, MRI images, or DR images; it can also include voice data input by the doctor during the diagnostic process to generate medical reports; and structured tabular data. This manual also does not limit the data type of patient information; it can contain only one type of data or multiple types of data simultaneously.

[0130] The report generation model may include one or more encoding layers for feature encoding, with different layers used to encode features for different types of patient information. This specification does not limit the specific network structure of the encoding layers; different encoding layers may have the same or different network structures.

[0131] The specific encoding layers included in the report generation model can be configured according to the actual application scenario. By configuring different encoding layers, the report generation model can be adapted to different application scenarios.

[0132] For example, in one application scenario where patient information includes text data, the report generation model can be configured to include a text encoding layer. The server inputs the text data into the text encoding layer to obtain text diagnostic features, and uses these text diagnostic features as diagnostic features. In another application scenario where patient information includes image data, the report generation model can be configured to include an image encoding layer. The server inputs the image data into the image encoding layer to obtain image diagnostic features, and uses these image diagnostic features as diagnostic features.

[0133] S1003: Based on the diagnostic features, determine the historical diagnostic information related to the patient information in the historical diagnostic records as associated information.

[0134] The hospital's database contains historical diagnostic records, which save the diagnostic information for each patient throughout the historical diagnostic process. When a medical report needs to be generated for the current patient, the server can search the historical medical records to identify historical diagnostic information related to the patient's information, and use this information as the linking information.

[0135] A historical diagnostic record contains patient information and corresponding medical reports from a single diagnostic process. This historical diagnostic record can include medical reports, examination results, medical images, etc., from the patient's previous diagnostic process. Therefore, historical diagnostic records can consist of various data types, such as text data and image data. The server pre-inputs the historical diagnostic record into the encoding layer of the corresponding data type in the report generation model to obtain the encoded features of the historical diagnostic record.

[0136] After the server obtains the diagnostic features of the current patient information, it determines the similarity between the encoded features of each historical diagnostic information and the diagnostic features determined in step S1001. The encoded features of historical diagnostic information with a similarity greater than or equal to a preset value are then used as associated information.

[0137] This association information refers to information about historical patients similar to the patient's information, or medical cases related to the patient's information, which serve as prompts during the medical report generation process to guide the report generation model to produce more accurate medical reports.

[0138] S1005: Based on the associated information, determine the prompt information, input the prompt information and the diagnostic features into the report generation model, and generate a medical report.

[0139] The report generation model in this specification includes a decoding layer that can be configured as a large language model of any structure to generate medical reports based on diagnostic features and related information.

[0140] Based on the associated information, the server determines the prompt information, inputs the prompt information and diagnostic features into the report generation model, and obtains the medical report output by the decoding layer of the report generation model.

[0141] The medical report generation method described in this manual can generate reports based on patient information of various types, making it applicable to a wider range of medical diagnostic scenarios. Furthermore, this method uses relevant patient information to guide report generation, thereby improving the accuracy of medical reports. It can identify medical reports that better reflect the diseases or abnormalities described by the patient information, improving diagnostic efficiency for doctors while ensuring the accuracy of the generated medical reports.

[0142] In step S1001 above, when the patient information contains data of multiple data types, the diagnostic features of each type of data are obtained through different encoding layers.

[0143] In one or more embodiments of this specification, patient information includes image data and text data, and the report generation model may accordingly include an image encoding layer and a text encoding layer.

[0144] The server inputs image data into an image encoding layer to obtain image diagnostic features, and inputs text data into a text encoding layer to obtain text diagnostic features. Then, based on the image and text diagnostic features, the server determines the diagnostic features corresponding to the patient information.

[0145] Specifically, the server can use a neural network to fuse image diagnostic features and text diagnostic features to determine the diagnostic features. Alternatively, it can concatenate image diagnostic features and text diagnostic features to determine the diagnostic features.

[0146] Because the diagnostic features in this embodiment are determined based on image diagnostic features and text diagnostic features, the diagnostic features in this embodiment simultaneously include the semantics of image-type patient data and the semantics of text-type patient data. By using the semantic dimensions of multiple data types, the semantics of patient information are represented more completely, thereby enabling the report generation model to determine a more accurate medical report based on these diagnostic features.

[0147] The report generation model described in this manual can be used for auto-completion tasks.

[0148] In the autocomplete task, the prompt information in step S1005 above may also include preceding information input by the user. This preceding information can be input by the user via text or voice, and is used to guide the report generation model to predict the subsequent information based on this preceding information, outputting a medical report that matches the preceding information, thereby improving the accuracy of the medical report. In this manual, "user" refers to medical personnel, medical researchers, medical data analysts, and other individuals who need to obtain medical reports.

[0149] Specifically, the server obtains the preceding information of the medical report to be generated, input by the user. The server uses this preceding information and the associated information determined in step S1003 as prompt information. This prompt information, along with diagnostic features, is input into the decoding layer of the report generation model. The decoding layer then predicts the subsequent content of the preceding information to obtain the following information. Finally, the server generates the medical report based on the preceding information input by the user and the following information predicted by the report generation model.

[0150] In this embodiment, the preceding information input by the user is a small part of the content in the medical report. By automatically completing the task to predict the following information, the amount of input by the user is reduced, which improves the accuracy of the medical report while ensuring the efficiency of the medical report generation.

[0151] To further improve the accuracy of medical reports generated by the report generation model in the automatic completion task, in one or more embodiments of this specification, the subsequent information generated by the report generation model consists of multiple candidate information items. The report generation model predicts the preceding information to obtain multiple candidate information items. The server displays each candidate information item to the user. In response to the user's selection operation on each candidate information item, the server determines the candidate information item selected by the user. Then, the server determines the medical report based on the candidate information item selected by the user and the preceding information. This specification does not limit the specific number of candidate information items generated and can be set according to actual needs.

[0152] Figure 2 is a schematic diagram of a completion task provided in an embodiment of this specification. As shown in Figure 2, when the user inputs the preceding information "see in the upper lobe of the right lung", the server automatically predicts the following information and obtains five candidate information items, namely "punctate high-density shadow", "linear high-density shadow", "strip-like increased density shadow", "multiple solid nodule shadows", and "multiple nodules". The user can select one of the candidate information items according to the actual diagnosis of the patient.

[0153] The autocomplete task in this manual is a dynamic process. The server continuously acquires the preceding information entered by the user based on the previously entered information, determines the updated preceding information, and regenerates the candidate information based on the updated preceding information. Thus, as the preceding information entered by the user changes, the server automatically updates its predictions of the subsequent information to generate a medical report that better matches the user's intent.

[0154] For example, if the user continues to input information based on the preceding information in Figure 2, and obtains the preceding information "a stripe is seen in the upper lobe of the right lung", the server will automatically update the five candidate information items shown in Figure 2 to two: "cord-like high-density shadow" and "patchy increased density shadow", thus improving the accuracy of the generated medical report.

[0155] In one or more embodiments of this specification, in order to meet diverse medical report generation needs, the server can generate various types of medical reports, such as image-based report generation, report generation, and report structuring, based on the user's task prompts and through the report generation model.

[0156] When the server performs the task of generating at least one of the above types of medical reports, it needs to obtain task prompts from the user. The server can obtain task prompts in various ways, such as determining the task prompts in response to the user's operation on the graphical interface, or determining them based on the user's text or voice input.

[0157] When the server responds to the user's operation on the graphical interface to confirm the task prompt, the graphical interface may contain interface elements corresponding to the image view generation task, report generation task, and report structure task. In response to the user's selection of a certain interface element, the server determines the preset keywords of the medical report generation task corresponding to that interface element as a task prompt.

[0158] When the server determines the task prompt based on the user's voice or text input, it directly uses the user's text or voice input as the task prompt. After determining the task prompt, the server uses the task prompt and associated information as the prompt information, and inputs the prompt information and diagnostic features into the report generation model, so that the report generation model can generate a medical report that matches the medical report generation task corresponding to the task prompt.

[0159] In one or more embodiments of this specification, in order to obtain more accurate medical reports, the server may also obtain input information that matches the medical report generation task corresponding to the task prompt.

[0160] Different types of medical report generation tasks require different input information.

[0161] In the image view generation task, the task prompt is used to characterize the image view generation task, and the input information can be the contextual information of the image view to be generated. The server takes the determined task prompt, the contextual information of the image view to be generated, and related information as prompt information, and inputs this prompt information and diagnostic features into the report generation model to generate the contextual information of the contextual information of the image view to be generated, thus obtaining the image view. Then, the server displays the generated image view as a medical report to the user. That is, in this embodiment, the server determines the medical report through the above-mentioned automatic completion task.

[0162] In the report generation task, the task prompt characterizes the task itself. Input information can be images seen or background information from the initial diagnostic report. The initial diagnostic report is an unstructured medical report.

[0163] When the input information is the image view, the server uses the determined task prompts, image view, and related information as prompt information. Because the image view is part of the prompt information in this embodiment, the report generation model can generate an initial diagnostic report based on the image view. When the input information is the context of the initial diagnostic report, the server predicts the context of the unstructured medical report by automatically completing a task, thus obtaining the initial diagnostic report. After determining the initial diagnostic report, the server displays it to the user as a medical report.

[0164] The medical reports obtained from performing the report structuring task are structured medical reports categorized by body part or disease. Therefore, to obtain a structured medical report, the input information for this task includes a report template.

[0165] The report template contains multiple pre-set diagnostic items. This instruction manual does not restrict the criteria for classifying diagnostic items; the criteria can be determined according to symptoms, diseases, body parts, etc. For example, a diagnostic item classified according to symptoms may include lung shadows, lung nodules, lung consolidation, and intrapulmonary masses.

[0166] The report template can be a report template that the user selects from multiple report templates pre-stored on the server, or it can be a user-defined report template.

[0167] In the structured reporting task, task prompts are used to characterize the structured reporting task. The input information can be the initial diagnostic report or the preceding information in the structured medical report.

[0168] When the input information is the preceding information in a structured medical report, the server predicts the following information in the preceding information through an autocomplete task, thus obtaining the medical report.

[0169] When the input information for the structured reporting task is the initial diagnostic report.

[0170] First, the server determines the report template and the initial diagnostic report, retrieves each diagnostic item from the report template, and, based on the initial diagnostic report, determines the corresponding diagnostic text for each diagnostic item. The diagnostic text is a specific diagnostic description of the diagnostic item.

[0171] Then, based on the diagnostic text of each diagnostic item, the server determines the diagnostic conclusion for each diagnostic item and obtains a medical report.

[0172] Specifically, the server adds the diagnostic conclusions for each diagnostic item to the corresponding position in the report template. Finally, the medical report is generated based on the diagnostic conclusions for each diagnostic item added to the report template.

[0173] In this embodiment, the structured medical report format described in this specification has multiple formats to meet diverse medical report generation needs.

[0174] In one example of a report format, the server can present the diagnostic text for each diagnostic item as the diagnostic conclusion for that item. The medical report in this example uses the diagnostic text as the diagnostic conclusion, providing detailed information to facilitate a more thorough review of each diagnostic item by the doctor.

[0175] Figure 3 is a flowchart illustrating the generation process of a structured report in this embodiment. The left side of Figure 3 shows an unstructured initial diagnostic report, while the right side shows a structured medical report. In the embodiment shown in Figure 3, the diagnostic conclusion is the diagnostic text.

[0176] As shown in the right image of Figure 3, the structured report template contains a total of 18 diagnostic items, namely: lung shadows, lung nodules, lung masses, lung calcification, radiolucent shadows, lung consolidation, pneumonia, pulmonary tuberculosis, interstitial changes, atelectasis, pulmonary edema, pulmonary lymphadenopathy, pulmonary bullae / air sacs, emphysema, mediastinal masses, mediastinal calcification, mediastinal lymphadenopathy, and cardiac and vascular calcification.

[0177] Taking the diagnostic item "Lung Shadow" in Figure 3 as an example, in the initial diagnostic report, the server determines the diagnostic text corresponding to "Lung Shadow" as "Mixed density nodules are seen in the middle lobe of the right lung, with cavitation and fluid level visible inside, and linear shadows are seen around them, with pleural traction; scattered small irregular nodules are seen in both lungs, and small calcifications are seen in the upper lobe of the right lung; scattered patchy linear shadows are seen in both lungs." The diagnostic text corresponding to "Lung Shadow" is then used as the diagnostic conclusion for this diagnostic item and added to the corresponding position of "Lung Shadow" in the report template.

[0178] For a diagnostic item, if the server does not determine the corresponding diagnostic text for that item in the initial diagnostic report, the diagnostic conclusion for that item is set to the default text. The default text can be set to "None", "0", or any other content. As shown in Figure 3, there are multiple diagnostic items with a conclusion of "None". These are the diagnostic items for which the corresponding diagnostic text was not determined in the initial diagnostic report. The example in Figure 3 uses "None" as the default text.

[0179] In one example of a report format, the structured report consists of diagnostic text representing abnormal diagnostic items and judgment results. Therefore, after determining the diagnostic text for a diagnostic item, the server determines whether the corresponding diagnostic text indicates an anomaly. The server then uses the diagnostic text and judgment result for that diagnostic item as the diagnostic conclusion. Based on the diagnostic conclusions for each diagnostic item indicating anomalies, the server determines the structured medical report. This example medical report only contains the diagnostic text representing anomalies and the corresponding judgment results. The report is concise, allowing doctors to quickly notice abnormal diagnostic items and focus their review on those items based on the diagnostic text.

[0180] Figure 4 is a schematic diagram of a structured report corresponding to Figure 3 provided in this embodiment. Compared with Figure 3, it can be seen that Figure 4 only contains diagnostic conclusions for diagnostic items that indicate abnormalities. In the embodiment shown in Figure 4, the diagnostic conclusion consists of diagnostic text and judgment results.

[0181] In one example of a report format, the structured report consists of diagnostic conclusions for each diagnostic item, but does not include the diagnostic text for each item. This example of a medical report is more concise and structured, which is beneficial for standardizing the format of medical reports.

[0182] After determining the diagnostic text for a given diagnostic item, the server checks whether the corresponding diagnostic text indicates an anomaly. The server uses the result of this check as the diagnostic conclusion for that item. Based on the diagnostic conclusions for each diagnostic item, a structured medical report is generated.

[0183] Figure 5 is a schematic diagram of another structured report corresponding to Figure 3 provided in this embodiment. Compared with Figure 3, Figure 5 only includes the judgment results of each diagnostic item. In the embodiment shown in Figure 5, the diagnostic conclusion is the judgment result.

[0184] In a single report generation task, a user can choose to execute one of the above tasks: image view generation, report generation, and report structuring, or multiple tasks can be selected simultaneously.

[0185] The input information in this manual can be entered by the user or obtained from the server's own calculation results.

[0186] If only one type of task—image visualization generation, report generation, or report structuring—is performed during a single report generation process, then the input information matching the medical report generation task needs to be entered by the user.

[0187] If, during a single report generation process, at least two of the following tasks are executed sequentially: image visualization generation, report generation, and report structuring, then the input information for the first executed report generation task needs to be input by the user. The input information for subsequent report generation tasks can be the server's calculation result of the previously executed report generation task. In this embodiment, the server responds to user input, determines the input information for the first executed report generation task, and for subsequent executed report generation tasks, the server obtains its own output result from the previously executed report generation task as the input information for that subsequent task.

[0188] The server then uses the input information, task prompts, and related information as a prompt message. This prompt message and diagnostic features are then input into the report generation model to obtain a medical report.

[0189] For example, if the patient information contains image data corresponding to medical images, and the user selects to perform the above three tasks at the same time, the execution order of the three tasks is as follows: image view generation task, report generation task, and report structuring task. The server executes these three tasks in sequence to obtain a structured medical report.

[0190] In this embodiment, when the server performs the image view generation task, it can determine the image view through a completion task. After obtaining the image view, the server uses the image view obtained from the image view generation task as input information for the report generation task, and uses this input information as part of the report generation task prompt information. The prompt information and diagnostic features of the report generation task are then input into the report generation model to obtain an initial diagnostic report. Finally, the server uses the initial diagnostic report obtained from the report generation task as input information for the report structuring task, and uses this input information as part of the report structuring task prompt information. The prompt information and diagnostic features of the report structuring task are then input into the report generation model to obtain a structured medical report.

[0191] Therefore, in this embodiment, the input information for the image perception generation task needs to be obtained through user input. The input information for the report generation task and the input information for the report structure task are both calculation results of the server itself, and do not require user input again.

[0192] As can be seen, when a user selects to perform all three tasks simultaneously, the user only needs to input information in the first task, namely the image visualization generation task. The report generation model can then automatically execute the subsequent report generation and report structuring tasks to obtain a structured medical report. In other words, the user's workload is minimal during the medical report generation process, greatly improving the efficiency of medical report generation.

[0193] In addition to the image data, text data, voice data, and tabular data mentioned in step S1001, the patient information in this manual may also include the processing results of medical images by any functional machine learning model. For example, medical images obtained through image segmentation models that have been divided into regions, or medical images with annotated abnormal regions obtained through image recognition models.

[0194] The processing results of machine learning models with different functions can correspond to different encoding layers, which are used to encode the features of different processing results and obtain the diagnostic features corresponding to different processing results.

[0195] For example, if the patient information includes medical images with region division obtained through an image segmentation model, the report generation model can be configured to include a segmentation result encoding layer. The server inputs the region-divided medical images into the segmentation result encoding layer to obtain the segmentation result diagnostic features. If the patient information includes medical images with annotated abnormal regions obtained through an image recognition model, the report generation model can be configured to include a recognition result encoding layer. The server inputs the medical images with annotated abnormal regions into the recognition result encoding layer to obtain the recognition result diagnostic features.

[0196] The report generation model in this manual can be configured as one or more combinations of text encoding layers, image encoding layers, speech encoding layers, table encoding layers, segmentation result encoding layers, and recognition result encoding layers. Of course, the types of encoding layers that the report generation model can include are not limited to those mentioned above, and can be expanded according to the data types of patient information involved in the actual application.

[0197] When patient information contains data of multiple data types, each data type obtains corresponding features through different encoding layers. The server can then fuse the features of all data types to obtain the diagnostic features of the patient information.

[0198] In one or more embodiments of this specification, the report generation model in this specification may also include a fusion layer when the server fuses different types of features through a neural network.

[0199] The server inputs the features of patient information of different data types obtained from each encoding layer into the fusion layer. Through the fusion layer, cross-attention calculation is performed on the features of patient information of different data types to obtain diagnostic features.

[0200] In one or more embodiments of this specification, when the report generation model includes an image coding layer, in order to encode different types of medical images more specifically and obtain more accurate image diagnostic features, the image coding layer may include multiple sub-coding layers, each of which is used for feature coding of different types of medical images.

[0201] The server determines the sub-model corresponding to the image data contained in the patient information based on the medical image type, and inputs the image data into the sub-model to obtain image diagnostic features.

[0202] When patient information contains data of multiple data types, the server needs to fuse the features of all data types. However, because patient information of different data types is encoded through different encoding layers, different data types correspond to different vector spaces. Since the semantics of features are represented by distance in the vector space, the features of patient information of different data types differ in semantic expression. If the features of patient information of different data types are directly fused, the semantic integration effect between the features is not good.

[0203] Therefore, when patient data contains both text and non-text data, in order to more accurately integrate the features of different types of patient information semantically, the report generation model in this specification may also include a transfer layer. This transfer layer is used to map the features corresponding to non-text data to the vector space of text data features.

[0204] This transfer layer is configured corresponding to the encoding layer that encodes features for non-text data. That is, each encoding layer for non-text data in the report generation model can be configured with a corresponding transfer layer. Thus, the report generation model can include an image transfer layer corresponding to the image encoding layer, a speech transfer layer corresponding to the speech encoding layer, a segmentation transfer layer corresponding to the segmentation result encoding layer, a recognition transfer layer corresponding to the recognition result encoding layer, and so on.

[0205] When the report generation model includes a transfer layer, the server continues to input the features output by each encoding layer into the transfer layer to obtain the transfer features output by the transfer layer, and then fuses the transfer features with the text diagnostic features.

[0206] Taking a report generation model that includes an image encoding layer, a text encoding layer, and an image transfer layer corresponding to the image encoding layer as an example, the server inputs the image diagnostic features into the image transfer layer to obtain image transfer features. Then, the image transfer features are fused with the text diagnostic features to obtain diagnostic features.

[0207] Figure 6 is a flowchart of a medical report generation process provided in an embodiment of this specification. The report generation model in Figure 6 includes an image encoding layer, a text encoding layer, an image transfer layer, a fusion layer, and a decoding layer.

[0208] As shown in Figure 6, patient information includes image data and text data. The server inputs the image data into the image encoding layer to obtain image diagnostic features, and inputs the text data into the text encoding layer to obtain text diagnostic features. Then, the image diagnostic features are input into the image transfer layer to obtain image transfer features. The image transfer features and text diagnostic features are then input into the fusion layer to integrate the features corresponding to patient data of different data types to obtain diagnostic features.

[0209] Based on diagnostic features, the server identifies historical diagnostic information related to the patient's information from historical diagnostic records, using this as association information. Then, it acquires task prompts for at least one medical report generation task among image visualization, report generation, and report structuring tasks, as well as input information matching the corresponding medical report generation task. The task prompts, input information, and association information are then used as hint information. Finally, the hint information and diagnostic features are input into the decoding layer to obtain the medical report.

[0210] The medical report generation method described in this manual automates the generation of medical reports through a report generation model, reducing the workload for users in the process of generating medical reports and improving diagnostic efficiency.

[0211] The above content has introduced various embodiments of the report generation model in the application stage for generating medical reports. The following will introduce the training method of the report generation model in this specification. The server uses a layer-by-layer training method to train each neural network layer contained in the report generation model. Different report generation models have different structures and correspondingly different training methods.

[0212] In one or more embodiments of this specification, the report generation model includes an encoding layer corresponding to patient information of at least one data type, and a decoding layer.

[0213] The server first trains the encoding layer. After the encoding layer training is complete, the decoding layer is trained with its parameters fixed. When the report generation model contains encoding layers corresponding to multiple data types, this specification does not restrict the training order of the encoding layers corresponding to each data type.

[0214] Figure 7 is a schematic diagram of the structure of a report generation model provided in an embodiment of this specification. As shown in Figure 7, the report generation model includes an image encoding layer, a text encoding layer, and a decoding layer. Taking the report generation model shown in Figure 7 as an example, the training method of the report generation model will be specifically explained.

[0215] The report generation model in this specification also includes an auxiliary coding layer, which is a text encoder. The server trains the image coding layer and the auxiliary coding layer through contrastive learning. The training method for the image coding layer is as follows:

[0216] First, the server retrieves each medical image from historical diagnostic records as a sample image and determines the corresponding text description for each sample image.

[0217] Next, the server inputs each sample image into the image encoding layer to be trained, and the corresponding text description for each sample image into the auxiliary encoding layer to be trained. Through a contrastive learning training method, the model parameters of both the image encoding layer and the auxiliary encoding layer are adjusted.

[0218] The report generation model in this specification also includes a first auxiliary decoding layer. The text encoding layer is trained together with the first auxiliary decoding layer. The training method of the text encoding layer is as follows:

[0219] First, the service retrieves historical diagnostic text from historical diagnostic records, masks certain text segments within the historical diagnostic text, and obtains sample text. Second, the server inputs this sample text into the text encoding layer to be trained, obtaining the text features of the sample text. Then, the text features of the sample text are input into the first auxiliary decoding layer to predict the masked local text segments, obtaining the predicted text segments. Finally, the server adjusts the model parameters of the text encoding layer and the first auxiliary decoding layer based on the difference between the masked local text segments and the predicted text segments.

[0220] The training order of the image coding layer and text coding layer is not important; they can be trained simultaneously or sequentially. After the image coding layer and text coding layer are trained, the server fixes the model parameters of the image coding layer and text coding layer, and trains the decoding layer as follows:

[0221] First, the server retrieves each historical diagnosis text from the historical diagnosis records, as well as the medical images and patient text data at the time each historical diagnosis text was obtained. For each historical diagnosis text, the medical image obtained at the time of the historical diagnosis text is used as a sample image, and the patient text data obtained at the time of the historical diagnosis text is used as a sample text.

[0222] Next, the server inputs the sample image into the trained image encoding layer to obtain image features, and the sample text into the trained text encoding layer to obtain text features. Then, the server fuses the image features and text features to obtain diagnostic features, which are then input into the decoding layer to be trained to obtain the predicted diagnostic text. Based on the difference between the historical diagnostic text and the predicted diagnostic text output by the decoding layer, the model parameters of the decoding layer to be trained are adjusted.

[0223] Based on the model structure shown in Figure 7, the training methods for two types of coding layers, namely text coding layer and image coding layer, are explained. Image coding layer is a non-text data coding layer. When there are non-text data coding layers such as speech coding layer, table coding layer, segmentation result coding layer, and recognition result coding layer that can be extended according to the patient's data type, each non-text data coding layer is trained separately. The training method is similar to that of image coding layer. For details, please refer to the description of the training process of image coding layer.

[0224] As discussed above, when generating medical reports during the application phase, the server can use the user's text or voice input as a task prompt and incorporate this task prompt as part of the prompt information for the generation of the medical report.

[0225] In this approach, the task prompts are diverse due to the flexibility in relying on user text or voice input. To ensure the report generation model can accurately identify the corresponding medical report generation task based on these diverse prompts and generate a medical report matching the prompt, further steps are needed.

[0226] In one or more embodiments of this specification, after the decoding layer is trained, the server fine-tunes the report generation model based on the task prompt samples.

[0227] For each type of medical report generation task, multiple task prompts with different expressions that match that type of medical report generation task are used as task prompt samples for that type of medical report generation task.

[0228] First, the server retrieves historical diagnostic texts for each type of medical report from historical diagnostic records, as well as medical images and patient text data at the time each historical diagnostic text was obtained.

[0229] The historical diagnostic records contain historical diagnostic texts corresponding to three types of medical reports for the three medical report generation tasks mentioned above: image findings for the image findings generation task, unstructured initial diagnostic reports for the report generation task, and structured reports for the report structure task.

[0230] For each historical diagnosis text, the medical image at the time of the diagnosis is obtained as the sample image, and the patient's text data at the time of the diagnosis is obtained as the sample text. Simultaneously, a task prompt sample corresponding to the historical diagnosis text is determined. This task prompt sample is used to prompt the report generation model to execute the medical report generation task corresponding to that medical report type.

[0231] Next, the server inputs the sample image into the trained image encoding layer to obtain image features. The sample text is then input into the trained text encoding layer to obtain text features. Finally, the image features and text features are fused to obtain diagnostic features.

[0232] Then, based on the diagnostic features, the server identifies historical diagnostic information related to the sample image and sample text in the historical diagnostic records, which is used as association information. The task prompt sample and association information are then used as prompt information. This prompt information and the diagnostic features are then input into the trained decoding layer to obtain the predicted diagnostic text output by the decoding layer.

[0233] Finally, the server determines the difference between the predicted diagnostic text and the historical diagnostic text, and fine-tunes the model parameters of the decoding layer with the goal of minimizing this difference.

[0234] Based on the discussion in the application section above, each encoding layer for non-text data in the report generation model can be configured with a corresponding transfer layer. The training method for the transfer layer is the same for different types of non-text data. Here, we will only take the image transfer layer as an example to explain the training method of the transfer layer.

[0235] After the entire coding layer has been trained, the server fixes the model parameters of the coding layer and then trains the image transfer layer.

[0236] Taking an image coding layer and a text coding layer as an example, the training process of the image transfer layer is explained. The server can train the image transfer layer according to various loss functions.

[0237] In one or more embodiments of this specification, the server trains the image transfer layer based on the Image-Text Contrastive Loss (ITC). The specific method for determining the Image-Text Contrastive Loss is as follows:

[0238] First, the server retrieves medical images from historical diagnostic records as sample images and determines the corresponding text descriptions for each sample image. Then, the server inputs each sample image into the trained image encoding layer and the corresponding text descriptions into the trained auxiliary encoding layer. Through a contrastive learning task, the image-text contrast loss is determined.

[0239] In one or more embodiments of this specification, the server trains an image transfer layer based on the Image-Text Matching Loss (ITM). The report generation model also includes a prediction layer, and the specific method for determining the Image-Text Matching Loss is as follows:

[0240] First, the server retrieves medical images from historical diagnostic records as sample images and determines the corresponding text descriptions for each sample image. Then, the server inputs each sample image into the trained image encoding layer and the corresponding text descriptions into the trained auxiliary encoding layer. Through an image-text matching task, the image-text matching loss is determined.

[0241] In one or more embodiments of this specification, the server trains an image transfer layer based on the Image-Grounded Text Generation (ITG) loss. The report generation model to be trained then includes an auxiliary encoding layer and a second auxiliary decoding layer. The specific method for determining the image-grounded text generation loss is as follows:

[0242] First, the server retrieves medical images from historical diagnostic records as sample images and determines the corresponding text descriptions for each sample image. Then, the server inputs each sample image into the trained image encoding layer and the corresponding text descriptions into the trained auxiliary encoding layer. Through an image-text generation task, the image-text generation loss is determined.

[0243] The above three examples of training image transfer layers can be used individually or in any combination.

[0244] When the server trains the image transfer layer using three loss functions simultaneously—image-text contrast loss, image-text matching loss, and image-text generation loss—the model structure of the report generation model is shown in Figure 8.

[0245] Figure 8 is a schematic diagram of another report generation model provided in the embodiments of this specification. The report generation model includes an image encoding layer, an auxiliary encoding layer, a text encoding layer, a first auxiliary decoding layer, an image transfer layer, a prediction layer, and a decoding layer.

[0246] In one or more embodiments of this specification, the report generation model may further include a fusion layer for fusing diagnostic features corresponding to various types of patient information. This fusion layer can be trained together with the decoding layer, as follows:

[0247] First, the server retrieves each historical diagnosis text from the historical diagnosis records, as well as the medical images and patient text data at the time the historical diagnosis text was obtained. For each historical diagnosis text, the medical image at the time the historical diagnosis text was obtained is used as a sample image, and the patient text data at the time the historical diagnosis text was obtained is used as a sample text.

[0248] Next, the server inputs the sample image into the trained image encoding layer to obtain image features. The sample text is then input into the trained text encoding layer to obtain text features. Finally, the image features and text features are input into the fusion layer to be trained to obtain the diagnostic features output by the fusion layer.

[0249] The server then inputs the diagnostic features into the decoding layer to be trained to obtain the predicted diagnostic text. Based on the difference between the historical diagnostic text and the predicted diagnostic text, the model parameters of the fusion layer and the decoding layer to be trained are adjusted.

[0250] The above is the method for generating medical reports provided in this manual. Based on the same idea, this manual also provides a corresponding medical report generation device, as shown in Figure 9.

[0251] Figure 9 is a schematic diagram of a medical report generation device provided in this specification, specifically including:

[0252] The diagnostic feature determination module 1021 is used to acquire patient information, input the patient information into the report generation model, and obtain diagnostic features.

[0253] The retrieval module 1023 is used to determine, based on the diagnostic features, historical diagnostic information related to the patient information in historical diagnostic records, as associated information;

[0254] The generation module 1025 is used to determine the prompt information based on the associated information, input the prompt information and the diagnostic features into the report generation model, and generate a medical report.

[0255] Optionally, the report generation model includes an image encoding layer and a text encoding layer, the patient information includes image data and text data, and the diagnostic feature determination module 1021 is specifically used to input the image data into the image encoding layer to obtain image diagnostic features, input the text data into the text encoding layer to obtain text diagnostic features, and determine diagnostic features based on the image diagnostic features and the text diagnostic features.

[0256] Optionally, the retrieval module 1023 is specifically used to determine the coding features of each historical diagnostic information contained in the historical diagnostic record, determine the similarity between the coding features of each historical diagnostic information and the diagnostic features, and use the coding features of historical diagnostic information with a similarity greater than or equal to a preset value as associated information.

[0257] Optionally, the generation module 1025 is specifically used to obtain the preceding information of the medical report to be generated input by the user, use the preceding information and the associated information as prompt information, input the prompt information and the diagnostic features into the report generation model, predict the subsequent content of the preceding information through the report generation model to obtain the following information of the preceding information, and determine the medical report based on the preceding information and the following information.

[0258] Optionally, the subsequent information consists of multiple candidate information items. The generation module 1025 is specifically used to respond to the user's selection operation on each candidate information item, determine the candidate information selected by the user, and determine the medical report based on the preceding information and the candidate information selected by the user.

[0259] Optionally, the medical report is a structured report. The generation module 1025 is specifically used to determine task prompts based on user operations. The task prompts are used to characterize the structured task of the report. The module determines the report template and the initial diagnostic report, uses the task prompts and the associated information as prompt information, and inputs the prompt information and the diagnostic features into the report generation model to obtain the medical report.

[0260] Optionally, the generation module 1025 is specifically used to determine a report template and an initial diagnostic report, obtain each diagnostic item in the report template, determine the diagnostic text corresponding to the diagnostic item based on the initial diagnostic report, determine the diagnostic conclusion of the diagnostic item based on the diagnostic text, and determine a medical report based on the diagnostic conclusion.

[0261] Optionally, the generation module 1025 is specifically used to determine that the judgment result of the diagnostic item is abnormal when it is determined that the diagnostic text corresponding to the diagnostic item indicates that there is an anomaly, and to take the diagnostic text of the diagnostic item and the judgment result of the diagnostic item as the diagnostic conclusion of the diagnostic item.

[0262] This specification also provides a computer-readable storage medium storing a computer program that can be used to execute the medical report generation method shown in Figure 1 above.

[0263] This specification also provides a schematic structural diagram of the electronic device shown in Figure 10. As shown in Figure 10, at the hardware level, the electronic device includes a processor, an internal bus, a network interface, memory, and non-volatile memory, and may also include other hardware required for business operations. The processor reads the corresponding computer program from the non-volatile memory into memory and then runs it to implement the medical report generation method described in Figure 1. Of course, in addition to the software implementation, this specification does not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. That is to say, the execution subject of the following processing flow is not limited to individual logic units, but can also be hardware or logic devices.

[0264] Improvements in a technology can be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many improvements to the methodology can now be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that an improvement in methodology cannot be implemented using hardware physical modules. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program and "integrate" a digital system onto a PLD themselves, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must also be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also understand that by simply performing some logic programming on the method flow using one of these hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0265] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0266] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0267] For ease of description, the above devices are described in terms of function, divided into various units. Of course, in implementing this specification, the functions of each unit can be implemented in one or more software and / or hardware components.

[0268] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0269] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in one or more flowchart illustrations and / or one or more block diagrams.

[0270] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowcharts and / or one or more block diagrams.

[0271] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide steps for implementing the functions specified in one or more flowcharts and / or one or more block diagrams.

[0272] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0273] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0274] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0275] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0276] Those skilled in the art will understand that the embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this specification may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0277] This specification can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This specification can also be practiced in distributed computing environments, where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0278] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0279] The above description is merely an embodiment of this specification and is not intended to limit this specification. Various modifications and variations can be made to this specification by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of the claims of this application.

[0280] Figure 11 is a flowchart illustrating another medical report generation method described in this specification, which includes the following steps:

[0281] S2001: Determine and display the initial diagnostic report.

[0282] All steps in the medical report generation method provided in this manual can be implemented by any electronic device with computing capabilities, such as a terminal or server. For ease of description, the following explanation focuses on a server as the executing entity.

[0283] Due to standardization requirements, medical reports typically contain many relatively fixed descriptive statements, such as diagnostic conclusions for normal body parts or commonly used diagnostic conclusions in specialist clinics. Therefore, these relatively fixed descriptive statements can be used as pre-defined diagnostic texts, and a pre-defined report template can be determined based on the pre-defined diagnostic texts for each body part.

[0284] This manual does not limit the specific content of each human body part included in the report template. It can be set to any human body location, such as the chest, lungs, spine, or any human organ, or a component structure of a human organ, such as the left lung or lymph nodes, according to actual diagnostic needs.

[0285] This specification does not limit the number of preset report templates. In this method, only one preset report template can be set, or multiple report templates can be set according to structural or functional similarity. When a user determines a medical report using the method provided in this specification, they can select one or more suitable report templates based on actual diagnostic needs. Alternatively, in one or more embodiments of this specification, the server can also support user-defined report templates to adapt to diagnostic needs; that is, which body part is included in the report template, and the preset diagnostic text for each body part, can be set by the user.

[0286] The server can then determine the preset report template for the medical report generation process based on the pre-stored report template or the user-defined report template, and determine the initial diagnostic report based on the preset diagnostic text for each body part contained in the report template.

[0287] There are several ways to determine the initial diagnostic report. In one or more embodiments of this specification, the server directly uses the report template as the initial diagnostic report and displays the initial diagnostic report to the user so that the user can view the initial diagnostic report and make adjustments to it.

[0288] S2003: Obtain the input information of the user based on the initial diagnostic report, predict the user's subsequent input based on the input information, and determine the prediction information.

[0289] In the medical report generation method provided in this manual, after the server displays the initial diagnostic report to the user, the user adjusts the initial diagnostic report by inputting information according to the actual diagnosis, thereby obtaining a medical report.

[0290] The input information in this manual can be either voice or text. Users can choose the appropriate input method as needed.

[0291] During the user's input process, the server predicts the user's subsequent input based on the user's input information and determines the predicted information.

[0292] This prediction information is based on the semantics of the input information and predicts the subsequent text content. The generation process of the prediction information can be implemented using language models in the field of natural language processing, such as the GPT series models and the BERT model. The server can use a large corpus of commonly used diagnostic conclusions in the medical field as sample data to train the above-mentioned language model, obtaining a first language model for subsequent text prediction tasks. This first language model can understand the semantics of the user input information and predict the subsequent text.

[0293] Specifically, the server inputs the user's input information into the first language model in real time, enabling the first language model to predict the subsequent content of the input information and obtain the predicted information output by the first language model.

[0294] The integrated input and predicted information can be used to adjust the initial diagnostic report. This method eliminates the need for users to write medical reports from scratch; they only need to input a small amount of background information to adjust the report, significantly reducing their workload during the report generation process.

[0295] S2005: Adjust the initial diagnostic report based on the input information and the prediction information to determine the medical report.

[0296] The server can integrate user input and prediction information to determine adjustment information, and use the integrated adjustment information to adjust the initial diagnostic report.

[0297] Specifically, when the input information is speech, the server converts it into text. Based on the semantics of the text and the semantics of the predicted information, the server integrates the text and the adjustment information to determine the adjustment information. When the input information is text, the server integrates the input text and the predicted information to determine the adjustment information.

[0298] Then, the server performs semantic analysis on the adjustment information. Based on the semantics of the adjustment information, it identifies a text segment in the initial diagnostic report that is semantically consistent with the adjustment information, and adjusts the text segment accordingly. Based on the adjusted initial diagnostic report, the medical report is determined.

[0299] Semantic analysis, or the identification and parsing of the contextual semantics of natural language text, enables the execution of various natural language processing tasks based on the results. The medical report generation method described in this specification involves multiple semantic analysis steps, each corresponding to a different natural language processing task. The semantic analysis process can be implemented using a pre-trained second language model. This second language model can perform semantic analysis based on different task prompts and execute the corresponding natural language processing task.

[0300] The semantic analysis operations described above are used to identify text fragments in the initial diagnostic report that are consistent with the adjustment information. These text fragments can be one or more words, or one or more sentences, within the initial diagnostic report, and the specific content of these text fragments must be semantically relevant to the adjustment information.

[0301] The adjustment operations here can include three types: deletion, addition, and partial modification of existing content. Addition and partial modification of existing content involve semantic integration of the adjusted information and the corresponding text fragments. Deletion is performed on the text fragments corresponding to the adjusted information and does not require semantic integration.

[0302] Therefore, in the adjustment operation described in this manual, the server performs semantic analysis on the adjustment information to determine whether the adjustment information is a deletion operation. If the adjustment information is not a deletion operation, semantic analysis is performed on the adjustment information again, and the adjustment information is semantically integrated with the corresponding text fragment based on the semantics of the adjustment information. The first semantic analysis operation in this step is used to identify the deletion operation, and the second semantic analysis operation is used to semantically integrate the adjustment information with the corresponding text fragment based on the semantics of the adjustment information.

[0303] Specifically, the server determines whether the adjustment information contains keywords that represent the semantics of deletion. If so, the text fragment in the initial diagnostic report is deleted. If not, semantic analysis of the adjustment information and the text fragment continues. If it is determined that the semantics of the adjustment information and the text fragment contradict each other, the text fragment is replaced with the adjustment information. If it is determined that the semantics of the adjustment information and the text fragment overlap, the adjustment information and the text fragment are merged.

[0304] In this embodiment, because the predicted information is a prediction of the subsequent content of the input information, absolute accuracy cannot be guaranteed. To improve the accuracy of adjusting the medical report, the server can first display the predicted information to the user, respond to the user's confirmation of the predicted information, and then determine the adjustment information based on the predicted information and the input information to adjust the initial diagnostic report.

[0305] The initial diagnostic report in this manual is based on a report template. The template contains fixed descriptions commonly used in diagnostic scenarios. However, in actual diagnosis, patients typically only have lesions or abnormalities in a few areas. Therefore, users do not need to modify most of the initial diagnostic report; they only need to adjust the text segments corresponding to the areas with lesions or abnormalities. Thus, users usually only need to adjust a small portion of the initial diagnostic report, improving the efficiency of medical report generation.

[0306] By applying the medical report generation method described in this manual, users only need to input part of the preceding text in the adjustment information used to adjust the initial diagnostic report. The server can then automatically predict the user's subsequent input. Users do not need to input all the adjustment information. Through the generation of predictive information during the user's input process, the efficiency of medical report generation is further improved.

[0307] In step S2003 above, to improve the accuracy of the predicted information, in one or more embodiments of this specification, the server can predict the user's subsequent input based on the input information, determine multiple candidate information, and display each candidate information to the user. This specification does not limit the number of candidate information; it can be set according to requirements.

[0308] Then, in response to the user's selection of each candidate information, the selected candidate information is used as prediction information to adjust the initial diagnostic report.

[0309] Furthermore, in order to improve the efficiency of users in making decisions on predictive information from among the candidate information, in another embodiment, the server can determine the matching probability between each candidate information and the input information.

[0310] In this embodiment, each candidate piece of information is also output by the first language model in step S2003 above. The server can obtain the confidence level of each candidate piece of information when the first language model predicts each candidate piece of information, as the matching probability between each candidate piece of information and the input information. The confidence level of a candidate piece of information is the probability that the first language model considers the candidate piece of information to be consistent with the subsequent content of the actual input information during the model inference process.

[0311] Then, based on the magnitude of each matching probability, the candidate information corresponding to each matching probability is sorted, and the sorted candidate information is displayed to the user. The sorting method can be selected as needed, either from highest to lowest matching probability or from lowest to highest.

[0312] Because the candidate information with the highest matching probability corresponds to the highest confidence level, it means that the first language model, during reasoning, considers the candidate information to be more likely to match the subsequent content of the actual input information. Therefore, this candidate information is the one the user is most likely to choose. This embodiment allows users to notice the candidate information with the highest matching probability more quickly and make a selection, thereby further improving the efficiency of medical report generation.

[0313] Figure 12 is a schematic diagram of a prediction information generation process provided in an embodiment of this specification. As shown in Figure 12, when the input information is "left lower lobe posterior basal segment", the server generates four candidate information, namely "solid nodule shadow", "ground-glass nodule shadow", "linear shadow", and "patchy high-density shadow". As shown on the right side of Figure 12, the server displays the four generated candidate information to the user. If the user selects the candidate information "solid nodule shadow", the server determines "solid nodule shadow" as the prediction information, which is used to adjust the initial diagnostic report.

[0314] In one or more embodiments of this specification, the prediction information may also include a missing value, which is a value related to the morphology of the actual lesion or abnormal area.

[0315] The server responds to the user's selection of each candidate information, determines the prediction information, and then the user inputs the missing values ​​in the prediction information according to the actual diagnosis to obtain the complete prediction information. The complete prediction information and the input information are then integrated to adjust the initial diagnosis report.

[0316] Figure 13 is a schematic diagram of another prediction information generation process provided in the embodiments of this specification. As shown in Figure 13, when the input information is "a solid nodule shadow is seen in the posterior basal segment of the left lower lobe", the server generates two candidate information, namely "size approximately cm × cm" and "long diameter approximately cm". As shown on the right side of Figure 3, the server displays the two generated candidate information to the user. If the user selects the candidate information "size approximately cm × cm", the server determines "size approximately cm × cm" as the prediction information.

[0317] In one or more embodiments of this specification, since the initial diagnostic texts in the report template are divided according to human body parts, in the above step S2005, before adjusting the initial diagnostic report, the server can first perform part identification to determine the human body parts corresponding to the input information and the predicted information, and adjust the initial diagnostic texts for the corresponding human body parts to improve the accuracy of the medical report.

[0318] The server determines adjustment information based on the input and prediction information. Then, based on the semantics of the adjustment information, it identifies the corresponding body part from among the various body parts included in the initial diagnostic report, designating it as the target body part. Based on this adjustment information, the initial diagnostic text for the target body part in the initial diagnostic report is adjusted to finalize the medical report.

[0319] This manual does not limit the number of target sites. That is, a single input message from the user may contain only the adjustment content of the initial diagnostic text for one target site, or it may contain the adjustment content of the initial diagnostic text for multiple target sites.

[0320] When there are multiple target areas, the server performs semantic parsing on the adjustment information and determines the adjustment content for each target area based on the semantics of the adjustment information. Then, for each target area, the initial diagnostic text for that target area is adjusted according to the adjustment content for that target area.

[0321] The operation of adjusting the initial diagnostic text of the target area is similar to the adjustment operation in step S2005 above, and may also include three types: deletion, addition, and partial modification of the original content.

[0322] Specifically, the server determines whether the adjustments made to the target area contain keywords that represent the meaning of deletion.

[0323] When the adjustment content of the target part contains keywords that represent deletion semantics, the initial diagnostic text of the target part is deleted.

[0324] When the adjustment content of the target part does not contain keywords that represent deletion semantics, the adjustment content of the target part is integrated with the initial diagnostic text of the target part.

[0325] To perform the integration operation more accurately, the server can continue to perform semantic analysis on the adjustment content of the target part to determine whether the adjustment content of the target part is semantically contradictory to the initial diagnostic text of the target part or semantically repetitive.

[0326] When the server determines that the adjustment content for the target part contradicts the semantics of the initial diagnostic text for the target part, it replaces the initial diagnostic text for the target part with the adjustment content. When the server determines that the semantics of the adjustment content for the target part overlaps with the initial diagnostic text for the target part, it merges the adjustment content for the target part with the initial diagnostic text for the target part.

[0327] In one or more embodiments of this specification, to improve the generation efficiency of the medical report generation method, in step S2003 above, before the server determines the predicted information, the input information can be type-identified to determine the type of the input information. Here, the type of input information includes report type and non-report type.

[0328] Especially when the input information is voice information, users are easily affected by environmental factors when inputting voice information. For example, in a noisy environment, the server may record casual conversations or background noise that are unrelated to report generation, interfering with the generation of medical reports.

[0329] When the server determines that the input information is a report, it predicts the subsequent content of that input. When the server determines that the voice information is not a report, it discards the voice information and does not process it further, thus saving computing resources.

[0330] Through this embodiment, the server can filter out non-report-related input information that the user may enter in advance, so as to prevent the server from generating irrelevant input information based on the medical report and affecting the accuracy of the report.

[0331] In one or more embodiments of this specification, to further facilitate user operation, the input information entered by the user can be instruction information. The user sends instructions to the server through the input information, and the server performs the corresponding operation according to the instructions sent by the user.

[0332] The input information can be classified as report-based, non-report-based, or instruction-based. When the server determines that the input information is an instruction, it identifies the matching instruction from among the preset instructions and uses it as the target instruction, then executes the operation corresponding to the target instruction.

[0333] Specifically, the server can determine the keywords of each preset instruction, then determine the text information corresponding to the input information, and judge whether the text information matches the semantics of the keywords of any preset instruction. If so, the preset instruction corresponding to the keyword matched by the text information is taken as the target instruction.

[0334] For example, in one scenario, after a user has adjusted the initial diagnostic text for various body parts, they typically need to manually send a confirmation command to the server via page operations. The server then responds to the user's confirmation command by generating a medical report. If the keyword in the confirmation command is "generate report," the server will execute the operation of generating the medical report if it determines that the user's input contains the phrase "generate report" or a phrase with the same semantic meaning.

[0335] Medical reports typically require standardized professional terminology, but user input may contain non-standard terms, such as non-professional terms or omitting descriptions.

[0336] When the input information is voice information, it may also contain some colloquial content, such as repeated words, interjections, pause words, etc.

[0337] Therefore, in one or more embodiments of this specification, before adjusting the initial diagnostic report in step S2005 above, the server can perform standardization processing on the input information. This standardization processing includes removing duplicate words, interjections, and pause words, and converting non-medical terms into corresponding medical terms. This allows for the determination of a more standardized medical report after adjusting the initial diagnostic report using input and predicted information.

[0338] For example, if the user's input information is "trachea normal", this input information is an abbreviated description. The standardized input information determined by this embodiment is "trachea and bronchi are unobstructed". If the user's input information is "thoracic vertebrae or lumbar vertebrae osteophyte", this input information contains the pause word "or lumbar vertebrae". The standardized input information determined by this embodiment is "lumbar vertebrae osteophyte".

[0339] In one or more embodiments of this specification, the initial diagnostic text of a human body part in the initial diagnostic report includes diagnostic results for different locations within that human body part. For example, if a human body part in the initial diagnostic report is the lung, the initial diagnostic text for that human body part may include diagnostic results for the left lung and diagnostic results for the right lung.

[0340] When a user adjusts the initial diagnostic report by inputting information, they may only adjust part of the diagnostic results or a few words in the initial diagnostic text for the target area, and will not adjust the entire content of the initial diagnostic text.

[0341] After the server identifies the affected area, it can determine the text segment corresponding to the adjustment information in the initial diagnostic text of the target area, following the method described in step S2005 above, and then adjust that text segment. The adjustments here are the same as those in step S2005 above, including three types: deletion, addition, and partial modification of existing content.

[0342] For example, in the initial diagnostic report, a body part called "pleural cavity / pleura / diaphragm" might be included. The initial diagnostic text for this body part might be: No pleural effusion was observed bilaterally; no significant thickening of the pleura was observed bilaterally. It is evident that this initial diagnostic text includes diagnostic results for two different structural regions: the pleural cavity and the pleura.

[0343] If the input information is "slight thickening of the right pleura", then using the method in this embodiment, the server can use "pleural cavity / pleura / diaphragm" as the target site, and determine "no obvious thickening of the bilateral pleura" as the text segment corresponding to the input information in the initial diagnostic text for this body part, "no effusion seen in both pleural cavities; no obvious thickening of the bilateral pleura". Since there is a semantic contradiction between the input information and the semantics of this text segment, the server replaces the text segment with the input information, resulting in the adjusted initial diagnostic text for the target site as "no effusion seen in both pleural cavities; slight thickening of the right pleura".

[0344] If the input information is "delete the description about the right pleura", then using the method in this embodiment, the server can use "pleural cavity / pleura / diaphragm" as the target site, and determine "no obvious thickening of the pleura on both sides" as the text segment corresponding to the input information in the initial diagnostic text for this body part, "no effusion in both pleural cavities; no obvious thickening of the pleura on both sides". Because the input information contains the semantics of deletion, the server deletes the text segment according to the input information, resulting in the adjusted initial diagnostic text for the target site being "no effusion in both pleural cavities".

[0345] In one or more embodiments of this specification, the medical report generation method provided herein can also be used to generate medical reports on medical images.

[0346] In step S2001 above, the server can also determine the initial diagnostic report according to the following method.

[0347] First, the server acquires medical images and identifies the region of interest within them.

[0348] This specification does not limit the specific type of medical image; it can be any type of medical image. In this embodiment, the specific method for determining the region of interest is not limited; it can be obtained using any machine learning model for image processing tasks, through manual annotation, or simultaneously using both machine learning models and manual annotation.

[0349] Secondly, the server identifies the corresponding regions of interest (ROIs) within each human body part, designating them as regions of focus. These ROIs are thus marked as regions of interest within the image area of ​​that human body part.

[0350] Due to the characteristics of medical images, the server can determine one or more regions of interest, and consequently, the number of areas of interest can also be one or more.

[0351] Then, the server determines the initial diagnostic text for the areas of interest based on the region of interest, and generates an initial diagnostic report based on the initial diagnostic text for the areas of interest. This initial diagnostic report consists of the initial diagnostic text for each area of ​​interest.

[0352] Because, in this embodiment, not all human body parts are necessarily marked with regions of interest, the determined areas of interest are only a portion of all human body parts, and the initial diagnostic report only includes the initial diagnostic text of this portion of the body parts.

[0353] In another embodiment, to make the initial diagnostic report more comprehensive, the server designates human body parts that do not correspond to a region of interest (ROI) as non-ROI areas. A human body part without a corresponding ROI is one whose ROI is not marked within the image area of ​​that body part.

[0354] The server uses the preset diagnostic text for each non-interested area in the report template as the initial diagnostic text for that area. Therefore, the initial diagnostic report includes the initial diagnostic text for each interest area determined based on the region of interest, as well as the preset diagnostic text for each non-interested area.

[0355] Compared to the method in step S2001, which determines the initial diagnostic text entirely based on a preset template, this embodiment uses the initial diagnostic text of the area of ​​interest determined by the region of interest. This makes the content of the initial diagnostic report more accurate, thereby reducing the amount of voice information or text information that users need to adjust through voice input or keyboard input, and further improving the efficiency of report generation.

[0356] When the server determines the region of interest based on the machine learning model, to facilitate user viewing, the server can highlight the initial diagnostic text of the corresponding human body part in the initial diagnostic report, reminding the user to verify. This manual does not limit the specific form of this highlighting effect; any display effect such as highlighting, bolding, or flashing can be selected according to actual needs.

[0357] The method of this embodiment will be described below with reference to Figure 14, using a CT chest scan as an example. Figure 14 is a schematic diagram of an initial diagnostic report provided in this embodiment. As shown in Figure 14, the initial template includes seven parts for each human body part: lungs, trachea, mediastinum, heart and major blood vessels, pleural cavity / pleura / diaphragm, bones, and others. The initial diagnostic text for each human body part is listed in the initial diagnostic report. The bolded parts in Figure 14, namely the initial diagnostic text corresponding to the lungs and bones, represent the initial diagnostic text for the human body part corresponding to the region of interest determined by the machine learning model.

[0358] In the context of generating medical reports from medical images, it is typically necessary to first determine the image findings. Image findings describe the directly observed image information, and compared to the medical report, the content in image findings is usually more specialized. For example, image findings may include specialized image data such as density, signal intensity, morphology, size, and location. After determining the image findings, the corresponding image information is then used to determine the medical report.

[0359] In one or more embodiments of this specification, the preset diagnostic text for each body part in the report template can be image information describing medical images. Then, the initial diagnostic report determined in step S2001 above is used to describe the image information of medical images and represent the initial content seen in the images.

[0360] Users view medical images and, based on the information observed in the images, verify the content of the initial diagnostic report displayed by the server. When it is determined that there is content in the initial diagnostic report that needs to be adjusted, users input voice or text information. The server determines the predicted information based on the user's voice or text information and the generated predicted information, and adjusts the initial diagnostic text for each part of the body accordingly.

[0361] In this embodiment, the server determines the adjusted initial diagnostic text for each body part, which is used to describe the adjusted image information. Then, the server inputs the adjusted initial diagnostic text for each body part into a third-language model to generate a medical image examination conclusion. Based on the adjusted initial diagnostic report and the generated medical image examination conclusion, the server determines a medical report. The third-language model in this embodiment can be any trained machine learning model for natural language processing tasks.

[0362] In this embodiment, since the content of the initial diagnostic report directly adjusted by the user represents the initial image content, after the user completes the adjustment, the server generates a medical report based on the adjusted initial diagnostic report using a third-party language model. To achieve a better user experience, after the user completes the adjustment of the initial diagnostic report, the server can respond to the user's confirmation instruction, generate a medical report, and display the generated medical report to the user.

[0363] For the convenience of users, in one or more embodiments of this specification, the medical report generated by the server may only include diagnostic results that characterize the presence of lesions or abnormalities in the patient.

[0364] Based on the above, the types of input information in this specification can include command types. Therefore, in this embodiment, the user's command to generate a medical report can be a manual operation by the user specifying an identifier on the graphical user interface, or it can be triggered by input information determined by the user's voice or text input.

[0365] The method of this embodiment will now be described with reference to Figure 15. Figure 15 is a schematic diagram of a medical report generation process provided in an embodiment of this specification.

[0366] As shown in Figure 15, the upper image is the adjusted initial diagnostic report, and the lower image is the medical report. The medical report includes the adjusted initial diagnostic report and the examination conclusions of the generated medical images. If the keyword of the confirmation instruction is "generate report," after the server determines the adjusted initial diagnostic report, it responds to the user's input information containing the semantics of "generate report" and confirms the input information as a confirmation instruction. Then, the server executes the confirmation instruction, generates the examination conclusions of the medical images, and displays the adjusted initial diagnostic report and the generated examination conclusions of the medical images as the medical report to the user.

[0367] Figure 15 shows the diagnostic results for each body part in the medical report, listed in a structured manner. It can be seen that the adjusted initial diagnostic report is not entirely the same as the findings of the medical imaging examination; the medical report only includes diagnostic results indicating the presence of lesions or abnormalities in the patient.

[0368] In one or more embodiments of this specification, in order to further improve the accuracy of medical reports, after determining the medical report in step S2005 above, the server can further check the determined medical report based on the patient information. When it is found that there is content in the medical report that does not match the patient information, the user is reminded so that the user can check the content of the medical report again.

[0369] The patient information mentioned above may include any kind of diagnosis and treatment-related information provided by the patient, such as basic information, chief complaint, relevant examination results, medical history, etc.

[0370] Specifically, after determining the medical report, the server retrieves patient information and performs semantic analysis on the distribution of patient information and medical report. This semantic analysis determines whether there are any mismatches between the medical report and the patient information. If so, a prompt message is determined based on the corresponding statement in the medical report. This mismatch may be content that the user overlooked and needs to be adjusted in the initial diagnosis report.

[0371] This manual does not limit the specific form of the prompt message. It can be a highlighted statement of the mismatch in the medical report or a pop-up message, as long as it can prompt the user to check the mismatch again.

[0372] If not, display the medical report to the user. If the medical report confirms that no problems were detected, inform the user that the medical report is now in its optimal state.

[0373] When mismatched content is highlighted in the corresponding sentence of a medical report, the server can use any highlighting effect to display the mismatched content in the corresponding sentence of the medical report, such as highlighting, bolding, flashing, etc.

[0374] For example, the server can determine that the patient is male from the basic information in the patient's profile. However, the initial diagnosis text determined based on the report template is "ovarian cyst." If the user does not adjust this initial diagnosis text by inputting information during the adjustment process, the server, after determining the medical report, will perform semantic analysis on both the patient information and the medical report. Based on the patient information, the server will identify that the initial diagnosis text does not match the patient's gender, and therefore will display the initial diagnosis text in bold to the user.

[0375] The chief complaint is the symptom that leads a patient to seek medical attention. When the patient information includes the chief complaint, the server can determine the body part corresponding to the symptom from the various body parts included in the report template based on the symptom corresponding to the chief complaint.

[0376] Then, the server checks the medical report to determine if the diagnostic results for the corresponding body part indicate an abnormality. If so, the medical report is deemed to have detected no problem. If not, a prompt message is generated based on the diagnostic results for the corresponding body part in the medical report.

[0377] In this embodiment, the server further verifies the content of the determined medical report based on the patient information, enabling the user to further adjust any omissions or errors in the medical report and obtain a more accurate medical report.

[0378] This embodiment represents a further optimization of the established medical report. While applying this embodiment can further improve the accuracy of the medical report, it also increases the report generation time. Therefore, to improve user experience, this embodiment allows the server to respond to user actions, enabling users to freely choose whether to use the method of this embodiment based on their actual medical report generation needs.

[0379] Based on the above, the types of input information in this specification can include command types. Executing the command corresponding to the method in this embodiment can be triggered by the user's manual operation on a specified identifier on the graphical user interface, or it can be triggered by voice information.

[0380] When triggered by voice information, in one or more embodiments of this specification, the server may pre-determine that the keyword for executing the instruction corresponding to the method of this embodiment is "one-click quality control." If the input information contains the phrase "one-click quality control" or a phrase with the same semantic meaning, the server executes the method of this embodiment to further verify the medical report. Of course, "one-click quality control" is just one way to set a keyword; any keyword can be set according to actual needs to trigger the method of this embodiment.

[0381] Figure 16 is a flowchart of a voice-triggered optimization instruction provided in an embodiment of this specification. As shown in Figure 16, the medical report contains the diagnostic result "No obvious abnormalities were found in the heart and major blood vessels". However, after the server performs semantic analysis on the patient's complaint, it determines that the body parts corresponding to the symptoms in the complaint are the heart and major blood vessels. But the diagnostic result of the body parts corresponding to the symptoms in the medical report does not indicate any abnormality. Therefore, the server will highlight the diagnostic result "No obvious abnormalities were found in the heart and major blood vessels" from the medical report.

[0382] Based on the embodiments mentioned above, another method for generating medical reports, as shown in Figure 17, is now provided. Figure 17 is a flowchart of another medical report generation process provided in the embodiments of this specification. As shown in Figure 17, for sites of interest corresponding to regions of interest, the server determines the initial diagnostic text based on the regions of interest; for sites of non-interest without corresponding regions of interest, the server determines the initial diagnostic text based on the preset diagnostic text of the report template; and based on the initial diagnostic text of each body part, the server determines the initial diagnostic report.

[0383] Then, the server obtains the user's input information, which can be voice or text, and performs type recognition on the input information. If the input information is determined to be a command, the corresponding operation is executed; if the voice information is determined to be a non-report type, the input information is discarded and not processed.

[0384] If the input information is determined to be a report type, it is used to adjust the initial diagnostic report. Further predictions are made based on the semantics of the input information to determine the predicted information. Based on the input information and the predicted information, the initial diagnostic report is adjusted to determine the final medical report.

[0385] After the medical report is finalized, the server can further optimize the report based on the patient information to obtain an optimized medical report.

[0386] As shown in Figure 17, users input information containing the semantics of "generate report" to enable the server to perform the operation of generating a medical report. Users can also input information containing the semantics of "one-click quality control" to enable the server to perform optimization operations on the medical report.

[0387] The above is the method for generating medical reports provided in this manual. Based on the same idea, this manual also provides a corresponding medical report generating device, as shown in Figure 18.

[0388] Figure 18 is a schematic diagram of another medical report generation device provided in this specification, specifically including:

[0389] The Initial Diagnostic Report Determination Module 2021 is used to determine and display the initial diagnostic report;

[0390] Input module 2023 is used to acquire input information from the user based on the initial diagnostic report, predict the user's subsequent input based on the input information, and determine the predicted information;

[0391] The medical report determination module 2025 is used to adjust the initial diagnostic report based on the input information and the prediction information, and determine the medical report.

[0392] Optionally, the initial diagnostic report determination module 2021 is specifically used to determine a report template containing preset diagnostic text for each part of the human body, determine an initial diagnostic report based on the report template, and display the initial diagnostic report.

[0393] Optionally, the initial diagnostic report determination module 2021 is specifically used to acquire medical images, determine the region of interest in the medical images, determine the human body part corresponding to the region of interest among the various human body parts as the part of interest, determine the initial diagnostic text of the part of interest based on the region of interest, determine the initial diagnostic report based on the initial diagnostic text of the part of interest, and display the initial diagnostic report.

[0394] Optionally, the input module 2023 is specifically used to predict the user's subsequent input based on the input information, determine the predicted candidate information, display the candidate information to the user, and, in response to the user's selection operation on the candidate information, use the candidate information selected by the user as the predicted information.

[0395] Optionally, the input module 2023 is specifically used to determine the matching probability between each candidate information and the input information, sort the candidate information corresponding to each matching probability according to the magnitude of each matching probability, determine the sorted candidate information, and display the sorted candidate information to the user.

[0396] Optionally, the medical report determination module 2025 is specifically used to determine adjustment information based on the input information and the prediction information, determine the text segment corresponding to the adjustment information in the initial diagnosis report, and adjust the text segment according to the adjustment information.

[0397] Optionally, the input information is voice information or text information. The medical report determination module 2025 is specifically used to determine the text information corresponding to the voice information when the input information is voice information, integrate the text information with the prediction information, and determine the adjustment information when the input information is text information.

[0398] Optionally, the medical report determination module 2025 is specifically used to determine adjustment information based on the input information and the prediction information, determine the human body part corresponding to the adjustment information among the various human body parts as the target part, adjust the initial diagnosis text of the target part in the initial diagnosis report according to the adjustment information, and determine the medical report.

[0399] Optionally, there may be multiple target sites. The medical report determination module 2025 is specifically used to determine the adjustment content of each target site in the adjustment information, and to adjust the initial diagnosis text of each target site according to the adjustment content of that target site.

[0400] Optionally, the medical report determination module 2025 is specifically used to delete the initial diagnosis text of the target part when the adjustment content of the target part contains keywords that represent deletion semantics, and to integrate the adjustment content of the target part with the initial diagnosis text of the target part when the adjustment content of the target part does not contain keywords that represent deletion semantics.

[0401] Optionally, the medical report determination module 2025 is specifically used to perform semantic analysis on the adjustment content of the target area and the initial diagnosis text of the target area. When it is determined that there is a semantic contradiction between the adjustment content of the target area and the initial diagnosis text of the target area, the adjustment content of the target area is used to replace the initial diagnosis text of the target area. When it is determined that there is semantic overlap between the adjustment content of the target area and the initial diagnosis text of the target area, the adjustment content of the target area and the initial diagnosis text of the target area are merged.

[0402] Optionally, the initial diagnostic report is used to describe the image information of the medical image. The medical report determination module 204 is specifically used to adjust the initial diagnostic report according to the input information and the prediction information, input the adjusted initial diagnostic report into the third language model to generate the examination conclusion of the medical image, and determine the medical report based on the adjusted initial diagnostic report and the examination conclusion of the medical image.

[0403] Optionally, the medical report determination module 2025 is specifically used to obtain patient information, which includes at least one of basic information, chief complaint, relevant examination results, and medical history, and to determine whether the medical report contains mismatched content with the patient information. If so, a prompt message is determined based on the statement corresponding to the mismatched content in the medical report. The prompt message is used to prompt the user to re-check the mismatched content. If not, the medical report is displayed to the user.

[0404] This specification also provides another computer-readable storage medium storing a computer program that can be used to execute the medical report generation method provided in Figure 11 above.

[0405] This specification also provides a schematic structural diagram of the electronic device shown in Figure 19. As shown in Figure 19, at the hardware level, the electronic device includes a processor, an internal bus, a network interface, memory, and non-volatile memory, and may also include other hardware required for business operations. The processor reads the corresponding computer program from the non-volatile memory into memory and then runs it to implement the medical report generation method described in Figure 11. Of course, in addition to the software implementation, this specification does not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. That is to say, the execution subject of the following processing flow is not limited to individual logic units, but can also be hardware or logic devices.

[0406] Improvements in a technology can be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many improvements to the methodology can now be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that an improvement in methodology cannot be implemented using hardware physical modules. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program and "integrate" a digital system onto a PLD themselves, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must also be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also understand that by simply performing some logic programming on the method flow using one of these hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0407] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0408] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0409] For ease of description, the above devices are described in terms of function, divided into various units. Of course, in implementing this specification, the functions of each unit can be implemented in one or more software and / or hardware components.

[0410] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0411] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in one or more flowchart illustrations and / or one or more block diagrams.

[0412] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowcharts and / or one or more block diagrams.

[0413] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide steps for implementing the functions specified in one or more flowcharts and / or one or more block diagrams.

[0414] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0415] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0416] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0417] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0418] Those skilled in the art will understand that the embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this specification may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0419] This specification can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data organs, etc., that perform a specific task or implement a specific abstract data type. This specification can also be practiced in distributed computing environments, where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0420] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0421] The above description is merely an embodiment of this specification and is not intended to limit this specification. Various modifications and variations can be made to this specification by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of the claims of this application.

Claims

1. A method for generating medical reports, characterized in that, include: Obtain patient information, input the patient information into the report generation model, and obtain diagnostic features; Based on the diagnostic features, historical diagnostic information related to the patient information is identified in the historical diagnostic records and used as associated information; Based on the associated information, a prompt message is determined, and the prompt message and the diagnostic features are input into the report generation model to generate a medical report.

2. The method as described in claim 1, characterized in that, The report generation model includes an image encoding layer and a text encoding layer, and the patient information includes image data and text data; The patient information is input into the report generation model to obtain diagnostic features, specifically including: The image data is input into the image encoding layer to obtain image diagnostic features, and the text data is input into the text encoding layer to obtain text diagnostic features; Diagnostic features are determined based on the image diagnostic features and the text diagnostic features.

3. The method as described in claim 1, characterized in that, Based on the diagnostic characteristics, historical diagnostic information related to the patient information is identified from historical diagnostic records as associated information, specifically including: Determine the encoding characteristics of each historical diagnostic information contained in the historical diagnostic records; The similarity between the encoding features of each historical diagnostic information and the diagnostic features is determined respectively; The encoded features of historical diagnostic information with a similarity greater than or equal to a preset value are used as associated information.

4. The method as described in claim 1, characterized in that, Based on the associated information, a prompt message is determined. The prompt message and the diagnostic features are then input into the report generation model to generate a medical report, specifically including: Obtain the context information of the medical report to be generated input by the user, and use the context information and the associated information as prompt information; The prompt information and the diagnostic features are input into the report generation model. The report generation model is used to predict the subsequent content of the preceding information to obtain the subsequent information of the preceding information. The medical report is determined based on the preceding and following information.

5. The method as described in claim 4, characterized in that, The following information consists of multiple candidate information items; Based on the foregoing and subsequent information, a medical report is determined, specifically including: In response to the user's selection operation on each candidate information, determine the candidate information selected by the user; Based on the aforementioned information and the candidate information selected by the user, a medical report is determined.

6. The method as described in claim 1, characterized in that, The medical report is a structured report; Based on the associated information, a prompt message is determined. The prompt message and the diagnostic features are then input into the report generation model to generate a medical report, specifically including: Based on user actions, a task prompt is determined, which is used to characterize the structured task of the report; The task prompt and the associated information are used as the prompt information; The prompt information and the diagnostic features are input into the report generation model to obtain the medical report.

7. The method as described in claim 6, characterized in that, The prompt information and the diagnostic features are input into the report generation model to obtain the medical report, which specifically includes: Determine the report template and initial diagnostic report; Obtain each diagnostic item in the report template, and determine the corresponding diagnostic text for each diagnostic item based on the initial diagnostic report; Based on the diagnostic text, determine the diagnostic conclusion for the diagnostic item; Based on the diagnostic conclusion, a medical report is determined.

8. The method as described in claim 7, characterized in that, Based on the diagnostic text, the diagnostic conclusion for the diagnostic item is determined, specifically including: When it is determined that the diagnostic text corresponding to the diagnostic item indicates an anomaly, the judgment result of the diagnostic item is determined to be abnormal. The diagnostic text and the judgment result of the diagnostic item shall be used as the diagnostic conclusion of the diagnostic item.

9. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the method described in any one of claims 1 to 7.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method described in any one of claims 1 to 7.

11. A method for generating medical reports, characterized in that, include: Determine and present the initial diagnostic report; Obtain input information from the user based on the initial diagnostic report, predict the user's subsequent input based on the input information, and determine the predicted information; Based on the input information and the prediction information, the initial diagnostic report is adjusted to determine the medical report.

12. The method as described in claim 11, characterized in that, Determine and present the initial diagnostic report, specifically including: Determine a report template that includes pre-defined diagnostic text for each part of the body; Based on the report template, an initial diagnostic report is determined and then displayed.

13. The method as described in claim 12, characterized in that, Based on the aforementioned report template, an initial diagnostic report is determined, specifically including: Acquire medical images and determine the region of interest within the medical images; Among the various human body parts, the human body parts corresponding to the region of interest are identified as the regions of interest; Based on the region of interest, determine the initial diagnostic text for the area of ​​interest; Based on the initial diagnostic text of the area of ​​interest, an initial diagnostic report is determined and displayed.

14. The method as described in claim 11, characterized in that, Based on the input information, predict the user's subsequent input and determine the prediction information, specifically including: Based on the input information, predict the user's subsequent inputs and determine the predicted candidate information; The candidate information is then displayed to the user. In response to the user's selection of each candidate information, the candidate information selected by the user is used as the prediction information.

15. The method as described in claim 14, characterized in that, Displaying the candidate information to the user specifically includes: Determine the matching probability between each candidate information and the input information; Based on the magnitude of each matching probability, the candidate information corresponding to each matching probability is sorted to determine the sorted candidate information. The sorted candidate information is then displayed to the user.

16. The method as described in claim 11, characterized in that, Based on the input information and the prediction information, the initial diagnostic report is adjusted to determine the medical report, specifically including: Based on the input information and the prediction information, the adjustment information is determined; In the initial diagnostic report, the text fragment corresponding to the adjustment information is identified; The text fragment is adjusted according to the adjustment information.

17. The method as described in claim 16, characterized in that, The input information is either voice information or text information; Based on the input information and the prediction information, adjustment information is determined, specifically including: When the input information is voice information, determine the text information corresponding to the voice information, integrate the text information with the prediction information, and determine the adjustment information; When the input information is text information, the text information is integrated with the prediction information to determine the adjustment information.

18. The method as described in claim 12 or 13, characterized in that, Based on the input information and the prediction information, the initial diagnostic report is adjusted to determine the medical report, specifically including: Based on the input information and the prediction information, the adjustment information is determined; Among the various human body parts, the human body part corresponding to the adjustment information is determined as the target part; Based on the adjustment information, the initial diagnostic text for the target site in the initial diagnostic report is adjusted to determine the medical report.

19. The method as described in claim 18, characterized in that, The number of target locations is multiple; Based on the adjustment information, the initial diagnostic text for the target site in the initial diagnostic report is adjusted, specifically including: The adjustment information specifies the adjustment details for each target part. For each target area, adjust the initial diagnostic text for that target area according to the adjustments made to that target area.

20. The method as described in claim 19, characterized in that, Based on the adjustments made to the target area, the initial diagnostic text for that target area is adjusted, specifically including: When the adjustment content of the target part contains keywords that represent the semantics of deletion, the initial diagnostic text of the target part is deleted; When the adjustment content of the target part does not contain keywords that represent deletion semantics, the adjustment content of the target part is integrated with the initial diagnostic text of the target part.

21. The method as described in claim 20, characterized in that, The adjustments made to the target area are integrated with the initial diagnostic text for that target area, specifically including: Semantic analysis is performed on the adjustment content of the target area and the initial diagnostic text of the target area; When it is determined that the adjustment content of the target part contradicts the semantics of the initial diagnostic text of the target part, the initial diagnostic text of the target part shall be replaced with the adjustment content of the target part. When it is determined that the adjustment content of the target part overlaps with the semantics of the initial diagnostic text of the target part, the adjustment content of the target part and the initial diagnostic text of the target part are merged.

22. The method as described in claim 11, characterized in that, The initial diagnostic report is used to describe the imaging information of the medical images; Based on the input information and the prediction information, the initial diagnostic report is adjusted to determine the medical report, specifically including: The initial diagnostic report is adjusted based on the input information and the prediction information. The adjusted initial diagnostic report is input into the third language model to generate the examination conclusion of the medical images; Based on the adjusted initial diagnostic report and the examination conclusions of the medical images, a medical report is determined.

23. The method as described in claim 11, characterized in that, After the medical report is confirmed, the method further includes: Obtain patient information, which includes at least one of the following: basic information, chief complaint, relevant examination results, and medical history; Determine whether there are any discrepancies between the medical report and the patient information; If so, then based on the statement in the medical report corresponding to the mismatched content, a prompt message is determined, which is used to prompt the user to check the mismatched content again; If not, then the medical report is displayed to the user.

24. A medical report generation device, characterized in that, include: The initial diagnostic report determination module determines and displays the initial diagnostic report. The input module acquires input information from the user based on the initial diagnostic report, predicts the user's subsequent input based on the input information, and determines the predicted information. The medical report determination module adjusts the initial diagnostic report based on the input information and the prediction information to determine the medical report.

25. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the method described in any one of claims 11 to 23.

26. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method described in any one of claims 11 to 23.

Images