Ultrasound report generation method, related equipment and computer program product
By combining the fast and slow thinking modes of a large model, ultrasound detection parameters are extracted and corrected in real time to generate high-quality ultrasound reports. This solves the problem of the trade-off between real-time performance and quality in existing technologies, and achieves real-time interaction and efficient report generation.
Patent Information
- Application Number
- CN202512054455.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-31
- Publication Date
- 2026-02-06
AI Technical Summary
Existing ultrasound report generation technologies struggle to balance real-time performance and report quality. Fast-thinking modes based on large models produce poor output quality, while slow-thinking modes disrupt the doctor's examination flow, failing to meet the clinical needs for real-time interaction.
The system employs a fast-thinking mode based on a large model to extract ultrasound detection parameters in real time and output a preview. It then combines this with a slow-thinking mode to generate a complete report. Finally, it uses speech recognition and an ultrasound knowledge graph for text correction and knowledge enhancement, integrating detection parameters, descriptions, and diagnoses to generate a high-quality report.
It enables real-time output of test parameters and interaction with doctors during ultrasound examinations, ensuring transparent test results, uninterrupted examination flow, and generating high-quality ultrasound reports, meeting the needs of real-time interaction and improving clinical work efficiency.
Smart Images

Figure CN121483481A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of artificial intelligence, and more particularly, to an ultrasound report generation method, related equipment and computer program product. BACKGROUND
[0002] With the in-depth application of speech recognition and natural language processing technology in medical scenarios such as electronic medical records and image reports, the efficiency and accuracy of its generation are directly related to the smoothness of clinical work and the reliability of diagnosis. Medical ultrasound report generation usually has strong structure, professional terminology, and needs to be completed in real time during the examination process. Therefore, realizing the rapid and accurate conversion from speech to structured report is the key to improving the efficiency and quality of clinical work.
[0003] Currently, the generation of the ultrasound report mainly relies on manual transcription after the doctor's dictation or uses a model to generate the ultrasound report. When using a model to generate the ultrasound report, the existing technology either uses a "fast thinking mode" of a large model or uses a "slow thinking mode" of a large model. The existing technical solutions have a core contradiction that is difficult to reconcile. First, the "fast thinking mode" solution based on a large model has strong real-time performance because it does not need to go through deep thinking to directly output an answer, but it lacks deep semantic understanding and the quality of the output report is poor. Second, the "slow thinking mode" solution based on a large model can generate accurate and standardized professional reports, but the deep thinking process consumes time, seriously disrupting the doctor's examination line, and cannot meet the clinical needs of real-time interaction. SUMMARY
[0004] In view of the above problems, the present application is proposed to at least overcome some of the defects existing in the prior art. The specific scheme is as follows:
[0005] In a first aspect, an ultrasound report generation method is provided, comprising:
[0006] obtaining real-time dictation speech of a doctor during an ultrasound detection process, the dictation speech being a description of an ultrasound image;
[0007] feeding the real-time dictation speech into a speech recognition model for streaming recognition to obtain a current speech transcription text segment;
[0008] calling a large model to extract ultrasound detection parameters based on the current speech transcription text segment using a fast thinking mode, and outputting the ultrasound detection parameters in real time for preview;
[0009] after the ultrasound detection is completed, calling a large model to generate an ultrasound description and diagnosis based on an original complete speech transcription text using a slow thinking mode, the original complete speech transcription text including all speech transcription text segments obtained during the ultrasound detection process.
[0010] By integrating the ultrasound detection parameters, ultrasound description, and diagnosis, an ultrasound report corresponding to this ultrasound detection is obtained.
[0011] In one possible design, in another implementation of the first aspect of the embodiments of this application, the process of calling the large model to extract ultrasound detection parameters based on the current speech-to-text segment using a fast-thinking mode includes:
[0012] The large model is invoked to correct the current speech-to-text segment using the fast thinking mode, and the ultrasound detection parameters are extracted to obtain the corrected current speech-to-text segment and ultrasound detection parameters output by the large model.
[0013] In one possible design, in another implementation of the first aspect of the embodiments of this application, the process of calling a large model to correct the current speech-to-text segment using a fast-thinking mode and extracting ultrasound detection parameters includes:
[0014] Extract the entities and relationships of the current speech-to-text segment, and retrieve relevant first knowledge information in the configured ultrasound knowledge graph based on the extracted entities and relationships;
[0015] The large model is invoked to use a fast thinking mode to correct the current speech-to-text segment based on the first knowledge information, and the ultrasound detection parameters are extracted to obtain the corrected current speech-to-text segment and ultrasound detection parameters output by the large model.
[0016] In one possible design, in another implementation of the first aspect of the embodiments of this application, the process of calling the large model to use a fast thinking mode to correct the current speech-to-text segment based on the first knowledge information and extracting ultrasound detection parameters includes:
[0017] Obtain the first prompt template, which includes the current speech-to-text segment slot, the historical speech-to-text slot, the knowledge slot, and the first prompt. The first prompt is used to instruct the large model to adopt a fast thinking mode, refer to the historical speech-to-text and knowledge information, correct the current speech-to-text segment, and extract the ultrasound detection parameters.
[0018] The current speech-to-text segment is filled into the current speech-to-text segment slot, the previously recognized historical speech-to-text segment is filled into the historical speech-to-text segment slot, and the first knowledge information is filled into the knowledge slot to obtain the edited first prompt instruction. The first prompt instruction is then sent into the large model.
[0019] In one possible design, in another implementation of the first aspect of the embodiments of this application, the process of invoking a large model to generate an ultrasound description and diagnosis based on the original complete speech-to-text using a slow-thinking mode includes:
[0020] The entities and relationships extracted from each speech-to-text segment of this ultrasound detection process are obtained to obtain a set of entities and relationships. Based on the set of entities and relationships, relevant second knowledge information is retrieved in the configured ultrasound knowledge graph.
[0021] The large model is invoked in a slow-thinking mode to generate an ultrasound description and diagnosis based on the original complete speech-to-text and the second knowledge information.
[0022] In one possible design, in another implementation of the first aspect of the embodiments of this application, the process of invoking a large model to generate an ultrasound description and diagnosis based on the original complete speech-to-text using a slow-thinking mode includes:
[0023] The entities and relationships extracted from each speech-to-text segment of this ultrasound detection process are obtained to obtain a set of entities and relationships. Based on the set of entities and relationships, relevant second knowledge information is retrieved in the configured ultrasound knowledge graph.
[0024] The large model is invoked in a slow-thinking mode. Based on the original complete speech-to-text, the corrected complete speech-to-text, and the second knowledge information, an ultrasound description and diagnosis are generated. The corrected complete speech-to-text includes each speech-to-text segment corrected during the ultrasound examination process.
[0025] In one possible design, another implementation of the first aspect of the embodiments of this application further includes:
[0026] Obtain a standard report corresponding to the ultrasound report, wherein the standard report is a report after the ultrasound report has been manually reviewed;
[0027] When the large model is idle, it is invoked to compare the differences between the ultrasound report and the standard report in slow thinking mode. Combining the original complete speech-to-text and the corrected complete speech-to-text, erroneous atlas knowledge is identified in the atlas knowledge set, and the atlas knowledge that needs to be supplemented is provided. The corrected complete speech-to-text includes each speech-to-text segment corrected during this ultrasound examination, and the atlas knowledge set includes all atlas knowledge referenced during this ultrasound examination.
[0028] The ultrasound knowledge map is updated based on the erroneous and necessary supplementary atlas knowledge output by the large model.
[0029] In one possible design, in another implementation of the first aspect of the embodiments of this application, when calling the large model to compare the differences between the ultrasound report and the standard report using a slow-thinking mode, combining the original complete speech-to-text and the corrected complete speech-to-text, identifying erroneous graph knowledge in the graph knowledge set, and providing the graph knowledge that needs to be supplemented, the large model reasoning is guided by a thought chain approach, which includes the following steps:
[0030] The large model is used to compare the differences between the ultrasound report and the standard report.
[0031] The large model is instructed to locate the target location in the corrected complete speech-to-text that may lead to the discrepancy content.
[0032] The large model is instructed to compare the differences between the original complete speech-to-text and the corrected complete speech-to-text. If the target position is a different point, it means that the correction of the target position is incorrect, and the incorrect graph knowledge needs to be found in the graph knowledge set, and the reason is explained. If the target position is not a different point, it means that the target position has not been corrected, and the graph knowledge that needs to be supplemented is given, and the reason is explained.
[0033] In one possible design, in another implementation of the first aspect of the embodiments of this application, the process of updating the ultrasound knowledge graph based on the erroneous atlas knowledge output by the large model and the atlas knowledge that needs to be supplemented includes:
[0034] The first cumulative occurrence count of erroneous atlas knowledge is counted. If the first cumulative occurrence count exceeds a first threshold or a user confirmation instruction for erroneous atlas knowledge is received, the erroneous atlas knowledge is deleted from the ultrasound knowledge atlas.
[0035] The second cumulative occurrence count of the atlas knowledge that needs to be supplemented is counted. If the second cumulative occurrence count exceeds the second threshold or a user confirmation instruction for the atlas knowledge that needs to be supplemented is received, the atlas knowledge that needs to be supplemented is added to the ultrasound knowledge atlas.
[0036] In a second aspect, an electronic device is provided, comprising: a memory and a processor;
[0037] The memory is used to store programs;
[0038] The processor is configured to execute the program to implement the various steps of the ultrasound report generation method described in any of the first aspects of this application.
[0039] Thirdly, a readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the ultrasound report generation method described in any of the first aspects of this application.
[0040] Fourthly, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the ultrasound report generation method described in any of the first aspects of this application.
[0041] As can be seen from the above technical solution, this application provides an ultrasound report generation scheme that can deeply integrate the fast thinking mode and slow thinking mode of a large model. Specifically, during the ultrasound examination, this application performs streaming recognition on the doctor's real-time spoken speech to obtain the current speech-to-text segment. The large model is then used in fast thinking mode to extract ultrasound examination parameters from the current speech-to-text segment. Taking cardiac ultrasound examination as an example, the extracted ultrasound examination parameters include: aortic sinus diameter 32, ascending aortic diameter 35, etc. The extracted ultrasound examination parameters are output for preview in real time, thereby enabling real-time interaction with the doctor and ensuring the transparency of the examination results. Furthermore, after the ultrasound examination, the large model is used in slow thinking mode to generate an ultrasound description and diagnosis based on the original complete speech-to-text (including all speech-to-text segments from the current ultrasound examination process). The ultrasound examination parameters, ultrasound description, and diagnosis extracted during the examination process are integrated to obtain the ultrasound report corresponding to this ultrasound examination. By using slow thinking mode with the large model, accurate, standardized, and high-quality ultrasound reports can be generated.
[0042] In summary, the proposed solution can output ultrasound detection parameters in real time during ultrasound detection through the large model fast thinking mode, enabling real-time interaction with doctors and ensuring that the detection results are transparent to doctors without interrupting their detection process. After the ultrasound detection is completed, a high-quality ultrasound report can be generated through the large model slow thinking mode. This not only meets the clinical needs for real-time interaction and improves clinical work efficiency, but also ensures the quality of the final ultrasound report. Attached Figure Description
[0043] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:
[0044] Figure 1 A schematic diagram of an implementation system architecture for the ultrasound report generation method provided in this application embodiment;
[0045] Figure 2This is a schematic flowchart of an ultrasound report generation method provided in an embodiment of this application;
[0046] Figure 3 This is a schematic diagram of another ultrasound report generation method provided in an embodiment of this application;
[0047] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation
[0048] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0049] It is understood that before using the technical solutions disclosed in the various embodiments of this application, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this application in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.
[0050] This application provides an ultrasound report generation scheme that can output ultrasound test parameters in real time during the ultrasound test performed by the doctor and generate a high-quality ultrasound report after the test is completed.
[0051] The ultrasound report generation scheme of this application can be applied to various types of ultrasound examination scenarios, such as cardiac ultrasound examination scenarios and abdominal ultrasound examination scenarios, and this application does not make strict limitations on them.
[0052] This application provides a method for generating ultrasound reports, which can be applied to, for example... Figure 1 The system architecture shown may include a terminal 100 and a server 200. The server 200 may include one or more servers (…). Figure 1 (This example uses a server as an illustration).
[0053] Either terminal 100 or server 200 can be used independently to execute the ultrasound report generation method provided in the embodiments of this application. Alternatively, terminal 100 and server 200 can also be used collaboratively to execute the ultrasound report generation method provided in the embodiments of this application.
[0054] In this application embodiment, the terminal 100 can be a computer, an ultrasound testing terminal, or other terminal device with data processing capabilities, and this application embodiment does not impose any restrictions on it.
[0055] This application provides a method for generating an ultrasound report. Taking the application of this method to a computer device as an example, the computer device can specifically be... Figure 1 The system consists of terminal 100 or a combination of terminal 100 and server 200. (Refer to...) Figure 2 The method for generating this ultrasound report specifically includes the following steps:
[0056] Step S100: Obtain the doctor's real-time spoken voice during the ultrasound examination process. The spoken voice is a description of the ultrasound image.
[0057] Specifically, this application allows for audio recording via a microphone device. During the ultrasound examination, the doctor can verbally describe the ultrasound images they see, and the microphone device can then capture the doctor's real-time spoken audio.
[0058] Step S110: The real-time spoken speech is fed into the speech recognition model for streaming recognition to obtain the current speech-to-text segment.
[0059] Specifically, real-time spoken speech is fed into a speech recognition model for streaming recognition to obtain the current speech-to-text segment.
[0060] In one optional example, real-time spoken speech can be segmented according to a set duration, such as segmenting into a speech fragment every 8 seconds. Streaming recognition can then be used to obtain the corresponding speech-to-text segments. Other streaming speech recognition schemes can also be used, which will not be described in detail here.
[0061] Step S120: Call the large model and use the fast thinking mode to extract ultrasound detection parameters based on the current speech-to-text segment, and output the ultrasound detection parameters for preview in real time.
[0062] This application provides two reasoning modes for a large-scale model: a fast-thinking mode and a slow-thinking mode. In fast-thinking mode, the large-scale model does not generate deep thinking processes (such as thought chains) and directly outputs the answer. In slow-thinking mode, the large-scale model generates deep thinking processes before generating the final answer. Because the large-scale model does not need to generate deep thinking processes in fast-thinking mode, its output response is more timely, and its answer accuracy is still relatively good for simple tasks. However, for complex tasks, its answer accuracy is lower than in slow-thinking mode. In slow-thinking mode, because the large-scale model needs to generate deep thinking processes, its reasoning time is longer, and its output response time is not as timely as in slow-thinking mode, but the quality of the generated answer is higher.
[0063] In this application, during the ultrasound examination, for the real-time acquired speech-to-text segment, a large model is invoked to extract the ultrasound examination parameters using a fast-thinking mode, and the ultrasound examination parameters are output and previewed in real time, providing doctors with immediate feedback on the examination results and meeting the clinical needs for real-time interaction.
[0064] Taking cardiac ultrasound as an example, an example of extracted ultrasound parameters is a left atrial diameter of 30mm.
[0065] Understandably, the task of extracting ultrasound detection parameters is relatively easy. This embodiment can reduce the inference time of the large model and improve the timeliness of the large model output while ensuring the accuracy of the extracted ultrasound detection parameters through the fast thinking mode of the large model. This will not interrupt the doctor's detection process and will meet the clinical needs for timely interaction.
[0066] Step S130: After the ultrasound examination is completed, the large model is invoked to generate an ultrasound description and diagnosis based on the original complete speech-to-text in slow thinking mode.
[0067] The original complete speech-to-text includes all speech-to-text segments obtained during this ultrasound examination process.
[0068] Understandably, from the start of this ultrasound examination, the identified speech-to-text segments can be sequentially pieced together until the ultrasound examination is completed, resulting in the original complete speech-to-text.
[0069] In this embodiment, after confirming the completion of the ultrasound examination, the doctor can initiate an end signal, such as by clicking the "End" icon on the screen, triggering the designated end button, or issuing an end command via voice. Upon receiving the end command, the system can confirm the end of the ultrasound examination, then collect the original complete speech-to-text, and use a large model in slow-thinking mode to generate an ultrasound description and diagnosis for this ultrasound examination.
[0070] In this step, after the ultrasound examination is completed, a large model is used to generate an ultrasound description and diagnosis in a slow-thinking mode, which ensures the quality of the generated results. Although this process takes longer than the fast-thinking mode, it does not interrupt the doctor's examination flow because it is not performed during the ultrasound examination.
[0071] Step S140: Integrate ultrasound detection parameters, ultrasound description and diagnosis to obtain the ultrasound report corresponding to this ultrasound detection.
[0072] All ultrasound detection parameters extracted using the fast-thinking mode of the large model during this ultrasound detection process can be combined with the ultrasound description and diagnosis generated by the slow-thinking mode of the large model to obtain a complete ultrasound report corresponding to this ultrasound detection.
[0073] The ultrasound report generation method provided in this application can output ultrasound detection parameters in real time through the large model fast thinking mode during ultrasound detection, and interact with doctors in real time, ensuring that the detection results are transparent to doctors and will not interrupt the doctor's detection process. After the ultrasound detection is completed, a high-quality ultrasound report can be generated through the large model slow thinking mode. This not only meets the clinical needs of real-time interaction and improves clinical work efficiency, but also ensures the quality of the final ultrasound report.
[0074] This application's method, through the synergy of fast and slow thinking modes in a large model, addresses the core contradiction in ultrasound diagnostic scenarios where real-time report generation and final draft quality are mutually exclusive.
[0075] In some embodiments of this application, the process of step S120, which involves calling a large model and using fast thinking mode to extract ultrasound detection parameters based on the current speech-to-text fragment, is described.
[0076] In one alternative implementation, a prompt instruction can be used to instruct the large model to adopt a fast-thinking mode to extract ultrasound detection parameters based on the current speech-to-text segment.
[0077] In another optional implementation, the large model can be invoked to correct the current speech-to-text segment using a fast-thinking mode, and the ultrasound detection parameters can be extracted to obtain the corrected current speech-to-text segment and ultrasound detection parameters output by the large model.
[0078] In this embodiment, it is considered that the current speech-to-text segment may contain erroneous or non-standard content. This noisy content may be due to misidentification by the speech recognition model, or it may be because the doctor's spoken voice inevitably contains errors and personal habits introduced by the doctor's personal accent, habitual abbreviations, and non-standard expressions. For example:
[0079] Pronunciation errors: These are caused by accents, such as pronouncing "valve" as "half-mo";
[0080] Common abbreviations: For example, "FS" is used to refer to "left ventricular shortening fraction";
[0081] Non-standard expressions: such as "poor left ventricular systolic and diastolic function".
[0082] To improve the accuracy of ultrasound detection parameters extracted by the fast thinking mode of the large model, this embodiment requires the large model to correct the current speech-to-text segment while extracting ultrasound detection parameters using the fast thinking mode. This corrects the errors in the current speech-to-text segment, and the ultrasound detection parameters are extracted based on this correction, which can improve the accuracy of the extracted ultrasound detection parameters.
[0083] In another optional implementation, this embodiment can further introduce an ultrasound knowledge graph to provide knowledge enhancement information for the large model's fast thinking mode to correct the current speech-to-text, thereby helping to improve the text correction effect and obtain high-quality corrected current speech-to-text and accurate ultrasound detection parameters.
[0084] The ultrasound knowledge graph includes professional knowledge in the field of ultrasound, for example:
[0085] Entities and Relationships in the Ultrasound Domain: This section defines the core entities (such as anatomical structures, measurement parameters, and case characteristics) and their relationships in the ultrasound domain. Several examples of entity relationship triples are shown below:
[0086] [Interventricular septum thickness] (location) [anterior part of examination], [mitral septum TDI] (location) [posterior part of examination], [interventricular septum] (range) [2-20], [mitral septum TDI] (range) [5-25].
[0087] Entity Synonyms: Defines synonyms for entities, such as [ventricular septum] (abbreviation) [ventricular septum thickness], [ventricular septum] (abbreviation) [mitral septum TDI], etc.
[0088] Error correction rules: Define possible errors and corrective measurements in the field of ultrasound, such as [fcc] (speech recognition error) [fac], [left side] (speech recognition error) [left atrium].
[0089] Based on the ultrasound knowledge graph, the process of using a large model in a fast-thinking mode to correct the current speech-to-text segment and extract ultrasound detection parameters may include:
[0090] Extract the entities and relationships of the current speech-to-text segment, and retrieve relevant primary knowledge information from the configured ultrasound knowledge graph based on the extracted entities and relationships.
[0091] The large model is invoked to use the fast thinking mode to correct the current speech-to-text segment based on the first knowledge information, and the ultrasound detection parameters are extracted to obtain the corrected current speech-to-text segment and ultrasound detection parameters output by the large model.
[0092] Specifically, this application can pre-configure a first prompt template, which includes a current speech-to-text segment slot, a historical speech-to-text segment slot, a knowledge slot, and a first prompt. The first prompt is used to instruct the large model to adopt a fast thinking mode, refer to historical speech-to-text and knowledge information, correct the current speech-to-text segment, and extract ultrasound detection parameters.
[0093] The process involves filling the current speech-to-text segment slot with the current speech-to-text transcribed text, filling the historical speech-to-text segment previously identified with the current speech-to-text transcribed text into the historical speech-to-text transcribed text slot, filling the knowledge slot with the first knowledge information, obtaining the edited first prompt instruction, and then feeding the first prompt instruction into the large model to obtain the corrected current speech-to-text segment and ultrasound detection parameters output by the large model.
[0094] The following is an example of an optional first prompt template, using the cardiac ultrasound examination process as an example:
[0095] First prompt template:
[0096] You are an intelligent assistant with extensive knowledge of cardiac ultrasound. During an ultrasound examination, you need to fill in the patient's ultrasound report by checking the examination items and values reported by the doctor in the "Existing Information". Based on the "Existing Information", you are required to extract the relationships that conform to the types defined in the "Standard Name Reference List" according to the "Task Requirements" and output the "structured results" of the input text according to the specified "Output Format".
[0097] The specific requirements are as follows:
[0098] 1. This is an extraction task. Please extract only the type names and corresponding values that match the "Standard Name Reference List" from the "Existing Information". Do not extract items that are not in the "Standard Name Reference List".
[0099] 2. There may be some punctuation errors during the transcription process. Ignore these errors and focus only on the logical content. Doctors usually report the examination items and values in the order they appear. The information should be integrated according to this order. When the same examination item and value are repeated, take the latter one.
[0100] 3. If a check item in "Existing Information" only has a numerical value, while the last check item in "Previous Information" only has a name, then it is necessary to combine "Previous Information" to complete the missing name in "Existing Information";
[0101] 4. Note that only the result after completing the "existing information" is output; other complete check results of the "previous information" are not required.
[0102] 5. Errors and personalized expressions may exist during the transcription process, which need to be corrected based on the "key information of the knowledge graph".
[0103] 6. No need to output the thought process, just output the final answer.
[0104] <Standard Name Reference List>
[0105] Numerical extraction item standard name:
[0106] Left ventricular shortening fraction
[0107] Left ventricular end-diastolic volume ...
[0109] # Output Format
[0110] The output should be in JSON format, where "name" is the type name, "value" is the extracted numerical value displayed in Arabic numerals (excluding Chinese characters), and "content" is the existing information modified from the <Knowledge Graph Key Information>. See the sample output format below.
[0111] {
[0112] "table" : [
[0113] {
[0114] "name" : "Indicator 1",
[0115] "value": "Value 1"
[0116] },
[0117] {
[0118] "name" : "Indicator 2",
[0119] "value": "Value 2"
[0120] },
[0121] {
[0122] "name": "Critical Value Name 1",
[0123] "value" : "Nan"
[0124] } ]
[0126] "content": ""
[0127] }
[0128] The preceding information is as follows:
[0129] {Historical speech-to-text transcript starting from the inspection}
[0130] The current information is as follows:
[0131] {Current speech-to-text snippet}
[0132] The key information about the knowledge graph is as follows:
[0133] {First knowledge information retrieved from the knowledge graph}
[0134] # Based on the provided "existing information", output the "structured results" of the medical text according to the output format requirements.
[0135] In this embodiment, during the fast-thinking phase, the large model can use the provided ultrasound knowledge information (first knowledge information) combined with previously identified historical speech-to-text segments to correct the current speech-to-text segment and extract ultrasound detection parameters. The corrections include, but are not limited to, correcting spelling errors, resolving spoken language errors, and standardizing medical terminology. With the assistance of ultrasound knowledge, multi-level real-time error correction from lexical to semantic levels can be achieved. This significantly improves the output accuracy of the large model's fast-thinking mode while maintaining response speed.
[0136] The large model's fast-thinking mode outputs ultrasound examination parameters that can be displayed to doctors in real time, enabling real-time interaction and ensuring transparency of the examination results. The corrected speech-to-text segment can be cached until the ultrasound examination is completed, at which point the cached corrected speech-to-text segments are sequentially combined to form a complete corrected speech-to-text.
[0137] In some embodiments of this application, the process of step S130, which involves calling a large model and using slow thinking mode to generate an ultrasound description and diagnosis based on the original complete speech-to-text, is described.
[0138] In one alternative implementation, a prompt instruction can be used to instruct the large model to adopt a slow-thinking mode to generate ultrasound descriptions and diagnoses based on the original complete speech-to-text transcription.
[0139] In another alternative implementation, during the process of generating ultrasound descriptions and diagnoses using the slow-thinking mode in the large model, the aforementioned ultrasound knowledge graph information can be introduced to provide knowledge enhancement information for the slow-thinking mode and help improve the quality of the generated results.
[0140] Specifically, entities and relationships can be extracted from each speech-to-text segment of the ultrasound detection process to obtain a set of entities and relationships. Based on the set of entities and relationships, relevant secondary knowledge information can be retrieved in the ultrasound knowledge graph.
[0141] The large model is invoked in a slow-thinking mode to generate ultrasound descriptions and diagnoses based on the original complete speech-to-text transcription and secondary knowledge information.
[0142] In one optional implementation of the process of obtaining the entity and relation set described above, if the aforementioned steps of the large model fast thinking mode processing refer to ultrasound knowledge graph information, that is, if entities and relations are extracted for each speech-to-text segment separately, then the entities and relations extracted for each speech-to-text segment can be cached. After the ultrasound detection is completed, the entities and relations extracted from each speech-to-text segment can be summarized to obtain the entity and relation set. In another optional implementation, after the ultrasound detection is completed, entities and relations can be extracted from the original complete speech-to-text of this ultrasound detection process (composed of each speech-to-text segment) to obtain the entity and relation set.
[0143] Based on entity and relation sets, relevant knowledge information is retrieved from the ultrasound knowledge graph and used as secondary knowledge information. This secondary knowledge information is then used as knowledge enhancement information to assist the large model in generating high-quality ultrasound descriptions and diagnoses using a slow-thinking mode, thereby improving the accuracy and consistency of ultrasound descriptions and diagnoses.
[0144] In another alternative implementation, during the process of generating ultrasound descriptions and diagnoses in the slow-thinking mode of the large model, the input data, in addition to containing the original complete speech-to-text and second knowledge information, can be further supplemented with the corrected complete speech-to-text generated by the large model in the fast-thinking mode (including each corrected speech-to-text fragment).
[0145] In this embodiment, the process of generating ultrasound descriptions and diagnoses based on the original complete speech-to-text using a large model in slow-thinking mode includes:
[0146] The entities and relationships extracted from each speech-to-text segment of the ultrasound examination process are obtained to form a set of entities and relationships. Based on the set of entities and relationships, relevant second knowledge information is retrieved from the configured ultrasound knowledge graph.
[0147] The large model is invoked in a slow-thinking mode to generate ultrasound descriptions and diagnoses based on the original complete speech-to-text, the corrected complete speech-to-text, and second-knowledge information.
[0148] To guide the generation of ultrasound descriptions and diagnoses from large models, this application designs structured prompt words, namely, a second prompt template. The following is an example of an optional second prompt template:
[0149] Second prompt template:
[0150] You are an ultrasound physician. Based on the <corrected speech-to-text> and <key information from the knowledge graph> of the ultrasound examination process, and referring to the <uncorrected original speech-to-text>, after careful consideration, generate the corresponding <ultrasound description> according to the <ultrasound description sequence number requirements>, and directly provide the <ultrasound diagnosis>. For common examination items in the <corrected speech-to-text> that are not specifically mentioned, the examination results are assumed to be normal.
[0151] ## Requirements for Ultrasound Description Number:
[0152] Article 1 describes the atria and ventricles;
[0153] Section 2 describes the interventricular septum and its motion.
[0154] Item 3 is a description related to valves;
[0155] Item 4 is a description related to blood vessels.
[0156] Article 5 contains descriptions related to the pericardium, etc.
[0157] Other information, such as heart rate test results, is listed in item 6.
[0158] ##Uncorrected original speech transcription
[0159] {Original complete speech-to-text transcript}
[0160] ## Corrected speech transcription
[0161] {Corrected complete speech-to-text}
[0162] ##Key Information in Knowledge Graphs
[0163] {Secondary knowledge information retrieved from the knowledge graph}
[0164] ## Output Requirements
[0165] Please generate the JSON in the following format, and do not generate any other irrelevant information:
[0166] {
[0167] "findings": "1. Ultrasound description\n2. Ultrasound description\n3. Ultrasound description\n4. Ultrasound description\n5. Ultrasound description",
[0168] "impression": ["Ultrasound Diagnosis 1", "Ultrasound Diagnosis 2", "Ultrasound Diagnosis 3"]
[0169] }
[0170] In this embodiment, by providing the corrected complete speech-to-text to the large model, the large model can reduce the token cost for correcting the speech-to-text in slow-thinking mode, thereby further reducing the waiting time for answer output and improving the efficiency of ultrasound report generation. Simultaneously, the original complete speech-to-text retains the original speech-to-text information. If the large model makes incorrect corrections when correcting the original speech-to-text in fast-thinking mode, the large model retains the possibility of restoring the incorrect correction to correct information using the original complete speech-to-text in slow-thinking mode, ensuring the quality of the final generated ultrasound description and diagnosis.
[0171] In some embodiments of this application, an update strategy for the ultrasound knowledge graph is further provided. Specifically:
[0172] After receiving the generated ultrasound report, it can be submitted for manual review (by a doctor or expert). The manually reviewed report serves as the standard report. Understandably, the information in the standard report is considered accurate and error-free after manual review.
[0173] The differences between the generated ultrasound reports and standard reports can be analyzed through large-scale model analysis, allowing for reflection on erroneous knowledge in the ultrasound knowledge graph and the missing knowledge that needs to be supplemented, thus enabling the updating of the ultrasound knowledge graph.
[0174] To avoid interfering with doctors' use, this application allows the large model to perform the above analysis tasks in idle mode. Specifically:
[0175] When the large model is idle, it is invoked to use a slow-thinking mode to compare the differences between the ultrasound report and the standard report. Combining the original complete speech-to-text and the corrected complete speech-to-text, it identifies erroneous spectroscopic knowledge in the spectroscopic knowledge set and provides the spectroscopic knowledge that needs to be supplemented.
[0176] The corrected complete speech-to-text includes each speech-to-text segment corrected during this ultrasound examination process.
[0177] The atlas knowledge set includes all atlas knowledge referenced in this ultrasound examination process. For example, the atlas knowledge set may include the first knowledge information and / or the second knowledge information described in the foregoing embodiments. The first knowledge information is the atlas knowledge retrieved from the ultrasound knowledge graph related to the entities and relationships of the current speech-to-text segment during the large model's fast-thinking mode's correction process for the current speech-to-text segment. The second knowledge information is the atlas knowledge retrieved from the ultrasound knowledge graph related to the entity and relationship sets of each speech-to-text segment in this ultrasound examination process during the large model's slow-thinking mode's generation of ultrasound descriptions and diagnoses.
[0178] During this ultrasound inspection process, relevant knowledge retrieved from the ultrasound knowledge graph can be cached in the graph knowledge set, representing all the graph knowledge referenced in this ultrasound inspection process.
[0179] This embodiment, by comparing the differences between the generated ultrasound report and the standard report, can identify missing and redundant hallucination content in the generated ultrasound report. Combining the original complete speech-to-text and the corrected complete speech-to-text, the causes of the differences can be analyzed, including two parts: one part is that the atlas knowledge set contains incorrect atlas knowledge, leading to incorrect corrections of the original complete speech-to-text based on this incorrect atlas knowledge, thus causing errors in the ultrasound report; the other part is that some atlas knowledge is missing from the atlas knowledge set, preventing the detection and correction of errors in the original complete speech-to-text, thus causing errors in the ultrasound report. This embodiment, by calling a large model and using a slow-thinking mode for analysis, can identify incorrect atlas knowledge in the atlas knowledge set and provide the atlas knowledge that needs to be supplemented.
[0180] Based on this, the ultrasound knowledge map is updated according to the erroneous and missing knowledge maps output by the large model.
[0181] Among them, the atlas knowledge that needs to be completed in the output of the large model represents the missing atlas knowledge in the ultrasound knowledge atlas, such as entity synonym knowledge corresponding to the doctor's new expression habits.
[0182] In some possible implementations, this application can obtain habitual expressions or synonyms input by humans, and add these habitual expressions or synonyms as new atlas knowledge to the ultrasound knowledge graph, thereby facilitating the subsequent ultrasound diagnostic process and enabling direct understanding and standardization of doctors' new habitual terms.
[0183] In some possible implementations, the large model can employ a thought chain approach to reason, completing the comparison of differences between ultrasound reports and standard reports. This involves combining the original complete speech-to-text and the corrected complete speech-to-text, identifying erroneous spectrogram knowledge within the spectrogram knowledge set, and assigning the task of completing the necessary spectrogram knowledge. The relevant thought chain may include the following steps:
[0184] 1. Indicate the differences between the large-scale ultrasound report and the standard report;
[0185] 2. The large model is instructed to locate the target location that may cause the discrepancies in the corrected complete speech-to-text.
[0186] 3. Instruct the large model to compare the differences between the original complete speech-to-text and the corrected complete speech-to-text. If the target position is different, it means that the correction of the target position is incorrect. The incorrect spectrogram knowledge needs to be found in the spectrogram knowledge set, and the reason should be explained. If the target position is not different, it means that the target position has not been corrected. Provide the spectrogram knowledge that needs to be supplemented, and explain the reason.
[0187] The mind chain strategy can guide large models to better complete complex tasks and improve the accuracy of task results.
[0188] In some possible implementations, to more accurately pinpoint the differences between ultrasound and standard reports, when instructing the large model to compare the differences between the ultrasound and standard reports, the large model can be required to first decompose and standardize the ultrasound descriptions and diagnoses within both the ultrasound and standard reports, and then compare the differences based on the decomposed and standardized information. Furthermore, the large model can also be instructed to output the decomposed and standardized ultrasound descriptions and diagnoses within both the ultrasound and standard reports.
[0189] Further optional, in the embodiments of this application, the co-occurrence frequency of each entity in the splitting and standardization processing results of ultrasound description and diagnosis in the standard report output by the large model can be statistically analyzed. For entity pairs whose co-occurrence frequency exceeds a set threshold (e.g., 50%), if it is determined that the ultrasound knowledge graph does not contain the graph knowledge of that entity pair, it can be added to the ultrasound knowledge graph to improve the ultrasound knowledge graph.
[0190] To guide the large model in identifying erroneous atlas knowledge within the atlas knowledge set and providing necessary atlas knowledge for completion, while simultaneously outputting the processed results of splitting and standardizing ultrasound descriptions and diagnoses exceeding the scope of both the report and the standard report, this application designs a structured prompt, namely a third prompt template. The following is an example of an optional third prompt template:
[0191] Third prompt template:
[0192] You are an ultrasound physician. Based on the differences between the <Standard Report> and <Generated Report>, and using key information from the <Knowledge Graph Set>, compare the <Corrected Speech-to-Text> and <Uncorrected Original Speech-to-Text>. After in-depth consideration, provide the missing knowledge graph information in the form of triples from the <Knowledge Graph Set>, and explain your thought process for each missing triple. Also, identify any incorrect <Key Knowledge Graph Information> and explain the reasons for the errors.
[0193] ## Standard Report
[0194] {Standard report content includes examination item values, ultrasound description, and ultrasound diagnosis}
[0195] ## Generate Report
[0196] The generated ultrasound report includes examination item values, ultrasound description, and ultrasound diagnosis.
[0197] ## Collection of Graphical Knowledge
[0198] {Atlas knowledge referenced in this ultrasound examination}
[0199] ## When completing the knowledge graph, supplement it according to the following candidate entities and candidate relations.
[0200] Candidate entities:
[0201] {Candidate entity content, such as the diameter of the aortic sinus, the diameter of the ascending aorta, the thickness of the interventricular septum, the motion of the interventricular septum...}
[0202] Candidate Relationship:
[0203] {Candidate relationship content, such as scope, abbreviation, personalized expression, speech recognition error, etc.}
[0204] ## Corrected speech transcription
[0205] {Corrected complete speech-to-text}
[0206] ## Uncorrected original speech transcription
[0207] {Original complete speech-to-text transcript}
[0208] ## Thinking Steps
[0209] 1. You need to first compare the differences between the <Standard Report> and the <Generated Report>.
[0210] 1.1 You need to first break down and standardize the ultrasound diagnosis. For example, break down "widening of the aortic sinus and ascending aorta" into "widening of the aortic sinus" and "widening of the ascending aorta"; standardize "slightly reduced left ventricular motion amplitude" and "significantly reduced left ventricular motion amplitude" into "reduced left ventricular motion amplitude".
[0211] 1.2 You need to compare the standardized ultrasound diagnoses one by one to find the differences. These differences include two types: 1. Omitted parts that are present in the <Standard Report> but not in the <Generated Report>. 2. Hallucination parts that are present in the <Generated Report> but not in the <Standard Report>.
[0212] 1.3 You need to compare the differences between each number in the ultrasound description, but ignore the different expressions of the same meaning. For example, "the rest of the ventricular walls are normal" and "all other ventricular walls are normal" are different expressions of the same meaning.
[0213] 1.4 You need to compare the values of each examination item in the ultrasound examination parameters one by one to identify any differences. These differences fall into three categories: 1. Omissions where the <Standard Report> is present but not in the <Generated Report>. 2. Illusory findings where the <Generated Report> is present but not in the <Standard Report>. 3. Errors where the examination item exists in both the <Standard Report> and the <Generated Report>, but the values differ.
[0214] 2. Based on the differences found in point 1, you need to identify the possible reasons for the discrepancies in the corrected speech-to-text transcript. (For example, the standard report may contain "incoordination of ventricular wall motion," while the generated report does not, but the speech-to-text transcript may contain "asynchronous motion," which could be due to the doctor's personalized expression habits.)
[0215] 3. Carefully compare the differences between the uncorrected original speech-to-text and the corrected speech-to-text.
[0216] 3.1 If the target location of the cause found in step 2 is different, it means that it has already been corrected. This proves that the previous correction was wrong, and it is necessary to find the incorrect triplet relation in the <Graph Knowledge Set> and explain why it was judged to be wrong.
[0217] 3.2 If the cause found in step 2 is located in the same target position, it means that it has not yet been corrected. This proves that this part of the knowledge is missing, and the graph knowledge that needs to be supplemented needs to be given in the form of triples in the <Graph Knowledge Set>, along with the analysis process.
[0218] 4. Output according to the required format.
[0219] ## Output Requirements
[0220] Please generate the JSON in the following format, and do not generate any other irrelevant information:
[0221] {
[0222] "Results of Ultrasound Diagnosis After Disassembly and Standardization": [
[0223] "Standard report for ultrasound diagnosis": [],
[0224] "Generate Ultrasound Diagnosis Report": [] ]
[0226] "Differences in the report": "", # List the numerical sequence according to the different order of appearance.
[0227] "Atlas knowledge that needs to be completed": [
[0228] { # Graph knowledge that needs to be completed 1
[0229] "Analysis Process": "", # Includes the process of comparing and analyzing differences.
[0230] "Key knowledge points to be added to the diagram": "" # Triplet
[0231] },
[0232] # 2 Atlas knowledge points that need to be completed
[0233] "Analysis Process": "", # Includes the process of comparing and analyzing differences.
[0234] "Key knowledge points to be added to the diagram": "" # Triplet
[0235] }
[0236] ],
[0237] "Incorrect knowledge of atlases": [
[0238] { # Incorrect spectral knowledge 1
[0239] "Incorrect knowledge of the graph": "", # Triplet
[0240] "Analysis Process": "" # Explain why the judgment was incorrect
[0241] },
[0242] { # Incorrect spectral knowledge 2
[0243] "Incorrect knowledge of the graph": "", # Triplet
[0244] "Analysis Process": "" # Explain why the judgment was incorrect
[0245] } ]
[0247] }
[0248] In some embodiments of this application, an optional implementation method is further described for updating the ultrasound knowledge graph based on the erroneous atlas knowledge output by the large model and the atlas knowledge that needs to be supplemented in the aforementioned steps.
[0249] Specifically, the first cumulative occurrence count of erroneous atlas knowledge output by the large model can be counted. If the first cumulative occurrence count exceeds a first threshold (e.g., 10 times or other values) or a user confirmation instruction for the erroneous atlas knowledge is received, the erroneous atlas knowledge will be deleted from the ultrasound knowledge graph.
[0250] The system counts the second cumulative occurrence count of the atlas knowledge that needs to be supplemented. If the second cumulative occurrence count exceeds the second threshold (e.g., 5 times or other values) or if the system receives a confirmation instruction from the user regarding the atlas knowledge that needs to be supplemented, then the atlas knowledge that needs to be supplemented is added to the ultrasound knowledge atlas.
[0251] In this embodiment, considering that the output results of the large model may contain errors, the graph is updated only after the cumulative occurrence of erroneous knowledge and the cumulative occurrence of knowledge that needs to be supplemented are counted. This avoids incorrect graph updates caused by erroneous word outputs from the large model and improves the robustness of the ultrasound knowledge graph.
[0252] The updated ultrasound knowledge graph can be applied to the report generation and processing of subsequent ultrasound diagnostic procedures, providing richer real-time error correction and information completion capabilities for the fast-thinking mode of the large model, enabling it to directly understand and standardize doctors' new idiomatic language. It also provides stronger logical reasoning support for the slow-thinking mode of the large model, enabling it to make more accurate and complete diagnoses.
[0253] Through the ultrasound knowledge graph update mechanism provided in this application, the system can continuously learn from actual clinical use, constantly adapt to the expression habits of different doctors, and at the same time strengthen its medical knowledge system, ultimately achieving a spiral increase in processing capacity and report quality.
[0254] The ultrasound knowledge graph update mechanism provided in this application forms a self-learning closed-loop system. By automatically updating the ultrasound knowledge graph through clinical use, this application enables the system to continuously adapt to different usage habits, significantly improving its long-term value and effectiveness.
[0255] Reference Figure 3 This example illustrates a complete workflow for generating ultrasound reports.
[0256] The system acquires the doctor's real-time spoken audio during the ultrasound examination and feeds it into a speech recognition model for streaming recognition, resulting in a current speech-to-text segment. This current speech-to-text segment is then combined with historical speech-to-text segments from previous moments in the ultrasound examination process until the examination concludes, resulting in the original complete speech-to-text. Alternatively, the current speech-to-text segment can be fed into a larger model, where entity and relation extraction is performed using a fast-thinking mode. Based on the extracted entities and relations, relevant primary knowledge information is retrieved from the ultrasound knowledge graph as knowledge enhancement information for use by the larger model's fast-thinking mode.
[0257] In the large-scale model's fast-thinking mode, the current speech-to-text segment is corrected, and ultrasound examination parameters are extracted to obtain the corrected speech-to-text segment and ultrasound examination parameters. All corrected speech-to-text segments from this ultrasound examination process are combined to form the complete corrected speech-to-text. The ultrasound examination parameters are output and displayed in real-time during the ultrasound examination for the doctor to review.
[0258] After the ultrasound examination, relevant secondary knowledge information was retrieved from the ultrasound knowledge graph based on the original complete speech-to-text transcript. The large model was then invoked in slow-thinking mode to generate an ultrasound description and diagnosis based on the original complete speech-to-text transcript, the corrected complete speech-to-text transcript, and the secondary knowledge information.
[0259] A complete ultrasound report is generated based on the ultrasound description and diagnosis generated by the large model, as well as the various ultrasound detection parameters obtained from the large model's fast thinking mode. The generated ultrasound report can be reviewed by doctors to obtain a standardized report.
[0260] During the idle time of the large model, the differences between the ultrasound report generated by the large model in slow thinking mode and the standard report after being reviewed by the doctor can be compared. Combining the original complete speech-to-text and the corrected complete speech-to-text, the erroneous atlas knowledge in the atlas knowledge set can be identified, and the atlas knowledge that needs to be supplemented can be provided. The atlas knowledge set includes all atlas knowledge referenced in this ultrasound examination process.
[0261] For erroneous graph knowledge output by the large model and graph knowledge that needs to be completed, counts can be accumulated separately. Once the accumulated counts reach the corresponding thresholds, the graph update conditions are considered met, and structured update instructions can be generated accordingly. For example, corresponding knowledge deletion instructions can be generated based on erroneous graph knowledge, and corresponding knowledge insertion instructions can be generated based on graph knowledge that needs to be completed.
[0262] Furthermore, it can also acquire doctors' new expression habits and synonyms, and generate structured update instructions based on this.
[0263] The ultrasound knowledge graph is updated according to the generated structured update instructions. The updated ultrasound knowledge graph can be applied to the ultrasound report generation process in the next ultrasound diagnostic process.
[0264] This application also provides an electronic device in its embodiments. (See reference...) Figure 4 The diagram illustrates a structural schematic suitable for implementing the electronic device in the embodiments of this application. The electronic device in the embodiments of this application may include, but is not limited to, fixed terminals such as computers, ultrasonic testing terminals, etc. Figure 4 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.
[0265] like Figure 4 As shown, the electronic device may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 1, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 2 or a program loaded from a storage device 8 into a random access memory (RAM) 3, to implement the ultrasound report generation method of the foregoing embodiments of this application. When the electronic device is powered on, the RAM 3 also stores various programs and data required for the operation of the electronic device. The processing unit 1, ROM 2, and RAM 3 are interconnected via a bus 4. An input / output (I / O) interface 5 is also connected to the bus 4.
[0266] Typically, the following devices can be connected to I / O interface 5: input devices 6 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 7 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 8 including, for example, memory cards, hard drives, etc.; and communication devices 9. Communication device 9 allows electronic devices to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 4 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or have instead.
[0267] This application also provides a computer program product including computer-readable instructions, which, when executed on an electronic device, cause the electronic device to implement any of the ultrasound report generation methods provided in this application.
[0268] This application also provides a computer-readable storage medium that carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement any of the ultrasound report generation methods provided in this application.
[0269] It should also be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. In addition, in the device embodiment drawings provided in this application, the connection relationship between modules indicates that they have a communication connection, which can be implemented as one or more communication buses or signal lines.
[0270] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware, or it can be implemented by special-purpose hardware including application-specific integrated circuits, special-purpose CPUs, special-purpose memory, special-purpose components, etc. Generally, any function performed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be diverse, such as analog circuits, digital circuits, or special-purpose circuits. However, for this application, software program implementation is more often the preferred implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk, or optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, training equipment, or network device, etc.) to execute the methods described in the various embodiments of this application.
[0271] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product.
[0272] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, training device, or data center to another website, computer, training device, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives (SSDs)).
[0273] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can be referred to each other.
Claims
1. A method for generating an ultrasound report, characterized in that, include: The system acquires real-time spoken audio from doctors during ultrasound examinations, which is a description of the ultrasound images. The real-time spoken speech is fed into a speech recognition model for streaming recognition to obtain the current speech-to-text segment. The large model is invoked to use the fast thinking mode to extract ultrasound detection parameters based on the current speech-to-text segment, and the ultrasound detection parameters are output as a real-time preview. After the ultrasound examination is completed, the large model is invoked to generate an ultrasound description and diagnosis based on the original complete speech-to-text transcript in slow thinking mode. The original complete speech-to-text transcript includes all speech-to-text fragments obtained during this ultrasound examination process. By integrating the ultrasound detection parameters, ultrasound description, and diagnosis, an ultrasound report corresponding to this ultrasound detection is obtained.
2. The ultrasound report generation method according to claim 1, characterized in that, The process of extracting ultrasound detection parameters based on the current speech-to-text segment using a large model and a fast-thinking mode includes: The large model is invoked to correct the current speech-to-text segment using the fast thinking mode, and the ultrasound detection parameters are extracted to obtain the corrected current speech-to-text segment and ultrasound detection parameters output by the large model.
3. The ultrasound report generation method according to claim 2, characterized in that, The process of calling the large model to correct the current speech-to-text segment using a fast-thinking mode and extracting ultrasound detection parameters includes: Extract the entities and relationships of the current speech-to-text segment, and retrieve relevant first knowledge information in the configured ultrasound knowledge graph based on the extracted entities and relationships; The large model is invoked to use a fast thinking mode to correct the current speech-to-text segment based on the first knowledge information, and the ultrasound detection parameters are extracted to obtain the corrected current speech-to-text segment and ultrasound detection parameters output by the large model.
4. The ultrasound report generation method according to claim 3, characterized in that, The process of calling the large model and using a fast-thinking mode to correct the current speech-to-text segment based on the first knowledge information, and extracting ultrasound detection parameters, includes: Obtain the first prompt template, which includes the current speech-to-text segment slot, the historical speech-to-text slot, the knowledge slot, and the first prompt. The first prompt is used to instruct the large model to adopt a fast thinking mode, refer to the historical speech-to-text and knowledge information, correct the current speech-to-text segment, and extract the ultrasound detection parameters. The current speech-to-text segment is filled into the current speech-to-text segment slot, the previously recognized historical speech-to-text segment is filled into the historical speech-to-text segment slot, and the first knowledge information is filled into the knowledge slot to obtain the edited first prompt instruction. The first prompt instruction is then sent into the large model.
5. The ultrasound report generation method according to claim 1, characterized in that, The process of generating ultrasound descriptions and diagnoses based on original, complete speech-to-text transcription using a large model and a slow-thinking approach includes: The entities and relationships extracted from each speech-to-text segment of this ultrasound detection process are obtained to obtain a set of entities and relationships. Based on the set of entities and relationships, relevant second knowledge information is retrieved in the configured ultrasound knowledge graph. The large model is invoked in a slow-thinking mode to generate an ultrasound description and diagnosis based on the original complete speech-to-text and the second knowledge information.
6. The ultrasound report generation method according to any one of claims 2-4, characterized in that, The process of generating ultrasound descriptions and diagnoses based on original, complete speech-to-text transcription using a large model and a slow-thinking approach includes: The entities and relationships extracted from each speech-to-text segment of this ultrasound detection process are obtained to obtain a set of entities and relationships. Based on the set of entities and relationships, relevant second knowledge information is retrieved in the configured ultrasound knowledge graph. The large model is invoked in a slow-thinking mode. Based on the original complete speech-to-text, the corrected complete speech-to-text, and the second knowledge information, an ultrasound description and diagnosis are generated. The corrected complete speech-to-text includes each speech-to-text segment corrected during the ultrasound examination process.
7. The ultrasound report generation method according to claim 3, characterized in that, Also includes: Obtain a standard report corresponding to the ultrasound report, wherein the standard report is a report after the ultrasound report has been manually reviewed; When the large model is idle, it is invoked to compare the differences between the ultrasound report and the standard report in slow thinking mode. Combining the original complete speech-to-text and the corrected complete speech-to-text, erroneous atlas knowledge is identified in the atlas knowledge set, and the atlas knowledge that needs to be supplemented is provided. The corrected complete speech-to-text includes each speech-to-text segment corrected during this ultrasound examination, and the atlas knowledge set includes all atlas knowledge referenced during this ultrasound examination. The ultrasound knowledge map is updated based on the erroneous and necessary supplementary atlas knowledge output by the large model.
8. The ultrasound report generation method according to claim 7, characterized in that, The large model employs a slow-thinking mode to compare the differences between the ultrasound report and the standard report. It combines the original complete speech-to-text transcript with the corrected complete speech-to-text transcript, identifies erroneous spectrogram knowledge within the spectrogram knowledge set, and identifies the spectrogram knowledge that needs to be supplemented. This process is guided by a thought chain approach, which includes the following steps: The large model is used to compare the differences between the ultrasound report and the standard report. The large model is instructed to locate the target location in the corrected complete speech-to-text that may lead to the discrepancy content. The large model is instructed to compare the differences between the original complete speech-to-text and the corrected complete speech-to-text. If the target position is different, it means that the correction of the target position is incorrect. The incorrect graph knowledge needs to be found in the graph knowledge set and the reason should be explained. If the target location is not a different point, it means that the target location has not been corrected. Provide the map information that needs to be completed and explain the reason.
9. The ultrasound report generation method according to claim 7, characterized in that, The process of updating the ultrasound knowledge map based on the erroneous and necessary supplementary atlas knowledge output by the large model includes: The first cumulative occurrence count of erroneous atlas knowledge is counted. If the first cumulative occurrence count exceeds a first threshold or a user confirmation instruction for erroneous atlas knowledge is received, the erroneous atlas knowledge is deleted from the ultrasound knowledge atlas. The second cumulative occurrence count of the atlas knowledge that needs to be supplemented is counted. If the second cumulative occurrence count exceeds the second threshold or a user confirmation instruction for the atlas knowledge that needs to be supplemented is received, the atlas knowledge that needs to be supplemented is added to the ultrasound knowledge atlas.
10. An electronic device, characterized in that, include: Memory and processor; The memory is used to store programs; The processor is used to execute the program to implement each step of the ultrasound report generation method as described in any one of claims 1 to 9.
11. A readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements each step of the ultrasound report generation method as described in any one of claims 1 to 9.
12. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the various steps of the ultrasound report generation method as described in any one of claims 1 to 9.
Citation Information
Patent Citations
Image report automatic generation method and system based on voice recognition and artificial intelligence
CN118506960A
Large model and interaction-based medical diagnosis report generation method and related equipment
CN120853787A
Voice interaction large model family health assistant dialogue method, device and equipment and medium
CN120954763A
Structured ultrasound report generation method based on large language model
CN121215155A
