Ultrasound report generation method, system, device, storage medium and program
By obtaining voice data and ultrasonic image feature data for verification, common error problems in the entry process of ultrasonic report sheets are solved, the entry efficiency and accuracy are improved, and the quality of the report is ensured.
Patent Information
- Application Number
- CN202410999508.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-24
- Publication Date
- 2024-12-17
AI Technical Summary
During the input process of ultrasound report, problems such as typos, left and right reversal, numerical input errors, and unit writing errors often occur, resulting in more typos and low entry efficiency.
Generate ultrasonic description mark text by obtaining speech data, extracting feature data from ultrasonic images for verification, and extracting image features in convolutional neural networks to ensure the accuracy and completeness of the report content.
It significantly improves the entry efficiency of ultrasound reports, reduces the error rate, ensures the comprehensiveness, completeness and accuracy of the reports, and provides a reliable basis for subsequent diagnosis and treatment plans.
Smart Images
Figure CN119153011A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of ultrasound report generation, and in particular to an ultrasound report generation method, system, device, storage medium and program. Background Art
[0002] At present, in most hospitals, the examining doctor verbally describes the ultrasound examination findings and possible ultrasound diagnosis in real time, and the data entry clerk beside him writes the ultrasound report immediately according to the doctor's description using a pre-customized fixed template. Due to the large workload of the ultrasound department, the tight time for issuing reports, the need to issue personalized reports in real time for different lesions, and the fact that reports for different examination sites contain many values of different orders of magnitude, when the examining doctor verbally conveys the contents of the report to the data entry clerk, the ultrasound report often contains typos, left-right reversals, numerical entry errors, and unit writing errors. There are technical problems in this field such as many errors in ultrasound reports and low data entry efficiency. Summary of the invention
[0003] The invention provides an ultrasound report generation method, system, equipment, storage medium and program, which solve the technical problems of many errors in ultrasound report sheets and low input efficiency.
[0004] In a first aspect, the present invention provides an ultrasound report generation method, comprising: acquiring voice data, and generating ultrasound description mark text based on the voice data; acquiring an ultrasound image, and extracting ultrasound image feature data from the ultrasound image; and verifying the ultrasound description mark text based on the ultrasound image feature data to generate ultrasound description text.
[0005] In some embodiments, the step of extracting ultrasonic image feature data from the ultrasonic image includes: preprocessing the ultrasonic image to obtain a preprocessed image; inputting the preprocessed image into a convolutional neural network to extract the ultrasonic image feature data.
[0006] In some embodiments, the step of verifying the ultrasound description mark text based on the ultrasound image feature data includes: determining whether all the ultrasound image feature data exist in the ultrasound description mark text; if at least one ultrasound image feature data does not exist in the ultrasound description mark text, then marking the image area corresponding to the ultrasound image feature data that does not exist in the ultrasound description mark text in the ultrasound image.
[0007] In some embodiments, the step of generating ultrasound description marked text based on voice data includes: obtaining ultrasound description input text; converting the voice data into ultrasound description voice text; marking the ultrasound description input text based on the ultrasound description voice text to obtain ultrasound description marked text.
[0008] In some embodiments, the step of marking the ultrasonic description input text based on the ultrasonic description voice text includes: extracting direction data and unit data in the ultrasonic description voice text to form an ultrasonic description voice feature set; extracting direction data and unit data in the ultrasonic description input text to form an ultrasonic description input feature set; judging whether the ultrasonic description voice feature set is equal to the ultrasonic description input feature set; if the ultrasonic description voice feature set is not equal to the ultrasonic description input feature set, subtracting the ultrasonic description voice feature set and the ultrasonic description input feature set to obtain a voice input marking set; marking the ultrasonic description input text based on the voice input marking set.
[0009] In some embodiments, the method further includes: acquiring ultrasound report template information; and generating an ultrasound report in a preset format based on the ultrasound report template information, the ultrasound image, and the ultrasound description text.
[0010] In a second aspect, the present invention provides an ultrasound report generation system, comprising: an acquisition module, used to acquire voice data, generate ultrasound description mark text based on the voice data, acquire ultrasound images, and extract ultrasound image feature data from the ultrasound images; an analysis module, used to verify the ultrasound description mark text based on the ultrasound image feature data and generate ultrasound description text.
[0011] In a third aspect, the present invention provides a computer device, comprising a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of any one of the methods in the above aspects.
[0012] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of any one of the methods in the above aspects when executed by a processor.
[0013] In a fifth aspect, the present invention provides a computer program product, comprising a computer program / instructions, which implements the steps of any one of the methods in the above aspects when executed by a processor.
[0014] The present invention provides an ultrasound report generation method, system, device, storage medium and program, which obtains voice data and generates ultrasound description mark text based on the voice data; obtains an ultrasound image and extracts ultrasound image feature data from the ultrasound image; verifies the ultrasound description mark text based on the ultrasound image feature data to generate ultrasound description text; significantly improves the efficiency of ultrasound report entry and greatly reduces the error rate. BRIEF DESCRIPTION OF THE DRAWINGS
[0015] The present invention will be described in more detail below based on embodiments and with reference to the accompanying drawings:
[0016] Figure 1It is a flowchart of a method for generating an ultrasound report provided in an embodiment of the present application;
[0017] Figure 2 It is a structural schematic diagram of an ultrasound report generation system provided in an embodiment of the present application;
[0018] Figure 3 It is a flowchart of an artificial intelligence-based ultrasound report writing assistance method provided in an embodiment of the present application;
[0019] Figure 4 This is a schematic diagram of the module structure of an artificial intelligence-based ultrasound report writing assistance system provided in an embodiment of the present application;
[0020] Figure 5 It is a schematic diagram of an ultrasound report provided in an embodiment of the present application.
[0021] In the drawings, the same reference numerals are used for the same components, and the drawings are not drawn to scale. DETAILED DESCRIPTION
[0022] In order to enable those skilled in the art to better understand the scheme of the present invention, and to fully understand and implement how the present invention applies technical means to solve technical problems and achieve the corresponding technical effects, the technical scheme in the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiment of the present invention. Obviously, the described embodiment is only an embodiment of a part of the present invention, not all of the embodiments. The embodiments of the present invention and the various features in the embodiments can be combined with each other without conflict, and the technical schemes formed are all within the scope of protection of the present invention. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work should fall within the scope of protection of the present invention.
[0023] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0024] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.
[0025] At present, in most hospitals, the examining doctor verbally describes the ultrasound examination findings and possible ultrasound diagnosis in real time, and the data entry clerk beside him writes the ultrasound report immediately according to the doctor's description using a pre-customized fixed template. Due to the large workload of the ultrasound department, the tight time for issuing reports, the need to issue personalized reports in real time for different lesions, and the fact that reports for different examination sites contain many values of different orders of magnitude, when the examining doctor verbally conveys the contents of the report to the data entry clerk, the ultrasound report often contains typos, left-right reversals, numerical entry errors, and unit writing errors. There are technical problems in this field such as many errors in ultrasound reports and low data entry efficiency.
[0026] Hereinafter, the technical solution of the present application will be described in conjunction with specific embodiments.
[0027] Embodiment 1
[0028] Figure 1 is a flow chart of a method for generating an ultrasound report provided in an embodiment of the present application, such as Figure 1 As shown, in the technical solution of this embodiment, a method for generating an ultrasound report is provided, including: acquiring voice data, and generating an ultrasound description mark text based on the voice data; acquiring an ultrasound image, and extracting ultrasound image feature data from the ultrasound image; and verifying the ultrasound description mark text based on the ultrasound image feature data to generate an ultrasound description text.
[0029] The technical problem solved by this embodiment is how to improve the efficiency of ultrasound input and reduce the error rate of ultrasound input text. When writing an ultrasound report, doctors often need to pay attention to the image and describe it at the same time when performing an ultrasound examination. By obtaining the doctor's voice data and converting it into a marked text, the time and energy of manual input can be greatly saved. At the same time, the feature data accurately extracted from the ultrasound image can be carefully compared and verified with the generated marked text. For example, when the doctor is examining the liver, the voice describes that there is an abnormal echo in a certain area of the liver and information such as the specific shape and size. If this part of the key content is omitted in the generated marked text or there is an erroneous statement, the verification mechanism of this embodiment can be quickly discovered and corrected in time. This method significantly improves the input efficiency of ultrasound reports and greatly reduces the error rate. It ensures the comprehensiveness, completeness and accuracy of the report content, and provides a reliable basis for subsequent doctors to diagnose diseases and formulate reasonable treatment plans.
[0030] Embodiment 2
[0031] On the basis of the above embodiment, the step of extracting ultrasonic image feature data from the ultrasonic image includes: preprocessing the ultrasonic image to obtain a preprocessed image; inputting the preprocessed image into a convolutional neural network to extract ultrasonic image feature data.
[0032] The technical problem solved by this embodiment is how to improve the accuracy of extracting ultrasonic image feature data from the ultrasonic image. In the process of writing an ultrasonic report, the quality of the ultrasonic image obtained will be affected by many factors, such as the performance of the ultrasonic equipment, the patient's position, etc., which leads to technical problems such as noise, blur or poor contrast in the image. At this time, it is particularly important to preprocess the ultrasonic image. Through a series of image processing operations, such as denoising, contrast enhancement, binarization, etc., a clearer and easier to analyze preprocessed image can be obtained. Subsequently, the preprocessed image is input into a pre-trained convolutional neural network, which can accurately extract various subtle and critical feature data from the image. For example, in cardiac ultrasound examination, the convolutional neural network can effectively extract important features such as the thickness of the myocardium, the size of the ventricle, and the morphology of the valve. This technical solution significantly improves the accuracy of extracting feature data from ultrasonic images, making the subsequent ultrasound reports generated based on these feature data more accurate and reliable.
[0033] Embodiment 3
[0034] On the basis of the above-mentioned embodiment, the step of verifying the ultrasonic description mark text based on the ultrasonic image feature data includes: determining whether all the ultrasonic image feature data exist in the ultrasonic description mark text; if at least one ultrasonic image feature data does not exist in the ultrasonic description mark text, then marking, in the ultrasonic image, the image area corresponding to the ultrasonic image feature data that does not exist in the ultrasonic description mark text.
[0035] The technical problem solved by this embodiment is how to verify the ultrasound description mark text based on the ultrasound image feature data. In the generation of ultrasound reports, due to the fact that the doctor's focus and description focus during the examination process may be different, or there may be deviations in the information transmission and recording process, some key ultrasound image feature data are not fully and accurately reflected in the ultrasound description mark text. In view of this situation, the technical solution of this embodiment determines whether all feature data extracted from the ultrasound image have corresponding records in the mark text. If it is found that at least one feature data is missing in the mark text, the technical solution of this embodiment will clearly mark the specific image area corresponding to this part of the missing feature data in the ultrasound image, such as a red frame selection. For example, when performing a thyroid ultrasound examination, the image shows that the edge of the thyroid nodule has an important feature of burr-like changes, but it is not mentioned in the mark text. Through verification and markings such as red frame selection, doctors and data entry personnel can quickly notice this omission and promptly supplement the relevant description, thereby effectively ensuring the high consistency between the content of the ultrasound report and the actual examination image, and reducing the risk of missed diagnosis.
[0036] Embodiment 4
[0037] Based on the above embodiment, the step of generating ultrasound description marked text based on voice data includes: obtaining ultrasound description input text; converting the voice data into ultrasound description voice text; marking the ultrasound description input text based on the ultrasound description voice text to obtain ultrasound description marked text.
[0038] The technical problem solved by this embodiment is how to convert the voice data into ultrasound description text. In the process of writing ultrasound reports, doctors usually need to operate ultrasound equipment while describing the observed situation in real time. In this process, it often happens that the ultrasound description input text is not comprehensive enough or fails to keep up with the doctor's description in time. By quickly converting the voice data dictated by the doctor into ultrasound description voice text, the latest information can be captured in time. Then, based on this voice text, the input text is marked and supplemented to ensure the completeness and timeliness of the report content. For example, when performing an ultrasound examination, the doctor suddenly discovers a new abnormal structure in the organ during the examination and describes it in detail. Through the conversion of voice data and the marking of the input text, this key information can be quickly integrated into the report, which improves the efficiency and accuracy of the generation of ultrasound description mark text, so that the report can more comprehensively and accurately reflect the actual situation of the examination.
[0039] Embodiment 5
[0040] On the basis of the above embodiment, the step of marking the ultrasonic description input text based on the ultrasonic description voice text includes: extracting direction data and unit data in the ultrasonic description voice text to form an ultrasonic description voice feature set; extracting direction data and unit data in the ultrasonic description input text to form an ultrasonic description input feature set; judging whether the ultrasonic description voice feature set is equal to the ultrasonic description input feature set; if the ultrasonic description voice feature set is not equal to the ultrasonic description input feature set, subtracting the ultrasonic description voice feature set and the ultrasonic description input feature set to obtain a voice input marking set; marking the ultrasonic description input text based on the voice input marking set.
[0041] The technical problem solved by this embodiment is how to mark the ultrasound description input text based on the ultrasound description voice text. In the writing of ultrasound reports, the accuracy of data such as direction and unit is crucial to describing the location and size of the lesion. However, due to the uncertainty of the doctor's oral description, these key data may be inconsistent between the voice text and the input text. The technical solution of this embodiment first accurately extracts key data such as direction and unit from the ultrasound description voice text and the input text, and forms feature sets respectively. Then, it is determined whether the two feature sets are completely equal. In some embodiments, in order to speed up the comparison, the two feature sets are compared to determine whether they are different by comparing whether the lengths of the two feature sets are equal. If it is found that they are not equal, the difference part, that is, the voice input mark set, is obtained by a difference operation, and the input text is accurately marked and corrected based on this. Figure 5 is a schematic diagram of an ultrasound report provided in an embodiment of the present application, such as Figure 5As shown in the figure, during an ultrasound examination, the doctor described the size of the gallbladder as "1.2cm×0.5cm, 1.3cm×0.5cm", but the input text was written as "1.2mm×0.5mm, 1.3cm×0.5cm". The ultrasound description voice feature set is {1.2cm, 0.5cm, 1.3cm, 0.5cm}, while the ultrasound description input feature set is {1.2mm, 0.5mm, 1.3cm, 0.5cm}. At this time, if it is determined that the ultrasound description voice feature set is not equal to the ultrasound description input feature set, the difference between the ultrasound description voice feature set and the ultrasound description input feature set is calculated to obtain the voice input mark set {1.2cm, 1.2mm, 0.5mm}; the ultrasound description input text is marked based on the voice input mark set, for example, the elements 1.2mm and 0.5mm in the ultrasound description input text that are repeated in the voice input mark set are highlighted, or the elements 1.2mm and 0.5mm in the ultrasound description input text that are repeated in the voice input mark set are added with a border, so that doctors and input personnel can quickly find the error. At the same time, a pop-up window prompt is given for example 1.2cm to prompt doctors and input personnel that the element of the ultrasound description voice feature set does not appear in the ultrasound description input feature set. It can be understood that the ultrasound description voice feature set and the ultrasound description input feature set of this embodiment can also include other elements. Here, the size of the gallbladder is used as an example to illustrate the concept of the technical solution of this embodiment, and it is not used to limit the technical solution of this application. This technical solution can quickly detect input errors and ensure the consistency and accuracy of key data such as direction and units in the report.
[0042] Embodiment 6
[0043] Based on the above embodiment, the method further includes: acquiring ultrasound report template information; and generating an ultrasound report in a preset format based on the ultrasound report template information, the ultrasound image and the ultrasound description text.
[0044] The technical problem solved by this embodiment is how to generate an ultrasound report. In the final generation stage of the ultrasound report, it is necessary to effectively integrate and present the scattered information obtained from various channels in a standardized manner. By obtaining the pre-set ultrasound report template information, clear guidance is provided for the format and content framework of the report. Then, the key contents such as accurate ultrasound image information, detailed and verified ultrasound description text, etc. are filled and organized in an orderly manner according to the preset format specified by the template. For example, in the abdominal ultrasound report, the template stipulates that the condition of the liver is described first, followed by the gallbladder, pancreas and other organs, and there is a preset format for the description of each organ. Generating an ultrasound report according to such a preset format can ensure that the format of the report is unified and standardized, the content is clear and organized, and the key points are highlighted, which improves the readability and usability of the report and facilitates doctors to quickly and accurately obtain key information.
[0045] Embodiment 7
[0046] Figure 2 is a schematic diagram of the structure of an ultrasound report generation system provided in an embodiment of the present application, such as Figure 2 As shown, based on the above embodiments, the technical solution of this embodiment provides an ultrasound report generation system, including: an acquisition module, used to acquire voice data, generate ultrasound description mark text based on the voice data, acquire ultrasound images, and extract ultrasound image feature data from the ultrasound images; an analysis module, used to verify the ultrasound description mark text based on the ultrasound image feature data and generate ultrasound description text.
[0047] The technical problem solved by this embodiment is how to improve the efficiency of ultrasound input and reduce the error rate of ultrasound input text. When writing an ultrasound report, doctors often need to pay attention to the image and describe it at the same time when performing an ultrasound examination. By obtaining the doctor's voice data and converting it into a marked text, the time and energy of manual input can be greatly saved. At the same time, the feature data accurately extracted from the ultrasound image can be carefully compared and verified with the generated marked text. For example, when the doctor is examining the liver, the voice describes that there is an abnormal echo in a certain area of the liver and the specific morphology and size and other information. If this part of the key content is omitted or there is an erroneous statement in the generated marked text, the verification mechanism of the ultrasound report generation system of this embodiment can be quickly discovered and corrected in time. This method significantly improves the input efficiency of ultrasound reports and greatly reduces the error rate. It ensures the comprehensiveness, completeness and accuracy of the report content, and provides a reliable basis for subsequent doctors to diagnose diseases and formulate reasonable treatment plans. The other technical features and technical effects of this embodiment correspond to the above-mentioned embodiments and are not repeated here.
[0048] Embodiment 8
[0049] In the technical solution of this embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory, and the processor executes the computer program to implement the steps of any method of the above embodiments.
[0050] In the technical solution of this embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of any method of the above embodiments are implemented.
[0051] In the technical solution of this embodiment, a computer program product is provided, including a computer program / instructions, and when the computer program is executed by a processor, the steps of any method of the above embodiments are implemented.
[0052] The processor may include, but is not limited to, one or more processors or microprocessors. Each processor may be an application specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), a controller, a microcontroller, a microprocessor or other electronic components to execute the method in the above embodiments.
[0053] The computer-readable storage medium may be implemented by any type of volatile or non-volatile storage device or a combination thereof. The computer-readable storage medium may include, but is not limited to, for example, random access memory (RAM), read-only memory (ROM), flash memory, EPROM memory, EEPROM memory, registers, computer storage media (e.g., hard disk, floppy disk, solid-state drive, removable disk, CDROM, DVDROM, Blu-ray disc, etc.).
[0054] The computer-readable storage medium may also store at least one computer executable program / instruction, which may be, for example, a computer-readable instruction. The computer-readable storage medium includes, but is not limited to, for example, a volatile memory and / or a non-volatile memory. The volatile memory may include, for example, a random access memory (RAM) and / or a cache memory (cache), etc. The computer-readable storage medium may include, for example, a read-only memory (ROM), a hard disk, a flash memory, etc. For example, a non-transitory computer-readable storage medium may be connected to a computing device such as a computer, and then, when the computing device runs the computer-readable instructions stored on the computer-readable storage medium, the various methods described above may be performed.
[0055] In addition, the computer device may also include (but not limited to) a data bus, an input / output (I / O) bus, a display, and input / output devices (eg, keyboard, mouse, speaker, etc.), etc.
[0056] The processor may communicate with external devices via an I / O bus via a wired or wireless network.
[0057] In one embodiment, the at least one computer executable instruction may also be compiled into or constitute a software product / computer program product, wherein one or more computer executable instructions are executed by a processor to perform the various functions and / or method steps in the embodiments described in the present technology.
[0058] Embodiment 9
[0059] Based on the above embodiments, this embodiment provides an application example.
[0060] The present invention relates to the technical field of report writing, and in particular to an ultrasound report writing auxiliary method and system based on artificial intelligence.
[0061] Ultrasound reports are usually divided into two parts. The first part is called ultrasound findings / ultrasound description, in which you can see some ultrasound descriptions, sonographic features, and ultrasound diagnostic terms. The second part is called ultrasound impression / ultrasound prompts or ultrasound diagnosis, which is the doctor's tendency diagnosis based on ultrasound features and the patient's clinical information. The ultrasound report is not only a summary of the ultrasound examination work, which can provide a reference for clinical diagnosis and treatment, but also a legal basis for exploring the laws of diseases and handling medical disputes. However, most hospitals currently have the examining doctor verbally describe the ultrasound findings and possible ultrasound diagnoses in real time, and the data entry clerk next to him writes the ultrasound report immediately based on the doctor's description using a pre-customized fixed template. Due to the heavy workload of the ultrasound department, the tight time for issuing reports, the need to issue personalized reports in real time for different lesions, and the fact that reports for different examination sites contain many values of different orders of magnitude, when the examining doctor verbally conveys the contents of the report to the data entry clerk, the ultrasound report often contains typos, left-right inversions, numerical entry errors, unit writing errors, etc. There are many technical problems such as clerical errors and low efficiency in ultrasound report entry, which are particularly serious in the ultrasound description part of the ultrasound report, and may even lead to subsequent incorrect treatment plans, aggravate medical disputes, and even medical accidents. This field urgently needs an ultrasound report writing assistance solution based on artificial intelligence to solve the technical problems of many clerical errors and low entry efficiency in ultrasound reports.
[0062] The present application provides an artificial intelligence-based ultrasound report writing assistance method and system, which relates to the report writing technology field. The system includes a data acquisition module, a data analysis module, a wireless communication module and a data storage module. The data acquisition module is used to obtain the patient's basic information, the ultrasound image of the target area and the real-time conversation voice information of the examining doctor; the data analysis module is used to analyze the acquired ultrasound image, extract all area information, numerical values and position information in the ultrasound image, and analyze the acquired conversation voice to extract tissue, organ or lesion information and effective numerical values and position information; the wireless communication module connects the data acquisition module and the data analysis module, and is used to send the basic information, ultrasound image area information, numerical values and position information, and effective voice information content to the data storage module; the data storage module is used to store the patient's basic information, ultrasound numerical information and effective voice information. The artificial intelligence-based ultrasound report writing assistance method and system provided by the present application improves the writing efficiency of ultrasound reports, avoids the problems of data entry errors and position description errors about organs, tissues and lesions in ultrasound reports, and reduces medical disputes.
[0063] In a first aspect, the present application provides an artificial intelligence-based ultrasound report writing assistance method, comprising the following steps:
[0064] S1: Obtain patient information, including the patient's basic personal information and clinical information, and send it to the storage module via wireless communication. Extract template information based on the patient's basic personal information and clinical information;
[0065] S2: Waiting for the input operator to edit the ultrasound report in real time according to the extracted template information to obtain the ultrasound description input text;
[0066] S3: Scan the area to be detected by an ultrasonic device to obtain an ultrasonic image of the target area;
[0067] S4: Real-time recognition and reading of ultrasound image information. Based on the local feature method, feature point (data measurement information, body mark information) detection is performed on the ultrasound image to obtain ultrasound image feature data. The feature points represent key points with stability and uniqueness. These feature points can be used as the basis for subsequent automatic capture. According to the read ultrasound image information, the image features, numerical values and body mark information are extracted, the area information, numerical value information, numerical unit information and position information in the ultrasound image are obtained, and the data is sent to the storage module via wireless communication;
[0068] S5: The voice receiving system is turned on in real time to obtain the voice data of the conversation between the clinician and the examiner in real time, extract the information of the measured tissue, organ or lesion described in the conversation content and all the corresponding numerical values, numerical units and position information to obtain the ultrasound description voice text, and send it to the storage module via wireless communication;
[0069] S6: Obtain the basic personal information and clinical information stored in the system, the area information, value, unit and location information in the ultrasound image, the measured tissue, organ or lesion information and value, value unit and location information in the voice data, and obtain the ultrasound description text, all of which are sent to the client in real time, and the content of the ultrasound report is checked in real time. If the extracted information is inconsistent with the content of the ultrasound report, the first, second... prompt information is obtained respectively, and finally the user is waited for verification and confirmation.
[0070] S7: Finally, the content of the ultrasound report is verified as a whole, including typos, whether the ultrasound description is consistent with the ultrasound prompt part and position description. After the verification is completed, wait for the user to confirm the final information and complete the report writing.
[0071] On the other hand, the present application also provides an artificial intelligence-based ultrasound report writing assistance system, which includes: a data acquisition module, a data analysis module, a wireless communication module and a data storage module.
[0072] The data acquisition module is used to acquire the patient's basic information, the ultrasound image of the target area, and the real-time conversation voice information of the examining doctor;
[0073] The data analysis module is used to analyze the acquired ultrasound image, extract all area information, numerical values and position information in the ultrasound image, and analyze the acquired conversation voice to extract tissue, organ or lesion information and valid numerical values and position information;
[0074] The wireless communication module is connected to the data acquisition module and the data analysis module, and is used to send basic information, ultrasonic image area information, numerical and position information, and effective voice information content to the data storage module.
[0075] The data storage module is used to store basic patient information, ultrasound numerical information and effective voice information.
[0076] Furthermore, the data acquisition module includes a patient basic information acquisition unit, an ultrasound data acquisition unit and a conversation voice information acquisition unit;
[0077] The patient basic information acquisition unit is used to acquire the patient's basic personal information and clinical information;
[0078] The ultrasonic data acquisition unit is used to scan the area to be detected by an ultrasonic device to acquire an ultrasonic image of the target area;
[0079] The conversation voice information acquisition unit is used to acquire the real-time conversation voice information of the examining doctor.
[0080] Further, the data analysis module includes an ultrasonic image region detection unit, an ultrasonic image value and position information extraction unit, and a conversation voice information extraction unit;
[0081] The ultrasonic image area detection unit is used to perform target area range detection on the ultrasonic image;
[0082] The ultrasonic image value and position information extraction unit is used to analyze the image features, values and body marks in the ultrasonic image, obtain the value information and position information in the ultrasonic image and store them;
[0083] The dialogue voice information extraction unit is used to extract the numerical information and position information of the dialogue information including the size description, parameter description and position description of the tissue, organ and lesion.
[0084] Furthermore, the data storage module includes a basic information storage unit, an ultrasonic image value and position information storage unit, and a valid voice information storage unit;
[0085] The basic information storage unit is used to store the basic information and clinical information of the patient in the hospital system and transmit it to the user end for display, and the data entry operator performs the final confirmation and verification.
[0086] The ultrasound image area information, value and position information storage unit is used to store the ultrasound image value and position information and transmit it to the user end, and compare it with the data and position information of the organ, tissue or lesion in the ultrasound report entered by the user end data entry operator, and prompt it through a pop-up window, and the data entry operator and the examining doctor make the final confirmation and verification.
[0087] The effective voice information storage unit is used to store effective numerical information and position information about numerical descriptions, parameter descriptions, and position descriptions of organs, tissues, and lesions extracted from voice conversations.
[0088] The beneficial technical effects of the present invention are: obtaining the patient's basic information and clinical information, ultrasound image information, and physician conversation voice information through multiple channels, extracting the patient's personal information and clinical information, extracting regional information, numerical values, numerical units, and position information in the ultrasound image, extracting the numerical values, numerical units, and position information of tissues, organs, and lesions in real-time voice, integrating the multiple information, and sending it to the user end in real time, thereby improving the efficiency of writing ultrasound reports, avoiding problems such as report data entry errors and location description errors, and reducing medical disputes.
[0089] like Figure 3 As shown, this embodiment provides an artificial intelligence-based ultrasound report writing assistance method, the method comprising:
[0090] S1: Obtain the patient's basic personal information and clinical information, and send it to the storage module via wireless communication. Extract template information based on the patient's basic personal information and clinical information;
[0091] S2: The data entry operator edits the ultrasound report content in real time based on the extracted template information;
[0092] S3: Scan the area to be detected by an ultrasonic device to obtain an ultrasonic image of the target area;
[0093] S4: Real-time recognition and reading of ultrasound image information, based on the local feature method, feature point (data measurement information, body mark information) detection of ultrasound images, the feature points represent key points with stability and uniqueness, these feature points can be used as the basis for subsequent automatic interception. According to the read ultrasound image information, the image features, values and body mark information are extracted, the area information, value information, value unit information and position information in the ultrasound image are obtained, and the data is sent to the storage module via wireless communication;
[0094] S5: The voice receiving system is turned on in real time to obtain the conversation information of the clinician during the examination in real time, extract the information of the measured tissue, organ or lesion described in the conversation content and all corresponding values, value units and location information, and send them to the storage module via wireless communication;
[0095] S6: The basic personal information and clinical information obtained and stored in the system, the area information, value, unit and location information in the ultrasound image, the measured tissue, organ or lesion information and the value, value unit and location information in the voice content are all sent to the client in real time, and the content of the ultrasound report is checked in real time, such as whether the value entry is correct, whether the value unit marking is correct, and whether the left and right directions of the lesion are accurately described. If the extracted information is inconsistent with the content of the ultrasound report, the first, second... prompt information is obtained respectively, and finally the user is waited for verification and confirmation.
[0096] S7: Finally, the contents of the ultrasound report are checked as a whole, including typos, whether the ultrasound description is consistent with the ultrasound prompt part and position description. After the check is completed, wait for the final information to be confirmed and the report is written.
[0097] like Figure 4 As shown, this embodiment provides an artificial intelligence-based ultrasound report writing assistance system, the system includes: a data acquisition module, a data analysis module, a wireless communication module and a data storage module. The data acquisition module is used to obtain the patient's basic information, the ultrasound image of the target area and the real-time conversation voice information of the examining doctor; the data acquisition module includes a patient basic information acquisition unit, an ultrasound data acquisition unit and a conversation voice information acquisition unit.
[0098] The patient basic information acquisition unit is used to acquire the patient's basic personal information and clinical information;
[0099] The ultrasonic data acquisition unit is used to scan the area to be detected by an ultrasonic device to acquire an ultrasonic image of the target area;
[0100] The conversation voice information acquisition unit is used to acquire the real-time conversation voice information of the examining doctor.
[0101] The data analysis module is used to analyze the acquired ultrasound image, extract all area information, numerical values and position information in the ultrasound image, and analyze the acquired conversational voice to extract tissue, organ or lesion information and valid numerical values and position information; the data analysis module includes an ultrasound image area detection unit, an ultrasound image numerical value and position information extraction unit and a conversational voice information extraction unit.
[0102] The ultrasonic image area detection unit is used to detect the target area range of the ultrasonic image;
[0103] The ultrasonic image value and position information extraction unit is used to analyze the image features, values and body marks in the ultrasonic image, obtain the value information and position information in the ultrasonic image and store them;
[0104] The dialogue voice information extraction unit is used to extract the numerical information and position information of the size description, parameter description and position description of the tissue, organ and lesion contained in the dialogue information.
[0105] The wireless communication module is connected to the data acquisition module and the data analysis module, and is used to send basic information, ultrasonic image area information, numerical and position information, and effective voice information content to the data storage module.
[0106] The data storage module is used to store basic patient information, ultrasound numerical information and effective voice information. The data storage module includes a basic information storage unit, an ultrasound image numerical value and position information storage unit, and an effective voice information storage unit.
[0107] The basic information storage unit is used to store the basic information and clinical information of patients in the hospital system and transmit it to the user end for display, and the data entry operator performs the final confirmation and verification.
[0108] The ultrasound image area information, value and position information storage unit is used to store the ultrasound image value and position information and transmit it to the user end, and compare it with the data and position information of the organ, tissue or lesion in the ultrasound report entered by the user end data entry operator, and prompt it through a pop-up window, and the data entry operator and the examining doctor make the final confirmation and verification.
[0109] The effective voice information storage unit is used to store effective numerical information and position information of numerical description, parameter description and position description of organs, tissues and lesions extracted from voice dialogues.
[0110] In the embodiments provided by the present invention, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the above-mentioned module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.
[0111] It should be noted that, in the present invention, the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element limited by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.
[0112] Although the embodiments disclosed in the present invention are as above, the above contents are only embodiments adopted for facilitating the understanding of the present invention and are not intended to limit the present invention. Any technician in the technical field to which the present invention belongs can make any modifications and changes in the form and details of the implementation without departing from the spirit and scope disclosed in the present invention, but the patent protection scope of the present invention shall still be subject to the scope defined in the attached claims.
Claims
1. A method for generating an ultrasound report, characterized in that: include: Acquire voice data, and generate ultrasound description markup text based on the voice data; Acquire an ultrasonic image, and extract ultrasonic image feature data from the ultrasonic image; The ultrasound description mark text is verified based on the ultrasound image feature data to generate an ultrasound description text.
2. The ultrasound report generation method according to claim 1, characterized in that: The step of extracting ultrasonic image feature data from the ultrasonic image comprises: Preprocessing the ultrasonic image to obtain a preprocessed image; The preprocessed image is input into a convolutional neural network to extract ultrasonic image feature data.
3. The method for generating an ultrasound report according to claim 2, characterized in that: The step of verifying the ultrasound description mark text based on the ultrasound image feature data comprises: Determining whether all ultrasound image feature data exist in the ultrasound description mark text; If at least one ultrasonic image feature data does not exist in the ultrasonic description mark text, then in the ultrasonic image, an image area corresponding to the ultrasonic image feature data that does not exist in the ultrasonic description mark text is marked.
4. The method for generating an ultrasound report according to claim 1, characterized in that: The step of generating ultrasonic description markup text based on the voice data comprises: Obtain ultrasound description input text; Converting the voice data into ultrasound description voice text; The ultrasound description input text is marked based on the ultrasound description voice text to obtain an ultrasound description marked text.
5. The method for generating an ultrasound report according to claim 4, characterized in that: The step of marking the ultrasound description input text based on the ultrasound description voice text comprises: Extracting direction data and unit data from the ultrasonic description speech text to form an ultrasonic description speech feature set; Extracting direction data and unit data from the ultrasound description input text to form an ultrasound description input feature set; Determining whether the ultrasound description speech feature set is equal to the ultrasound description input feature set; If the ultrasonic description speech feature set is not equal to the ultrasonic description recording feature set, subtracting the ultrasonic description speech feature set from the ultrasonic description recording feature set to obtain a speech recording mark set; The ultrasound description entry text is marked based on the speech entry mark set.
6. The ultrasound report generation method according to claim 1, characterized in that: The method further comprises: Get ultrasound report template information; An ultrasound report is generated in a preset format based on the ultrasound report template information, the ultrasound image and the ultrasound description text.
7. An ultrasound report generation system, characterized in that: include: An acquisition module, used for acquiring voice data, generating ultrasound description markup text based on the voice data, acquiring an ultrasound image, and extracting ultrasound image feature data from the ultrasound image; The analysis module is used to verify the ultrasound description mark text based on the ultrasound image feature data to generate an ultrasound description text.
8. A computer device comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 6.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.
10. A computer program product comprising a computer program / instructions, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.