A program that guides the operation of an ultrasound probe, and an ultrasound image generation system.

Abstract

This guides you through the operation of the ultrasound probe. [Solution] The system analyzes a first ultrasound image at a first time point in time using an ultrasound probe to identify a predetermined structure within the imaging target and determine its first position. A first area indicator corresponding to the first area of ​​the predetermined structure in the first ultrasound image is also output. The system outputs guide information instructing the operator to move the ultrasound probe to a position where an ultrasound image in which the predetermined structure is better displayed can be obtained. A second area indicator having attributes corresponding to the second area of ​​the predetermined structure in a second ultrasound image at a second time point is also output in a manner that allows for recognition of changes over time relative to the first area indicator.

Description

Technical Field

[0001] The present invention relates to a program for guiding the operation of an ultrasonic probe, and particularly to a method for favorably displaying a target structure displayed in an ultrasonic image obtained using the ultrasonic probe.

Background Art

[0002] When performing an ultrasonic inspection, an operator can place an ultrasonic probe at a free position of a scanning target, orient it in a free direction, and perform imaging to obtain a non-destructive / non-invasive ultrasonic image.

[0003] On the other hand, for example, when it is necessary to uniformly perform an ultrasonic inspection of an entire target site for the purpose of a health check or the like, the operator may be required to place the ultrasonic probe at a predetermined position and direction. For example, the Japanese Society for Ultrasonic Diagnosis of Digestive Organs has defined 25 types of recommended recording cross-sections (ultrasonic B-mode images), and for each of these cross-sections, the position and direction in which the operator places the ultrasonic probe on the subject are defined. Images recorded according to such procedures serve as conclusive evidence of the inspection and are used to prepare a report indicating the presence or absence of abnormal findings. At this time, it is required to depict and record the organs included in each cross-section at the center of the screen and with the largest cross-sectional area as much as possible.

[0004] However, due to differences in the technical skills of operators or fatigue caused by repeating inspections on a large number of people, appropriate image recording may not be performed in some cases.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0006] Therefore, in order to obtain desirable ultrasound images, there is a need for a system that provides easy-to-understand support for operators of ultrasound probes to perform appropriate operations. [Means for solving the problem]

[0007] In a first aspect of this disclosure, an ultrasound image generation system is provided for displaying an ultrasound image on a display device. The ultrasound image generation system includes a processor and a non-temporary storage medium for storing a program. The program is configured to cause the processor to perform the following steps: analyze a first ultrasound image of an ultrasound probe at a first time point in time to identify a predetermined structure within an imaging target; identify a first position of the predetermined structure in the first ultrasound image; output a first area indicator having attributes corresponding to a first area of ​​the predetermined structure in the first ultrasound image; output guide information instructing an operator operating the ultrasound probe to move the ultrasound probe to a position where an ultrasound image in which the predetermined structure is better displayed can be obtained based on the identified first position; and output a second area indicator having attributes corresponding to a second area of ​​the predetermined structure in a second ultrasound image at a second time point in time, in a manner that allows for recognition of changes over time relative to the first area indicator.

[0008] A second aspect of this disclosure provides a program for displaying an ultrasound image on a display device of an ultrasound image generation system. The program is configured to cause a processor to perform the following actions: analyze a first ultrasound image at a first time point in time with an ultrasound probe to identify a predetermined structure within the imaging target; identify a first position of the predetermined structure in the first ultrasound image; output a first area indicator having attributes corresponding to a first area of ​​the predetermined structure in the first ultrasound image; output guide information instructing an operator operating the ultrasound probe to move the ultrasound probe to a position where an ultrasound image in which the predetermined structure is better displayed can be obtained based on the identified first position; and output a second area indicator having attributes corresponding to a second area of ​​the predetermined structure in a second ultrasound image at a second time point in time, in a manner that allows for recognition of changes over time relative to the first area indicator. [Brief explanation of the drawing]

[0009] [Figure 1] This is a block diagram showing an overview of the overall system configuration in an embodiment of the present invention. [Figure 2] This block diagram shows the configuration of the ultrasound equipment that makes up the system shown in Figure 1. [Figure 3] This figure shows an image generated by an ultrasonic image generation system according to an embodiment of the present invention. [Figure 4] This figure shows an image generated by an ultrasonic image generation system in another embodiment of the present invention. [Figure 5] This figure shows an image generated by an ultrasonic image generation system in another embodiment of the present invention. [Figure 6] This is a magnified view of the area indicator set. [Figure 7] This figure shows an image at a different point in time generated by the ultrasound image generation system in an embodiment of the present invention. [Figure 8] This figure shows an image at a different point in time generated by the ultrasound image generation system in an embodiment of the present invention. [Modes for carrying out the invention]

[0010] Embodiments of the invention will be described below. However, the claimed invention is not limited to the embodiments described herein. In particular, this disclosure will use a medical ultrasound diagnostic system as an example, but the present invention can be applied to ultrasound inspection systems, ultrasound inspection devices, and ultrasound probes for non-destructive testing of buildings, structures, various machinery and equipment, etc.

[0011] Embodiments of the present invention will now be described with reference to the drawings. The system 100 shown in Figure 1 comprises a plurality of ultrasound examination devices 101, 102, and 103 and a server 104. The plurality of ultrasound examination devices 101, 102, and 103 may be connected to each other via a network 105. In addition, each of the plurality of ultrasound examination devices 101, 102, and 103 may also be connected to the server 104 via the network 105. A trained model production terminal 107 that produces trained models may also be connected to the server 104 via the network 105. Trained models used in the present invention can be transmitted from the trained model production terminal 107 to the plurality of ultrasound examination devices 101, 102, and 103 via the server 104 or without going through the server 104. In addition, the trained model production terminal 107 can receive training data from the plurality of ultrasound examination devices 101, 102, and 103 via the server 104 or without going through the server 104. Furthermore, other software related to the present invention can be downloaded from server 104 to multiple ultrasound inspection devices 101, 102, and 103.

[0012] Figure 2 shows the configurations of the ultrasound examination devices 101, 102, and 103, respectively, as ultrasound examination device 200. Ultrasound examination device 200 includes a transmitting beamformer 203 that drives a plurality of vibrating elements 201 arranged in an ultrasound probe 202 to generate a pulsed ultrasonic signal, and a transmitter 204 that radiates the generated pulsed ultrasonic signal to a subject (not shown). The pulsed ultrasonic signal generates echoes that are reflected within the subject and return to the vibrating elements 201. The echoes are converted into electrical signals by the vibrating elements 201, and the electrical signals are received by a receiver 205. The electrical signals representing the received echoes, i.e., the echo signals, are amplified by the required gain in the receiver 205 and then input to a receiving beamformer 206, where receiving beamforming is performed. The receiving beamformer 206 outputs ultrasonic data after receiving beamforming.

[0013] The receiving beamformer 206 may be a hardware beamformer or a software beamformer. If the receiving beamformer 206 is a software beamformer, it may comprise one or more processors 207, including any one or more of the following: a graphics processing unit (GPU), a microprocessor, a central processing unit (CPU), a digital signal processor (DSP), or other types of processors capable of performing logical operations. The processors comprising the receiving beamformer 206 may consist of processors other than the processors 207 described below, or they may consist of processors 207. The echo signal before receiving beamforming and the ultrasonic data after receiving beamforming are stored in memory 209.

[0014] When an echo signal is received, the processor 207 can process the data in real time during the scanning session. For the purposes of this disclosure, the term “real time” is defined to include procedures performed without any intentional delay.

[0015] Also, the data can be temporarily stored in a buffer (not shown) during the ultrasonic scan and processed in a live or offline operation, not in real time. In this disclosure, the term "data" can be used to refer to one or more data sets acquired using an ultrasonic inspection device in this disclosure.

[0016] The ultrasonic data can be processed by the processor 207 in other or different mode-related modules (e.g., B-mode, color Doppler, M-mode, color M-mode, spectral Doppler, contrast mode, elastography, TVI, strain, strain rate, etc.) to create data for ultrasonic images. For example, one or more modules can generate ultrasonic images such as B-mode, color Doppler, M-mode, color M-mode, spectral Doppler, contrast mode, elastography, TVI, strain, strain rate, and combinations thereof.

[0017] A video processor module may be provided that reads an image frame from the memory while a treatment is being performed on the subject and displays the image frame in real time. The video processor module can save the image frame in the image memory, and the ultrasonic image is read from the image memory and displayed on the display 208.

[0018] Note that, as used in this specification, the term "image" broadly refers to both visible images and data representing visible images. Also, the term "data" can include raw data, which is ultrasonic data (echo signals or beam signals) before scan conversion calculation, and image data, which is data after scan conversion calculation. The processor 207 can further process information obtained by analyzing the ultrasonic image and display it on the display 208 together with the ultrasonic image.

[0019] When the processor 207 includes a plurality of processors, the above-described processing tasks assigned to the processor 207 may be assigned to the plurality of processors. For example, a first processor can be used to demodulate and decimate the RF signal, and a second processor can be used to further process the data and then display an image. Also, for example, when the receiving beamformer 206 is a software beamformer, its processing function may be executed by a single processor or by a plurality of processors.

[0020] The display 208 is an LED (Light Emitting Diode) display, an LCD (Liquid Crystal Display), a micro-LED display, an organic EL (Electro-Luminescence) display, or the like. The display 208 does not need to be single, and a plurality of displays may be provided. When a plurality of displays are provided, all or many of the displays may be used as the main display, or one display may be used as the main display and one or more other displays may be used as auxiliary displays. The auxiliary display may be, for example, one or more LED elements arranged on the keyboard of the user interface 210, the ultrasonic probe 202, and / or other components described later.

[0021] Memory 209 is any known data storage medium. For example, the ultrasound machine 200 includes multiple memories 209, including non-transient and transient storage media as memory 209. Non-transient storage media are non-volatile storage media such as HDD (Hard Disk Drive) and ROM (Read Only Memory). Non-transient storage media may also include portable storage media such as CD (Compact Disk), DVD (Digital Versatile Disk), and Blu-ray Disc (registered trademark). Programs executed by the processor 207 are stored in the non-transient storage media. Protocols, learning models, image data, etc., necessary for carrying out the present invention are also stored in the non-transient storage media. Transient storage media can be volatile storage media such as RAM (Random Access Memory). All of these may be stored in the same memory 209, or at least one of them may be stored in a different memory 209. Furthermore, memory 209 may be multiple data storage media deployed on the cloud.

[0022] The user interface 210 can accept input from an operator. For example, the user interface 210 accepts instructions and information input from an operator. The user interface 210 is composed of a keyboard, hard keys, and soft keys, etc. The user interface 210 may also include various input devices such as a mouse, touch panel, pen tablet, touchpad, trackball, joystick, pointing device, eye tracking, and voice input.

[0023] The speaker 211 outputs sound under the control of the processor 207. In one example, the speaker 211 outputs sound based on a signal input from the processor 207.

[0024] As shown in Figure 1, a communication interface (not shown) may be provided to enable the ultrasound inspection devices 101 to 103 (each corresponding to the ultrasound inspection device 200 in Figure 2) to communicate with a server 104 or the like. However, in other embodiments, the ultrasound inspection device 200 can operate in a standalone state, and the present invention can be implemented in this manner. In this case, a communication interface is not required.

[0025] Figure 3 shows an image 300 generated by an ultrasound image generation system in an embodiment of the present invention. In this embodiment, a navigation function as described in Patent Document 2 is used. The navigation function guides the operator to perform the examination in the correct location by showing the organs that need to be measured in a pre-registered order, so as to ensure that no images that need to be acquired for diagnosis are missed. In this example, the imaging target is the human body, and the structure to be examined is an organ of the human body or a part thereof.

[0026] For example, when it is necessary to perform an ultrasound examination of the entire target area for purposes such as a health checkup, the areas to be examined are registered in order to ensure that the examination proceeds according to a predetermined procedure in order to prevent omissions. The operator can proceed by referring to the displayed comments and reference images. The operator examines the necessary areas according to these instructions and records the images. The recorded images serve as evidence that the examination was completed without omissions and are used to report whether or not any abnormalities were detected. At this time, it is required that the organs indicated in each step be depicted as far as possible in the center of the screen and with the maximum possible cross-sectional area, and recorded as images.

[0027] In the upper left of Figure 3, the inspection step display box 320 is displayed. The inspection step display box 320 displays the names of the inspection structures 321 to 329 that are to be inspected at each step of the navigation. The inspection is scheduled to be performed in the order of the inspection structure names 321 to 329 displayed in the inspection step display box 320, and the navigation is performed in this order. In the example in Figure 3, step 321 of the liver inspection 331 has been completed, and step 322 of the right kidney inspection 335 has started. In the embodiment for carrying out the invention, the explanation will focus on the right kidney 335, but other target organs can be processed in the same way. Furthermore, as mentioned above, when applied to an industrial ultrasound inspection device, the target organ can be replaced with internal structures such as pipes and valves.

[0028] Referring to Figure 3, the current examination step number 351 and the current examination step name 353 are displayed at the bottom of the screen 300 of the display 208. In this example, the current examination step number 351 is the 25th, and the right kidney, which is the structure (organ) to be examined, is registered as the step name 353. The organs to be selected as examination items and the order of examinations are customizable, and the setting of the examination step name 353 can also be customized. In certain embodiments of the present invention, in addition to customizing which organs to examine and in what order, it is also possible to execute routines that prompt the user to switch to a mode at each step (e.g., Doppler mode) or to add comments at specific steps (e.g., presence or absence of fatty liver, cholecystectomy). At each examination step, in addition to a cross-sectional image of the target organ, comments and annotations can be recorded automatically, semi-automatically, or manually in the memory 209. Comments and annotations can include changes in lesions, the percentage of change, the name of the suspected lesion, its classification, stage, congenital deformity, the width of bile duct or aortic aneurysm, arrows placed in specific locations, and shapes such as circular enclosures. Comments and annotations are displayed in a manner that allows the operator to edit them based on AI-generated automatic recognition. After reviewing the content, the operator can edit or save the comments and annotations without making any changes.

[0029] The current step can be terminated and the next step switched, i.e., the target organ can be switched, in response to various events. In certain embodiments of the present invention, the system can move to the next target organ in response to an event that saves an image of the current target organ. After moving to the next target organ, only the next target organ can be tracked. Also, for example, the operator can skip the left kidney examination step 323 and perform the steps from the spleen examination step 324 onward by instructing the software to select the spleen examination step 324 displayed in the examination step display box 320 and set it as the current target organ. In certain embodiments of the present invention, if the target organ is set to a specific single organ, other organs or structures that are not the target organ will be ignored even if other organs are recognized by the AI, and only the target organ will be tracked. In certain embodiments of the present invention, the names of all organs and structures detected in the B-mode image 310 and their detection accuracy are displayed.

[0030] In the example in Figure 3, the ultrasound probe 202 is placed on the subject, and the ultrasound probe 202 receives echo signals from the portion of the subject including the liver 331 and the right kidney 335, generating a B-mode image 310. At this point, the B-mode image 310 is analyzed. As a result of this analysis, the presence of the subject's liver 331 and right kidney 335 is identified within the B-mode image 310. In the example in Figure 3, the bounding box 333 surrounding the liver 331 is shown in, for example, dark gray, and the bounding box 337 surrounding the right kidney 335 is shown in, for example, light green, so that the operator can recognize that step 321 of the liver examination has been completed and step 322 of the right kidney examination has begun. The bounding boxes 333 and 337 indicate that these structures are recognized as specific structures. These boxes do not need to be rectangular; they can be circular, elliptical, polygonal, etc. Furthermore, as shown in Figure 4, the recognized structure can also be shown to the operator by segmenting the recognized structure and tracing the resulting figures 332 and 336, displaying them semi-transparently and / or blinking, and superimposing them on the B-mode image. In Figure 4, figures 332 and 336 are drawn with shapes and sizes that perfectly match the cross-sections of the liver 331 and the right kidney 335, but their sizes may be made 1-10% larger or smaller. The central part can also be removed, leaving only an annular shape along the contour.

[0031] In a specific embodiment, an AI learning model determines whether the organ set for each step of this navigation software is depicted on the screen. The neural network of the AI ​​learning model can be various types of neural networks, such as deep learning, DeepDream, RNN, CNN, diffusion, and GAN. The AI ​​learning model can detect which types of organs are included in the ultrasound image. It can also obtain the probability (certainty) that the detected organ is that organ. The detected organ and its probability can be displayed in association with the image of that organ.

[0032] In certain embodiments, AI detects the region and shape of one or more organs included in the ultrasound image, and checks whether the target organ specified in each step is included. If the current target organ is included, an instruction is output to move the probe in a direction that brings that region as close to the center of the screen as possible. In the example in Figure 3, an arrow 340 is displayed on the screen prompting the operator to move the probe so that the frame 337 surrounding the right kidney 335 is depicted in the center of the B-mode image 310. The arrow 340 is depicted in the same color as the frame 337 surrounding the right kidney 335, making it easier for the operator to understand that the instruction is for the right kidney 335. The length of the arrow 340 can be changed according to the required movement distance. That is, if the required movement distance is long, the arrow 340 is depicted as longer, and if the required movement distance is short, the arrow 340 is depicted as shorter. The required movement distance can also be expressed not by the length of the arrow 340, but by its thickness, brightness, color, etc.

[0033] The position of a predetermined structure, exemplified by the right kidney 335, in an ultrasound image can be determined by various methods. For example, the intersection of the diagonals of the frame 337 shown in Figure 3 can be taken as the center of the right kidney 335. Alternatively, the center of the right kidney 335 can be taken as the average of the pixel positions included in the traced figure 336 shown in Figure 4. In the former case, the determined position is not necessarily accurate, but the computational load is small and the calculation can be completed quickly. In the latter case, the computational load is large, but a more accurate position can be determined. Furthermore, in certain embodiments, if a lesion is detected in the right kidney 335, the center of the right kidney 335 can be taken into consideration by considering the location of the lesion. For example, the midpoint between the center of the right kidney 335 without considering the lesion and the center of the area occupied by the lesion can be taken as the center of the right kidney 335 with the lesion. In other embodiments, the presence or absence of a lesion in the right kidney 335 is not considered, and the center of the right kidney 335 is calculated.

[0034] In Figures 3 and 4, the guide information that instructs the operator of the ultrasound probe 202 to move the ultrasound probe to a position where an ultrasound image showing a predetermined structure is obtained more clearly is embodied by an arrow 340. However, the guide information can be embodied in various other forms. In the example in Figure 5, a zebra line 344 is displayed prompting the operator to move the B-mode image 310 itself to the right. The zebra line 344 may be fixed, but it can also be animated or blinked. The guide information may also be text; for example, the text information "(move the ultrasound probe 202) 4cm to the right" can be displayed on the screen 300. Furthermore, LEDs can be placed on both sides of the keyboard of the ultrasound probe 202 and / or the user interface 210, and the direction to move can be indicated by lighting up the LEDs. The movement direction guide display is not limited to the arrow 340, zebra line 344, and LED described here, but can be embodied in various forms. For example, the system can provide voice guidance for movement, or use voice or animation to indicate the patient's location where the ultrasound probe 202 should be applied (e.g., between the right 7th and 8th ribs).

[0035] As described above, in certain embodiments, AI detects the region and shape of one or more organs included in the ultrasound image, and it is confirmed whether the specified target organ is included in each step. The area of ​​a predetermined structure in the ultrasound image, as exemplified by the right kidney 335 in Figure 3, can be calculated using various methods. For example, the area of ​​the frame 337 shown in Figure 3 can be approximated as the drawing area of ​​the right kidney 335. Alternatively, for example, the number of pixels included in the traced figure 336 shown in Figure 4 can be used as its area. In the former case, the calculated area is not necessarily accurate, but the computational load is small and the calculation can be completed quickly. In the latter case, the computational load is large, but a more accurate area can be identified.

[0036] The calculated area of ​​the right kidney 335 is displayed on screen 300 as a set of area indicators 360. Figure 6 is a magnified view of the set of area indicators 360. In this example, the area indicator 361 on the far right is the most recent, and the area indicator 365 on the far left is the oldest. Area indicators 361-365 are generated at a predetermined sampling rate. Area indicators 361-365 can be set to different colors for each type of organ, or to the same color for all organs. The predetermined sampling rate should preferably be equal to the image frame rate. When a new area indicator is generated, the existing area indicators 361-365 are shifted to the right, and area indicator 365 disappears from screen 300. The horizontal axis 368 represents time, and the vertical axis 369 represents the area of ​​the target organ. It is possible to change the vertical axis 369 to represent time and the horizontal axis 368 to represent the area of ​​the target organ, or to reverse the time axis (i.e., shift left to shift right). These changes can also be customized by the operator. By displaying multiple area indicators simultaneously on the display device, the operator can confirm how the area of ​​the target organ depicted changes over time. In other words, the set of area indicators 360 outputs in a manner that allows the temporal changes in the area of ​​the target organ at multiple points in time to be recognized.

[0037] In the example in Figure 6, the area indicator is implemented in the form of a bar graph, and the area of ​​the target organ is reflected in the attribute of the bar graph's length. The area indicator can be implemented in various forms. For example, the bar graph in Figure 6 can be a line graph or a simple plot. In this case, the area of ​​the target organ will be reflected in the attribute of the line graph or the height (coordinate position) of the plot. Also, image items of the same shape can be arranged vertically or horizontally, and the area of ​​the target organ can be made green when it is large and red when it is small, or high brightness when it is large and low brightness when it is small. Furthermore, the area indicator can also be represented by animation. For example, when the area of ​​the target organ is large, the sign, symbol, or marker representing that organ can become active or enlarged. In such a case, the area of ​​the target organ will be reflected in the attribute of the area indicator's shape. The area of ​​the target organ can also be simply displayed numerically. The examiner can refer to the temporal change of the area indicator and record the largest possible cross-sectional area. The examiner can refer to the temporal changes in the area indicator and move the probe (perpendicular to the cross-section) to save the image when the cross-sectional area is larger.

[0038] Returning to Figure 3 and continuing the explanation, the image quality (IQ) gauge 380 is displayed to the right of the examination step display box 320. In this example, the image quality gauge 380 is configured such that a higher image quality (IQ) score results in more illuminated squares on the gauge 380, and a lower image quality score results in fewer illuminated squares on the gauge 380. The image quality score is calculated according to the area of ​​the target organ, the location of the target organ, and the degree of noise and / or artifacts contained in the target organ. Noise generated in ultrasound images can be caused by EMI, power supply issues, etc., as is well known to those skilled in the art. Artifacts generated in ultrasound images can be caused by various factors, such as poor probe contact, the presence of calcified organs, metals in the body, and bone fragments, and appear in the image as acoustic shadows, reverberation, haze, etc., as is well known to those skilled in the art.

[0039] Each score that forms the basis of the image quality score can be calculated using various functions. For example, the area score can be calculated based on "current area ÷ maximum area observed over a certain period." The position score can be obtained by substituting the distance from the center into a function of a parabola that is concave downwards with a y-intercept of 1. The noise and / or artifact score can be calculated based on "(total image area - area of ​​noise or artifacts) ÷ total image area."

[0040] The image quality score can be calculated, for example, by weighting the area score, the location score, and the noise and / or artifact score. In certain embodiments, the noise and / or artifact score may be excluded from the calculation of the image quality score, and the image quality score may be calculated using only the area score and the location score. Each score may be a gauge indicating the level of each item. Alternatively, for example, a gauge may be displayed that combines the location score and the noise and / or artifact score, excluding the area score, or other combinations of gauges may be displayed. The image quality gauge 380 can also change color, such as red for low scores, blue for good scores, and yellow for intermediate ranges, or its brightness may be changed. The image quality gauge 380 can also have other shapes, such as annular rather than linear.

[0041] Figure 7 shows an image generated by the ultrasound image generation system in an embodiment of the present invention at a different point in time. As shown in Figure 7, the frame 337 surrounding the right kidney 335 is shifted to the left. An arrow 342 appears on the screen prompting the operator to move the frame 337 surrounding the right kidney 335 so that it is depicted in the center of the B-mode image 310. The image quality score shown by the image quality gauge 380 is higher than that shown in Figure 3, etc. The arrow 342 is also depicted in the same color as the frame 337 surrounding the right kidney 335, making it easier for the operator to understand that it is an instruction for the right kidney 335. The arrow 342 is shorter than the arrow 340 in Figure 3. In this example, the ultrasound probe 202 was moved according to the guide information 340 and 344 in Figures 3-5, but the operation was incorrect and it was moved too far, so the arrow 342 in Figure 7 points in the opposite direction to the arrow 340 in Figure 3.

[0042] The image quality gauge can also be displayed in association with the examination step of the target organ. In the example in Figure 7, the image quality gauge 343 for the right kidney is displayed below the examination step 322 for the right kidney. Displaying the image quality gauge in association with the examination step of the target organ has the advantage of allowing confirmation of which organ is currently being depicted in the B-mode image 310. Also, as in the example in Figure 3, when the examination step 321 for the liver 331 has been completed and the examination step 322 for the right kidney 335 has started, it has the advantage of allowing confirmation of the image quality of the target organ for the previous examination step and the current examination step.

[0043] Figure 8 shows an image generated by the ultrasound image generation system in an embodiment of the present invention at a different point in time. As shown in Figure 8, a frame 337 surrounding the right kidney 335 is depicted in the center of the B-mode image 310. The image quality score shown by the image quality gauge 380 is higher than that shown in Figure 7. In a particular embodiment of the present invention, if the image quality score exceeds a predetermined value, an image saving event is automatically initiated. Also, when the AI ​​recognizes that the target organ is depicted in the maximum cross-section, an indicator 348 is displayed on the screen 300 to show the examiner that it is depicted in the maximum cross-section (maximum cross-section indicator 348). In other embodiments, cross-sectional area estimation by AI is not performed, and the maximum cross-section is identified by estimating the maximum value using the area change curve created by moving the ultrasound probe 202. In a particular embodiment of the present invention, in response to the completion of the image saving event, the current examination step is automatically terminated and the examination step for the next target organ is automatically initiated. In another particular embodiment of the present invention, in response to the completion of the image saving event, an indicator is output prompting the operator to terminate the current examination step and start the examination step for the next target organ. The operator can instruct the software to proceed to the next examination step accordingly. In Figure 8, the next examination step is the left kidney examination step 323. In this example, the left kidney is not depicted in the B-mode image 310. Therefore, no movement direction guide indicators such as arrows 340, zebra lines 344, or LEDs are output. Instead, the examiner can be shown, for example, the specific position of the patient to which the ultrasound probe 202 should be placed and what instructions to give the patient (e.g., moving the body from a position with the right side of the waist lifted to a position with the left side of the waist lifted) through a tutorial using voice and / or animation.

[0044] In certain embodiments, an additional examination step may be automatically added. For example, if a malignant tumor is detected in the right kidney 335, the region recognized as the malignant tumor becomes the next target organ, and that examination step is performed. This examination step involves navigation to obtain the maximum cross-sectional area of ​​the region recognized as the malignant tumor. The examiner is output a message indicating that a special examination step has been added.

[0045] The image saving process event can be executed in various ways. For example, multiple images with image quality scores exceeding a predetermined value can be saved as a background process unnoticed by the inspector. The inspector can select one or more images from the multiple images as the final image to be saved. In another embodiment, when the image quality score exceeds a predetermined value, the B-mode image 310 freezes and outputs a message prompting the inspector to save the frozen B-mode image 310. The inspector can save the B-mode image 310 accordingly. When the B-mode image 310 freezes, its updating is stopped, and the frozen state is maintained until the inspector saves and unfreezes it as needed, or until the power is turned off. In another embodiment, the B-mode image 310 remains frozen for a predetermined time. This predetermined time can be customized.

[0046] It should be noted that the invention is not limited to this embodiment, and various modifications are possible without departing from the spirit of the invention. [Explanation of symbols]

[0047] 100: System 101, 102, 103: Ultrasound examination equipment 104: Server 105: Network 107: Pre-trained model production terminal 200: Ultrasound examination equipment 201: Vibration element 202: Ultrasound probe 203: Transmitting beamformer 204: Transmitter 205: Receiver 206: Receiving beamformer 207: Processor 208: Display 209: Memory 210: User Interface 211: Speaker 300: Image 310: B-mode image 320: Inspection step display box 321: Name of the first inspection structure 322: Second inspection structure name 323~329: Other inspection structure names 331: First inspection structure 332: Figure corresponding to the first inspection structure 333, 337: Frame 335: Second inspection structure 336: Figure corresponding to the second inspection structure 340, 342: Arrows 341, 343: Image Quality Gauge 344: Zebra line 348: Display of maximum cut surface area 351: Current inspection step number 353: Current test step name 360: A collection of area indicators 361-365: Area Indicator 368: Horizontal axis 369: Vertical axis 380: Image Quality Gauge

Claims

1. An ultrasound image generation system that displays ultrasound images on a display device, It includes a processor and a non-temporary storage medium for storing programs, The aforementioned program, The steps include: analyzing a first ultrasound image at a first time point using an ultrasound probe to identify a predetermined structure within the imaging target; A step of identifying the first position of the predetermined structure in the first ultrasound image, A step of outputting a first area indicator having attributes corresponding to a first area in the first ultrasound image of the predetermined structure, The steps include outputting guide information instructing the operator of the ultrasound probe to move the ultrasound probe to a position where an ultrasound image can be obtained in which the predetermined structure is better displayed, based on the identified first position, The steps include outputting a second area indicator having attributes corresponding to a second area in a second ultrasound image of the predetermined structure at a second time point in time, in a manner that allows for recognition of changes over time relative to the first area indicator, An ultrasonic image generation system configured to cause the processor to perform the following operation.

2. The display device and the ultrasonic probe are included, The first area indicator and the second area indicator are displayed simultaneously on the display device. The ultrasonic image generation system according to claim 1, wherein the attribute includes any of the shape, coordinate position, brightness, and color of the first area indicator and / or the second area indicator.

3. The ultrasonic image generation system according to claim 1, wherein the program is configured to cause the processor to perform the step of outputting a corresponding movement direction guide display indicating the direction of the second ultrasonic probe position relative to the first ultrasonic probe position as at least part of the guide information to the display device.

4. The ultrasonic image generation system according to claim 1, wherein the program is configured to cause the processor to perform the step of outputting a corresponding movement direction guide sound indicating the direction of the second ultrasonic probe position relative to the first ultrasonic probe position as at least part of the guide information to the speaker of the ultrasonic image generation system.

5. The ultrasound image generation system according to claim 1, wherein the non-temporary storage medium stores a learned model for identifying the predetermined structure within the imaging target.

6. The ultrasound image generation system according to claim 1, wherein the object to be imaged is a human body, and the structure is an organ or a part of the human body.

7. The program is configured to cause the processor to perform a step that refers to a workflow that defines the order in which to image at least a first structure and a second structure within the object to be imaged, When the workflow is in the step of imaging the first structure, The step of analyzing the first ultrasound image to identify the predetermined structure within the imaging target is: The steps include analyzing the first ultrasound image to identify the first structure and the second structure within the imaging target, The steps include: ignoring the second structure and making the first structure the predetermined structure; The ultrasonic image generation system according to claim 1, including the following:

8. The ultrasound imaging system according to claim 7, wherein the workflow is customizable, the workflow includes changing the scan mode of the ultrasound imaging system, and the structure is an organ or part of the human body.

9. The program includes the step of confirming whether the second position and the second area of ​​the predetermined structure in the second ultrasound image meet predetermined criteria, If the second position and the second area satisfy the predetermined criteria, The step of automatically saving the second ultrasound image, Or, The steps of maintaining the display of the second ultrasound image on the display device for a predetermined time interval or longer, The ultrasonic image generation system according to claim 1, configured to cause the processor to execute the above.

10. The program includes the step of confirming whether the second image quality, second position, and second area of ​​the predetermined structure in the second ultrasound image meet predetermined criteria, If the second image quality, the second position, and the second area satisfy the predetermined criteria, The step of automatically saving the second ultrasound image, Or, The steps of maintaining the display of the second ultrasound image on the display device for a predetermined time interval or longer, The ultrasonic image generation system according to claim 1, configured to cause the processor to execute the above.

11. The ultrasonic image generation system according to claim 8, wherein the first area is the area of ​​the predetermined structure or the area of ​​a figure corresponding to the predetermined structure.

12. The ultrasonic image generation system according to claim 11, wherein the figure corresponding to the predetermined structure is a rectangle or a polygon obtained by segmenting the predetermined structure.

13. The aforementioned program, A step of calculating a second score based on the area, position, and image quality of the predetermined structure in the second ultrasound image, The steps include displaying a second score indicator corresponding to the second score on the display device, If the second score indicates a value greater than or equal to a predetermined value, the second score indicator is displayed in a first color and / or a first brightness; If the second score indicates a value less than a predetermined value, the second score indicator is displayed in a second color and / or a second brightness; The ultrasonic image generation system according to claim 7, configured to cause the processor to perform the following.

14. The ultrasonic image generation system according to claim 13, wherein the image quality includes noise and / or artifacts of the predetermined structure.

15. The ultrasound image generation system according to claim 13, wherein the program is configured to cause the processor to perform the step of automatically saving the second ultrasound image if the second score is greater than or equal to a predetermined value.

16. The aforementioned program, If the second score shows a value greater than or equal to a predetermined value, the second ultrasound image is continued to be displayed as a still image on the display device. or If the second score is above a predetermined value, the procedure generates an output prompting the operator to store the second ultrasound image as a still image in a storage device. The ultrasonic image generation system according to claim 13, configured to cause the processor to perform the following.

17. The ultrasound image generation system according to claim 16, wherein the imaging target is a patient, and the still image is stored in association with patient identification information associated with the patient and step identification information identifying a step in the workflow, and is referenced when an ultrasound examination is performed on the patient afterward.

18. The ultrasonic image generation system according to claim 1, wherein the first area indicator and the second area indicator are displayed along a time axis.

19. The ultrasonic image generation system according to claim 1, wherein the first area indicator and the second area indicator are generated and displayed at predetermined sampling intervals.

20. A program for displaying ultrasound images on the display device of an ultrasound image generation system, The steps include: analyzing a first ultrasound image at a first time point using an ultrasound probe to identify a predetermined structure within the imaging target; A step of identifying the first position of the predetermined structure in the first ultrasound image, A step of outputting a first area indicator having attributes corresponding to a first area in the first ultrasound image of the predetermined structure, The steps include outputting guide information instructing the operator of the ultrasound probe to move the ultrasound probe to a position where an ultrasound image can be obtained in which the predetermined structure is better displayed, based on the identified first position, The steps include outputting a second area indicator having attributes corresponding to a second area in a second ultrasound image of the predetermined structure at a second time point in time, in a manner that allows for recognition of changes over time relative to the first area indicator, A program configured to cause a processor to execute.

Images