Ultrasound probe operation guide program and ultrasound image generation system

Abstract

Guides the operation of the ultrasound probe. [Solution] A first ultrasound image of an ultrasound probe at a first time point is analyzed to identify a predetermined structure within an imaging target and determine its first location. A first area indicator corresponding to a first area of ​​the predetermined structure in the first ultrasound image is output. The system outputs guide information instructing an operator to move the ultrasound probe to a position where an ultrasound image can be obtained that better displays the predetermined structure. A second area indicator having attributes corresponding to a second area of ​​the predetermined structure in a second ultrasound image at a second time point is also output in a manner that allows changes in the first area indicator over time to be recognized.

Description

[Technical Field]

[0001] The present invention relates to a program for guiding the operation of an ultrasonic probe, and more particularly to a method for properly displaying a target structure in an ultrasonic image obtained using an ultrasonic probe. [Background technology]

[0002] When performing an ultrasound examination, an operator can freely place an ultrasound probe on a scanning object, orient it in any direction, and perform imaging, thereby obtaining non-destructive / non-invasive ultrasound images.

[0003] On the other hand, when ultrasound examination of the entire target area is required for purposes such as health checkups, the operator may be required to position the ultrasound probe in a predetermined position and direction. For example, the Japanese Society of Digestive Cancer Screening has established 25 recommended recording sections (ultrasound B-mode images), and for each section, the operator is required to position and direction the ultrasound probe on the subject. Images recorded according to this procedure serve as evidence of a complete examination and are used to report whether or not any abnormalities were detected. In this case, it is required that the organs included in each section be depicted in the center of the screen and at the maximum cut plane as much as possible, and the image be recorded.

[0004] However, there are cases where images are not recorded properly due to differences in the technical skills of the operators or fatigue caused by repeatedly examining a large number of people. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Publication No. 2019-48211 [Patent Document 2] Japanese Patent Application Publication No. 2017-153818 Summary of the Invention [Problem to be solved by the invention]

[0006] Therefore, there is a need for a system that provides easy-to-understand support for the operator of the ultrasound probe to perform appropriate operations in order to obtain desirable ultrasound images. [Means for solving the problem]

[0007] In a first aspect of the present disclosure, there is provided an ultrasound image generating system for displaying an ultrasound image on a display device. The ultrasound image generating system includes a processor and a non-transitory storage medium storing a program. The program is configured to cause the processor to execute the steps of: analyzing a first ultrasound image taken by an ultrasound probe at a first time point to identify a predetermined structure within an imaging target; identifying a first position of the predetermined structure in the first ultrasound image; outputting a first area indicator having an attribute corresponding to a first area of ​​the predetermined structure in the first ultrasound image; outputting guide information instructing an operator operating the ultrasound probe to move the ultrasound probe to a position where an ultrasound image that better displays the predetermined structure can be obtained based on the identified first position; and outputting a second area indicator having an attribute corresponding to a second area of ​​the predetermined structure in a second ultrasound image at a second time point in a manner that allows a change in the first area indicator over time to be recognized.

[0008] In a second aspect of the present disclosure, there is provided a program for displaying an ultrasound image on a display device of an ultrasound image generating system, the program being configured to cause a processor to: analyze a first ultrasound image of an ultrasound probe at a first time point to identify a predetermined structure within an imaging target; identify a first position of the predetermined structure in the first ultrasound image; output a first area indicator having an attribute corresponding to a first area of ​​the predetermined structure in the first ultrasound image; output guide information instructing an operator operating the ultrasound probe to move the ultrasound probe to a position where an ultrasound image that better displays the predetermined structure can be obtained based on the identified first position; and output a second area indicator having an attribute corresponding to a second area of ​​the predetermined structure in a second ultrasound image of the predetermined structure at a second time point in a manner that allows a change in the first area indicator over time to be recognized. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a block diagram showing an outline of the overall configuration of a system according to an embodiment of the present invention. [Figure 2] 2 is a block diagram showing the configuration of an ultrasonic inspection device that constitutes the system shown in FIG. 1. FIG. [Figure 3] 1A and 1B are diagrams illustrating images generated by an ultrasound image generation system according to an embodiment of the present invention. [Figure 4] 10A and 10B are diagrams illustrating images generated by an ultrasound imaging system according to another embodiment of the present invention. [Figure 5] 10A and 10B are diagrams illustrating images generated by an ultrasound imaging system according to another embodiment of the present invention. [Figure 6] FIG. 10 is a close-up view of a collection of area indicators. [Figure 7] 10A and 10B are diagrams showing images at different time points generated by an ultrasound imaging system according to an embodiment of the present invention. [Figure 8] 10A and 10B are diagrams showing images at different time points generated by an ultrasound imaging system according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments of the present invention will be described. However, the claimed invention is not limited to the embodiments described here. In particular, in this disclosure, a medical ultrasound diagnostic system will be described as an example, but the present invention can be applied to ultrasound inspection systems, ultrasound inspection devices, and ultrasound probes for non-destructive testing of buildings, structures, various mechanical devices, etc.

[0011] An embodiment of the present invention will be described below with reference to the drawings. A system 100 shown in FIG. 1 includes multiple ultrasound inspection devices 101, 102, and 103 and a server 104. The multiple ultrasound inspection devices 101, 102, and 103 may be communicatively connected to one another via a network 105. Each of the multiple ultrasound inspection devices 101, 102, and 103 may also be communicatively connected to the server 104 via the network 105. A trained model production terminal 107 that produces trained models may also be communicatively connected to the server 104 via the network 105. The trained model used in the present invention can be transmitted from the trained model production terminal 107 to the multiple ultrasound inspection devices 101, 102, and 103 via the server 104 or without the server 104. The trained model production terminal 107 can receive training data from the multiple ultrasound inspection devices 101, 102, and 103 via the server 104 or without the server 104. Additionally, other software related to the present invention can be downloaded from the server 104 to the multiple ultrasound examination devices 101 , 102 , 103 .

[0012] The configuration of each of the ultrasonic inspection devices 101, 102, and 103 is shown in FIG. 2 as an ultrasonic inspection device 200. The ultrasonic inspection device 200 includes a transmit beamformer 203 that drives a plurality of transducer elements 201 arranged in an ultrasonic probe 202 to generate pulsed ultrasonic signals, and a transmitter 204 that emits the generated pulsed ultrasonic signals to a subject (not shown). The pulsed ultrasonic signals generate echoes that are reflected within the subject and return to the transducer elements 201. The echoes are converted into electrical signals by the transducer elements 201, and the electrical signals are received by a receiver 205. The electrical signals representing the received echoes, i.e., echo signals, are amplified by a required gain in the receiver 205 and then input to a receive beamformer 206, where receive beamforming is performed. The receive beamformer 206 outputs ultrasound data after receive beamforming.

[0013] The receive beamformer 206 may be a hardware beamformer or a software beamformer. If the receive beamformer 206 is a software beamformer, the receive beamformer 206 may include one or more processors 207, including any one or more of a graphics processing unit (GPU), a microprocessor, a central processing unit (CPU), a digital signal processor (DSP), or other types of processors capable of performing logical operations. The processor(s) constituting the receive beamformer 206 may be a processor separate from the processor 207 described below, or may be comprised of the processor 207. Echo signals before receive beamforming and ultrasound data after receive beamforming are stored in a memory 209.

[0014] The processor 207 can process data in real time during a scanning session as the echo signals are received. For purposes of this disclosure, the term "real time" is defined to include procedures that occur without any intentional delay.

[0015] Additionally, data may be temporarily stored in a buffer (not shown) during an ultrasound scan and processed non-real-time for live or offline operation. In this disclosure, the term "data" may be used to refer to one or more data sets acquired using an ultrasound device.

[0016] The ultrasound data may be processed by the processor 207 in other or different mode-related modules (e.g., B-mode, color Doppler, M-mode, color M-mode, spectral Doppler, contrast-enhanced mode, elastography, TVI, strain, strain rate, etc.) to produce data for an ultrasound image. For example, one or more modules may generate an ultrasound image in B-mode, color Doppler, M-mode, color M-mode, spectral Doppler, contrast-enhanced mode, elastography, TVI, strain, strain rate, and combinations thereof.

[0017] A video processor module may be provided that reads image frames from the memory and displays the image frames in real time while a procedure is being performed on the subject. The video processor module may store the image frames in an image memory, and the ultrasound images may be read from the image memory and displayed on a display 208.

[0018] As used herein, the term "image" broadly refers to both a visible image and data representing a visible image. The term "data" may also include raw data, which is ultrasound data (echo signals or acoustic ray signals) before scan conversion, and image data, which is data after scan conversion. The processor 207 may further process information obtained by analyzing the ultrasound image and display the information on the display 208 together with the ultrasound image.

[0019] If processor 207 includes multiple processors, the processing tasks described above for processor 207 may be performed by the multiple processors. For example, a first processor may be used to demodulate and decimate RF signals, and a second processor may be used to further process the data before displaying an image. Also, for example, if receive beamformer 206 is a software beamformer, its processing functions may be performed by a single processor or multiple processors.

[0020] The display 208 may be a light emitting diode (LED) display, a liquid crystal display (LCD), a micro LED display, an organic electroluminescence (EL) display, or the like. The display 208 does not need to be a single display; multiple displays may be provided. When multiple displays are provided, all or many of the displays may be primary displays, or one may be primary and one or more of the other displays may be auxiliary displays. The auxiliary displays may be, for example, one or more LED elements disposed on a keyboard, the ultrasound probe 202, and / or other components of the user interface 210, which will be described later.

[0021] The memory 209 is any known data storage medium. In one example, the ultrasound inspection device 200 includes multiple memories 209, including non-transitory and transient storage media. The non-transitory storage medium is, for example, a non-volatile storage medium such as a hard disk drive (HDD) or a read-only memory (ROM). The non-transitory storage medium may also include a portable storage medium such as a compact disk (CD), a digital versatile disk (DVD), or a Blu-ray Disc (registered trademark). The non-transitory storage medium stores a program executed by the processor 207. The non-transitory storage medium also stores protocols, learning models, image data, and the like required to implement the present invention. The transient storage medium may be a volatile storage medium such as a random access memory (RAM). These may all be stored in the same memory 209, or at least one of them may be stored in a different memory 209. The memory 209 may also be multiple data storage media deployed on the cloud.

[0022] The user interface 210 can accept input from an operator. For example, the user interface 210 accepts input of instructions and information from the operator. The user interface 210 is configured to include a keyboard, hard keys, soft keys, etc. The user interface 210 may include various input devices such as pointing devices such as a mouse, touch panel, pen tablet, touchpad, trackball, and joystick, as well as eye tracking and voice input.

[0023] The speaker 211 outputs sound under the control of the processor 207. In one example, the speaker 211 outputs sound based on a signal input from the processor 207.

[0024] As shown in Figure 1, a communication interface (not shown) may be provided to enable the ultrasonic inspection devices 101 to 103 (each corresponding to the ultrasonic inspection device 200 in Figure 2) to communicate with the server 104, etc. However, in other embodiments, the ultrasonic inspection device 200 can operate in a standalone state to implement the present invention. In this case, the communication interface is not required.

[0025] FIG. 3 is a diagram showing an image 300 generated by an ultrasound imaging system according to an embodiment of the present invention. In this embodiment, a navigation function such as that described in Patent Document 2 is utilized. The navigation function displays organs to be measured in a pre-registered order to ensure that no images that must be acquired for diagnosis are omitted, and navigates (guides) the operator so that the examination is performed at the correct position. In this example, the imaging target is a human body, and the structure to be examined is an organ or part of the organ.

[0026] For example, when it is necessary to perform an ultrasound examination of the entire target area thoroughly for purposes such as a health check, the areas to be examined are registered in order so that the examination can proceed according to a predetermined procedure to prevent omissions. The operator can proceed by referring to the displayed comments and reference images. The operator follows these instructions to examine the necessary areas and leaves a record of the images. The recorded images serve as evidence that the examination was complete and are used to report whether or not any abnormalities were detected. At this time, it is required that the organs specified in each step are depicted in the center of the screen and in the largest cut plane possible, and the images are recorded.

[0027] An examination step display box 320 is displayed in the upper left of FIG. 3. The examination step display box 320 displays names 321 to 329 of examination structures to be examined at each step of the navigation. The examinations are scheduled to be performed in the order of the examination structure names 321 to 329 displayed in the examination step display box 320, and navigation is performed in this order. The example in FIG. 3 shows a state in which step 321 of the examination of the liver 331 has ended and step 322 of the examination of the right kidney 335 has started. In the description of the preferred embodiment of the invention, the right kidney 335 will be mainly described, but other target organs can be processed in a similar manner. Furthermore, as described above, when applied to an industrial ultrasound examination device, the target organ can be replaced with an internal structure such as a pipe or a valve.

[0028] Referring to FIG. 3 , the current examination step number 351 and the current examination step name 353 are displayed at the bottom of the screen 300 of the display 208. In this example, the current examination step number 351 is the 25th, and the right kidney, which is the structure (organ) to be examined, is registered as the current examination step name 353. The organs selected as examination items and the order of the examinations can be customized, and the way the examination step name 353 is set can also be customized. In certain embodiments of the present invention, in addition to customizing which organs are examined and in what order, it is also possible to execute routines to switch to which mode at which step (e.g., Doppler mode) and to prompt the user to enter comments at certain steps (e.g., whether or not there is fatty liver, cholecystectomy). At each examination step, comments and annotations, in addition to cross-sectional images of the target organ, can be automatically, semi-automatically, or manually recorded in the storage device (memory) 209. Comments and annotations can be changes in the lesion, the rate of change, the name of the suspected lesion, its classification, stage, congenital deformity, the width of the bile duct or aortic aneurysm, arrows placed at specific positions, circular enclosures, and other shapes. Comments and annotations are displayed in a manner that can be edited by the operator based on automatic recognition by AI, and after checking the content, the operator can save the comment or annotation with or without editing.

[0029] Ending the current step and switching to the next step, i.e., switching the target organ, can be performed in response to various events. In certain embodiments of the present invention, the software can switch to the next target organ in response to the occurrence of an image storage event for the current target organ. After switching to the next target organ, only the next target organ can be tracked. Furthermore, for example, the operator can select the spleen examination step 324 displayed in the examination step display box 320 and instruct the software to set it as the current target organ, thereby skipping the left kidney examination step 323 and performing the spleen examination step 324 and subsequent steps. In certain embodiments, when a specific organ is set as the target organ, even if other organs are recognized by the AI, other organs and structures that are not target organs are ignored, and only the target organ is tracked. In certain embodiments of the present invention, the names of all organs and structures detected in the B-mode image 310 and their detection accuracy are displayed.

[0030] In the example of FIG. 3 , an ultrasound probe 202 is placed on a subject, and the ultrasound probe 202 receives echo signals from a portion of the subject including a liver 331 and a right kidney 335, generating a B-mode image 310. At this point, the B-mode image 310 is analyzed. As a result of this analysis, the presence of the subject's liver 331 and right kidney 335 is identified within the B-mode image 310. In the example of FIG. 3 , a bounding box 333 surrounding the liver 331 is shown in, for example, dark gray, and a bounding box 337 surrounding the right kidney 335 is shown in, for example, light green, so that the operator can recognize that step 321 of the examination of the liver 331 has ended and step 322 of the examination of the right kidney 335 has begun. The bounding box 333 and the bounding box 337 indicate that these structures are recognized as specific structures. These bounding boxes do not have to be rectangular and can be circular, elliptical, polygonal, or the like. Furthermore, as shown in Fig. 4, the recognized structure can be segmented, traced, and superimposed on a B-mode image as semi-transparent and / or blinking figures 332 and 336, thereby showing the recognized structure to the operator. In Fig. 4, figures 332 and 336 are drawn with a shape and size that perfectly matches the cross-sections of liver 331 and right kidney 335, but these sizes may be made 1 to 10% larger or smaller. It is also possible to remove the central portion and create a ring shape that follows the contour.

[0031] In a specific embodiment, an AI learning model determines whether the organ set for each step of the navigation software is depicted on the screen. The neural network of the AI ​​learning model can be various types of neural networks, such as deep learning, DeepDream, RNN, CNN, diffusion, and GAN. The AI ​​learning model can detect which organ is included in the ultrasound image and obtain the probability (accuracy) that the detected organ is the organ. The detected organ and the accuracy can be displayed in association with the image of the organ.

[0032] In certain embodiments, AI detects the regions and shapes of one or more organs contained in the ultrasound image and confirms whether the target organ specified at each step is included. If the current target organ is included, an instruction to move the probe in a direction to center the region as close to the screen as possible is output. In the example of FIG. 3 , an arrow 340 is displayed on the screen to prompt the operator to center the B-mode image 310 on the frame 337 surrounding the right kidney 335. The arrow 340 is displayed in the same color as the frame 337 surrounding the right kidney 335, making it easier for the operator to understand that the instruction is directed to the right kidney 335. The length of the arrow 340 can be changed depending on the required movement distance. That is, if the required movement distance is long, the arrow 340 is displayed accordingly longer, and if the required movement distance is short, the arrow 340 is displayed accordingly shorter. The required movement distance can also be expressed not by the length of the arrow 340, but by its thickness, brightness, color, etc.

[0033] The position of a predetermined structure in an ultrasound image, exemplified by the right kidney 335, can be determined by various methods. For example, the intersection of the diagonal lines of the frame 337 shown in FIG. 3 can be determined as the center of the right kidney 335. Alternatively, the center of the right kidney 335 can be determined as the average of the pixel positions included in the traced figure 336 shown in FIG. 4. In the former case, the calculated position is not necessarily accurate, but the calculation amount is small and can be completed quickly. In the latter case, the calculation amount is large, but a more accurate position can be determined. Furthermore, in certain embodiments, if a lesion is detected in the right kidney 335, a point that takes the location of the lesion into consideration can be determined as the center of the right kidney 335. For example, the midpoint between the center of the right kidney 335 without considering the lesion and the center of the area occupied by the lesion can be determined as the center of the right kidney 335 with the lesion. In other embodiments, the center of the right kidney 335 is calculated without considering whether or not a lesion is present in the right kidney 335.

[0034] In FIGS. 3 and 4 , the guide information instructs the operator operating the ultrasound probe 202 to move the ultrasound probe to a position where an ultrasound image showing a predetermined structure can be obtained. However, the guide information can be embodied in various other ways. In the example of FIG. 5 , a zebra line 344 is displayed to prompt the operator to shift the B-mode image 310 itself to the right. The zebra line 344 may be fixed, or the zebra line 344 may be animated or flashing. The guide information may be text. For example, text information such as "Move (the ultrasound probe 202) 4 cm to the right" may be displayed on the screen 300. Furthermore, LEDs may be arranged on both sides of the keyboard of the ultrasound probe 202 and / or the user interface 210, and the direction of movement may be indicated by illuminating the LEDs. The movement direction guide display may be embodied in various ways, not limited to the arrow 340, zebra line 344, and LED described here. For example, the direction of movement can be guided by voice, and the part of the patient where the ultrasound probe 202 should come into contact (for example, the position between the seventh and eighth ribs on the right) can be shown to the examiner by voice or animation.

[0035] As described above, in certain embodiments, AI detects the regions and shapes of one or more organs contained in an ultrasound image and confirms whether the target organ specified at each step is included. The area of ​​a given structure in an ultrasound image, exemplified by the right kidney 335 in FIG. 3, can be calculated using various methods. For example, the area of ​​the frame 337 shown in FIG. 3 can be approximated as the drawing area of ​​the right kidney 335. Alternatively, the number of pixels contained in the traced figure 336 shown in FIG. 4 can be used as the area. In the former case, the calculated area is not necessarily accurate, but the calculation can be completed quickly with a small amount of calculation. In the latter case, the calculation is more complex, but a more accurate area can be determined.

[0036] The calculated area of ​​the right kidney 335 is displayed on the screen 300 as a set of area indicators 360. FIG. 6 is an enlarged view of the set of area indicators 360. In this example, the rightmost area indicator 361 is the most recent, and the leftmost area indicator 365 is the oldest. The area indicators 361-365 are generated at a predetermined sampling rate. The area indicators 361-365 can be set to different colors for each organ type, or the same color for all organs. The predetermined sampling rate is preferably equal to the frame rate of the image. When a new area indicator is generated, the existing area indicators 361-365 shift to the right, and the area indicator 365 disappears from the screen 300. The horizontal axis 368 represents time, and the vertical axis 369 represents the area of ​​the target organ. It is possible to transform the vertical axis 369 to represent time and the horizontal axis 368 to represent the area of ​​the target organ, or to reverse the time axis (i.e., change a left shift to a right shift). These changes can also be customized by the operator. By simultaneously displaying multiple area indicators on the display device, the operator can confirm how the area in which the target organ is depicted changes over time. That is, the set of area indicators 360 outputs the time-dependent changes in the area of ​​the target organ at multiple points in time in a manner that allows the operator to recognize them.

[0037] In the example of FIG. 6, the area indicator is embodied in the form of a bar graph, and the area of ​​the target organ is reflected in the attribute, which is the length of the bar graph. The area indicator can be implemented in various ways. For example, the bar graph in FIG. 6 can be a line graph or a simple plot. In this case, the area of ​​the target organ is reflected in the attribute, which is the height (coordinate position) of the line graph or plot. Alternatively, image items of the same shape can be arranged vertically or horizontally, and green indicates a larger area of ​​the target organ and red indicates a smaller area of ​​the target organ, or the indicator can be displayed as a high brightness and a low brightness, respectively. Furthermore, the area indicator can be animated. For example, as the area of ​​the target organ increases, a sign, symbol, or marker representing that organ can become more active or intensified. In such cases, the area of ​​the target organ is reflected in the attribute, which is the shape of the area indicator. The area of ​​the target organ can also be displayed simply as a numerical value. The examiner can refer to the temporal change in the area indicator and record the largest possible section. The examiner can refer to the change in the area indicator over time, move the probe (in a direction perpendicular to the cross section), and save an image where the cross-sectional area increases.

[0038] Returning to FIG. 3 , an image quality (IQ) gauge 380 is displayed on the right side of the examination step display box 320. In this example, the image quality gauge 380 is configured such that a higher image quality (IQ) score indicates more illuminated squares, whereas a lower image quality score indicates fewer illuminated squares. The image quality score is calculated based on the area of ​​the target organ, the position of the target organ, and the level of noise and / or artifacts contained in the target organ. As is well known to those skilled in the art, noise in ultrasound images can be caused by EMI, power sources, etc. Furthermore, as is well known to those skilled in the art, artifacts in ultrasound images can be caused by various factors, such as poor probe contact, the presence of calcified organs, metal or bone fragments present in the body, and appear in the image as acoustic shadows, reverberation, haze, etc.

[0039] Each score underlying the image quality score can be calculated using various functions. For example, the area score can be calculated based on the current area divided by the maximum area observed over a certain period of time. The location score can be obtained by substituting the distance from the center into a parabolic function with a y-intercept of 1 and an upward convex curve. The noise and / or artifact score can be calculated based on the total image area minus the area of ​​the noise or artifacts divided by the total image area.

[0040] The image quality score can be calculated, for example, by taking a weighted average of the area score, the location score, and the noise and / or artifact score. In certain embodiments, the noise and / or artifact score can be excluded from the image quality score calculation, and the image quality score can be calculated based only on the area score and the location score. Each score can be represented as a gauge indicating the level of that item. Alternatively, for example, a gauge can be displayed that combines the location score and the noise and / or artifact score and excludes the area score, or a gauge with other combinations. The image quality gauge 380 can also change color, such as red for a low score, blue for a good score, and yellow for a mid-range score, or can vary in brightness. The image quality gauge 380 can also be formed in a shape other than a line, such as a circle.

[0041] FIG. 7 is a diagram showing an image at another time point generated by an ultrasound image generation system according to an embodiment of the present invention. As shown in FIG. 7, the frame 337 surrounding the right kidney 335 has shifted to the left. An arrow 342 appears on the screen to prompt the operator to perform an operation so that the frame 337 surrounding the right kidney 335 is centered in the B-mode image 310. The image quality score indicated by the image quality gauge 380 is higher than that shown in FIG. 3 and other figures. The arrow 342 is also displayed in the same color as the frame 337 surrounding the right kidney 335, making it easier for the operator to understand that the instruction is directed to the right kidney 335. The arrow 342 is shorter than the arrow 340 in FIG. 3. In this example, the ultrasound probe 202 was moved according to the guide information 340 and 344 in FIGS. 3 to 5, but an error occurred and the probe was moved too far, so the arrow 342 in FIG. 7 points in the opposite direction to the arrow 340 in FIG. 3.

[0042] The image quality gauge can also be displayed in association with the step of the examination of the target organ. In the example of Fig. 7, the image quality gauge 343 for the right kidney is displayed below the examination step 322 for the right kidney. Displaying the image quality gauge in association with the step of the examination of the target organ has the advantage of making it possible to confirm which organ is currently depicted in the B-mode image 310. Another advantage is that, as in the example of Fig. 3, when the examination step 321 for the liver 331 has ended and the examination step 322 for the right kidney 335 has started, it is possible to check the image quality of the target organ in the previous examination step and the current examination step.

[0043] FIG. 8 is a diagram showing an image at another time point generated by an ultrasound imaging system according to an embodiment of the present invention. As shown in FIG. 8, a frame 337 surrounding the right kidney 335 is depicted in the center of the B-mode image 310. The image quality score indicated by the image quality gauge 380 is higher than that shown in FIG. 7. In a specific embodiment of the present invention, if the image quality score exceeds a predetermined value, an image storage event is automatically initiated. Furthermore, when the AI ​​recognizes that the target organ is depicted at its maximum section, a display 348 is displayed on the screen 300 to inform the examiner that the target organ is depicted at its maximum section (maximum section display 348). In another embodiment, the cross-sectional area is not estimated by the AI, and the maximum section is identified by estimating a local maximum using an area change curve generated by moving the ultrasound probe 202. In a specific embodiment of the present invention, in response to the completion of the image storage event, the current examination step is automatically terminated, and the examination step for the next target organ is automatically initiated. In another specific embodiment of the present invention, in response to the completion of the image storage event, a display is output prompting the operator to terminate the current examination step and begin the examination step for the next target organ. The operator can then instruct the software to proceed to the next examination step. In the case of Figure 8, the next examination step is the left kidney examination step 323. In this example, the left kidney is not depicted in the B-mode image 310. For this reason, no movement direction guide display indicating the movement direction, such as an arrow 340, a zebra line 344, or an LED, is output. Instead, the examiner can be shown, for example, by a tutorial using audio and / or animation, the specific position on the patient where the ultrasound probe 202 should be brought into contact and the instructions to give to the patient (e.g., moving the body from a position where the right side of the waist is raised to a position where the left side of the waist is raised).

[0044] In certain embodiments, an examination step may be automatically added. For example, if a malignant tumor is detected in the right kidney 335, the area identified as the malignant tumor becomes the next target organ, and the examination step is executed. In this examination step, navigation is performed to obtain the maximum section of the area identified as the malignant tumor. A message is output to the examiner indicating that a special examination step has been added.

[0045] The image saving process event can be executed in various ways. For example, multiple images whose image quality score exceeds a predetermined value can be saved as a background process that is not recognized by the examiner. The examiner can select one or more images from the multiple images as the images to be finally saved as needed. In another embodiment, when the image quality score exceeds a predetermined value, the B-mode image 310 is frozen, and a message is output prompting the examiner to save the frozen B-mode image 310. The examiner can save the B-mode image 310 in response. When the B-mode image 310 is frozen, updating of the B-mode image 310 is stopped, and the frozen state is maintained until the examiner saves and unfreezes it as needed, or until the power is turned off. In another example, the B-mode image 310 is maintained in a frozen state for a predetermined period of time. This predetermined period of time can be customized.

[0046] The invention is not limited to the present embodiment, and various modifications are possible within the scope of the invention. [Explanation of symbols]

[0047] 100: System 101, 102, 103: Ultrasonic inspection equipment 104: Server 105: Network 107: Trained model production terminal 200: Ultrasonic inspection equipment 201: Vibration element 202: Ultrasound probe 203: Transmit beamformer 204:Transmitter 205: Receiver 206: Receive beamformer 207: Processor 208: Display 209: Memory 210: User Interface 211: Speaker 300:Image 310: B-mode image 320: Inspection step display box 321: First test structure name 322: Second test structure name 323~329: Other test structure names 331: First inspection structure 332: A figure corresponding to the first test structure 333, 337: Frame 335: Second test structure 336: A figure corresponding to the second test structure 340, 342: Arrows 341, 343: Image quality gauge 344: Zebra Line 348: Display of maximum cut surface 351: Current inspection step number 353: Current inspection step name 360: Collection of area indicators 361~365: Area indicator 368: Horizontal axis 369: Vertical axis 380: Image quality gauge

Claims

1. An ultrasound image generating system for displaying an ultrasound image on a display device, comprising: a processor and a non-transitory storage medium for storing a program; The program analyzing a first ultrasound image of the ultrasound probe at a first time point to identify a predetermined structure (excluding the heart) within the imaging subject; Identifying a first location in the first ultrasound image of the predetermined structure; outputting a first area indicator having attributes corresponding to a first area in the first ultrasound image of the predetermined structure; a step of outputting guide information instructing an operator operating the ultrasonic probe to move the ultrasonic probe to a position where an ultrasonic image showing the predetermined structure more clearly can be obtained based on the identified first position; outputting a second area indicator having attributes corresponding to a second area of ​​the predetermined structure in a second ultrasound image at a second time point in a manner that allows a change over time relative to the first area indicator to be recognized; an ultrasound imaging system configured to cause the processor to execute the steps of:

2. the display device and the ultrasound probe, the first area indicator and the second area indicator are displayed simultaneously on the display device; The ultrasound imaging system of claim 1 , wherein the attributes include any one of a shape, a coordinate position, a brightness, and a color of the first area indicator and / or the second area indicator.

3. 2. The ultrasound image generation system of claim 1, wherein the program is configured to cause the processor to execute a step of outputting, to the display device as at least part of the guide information, a movement direction guide display corresponding to an indication of a direction of the second ultrasound probe position relative to the first ultrasound probe position.

4. 2. The ultrasound imaging system of claim 1, wherein the program is configured to cause the processor to execute a step of outputting, as at least part of the guide information, a movement direction guide voice corresponding to an indication of the direction of the second ultrasound probe position relative to the first ultrasound probe position to a speaker of the ultrasound imaging system.

5. The ultrasound imaging system of claim 1 , wherein the non-transitory storage medium stores a trained model for identifying the predetermined structure within the imaging subject.

6. 10. The method of claim 1, wherein the imaging target is a human body and the structure is an organ or part of an organ of the human body.

1. An ultrasound imaging system according to claim 1.

7. An ultrasound image generating system for displaying an ultrasound image on a display device, comprising: a processor and a non-transitory storage medium for storing a program; The program analyzing a first ultrasound image of the ultrasound probe at a first time point to identify a predetermined structure within the imaging subject; Identifying a first location in the first ultrasound image of the predetermined structure; outputting a first area indicator having attributes corresponding to a first area in the first ultrasound image of the predetermined structure; a step of outputting guide information instructing an operator operating the ultrasonic probe to move the ultrasonic probe to a position where an ultrasonic image showing the predetermined structure more clearly can be obtained based on the identified first position; outputting a second area indicator having attributes corresponding to a second area of ​​the predetermined structure in a second ultrasound image at a second time point in a manner that allows a change over time relative to the first area indicator to be recognized; configured to cause the processor to execute the program is configured to cause the processor to execute a step of referencing a workflow that defines an order in which at least a first structure and a second structure within the imaging target are to be imaged; When the workflow is in the step of imaging the first structure, The step of analyzing the first ultrasound image to identify the predetermined structure within the imaging subject includes: analyzing the first ultrasound image to identify the first structure and the second structure within the imaged subject; making the first structure the predetermined structure while ignoring the second structure; 1. An ultrasound imaging system comprising:

8. The ultrasound imaging system of claim 7 , wherein the workflow is customizable, the workflow includes changing a scan mode of the ultrasound imaging system, and the structure is an organ or portion of the body.

9. The program includes a step of confirming whether a second position and a second area of ​​the predetermined structure in the second ultrasound image satisfy predetermined criteria; If the second position and the second area satisfy the predetermined criteria, automatically saving the second ultrasound image; Or, maintaining the display of the second ultrasound image on the display device for a predetermined time interval or longer; The ultrasound imaging system of claim 1 , configured to cause the processor to execute:

10. The program includes a step of confirming whether a second image quality, a second position, and a second area of ​​the predetermined structure in the second ultrasound image satisfy predetermined criteria; automatically saving the second ultrasound image if the second image quality, the second location, and the second area meet the predetermined criteria; Or, maintaining the display of the second ultrasound image on the display device for a predetermined time interval or longer; The ultrasound imaging system of claim 1 , configured to cause the processor to execute:

11. The ultrasound imaging system of claim 8 , wherein the first area is an area of ​​the predetermined structure or an area of ​​a figure corresponding to the predetermined structure.

12. The ultrasound image generating system according to claim 11 , wherein the figure corresponding to the predetermined structure is a rectangle or a polygon obtained by segmenting the predetermined structure.

13. The program calculating a second score based on the area, position, and image quality of the predetermined structure in the second ultrasound image; displaying a second score indicator on the display device corresponding to the second score; If the second score indicates a value equal to or greater than a predetermined value, displaying the second score indicator in a first color and / or a first brightness; if the second score indicates a value less than a predetermined value, displaying the second score indicator in a second color and / or a second brightness; The ultrasound imaging system of claim 7 , configured to cause the processor to execute the following:

14. The ultrasound imaging system of claim 13 , wherein the image quality includes noise and / or artifacts of the predetermined structure.

15. The ultrasound image generation system of claim 13, wherein the program is configured to cause the processor to execute a step of automatically saving the second ultrasound image if the second score indicates a predetermined value or greater.

16. The program If the second score indicates a predetermined value or more, continuing to display the second ultrasound image on the display device as a still image; or a step of generating an output prompting an operator to perform an operation to store the second ultrasonic image as a still image in a storage device when the second score indicates a value equal to or greater than a predetermined value; The ultrasound imaging system of claim 13 , configured to cause the processor to execute:

17. 17. The ultrasound image generating system according to claim 16, wherein the imaging subject is a patient, the still image is stored in association with patient identification information associated with the patient and step identification information identifying a step in the workflow, and is referenced during a subsequent ultrasound examination of the patient.

18. The ultrasound imaging system of claim 1 , wherein the first area indicator and the second area indicator are displayed along a time axis.

19. The ultrasound imaging system of claim 1 , wherein the first area indicator and the second area indicator are generated and displayed at a predetermined sampling interval.

20. A program for displaying an ultrasound image on a display device of an ultrasound image generating system, analyzing a first ultrasound image of the ultrasound probe at a first time point to identify a predetermined structure (excluding the heart) within the imaging subject; Identifying a first location in the first ultrasound image of the predetermined structure; outputting a first area indicator having attributes corresponding to a first area in the first ultrasound image of the predetermined structure; a step of outputting guide information instructing an operator operating the ultrasonic probe to move the ultrasonic probe to a position where an ultrasonic image showing the predetermined structure more clearly can be obtained based on the identified first position; outputting a second area indicator having attributes corresponding to a second area of ​​the predetermined structure in a second ultrasound image at a second time point in a manner that allows a change over time relative to the first area indicator to be recognized; A program configured to cause a processor to execute the program.

Images