Ultrasound data processing device that generates an ultrasound image to be displayed in the display area of ​​a display device, and program that generates an ultrasound image to be displayed in the display area of ​​a display device.

Abstract

This invention provides an ultrasound data processing device that generates an ultrasound image to be displayed in the display area of ​​a display device, and a program that generates an ultrasound image to be displayed in the display area of ​​a display device. [Solution] In this disclosure, a rectangular bounding box 205 is generated for a structure 203 in the ultrasonic image of the first image mode, with sides parallel to each side of the rectangular display area 200. By applying a transformation function corresponding to the structure 203 to the information defining the bounding box 205, an ROI 210 corresponding to the structure 203 is defined. An ultrasonic image 211 of the second image mode is displayed within the ROI 210.

Description

Technical Field

[0001] The present disclosure relates to an ultrasonic data processing device that generates an ultrasonic image displayed in a display area of a display device and a program that generates an ultrasonic image displayed in a display area of a display device. More specifically, the present disclosure relates to an ultrasonic data processing device and a program that generate ultrasonic images in different image modes and display them in combination in a display area of a display device.

Background Art

[0002] Conventionally, an ROI (Region of Interest) is set on an ultrasonic image such as a B-mode image displayed on an ultrasonic diagnostic device, and an ultrasonic image in another image mode such as a flow image (e.g., Color Doppler or Power Doppler) is overlaid and displayed thereon. The procedure is somewhat different depending on the manufacturer and the model of the device, but it is troublesome and time-consuming for the operator. Spending a long time on an examination increases the burden on the examination subject and has a negative impact on the operation of the hospital.

[0003] For example, when overlaying and displaying a color Doppler image on an organ of interest displayed in a B-mode image, first, the probe is placed so that the target organ of the examination subject is within the field of view, and a B-mode image is acquired. Next, a key for entering the color Doppler mode is pressed to turn on the color Doppler function. The ROI displayed in response to the color Doppler function being turned on may not necessarily be at the position and size desired by the operator. In that case, the operator uses an input device such as a trackball or arrow keys to align the position of the ROI with the target site on the B-mode image. Then, the vertical and horizontal sizes of the ROI are also adjusted using the input device.

[0004] Such operations are troublesome and may take time, and may also require a high level of proficiency from the operator.

Prior Art Documents

Patent Documents

[0005] [Patent Document 1] Japanese Patent Publication No. 2022-159993 [Overview of the Initiative] [Problems that the invention aims to solve]

[0006] Therefore, an ultrasonic data processing device and a program for setting ROIs that are intuitive and easy for the operator to use are desired. Furthermore, reducing the computational resource usage required for setting ROIs is also desirable. [Means for solving the problem]

[0007] In a first aspect of this disclosure, an ultrasonic data processing device is provided that generates an ultrasonic image to be displayed in the display area of ​​a display device. The ultrasonic data processing device is A first image generation unit generates a first ultrasonic image from a first echo signal generated in response to irradiating an object containing multiple types of structures with a first ultrasonic signal, A structure detection unit that analyzes the first ultrasonic image to detect a structure, A bounding box generation unit that generates a rectangular bounding box corresponding to the aforementioned structure, having sides parallel to each side of the rectangular display area, An ROI defining unit defines an ROI corresponding to the structure by applying a first transformation function corresponding to the structure to the information defining the bounding box, A second image generation unit generates a second ultrasound image from a second echo signal generated in response to irradiating the ROI of the target with a second ultrasound signal, Includes. The second ultrasound image is displayed in the display area. The second ultrasonic signal is projected along multiple sound lines, The first transformation function transforms the information defining the bounding box such that the ROI is defined by a left boundary along one of the plurality of ray lines of the second ultrasonic signal and a right boundary along the other of the plurality of ray lines of the second ultrasonic signal.

[0008] In a second aspect of this disclosure, a program is provided which causes a processor of an ultrasonic data processing device to perform a plurality of steps. The execution of the program by the processor generates an ultrasonic image that is displayed in the display area of ​​the display device. The aforementioned steps are: The steps include: detecting structures by analyzing a first ultrasonic image generated from a first echo signal that occurs in response to irradiating an object containing multiple types of structures with a first ultrasonic signal; The steps include generating a rectangular bounding box corresponding to the aforementioned structure, having sides parallel to each side of the rectangular display area, The steps include defining an ROI corresponding to the structure by applying a first transformation function corresponding to the structure to the information defining the bounding box, The steps include generating a second ultrasound image from a second echo signal generated in response to irradiating the ROI of the target with a second ultrasound signal, Includes. The second ultrasound image is displayed in the display area. The second ultrasonic signal is projected along multiple sound lines, The first transformation function transforms the information defining the bounding box such that the ROI is defined by a left boundary along one of the plurality of ray lines of the second ultrasonic signal and a right boundary along the other of the plurality of ray lines of the second ultrasonic signal.

[0009] A third aspect of this disclosure provides a non-temporary computer-readable medium for storing the programs of the second aspect of this disclosure.

[0010] A fourth aspect of this disclosure provides an ultrasonic data processing device that generates an ultrasonic image to be displayed in the display area of ​​a display device. The ultrasonic data processing device is A first image generation unit generates ultrasound data for B-mode imaging from echo signals generated in response to irradiating an object containing multiple types of structures with an ultrasound signal for B-mode imaging, A structure detection unit that analyzes ultrasonic data for the B-mode image to detect structures, A bounding box generation unit that generates a rectangular bounding box corresponding to the aforementioned structure, having sides parallel to each side of the rectangular display area, An ROI defining unit defines an ROI corresponding to the structure by applying a first transformation function corresponding to the structure to the information defining the bounding box, A second image generation unit analyzes the texture of the ultrasonic data for the B-mode image of the structure located at the ROI of the target, and generates an ultrasonic image for texture analysis. Includes. The ultrasonic image for texture analysis is displayed in the display area as at least a portion of the B-mode image. The ultrasonic signal for the B-mode image is emitted along multiple sound lines. The first transformation function transforms the information defining the bounding box such that the ROI is defined by a left boundary along one of the plurality of ray lines of the second ultrasonic signal and a right boundary along the other of the plurality of ray lines of the second ultrasonic signal. [Brief explanation of the drawing]

[0011] [Figure 1] This is a block diagram showing the configuration of an ultrasound diagnostic device in an embodiment of the present invention. [Figure 2] This is a conceptual diagram showing a functional block in an embodiment of the present invention. [Figure 3A] This is a conceptual diagram showing the first ultrasound image, etc., displayed in the display area of ​​a display device. [Figure 3B] It is a conceptual diagram showing a first ultrasonic image or the like displayed in the display area of the display device. [Figure 3C] It is a conceptual diagram showing a first ultrasonic image or the like displayed in the display area of the display device. [Figure 4] In a specific embodiment of the present invention, it is a flowchart showing the procedure for the ROI defining unit to set the ROI in the target lock-on mode. [Figure 5] It is a conceptual diagram explaining that the ROI defining unit defines the position and size of the ROI based on the position information of the bounding box. [Figure 6] It is a conceptual diagram explaining a conversion function specifying table for specifying a conversion function corresponding to a structure. [Figure 7] Figures 7A to 7E are conceptual diagrams explaining how the conversion function converts the bounding box to form the ROI. [Figure 8] Figures 8A to 8E are conceptual diagrams explaining how the conversion function converts the bounding box to form the ROI. [Figure 9] In a specific embodiment of the present invention, it is a conceptual diagram explaining a selectively settable target lock-on mode and target unlock mode.

Mode for Carrying Out the Invention

[0012] Embodiments of the present invention will be described below with reference to the drawings. The ultrasound diagnostic apparatus 100 shown in Figure 1 includes a transmitting beamformer 103 that drives a plurality of vibrating elements 102a arranged in an ultrasound probe 102 to generate a pulsed ultrasound signal, and a transmitter 104 that radiates the generated pulsed ultrasound signal to a subject (not shown). The pulsed ultrasound signal generates echoes that are reflected within the subject and return to the vibrating elements 102a. In a particular embodiment, the subject is a human or a non-human mammal, and the structure being observed is an organ, a part of an organ, a blood vessel, or a lesion. The echoes are converted into electrical signals by the vibrating elements 102a, and the electrical signals are received by a receiver 105. The electrical signals representing the received echoes, i.e., the echo signals, are amplified by the required gain in the receiver 105 and then input to a receiving beamformer 106, where receiving beamforming is performed. The receiving beamformer 106 outputs ultrasound data after receiving beamforming.

[0013] The receiving beamformer 106 may be a hardware beamformer or a software beamformer. If the receiving beamformer 106 is a software beamformer, it may comprise one or more processors 107, including any one or more of the following: a graphics processing unit (GPU), a microprocessor, a central processing unit (CPU), a digital signal processor (DSP), or other types of processors capable of performing logical operations. The processors comprising the receiving beamformer 106 may consist of processors other than the processors 107 described below, or they may consist of processors 107. The echo signal before receiving beamforming and the ultrasonic data after receiving beamforming are stored in the memory 109. One or more of the multiple processors 107 may reside on the network to which the communication interface 336 (Figure 3) is connected.

[0014] When an echo signal is received, the processor 107 can process the data in real time during the scanning session. For the purposes of this disclosure, the term “real time” is defined to include procedures performed without any intentional delay.

[0015] Furthermore, the data can be temporarily stored in a buffer (not shown) during ultrasound scanning and processed in live or offline operation, but not in real time. In this disclosure, the term “data” may be used to refer to one or more datasets acquired using an ultrasound diagnostic device.

[0016] Ultrasound data can be processed by processor 107 using other or different mode-related modules (e.g., B-mode, color Doppler, M-mode, color M-mode, spectral Doppler, contrast-enhanced mode, elastography, TVI, strain, strain rate, etc.) to create ultrasound image data. For example, one or more modules can generate ultrasound images using B-mode, color Doppler, M-mode, color M-mode, spectral Doppler, contrast-enhanced mode, elastography, TVI, strain, strain rate, and combinations thereof.

[0017] A video processor module may be provided that reads image frames from memory while a procedure is being performed on the subject and displays those image frames in real time. The video processor module can store image frames in image memory, and the ultrasound image is read from the image memory and displayed on the display device 108.

[0018] In this specification, the term "image" broadly refers to both visible images and the data representing visible images. The term "data" may include raw data, which is the ultrasonic data (echo signals or sound ray signals) before the scan transformation operation, and image data, which is the data after the scan transformation operation.

[0019] If processor 107 includes multiple processors, the processing tasks described above that are handled by processor 107 may be handled by multiple processors. For example, the first processor can be used to demodulate and decimate the RF signal, and the second processor can be used to further process the data and then display the image. Also, if, for example, the receiving beamformer 106 is a software beamformer, its processing functions may be performed by a single processor or by multiple processors.

[0020] Display devices 108 include LED (Light Emitting Diode) displays, LCD (Liquid Crystal Display), microLED displays, and organic EL (Electro-Luminescence) displays.

[0021] Memory 109 is any known data storage medium. For example, the ultrasound image display device 1 includes multiple memory 109s, including non-transient and transient storage media. Non-transient storage media are non-volatile storage media such as HDDs (Hard Disk Drives) and ROMs (Read Only Memory). Non-transient storage media may also include portable storage media such as CDs (Compact Disks), DVDs (Digital Versatile Disks), and Blu-ray Discs (registered trademarks). Programs executed by the processor 107 are stored in the non-transient storage media. Protocols described later are also stored in the non-transient storage media. Transient storage media can be volatile storage media such as RAM (Random Access Memory). These may all be stored in the same memory 109, or at least one of them may be stored in a different memory 109. Memory 109 may also be multiple data storage media deployed on the cloud.

[0022] The user interface 110 can accept input from an operator. For example, the user interface 110 accepts instructions and information input from an operator. The user interface 110 is composed of a keyboard, hard keys, and soft keys, etc. The user interface 110 may also include various input devices such as a mouse, touch panel, pen tablet, touchpad, trackball, joystick, eye tracking, and voice input.

[0023] The speaker 111 outputs sound under the control of the processor 107. Specifically, the speaker 111 outputs sound based on a signal input from the processor 107. The ultrasound diagnostic apparatus 100 shown in Figure 1 can function as an ultrasound data processing device that generates a first ultrasound image 201 and a second ultrasound image 211 (Figure 3C) displayed in the display area 200 (Figure 3A) of the display device 108.

[0024] Figure 2 shows a functional block 300 that is realized when the program 112 stored in memory 109 is executed by the processor 107. Figures 3A-3C are conceptual diagrams showing a first ultrasound image 201 and a second ultrasound image 211, etc., displayed in the display area 200 of the display device 108. In the example of Figure 3C, a B-mode image 201 is displayed as one example of the first ultrasound image, and a color-mode image 211 is shown as one example of the second ultrasound image, displayed in a portion of the display area of ​​the B-mode image 201 (first ultrasound image). In the following description of the embodiment, in order to facilitate the reader's understanding, the explanation will focus on the case where the first ultrasound image is a B-mode image 201 and the second ultrasound image is a color-mode image 211.

[0025] In a preferred embodiment of the present invention, the functional block 300 includes a B-mode image processing unit 302, an ultrasonic signal control unit for B-mode images 304, a color Doppler image processing unit 306, an ultrasonic signal control unit for color Doppler images 308, an image generation unit / superimposed image generation unit 310, a structure detection unit 312, a bounding box generation unit 314, a trained deep learning model 316, an ROI definition unit 318, an event detection unit 320, an input interface 332, an output interface 334, and a communication interface 336.

[0026] The B-mode image processing unit 302 processes the echo signal received from the receiving beamformer 106 to generate a B-mode image 201. As shown in Figure 3A, the generated B-mode image 201 is displayed in the display area 200 by the image generation unit 310. When the B-mode image processing unit 302 receives the echo signal from the receiving beamformer 106, the operator positions the probe so that the target structure 203 to be examined is within the field of view and irradiates it with an ultrasound signal for the B-mode image. In this embodiment, the subject of examination is a human or a non-human mammal, and the structure 203 is an organ, part of an organ, blood vessel, or lesion. The structure 203 may be a single type of organ, part of an organ, blood vessel, or lesion, or it may be a combination of multiple types of organs, parts of organs, blood vessels, or lesions, such as the spleen and the inferior vena cava (IVC). The ultrasound signal for the B-mode image is controlled by the B-mode image ultrasound signal control unit 304 to be optimal for the examination. In the embodiment shown in Figure 3A, a convex-shaped ultrasonic probe is used as the ultrasonic probe 102, and the B-mode image 201 is fan-shaped.

[0027] Specifically, the ultrasonic signal control unit 304 for B-mode imaging can adjust the frequency, number and position of focal points, output power, pulse width, etc. of the ultrasonic signal for B-mode imaging output by the transmitter 104 (Figure 1) via the transmitting beamformer 103 (Figure 1).

[0028] The color Doppler image processing unit 306 processes the echo signal for the color Doppler image received from the receiving beamformer 106 to generate a color Doppler image 211. As shown in Figure 3C, the generated color Doppler image 211 is displayed on top of the B-mode image 201 in the display area 200 by the superimposed image generation unit 310.

[0029] The ultrasonic signal control unit 308 for color Doppler imaging can adjust the frequency, number and position of focal points, output power, pulse width, etc. of the ultrasonic signal for color Doppler imaging output by the transmitter 104 (Figure 1) via the transmitting beamformer 103 (Figure 1).

[0030] The image generation unit / superimposed image generation unit 310 works in conjunction with the B-mode image processing unit 302 and the color Doppler image processing unit 306 to display an image in the display area 200 in which the color Doppler image 211 is superimposed on the B-mode image 201.

[0031] The structure detection unit 312 identifies the type of structure 203 and its location information contained in the B-mode image 201. In a particular embodiment, the structure detection unit 312 inputs the B-mode image 201 to a trained deep learning model 316 and obtains the type of structure and its location information contained in the B-mode image 201 as the output of the trained deep learning model 316. This trained deep learning model 316 is a deep neural network trained using training data in which each of a large number of B-mode images is annotated with the type of structure and the coordinate information of the bounding box corresponding to that structure. In another embodiment, the structure detection unit 312 identifies the type of structure and its location information contained in the B-mode image 201 using known segmentation or pattern recognition techniques, without using the trained deep learning model 316. Identification of the type of structure and its location information contained in the B-mode image 201 can be performed at predetermined intervals (e.g., every 10 frames, every second, etc.).

[0032] As shown in Figure 3B, when the structure detection unit 312 identifies the type of structure 203 and its position information included in the B-mode image 201, the bounding box generation unit 314 draws a bounding box 205 to be displayed in the display area 200 based on this information. In a particular embodiment, the bounding box generation unit 314 generates a rectangular bounding box 205 with a color corresponding to the type of structure 203. Each side of the rectangular bounding box 205 is parallel to each side of the rectangular display area 200. The display device 108 includes a display panel having pixels arranged two-dimensionally in horizontal rows and vertical columns, and each side of the rectangular bounding box 205 is aligned with the arrangement of the display panel. The display area 200 may be the display panel of the display device 108 itself. In other embodiments, the display area 200 may be one of several windows displayed on the display panel of the display device 108. The method of using bounding boxes for object detection is highly versatile and highly compatible with deep learning models. When training a deep learning model, as described above, training data can be used in which each of a large number of B-mode images is annotated with the type of structure and the coordinate information of the bounding box corresponding to that structure. This enables high-speed object detection, allows execution even on small systems, and reduces costs. In another embodiment, information that identifies the bounding box 205 for a structure 203 (e.g., the coordinates of the top-left and bottom-right) is calculated and stored, but the bounding box 205 is not displayed.

[0033] The bounding box generation by the bounding box generation unit 314 can be performed at the same time as the identification of the type of structure and its position information included in the B-mode image 201. This timing can be at predetermined intervals (for example, every 10 frames, every second, etc.). In another embodiment, the bounding box generation by the bounding box generation unit 314 is performed when there is a predetermined change in either the type of structure to be identified or its position information.

[0034] As shown in Figure 3C, the ROI definition unit 318 sets the ROI 210 based on the type and location information of the structures 203 included in the B-mode image 201, which are identified by the structure detection unit 312. A color Doppler image 211 is displayed inside the ROI 210.

[0035] As described above, the second ultrasound image displayed within ROI210 shown in Figure 3C is explained using the color Doppler image 211 as an example. However, the second ultrasound image displayed within ROI210 by displaying (setting) ROI210 on the B-mode image 201 (first ultrasound image) may be a different type of ultrasound image than the color Doppler image. For example, it may be another blood flow display image (pulse wave (PW) Doppler, continuous wave (CW) Doppler, power Doppler, etc.), elasticity images (shear wave elastography, strain elastography, etc.), attenuation images, or texture analysis images.

[0036] Shear wave elastography (SWE) involves generating "shear waves" using ultrasound and measuring their propagation speed. It utilizes the principle that shear waves travel faster through rigid tissue and slower through softer tissue. It can be used for observing liver fibrosis (assessing liver stiffness), breast tissue, thyroid gland, and other organs.

[0037] Strain elastography involves applying "deformation (strain)" to tissue by gently pressing with a probe or utilizing the body's natural movements. It takes advantage of the fact that hard tissues are less deformable, while softer tissues are more easily deformable. It can be used to observe conditions such as breast cancer and thyroid nodules.

[0038] Attenuation images are created by utilizing the phenomenon where an ultrasound beam is weakened (i.e., attenuated) as its energy is absorbed and scattered by tissues as it travels through the body. Generally, tissues with strong attenuation (high reduction in ultrasound energy) appear dark, while tissues with weak attenuation appear bright.

[0039] Texture analysis images are images obtained by mathematically analyzing the patterns (roughness, smoothness, arrangement of fine dots and lines, etc.) of conventional ultrasound images such as B-mode images, extracting information and quantifying and visualizing the distribution of light and dark, roughness, regularity, and irregularity.

[0040] The event detection unit 320 detects specific suggestive inputs via the input interface 332 that indicate a desire to display the ROI 210 on the B-mode image 201 and to start a function that displays images of other modes within the ROI 210. The output interface 334, in cooperation with the display device 108 and / or speaker 111, executes instructions that prompt the operator to give specific inputs.

[0041] The communication interface 336 makes it possible to use other processors 107 and storage devices present on the network.

[0042] Figure 4 is a flowchart showing the procedure by which the ROI definition unit 318 sets the ROI 210 in target lock-on mode. In certain embodiments, the operator can pre-select between target lock-on mode and target unlock mode. In other embodiments, target lock-on mode is set by default, and the system transitions to target unlock mode when target lock-on mode is released at the operator's instruction.

[0043] As shown in Figure 4, first, the structure 203 is detected (step 501). Next, the event detection unit 320 determines whether there is a specific suggestive input indicating a desire to start the function of displaying images in other modes within the ROI 210 (step 502). If it is determined that there is no suggestive input (No in step 502), the processes of steps 501 and 502 are performed. In step 502, the processes of steps 501 and 502 are repeated at predetermined intervals until a suggestive input is detected (Yes in step S502). In step 502, when a suggestive input is detected, the target lock-on mode is started (step 502). In a particular embodiment, each time the structure 203 is detected repeatedly, the type of structure 203 and its location information are repeatedly identified in step 501. Also, as will be described later, a bounding box 205 corresponding to the structure 203 is generated. The bounding box 205 may or may not be displayed in the display area 200.

[0044] This particular suggestive input is, 1) An input instructing the system to enter color Doppler mode when a specific structure 203 is recognized. 2) There is an input to instruct a freeze, followed by an input to instruct an unfreeze, 3) When a specific structure 203 is recognized, input of a specific trigger key, 4) When a specific structure 203 is recognized, input to select the flow mode, This includes any of the above. When the present invention is applied to imaging modes other than color Doppler mode, the "input instructing to enter color Doppler mode" can be the "input instructing to enter another imaging mode." Other imaging modes other than color Doppler mode include at least other blood flow visualization imaging modes (pulse wave (PW) Doppler, continuous wave (CW) Doppler, power Doppler, etc.), elastic imaging modes (shear wave elastography, strain elastography, etc.), and attenuation imaging modes.

[0045] "1) An input instructing the system to enter color Doppler mode when a specific structure 203 is recognized" includes, for example, pressing a button labeled "Color," "Color Doppler," "CFM," or "CD" on the keyboard (or touch panel, etc.) 110 of the ultrasound diagnostic device 100 while a B-mode image 201 is being displayed. The color Doppler function can also be turned on by voice command or the like. The event detection unit 320 detects a command instructing the system to start the color Doppler mode via the input interface 332.

[0046] "2) When a specific structure 203 is recognized, there is an input to instruct a freeze, followed by an input to instruct an unfreeze" includes, for example, performing a freeze by pressing a button labeled "Freeze" or "FRZ" on the keyboard (or touch panel, etc.) 110 of the ultrasound diagnostic device 100 while a B-mode image 201 is being displayed, and then releasing the freeze (unfreezing) by pressing the same button again. The structure 203 on which the ROI 210 is set is acquired by the structure detection unit 312 recognizing it by inputting at least a portion of the B-mode image 201 into the trained deep learning model 316 immediately after the freeze is released. "Immediately after" here means within 1 second, preferably within 0.5 seconds.

[0047] "3) Input of a specific trigger key when a specific structure 203 is recognized" refers, for example, to a key input for saving an image. Often, an operator saves the image in which a specific structure (organ, etc.) is best depicted before finishing observation of a specific structure 203 and starting observation of the next structure 203. After detecting the key input for saving an image, the system unlocks the currently targeted structure 203 that was the target of the image saving and locks onto the next structure 203.

[0048] "4) Input for selecting a flow mode when a specific structure 203 is recognized" includes, for example, pressing a button on the keyboard (or touch panel, etc.) 110 of the ultrasound diagnostic device 100 that displays a flow mode while a B-mode image 201 is being displayed.

[0049] In certain embodiments, the operator can select from various flow modes. "(1) Color (Color Doppler) flow mode" displays the direction and relative velocity of blood flow using color (red when moving towards the probe, blue when moving away from the probe). It is mainly used to assess the heart and blood vessels when confirming the presence and direction of blood flow. The Color Doppler flow mode is entered when no other flow mode is selected after entering Color Doppler mode.

[0050] The "(2) Pulsed Wave Doppler (PW) mode" displays the blood flow velocity at a specific location (sample volume) as a waveform. It enables quantitative evaluation of local blood flow velocity within blood vessels and allows for measurement of the degree of arterial stenosis, etc. It can be selected by pressing the button labeled "PW" located on the keyboard (or touch panel, etc.) 110 of the ultrasound diagnostic device 100.

[0051] The "(3) Continuous Wave Doppler (CW) mode" continuously transmits and receives ultrasound waves, enabling the measurement of high-velocity blood flow. This allows for the evaluation of high-velocity blood flow (e.g., blood flow in heart valve disease). This mode can be selected by pressing the button labeled "CW" located on the keyboard (or touch panel, etc.) 110 of the ultrasound diagnostic device 100.

[0052] "(4) Power Doppler mode" focuses on the presence or absence of blood flow (strength of the Doppler signal) and displays it in color regardless of direction. It is effective for detecting minute blood flows or low flow velocities (e.g., intratumor blood flow, renal blood flow). It can be selected by pressing the button labeled "PD" located on the keyboard (or touch panel, etc.) 110 of the ultrasound diagnostic device 100.

[0053] "(5) Tissue Doppler Imaging (TDI)" allows for the measurement of tissue movement, such as myocardium and blood vessel walls, using Doppler. It is used for evaluating cardiac function, particularly left ventricular diastolic function. It can be selected by pressing the button labeled "TDI" on the keyboard (or touch panel, etc.) 110 of the ultrasound diagnostic device 100.

[0054] In a particular embodiment, after an indicative input is detected in step 502, the system can determine which structure 203 is being tracked by storing the type of structure 203 in step 504.

[0055] Some ultrasound diagnostic devices offered by GE Healthcare (trademark) are equipped with a Scan Assistant function. This function is an automated support tool designed to improve the efficiency and consistency of examinations. Based on a pre-configured examination flow, it sequentially sequences the organs to be imaged and automatically guides the user through image acquisition, labeling, and measurement, standardizing the operating procedure. It automatically labels images appropriately during acquisition and saves them in a specified format. This reduces working time and minimizes errors.

[0056] In certain embodiments, the target lock-on mode function is combined with the scan assistant function. Imaging in other modes, such as color Doppler mode, is set for some organs included in the examination flow. For example, the stomach and intestines can be preset for observation in B-mode only, while organs such as the kidneys can be preset for imaging in color Doppler mode. When the examination flow reaches the stage for an organ such as the kidneys, the target lock-on mode is activated, an ROI 210 is set for that organ, and the system tracks that organ until the examination flow reaches the next organ, maintaining the ROI 210 for that organ and continuously displaying images of other modes within the ROI 210. Tracking ends when the operator instructs the system to proceed to the next stage of the examination flow.

[0057] When the target lock-on mode function is combined with the scan assistant function, after a structure 203 is released from being locked onto as a target, the next target structure 203 will, in principle, be the next structure 203 in the inspection flow. The system can provide voice and image guidance as needed to help the operator properly position the ultrasonic probe 102 to the next structure 203. The operator can make any desired structure 203 the next target to be locked onto by disabling the scan assistant function or by instructing the scan assistant to change the inspection flow. Since information about the structure 203 in the current inspection flow can be obtained from the scan assistant function, step 504, which stores the type of structure 203, can be omitted.

[0058] If the target lock-on mode function is not combined with the scan assistant function, after a structure 203 is released from being locked on, a structure 203 can be set as the next target to be locked on under various conditions. If, after the lock-on target is released, structure 203 continues to be detected with the highest likelihood for a predetermined period of time or longer, the system recognizes that structure 203 as a target to be locked on. This predetermined period can be 0.2 seconds to 30 seconds, preferably 1 second to 10 seconds, and more preferably 2 seconds to 5 seconds.

[0059] Next, in step 510, the bounding box generation unit 314 generates (or updates) the bounding box 205. The bounding box 205 may not be displayed in the display area 200. In a particular embodiment, the position information of the bounding box 205 is temporarily stored as the current bounding box position for comparison.

[0060] The process proceeds to step 512, where it is determined whether this is the first time the process is running. If it is the first time, step 514 is skipped, and the process proceeds to step 516. If it is not the first time, in step 514, the current position information of the bounding box 205 is compared with the position information of the bounding box 205 temporarily stored in step 518 (described later), and it is determined whether the overlapping area of ​​the two has changed by a predetermined percentage or more. The predetermined percentage may be a value in the range of 1 to 30%. More preferably, it may be 2 to 10%, and even more preferably, 5%. If the change in the bounding box 205 is small (including cases where there is no change), steps 516 to 522 (described later) are skipped, and the process proceeds to step 524, thereby improving the processing speed and enabling the maintenance of a high frame rate. In other words, it is possible to avoid frequent adjustments of the ROI 210 in response to slight changes in the bounding box 205 due to breathing, etc., and to prevent frequent stops in the scan (time is required to change the scanning parameters as the ROI 210 is updated).

[0061] If the overlap area between the location information of the new bounding box 205 and the temporarily stored location information of the bounding box 205 changes by a predetermined percentage or more, or if it is determined in step 512 that this is the first time, the ROI 210 is created or updated in step 516.

[0062] When ROI210 is created or updated, steps 516-524 are performed, and an ROI210 corresponding to the type of structure 203 is set for the structure 203, as shown in Figure 3C, and an image in a different mode than the B-mode image 201 is displayed within the ROI210. The bounding box 205 shown in Figure 3B may continue to be displayed or may be hidden as shown in Figure 3C. In certain embodiments, no operator input is required from the time a predetermined event is detected by the event detection unit 320 until an image in a different mode than the B-mode image 201 (a color Doppler mode image in the example of Figure 4) is displayed within the ROI210. The event detection unit 320 activates the ROI definition unit 318, the ultrasonic signal control unit 308 for color Doppler images, the color Doppler image processing unit 306, and the superimposed image generation unit 310. Without operator intervention, an ROI 210 corresponding to the type of structure 203 is set, an ultrasonic signal for color Doppler images is transmitted with parameters corresponding to the type of structure 203 and ROI 210, and a color Doppler image is generated. When in target lock-on mode, ROI 210 adjustment is performed exclusively on recognized structures 203 until a new user input is received to release it.

[0063] Figure 5 is a conceptual diagram illustrating how the ROI definition unit 318 defines the position and size of the ROI 210 based on the position information of the bounding box 205. Similar to the embodiments in Figures 3A-3C, the embodiment in Figure 5 uses a convex ultrasonic probe 102, and the sound rays are radial. In this example, all sound rays can be represented as half-lines passing through point 260. Both the ultrasonic signal for B-mode imaging and the ultrasonic signal for color Doppler imaging are emitted along these multiple sound rays. Furthermore, each sound ray can be defined by an angle 266 with respect to the central sound ray 265. Sound rays located to the right of the central sound ray 265 are defined by a positive angle, and sound rays located to the left of the central sound ray 265 may be defined by a negative angle.

[0064] The bounding box 205 can be defined using two coordinates: the coordinates of the top-left vertex 251 (d1, a1) and the coordinates of the bottom-right vertex 254 (d2, a2). In another embodiment, the bounding box 205 can be defined using four coordinates, as shown in Figure 5: the coordinates of the top-left vertex 251 (d1, a1), the coordinates of the top-right vertex 252 (d2, a2), the coordinates of the bottom-left vertex 253 (d3, a3), and the coordinates of the bottom-right vertex 254 (d4, a4). Alternatively, the bounding box 205 can be defined by the coordinates of its center, height, and width.

[0065] The ROI 210 can be defined by the coordinate information of the bounding box 205. In Figure 5, the ROI 210 is described as having an isosceles trapezoidal shape, but the ROI 210 can be a part of a ring centered on point 260 (which can also be called a sector-shaped annular region, a sector-shaped band, or an annular sector). If the ROI 210 is a part of a ring centered on point 260, the depth range can be kept constant for each sound ray, thereby improving the calculation speed. In a particular embodiment, the ROI definition unit 318 can define the ROI 210 by applying a transformation function to the coordinate information that defines the bounding box 205. In a particular embodiment, the transformation function transforms the information that defines the bounding box such that the ROI 210 is defined by a left boundary along one of the multiple sound rays of the ultrasonic signal for the color Doppler image and a right boundary along one of the other multiple sound rays of the ultrasonic signal for the color Doppler image.

[0066] The ROI 210 can be defined by applying various transformation functions to the coordinate information of the bounding box 205. For example, a tone line passing through the midpoint between the top-left vertex 251 and the bottom-left vertex 253 of the bounding box can be selected as tone line 271 corresponding to the left side of the ROI 210, and a tone line passing through the midpoint between the top-right vertex 252 and the bottom-right vertex 254 of the bounding box can be selected as tone line 273 corresponding to the right side of the ROI 210.

[0067] The top-left vertex 281 of the ROI can be a point on the tone line 271 corresponding to the left side of ROI 210, where the distance from point 260 is equal to the average of the distance between point 260 and the top-left vertex 251 of the bounding box and the distance between point 260 and the top-right vertex 252 of the bounding box.

[0068] The upper right vertex 282 of the ROI can be a point on the tone line 273 corresponding to the right-hand side of ROI 210, where the distance from point 260 is equal to the average of the distance between point 260 and the upper left vertex 251 of the bounding box and the distance between point 260 and the upper right vertex 252 of the bounding box.

[0069] The bottom-left vertex 283 of the ROI can be a point on the tone line 271 corresponding to the left side of ROI 210, where the distance from point 260 is equal to the average of the distance between point 260 and the bottom-left vertex 253 of the bounding box and the distance between point 260 and the bottom-right vertex 254 of the bounding box.

[0070] The bottom-right vertex 284 of the ROI can be a point on the tone line 273 corresponding to the right-hand side of ROI 210, where the distance from point 260 is equal to the average of the distance between point 260 and the bottom-left vertex 253 of the bounding box and the distance between point 260 and the bottom-right vertex 254 of the bounding box.

[0071] In another embodiment, the tone line 271 corresponding to the left side of ROI 210 can be the tone line 271 that bisects the angle between the tone line 261 corresponding to the upper left vertex 251 of the bounding box 205 and the tone line 263 corresponding to the lower left vertex 253 of the bounding box 205. Similarly, the tone line 273 corresponding to the right side of ROI 210 can be the tone line 273 that bisects the angle between the tone line 262 corresponding to the upper right vertex 252 of the bounding box 205 and the tone line 264 corresponding to the lower right vertex 254 of the bounding box 205.

[0072] If either or both of the absolute values ​​of the angle between the central tone line 265 and the tone line 271 corresponding to the left side of ROI 210, and the absolute values ​​of the angle between the central tone line 265 and the tone line 273 corresponding to the right side of ROI 210 are greater than or equal to a predetermined value, the angles can be adjusted so that the absolute values ​​become larger. The method of defining ROI 210 along the tone lines using the bounding box 205 requires less computation, reducing the need for expensive computing resources and enabling faster computation.

[0073] Figure 6 is a conceptual diagram illustrating the transformation function identification table 350 for identifying transformation functions corresponding to structures. Figure 7 is a conceptual diagram illustrating how the transformation function transforms the bounding box 205 to form an ROI.

[0074] As shown in Figure 6, a corresponding transformation function is assigned depending on the type of structure 203. The first transformation coefficient applied to the first structure shown in Figure 7A is a transformation coefficient set so that the width and height are 90% of the transformation coefficient explained in Figure 5. The area of ​​ROI 210 in Figure 7A may be smaller than the area of ​​the bounding box 205. The second transformation coefficient applied to the second structure shown in Figure 7B is a transformation coefficient set so that the width and height are 100% of the transformation coefficient explained in Figure 5. The third transformation coefficient applied to the third structure shown in Figure 7C is a transformation coefficient set so that the width and height are 110% of the transformation coefficient explained in Figure 5. For example, in the case of the gallbladder, the structure 203 itself is not large, and the surrounding blood flow is often observed, so a large ROI 210 setting may be required for the structure 203. The area of ​​ROI 210 in Figure 7C may be larger than the area of ​​the bounding box 205. The fourth conversion factor applied to the fourth structure shown in Figure 7D is a conversion factor in which the width of the bounding box 205 is ignored, a fixed width of ROI 210 is applied, and the height of ROI 210 is set to 120% of the standard. For example, when observing a long, horizontally running blood vessel, it is usually not necessary to observe both ends simultaneously, but rather to observe only the central part. In such cases, the optimal ROI 210 can be set by ignoring the width of the bounding box 205.

[0075] The conversion function can be a function that makes the ROI210 have a predetermined height or width relative to at least one of the height and width of the bounding box 205. However, if the area of ​​ROI210 is too large, it may be difficult to maintain a high frame rate, and if the area of ​​ROI210 is too small, it may not be suitable for observation. For this reason, the conversion function defines ROI210 such that the height of ROI210 does not exceed a predetermined maximum height threshold, the height of ROI210 does not fall below a predetermined minimum height threshold, the width of ROI210 does not exceed a predetermined maximum width threshold, and the width of ROI210 does not fall below a predetermined minimum width threshold. In certain embodiments, the minimum and maximum width thresholds are set by the angle between the tone line 271 corresponding to the left side of ROI210 and the tone line 273 corresponding to the right side of ROI210. In another embodiment, the minimum width threshold and the maximum width threshold are set by the straight-line distance between the upper left vertex 281 and the upper right vertex 282 of the ROI 210.

[0076] In the examples shown in Figures 7A to 7D, the center of the ROI 210 almost coincides with the center of the bounding box 205. However, depending on the type of structure 203, the ROI 210 may be set to the periphery. For example, the renal artery and renal vein are located on the periphery of the kidney, not the center. Therefore, if the structure detection unit 312 determines that the structure 203 within the bounding box 205 is a kidney, the ROI 210 may be set to the position of the renal artery, which is offset from the center.

[0077] The fifth transformation coefficient applied to the fifth structure shown in Figure 7E ignores the bounding box 205 itself and sets the fixed ROI 210 with a fixed height and width to a predetermined position independent of the bounding box 205. If the fifth structure is, for example, the bladder, the system may not be able to determine whether to set the ROI 210 to focus on the ureteral orifice or to the outer wall of the bladder. For example, observing the urine jet phenomenon (ureteral jet) from the ureteral orifice into the bladder may help determine the possibility of urinary tract stones or ureteral tumors. Also, if there is a neoplastic lesion on the outer wall of the bladder, neovascularization may occur, and abnormal blood flow signals may be observed. In such cases, it may be easier to operate if the ROI 210 is set to a predetermined position independent of the bounding box 205, as shown in Figure 6E, and its position and size can be adjusted by the operator. In other embodiments, when the structure 203 is a bladder, a function is performed to display two ROIs 210 and prompt the operator to make a selection via the output interface 334. The two ROIs 210 differ from each other in their central position, height, and width. In other embodiments, considering that the observation of neoplastic lesions on the outer wall of the bladder is much less frequent than the observation of the urine jet, when the type of structure 203 is a bladder, only an ROI 210 for observing the urine jet is set. When the type of structure 203 is a kidney, it may also be desirable to set two types of ROIs 210, such as when observing the renal pelvis and when observing the outer wall of the kidney. In such cases as well, a function is performed to display two selectable ROIs 210 and prompt the operator to make a selection via the output interface 334. Often, the desired location (including the central position) of the ROI 210 does not coincide with the outer boundary of the structure 203, such as when observation of the ureteral orifice is desired when the structure 203 is a bladder, or when the renal pelvis is desired when the structure 203 is a kidney. Providing a conversion function for structure 203 would be a solution that easily and quickly addresses such requirements.

[0078] In certain embodiments, the structure 203 detected by the structure detection unit 312 and to which the bounding box 205 is applied may be different from the structure 203 to which the ROI 210 is set. The pancreas is located behind the stomach and may not be easy to visualize. For this reason, the spleen and / or the left lobe of the liver may be used as landmarks to set the ROI 210 on the pancreas. That is, the structure detection unit 312 may detect the spleen and set the bounding box 205 on the spleen, and the ROI 210 in color Doppler mode may be set to the position of the pancreas relative to the spleen.

[0079] Figures 7A to 7E illustrate the case where the ultrasonic probe 102 is a convex type ultrasonic probe. However, the ultrasonic probe 102 may also be a linear type ultrasonic probe. In this case, the multiple sound lines will be parallel to the vertical sides of the rectangle of the display area 200.

[0080] Figures 8A to 8E show several examples of ROI 210 settings for linear ultrasonic probes. In the example in Figure 8A, the shape of the bounding box 205 and the shape of the ROI 210 are perfectly matched. The sound ray of the linear ultrasonic probe is parallel to the vertical sides of the display area 200 and also parallel to the vertical sides of the bounding box 205.

[0081] As shown in Figure 8B, the area of ​​ROI 210 may be smaller than the area of ​​the bounding box 205. The conversion factor applied to the structure 203 shown in Figure 8B is set so that the width and height are 90% of the bounding box 205. The conversion factor applied to the structure 203 shown in Figure 8C is set so that the width and height are 110% of the bounding box 205. The area of ​​ROI 210 in Figure 8C may be larger than the area of ​​the bounding box 205. The conversion factor applied to the structure 203 shown in Figure 8D ignores the width of the bounding box 205, applies a fixed width for ROI 210, and sets the height of ROI 210 to 130% of the standard.

[0082] Even with linear ultrasound probes, if the area of ​​ROI210 is too large, it becomes difficult to maintain a high frame rate, and if the area of ​​ROI210 is too small, it may become unsuitable for observation. Therefore, the conversion function defines ROI210 such that the height of ROI210 does not exceed a predetermined maximum height threshold, the height of ROI210 does not fall below a predetermined minimum height threshold, the width of ROI210 does not exceed a predetermined maximum width threshold, and the width of ROI210 does not fall below a predetermined minimum width threshold. In a particular embodiment, the minimum and maximum width thresholds are set by the distance between the sound line 271 corresponding to the left side of ROI210 and the sound line 273 corresponding to the right side of ROI210.

[0083] The conversion coefficient applied to the structure 203 shown in Figure 8E is a conversion coefficient that ignores the bounding box 205 itself and sets the fixed ROI 210, with its height and width, to a predetermined position independent of the bounding box 205.

[0084] The ROI definition unit 318 can define the ROI 210 at the same time as the bounding box generation unit 314 generates the bounding box 205. This timing can be at predetermined intervals (e.g., every 10 frames, every second). In other embodiments, this is performed when there is a predetermined change in either the ROI definition unit 318 defining the ROI 210, the type of structure identified, or its location information.

[0085] Returning to Figure 4 and continuing the explanation, in step 518, the location information of the bounding box 205 that last updated (or newly created) ROI 210 is temporarily stored. In step 518, the stored location information of the bounding box 205 becomes the subject (reference) of the comparison process in step 514. In other embodiments, the subject (reference) of the comparison process in step 514 may be the location information of the bounding box 205 that was generated / updated in the previous (or n times previous) loop, rather than the location information of the bounding box 205 that last updated (or newly created) ROI 210.

[0086] In a specific embodiment of the present invention, the ultrasonic signal control unit 308 for color Doppler imaging adjusts the parameters of the ultrasonic signal for color Doppler imaging based on the type of identified structure 203 and the position and size of the defined ROI 210 (step 522). For example, if the area of ​​the ROI 210 is small, the range over which the ultrasonic signal is transmitted can be concentrated, which can improve the frame rate and improve responsiveness to motion (temporal resolution). Conversely, if the area of ​​the ROI 210 is large, the frame rate may decrease, and tracking of changes in flow velocity may worsen. The ROI definition unit 318 sets a small ROI 210 for structures 203 where fast-moving blood flow is expected, such as the heart or aorta. By changing the excitation delay timing of individual vibrating elements 102a or groups of vibrating elements 102a, the direction (refraction angle) and focus point of the ultrasonic beam can be electronically steered, and this steering can be controlled according to the position of the ROI 210.

[0087] Furthermore, if the ROI 210 is located deep, the round trip of the ultrasound pulse takes time, so adjustments are made to lower the pulse repetition frequency (PRF). Also, since the expected flow velocity of blood, etc., differs depending on the structure 203 (for example, blood flow in the abdominal aorta is fast, and the velocity of urine flowing through the ureter is slow), an appropriate pulse repetition frequency can be selected according to the flow velocity to prevent aliasing.

[0088] Depending on the type of structure 203, the importance of observing microcirculation may differ (for example, observation of microcirculation is not required in the abdominal aorta, but in the case of malignant tumors such as kidney cancer, observation of neovascularization, which constitutes microcirculation, may be required). Therefore, the gain (sensitivity) is set to an appropriate value accordingly. If the gain is too high, noise (observed as false blood flow) increases, and if it is too low, microcirculation cannot be detected. If the gain is too high, not only noise but also "blurring" of the color flow may occur.

[0089] If the ROI 210 is close to the heart or major blood vessels, it may be necessary to set the wall filter higher to remove slow-moving components (tissue movement).

[0090] Furthermore, if structure 203 is a kidney, it is necessary to appropriately adjust the overage and frame rate. Overage refers to the ratio or degree to which multiple pulse Doppler measurements (multiple lines) are taken and the results are averaged and combined to create one color Doppler frame. Increasing the overage reduces noise, makes the image smoother, and makes the blood flow image clearer. However, increasing the overage takes longer to create one frame, so the frame rate decreases. A low frame rate can result in jerky motion, which can cause problems in tracking the flow.

[0091] The ultrasonic signal control unit 308 for color Doppler imaging controls the ultrasonic probe 102 to output an ultrasonic signal with set parameters. The second ultrasonic image displayed inside the ROI 210 is an ultrasonic signal with a different image display mode than the ultrasonic signal for the B-mode image. The ultrasonic signal for color Doppler imaging is generally an ultrasonic signal with a narrower range, lower frequency, and higher frame rate compared to the ultrasonic signal for B-mode imaging. The echo signals corresponding to the ultrasonic signal for B-mode imaging and the echo signals corresponding to the ultrasonic signal for color Doppler imaging are generally processed separately.

[0092] In addition to color Doppler mode, other imaging modes such as other blood flow visualization images (pulse wave (PW) Doppler, continuous wave (CW) Doppler, power Doppler, etc.), elastic images (shear wave elastography, strain elastography, etc.), and attenuated images also transmit a separate ultrasound signal from the B-mode ultrasound signal, and this echo signal is processed separately.

[0093] Texture analysis images are images obtained by mathematically analyzing the patterns (roughness, smoothness, arrangement of fine dots and lines, etc.) of normal ultrasound images such as B-mode images, extracting information and quantifying and visualizing the distribution of density, roughness, regularity, and irregularity. It is not necessarily required to transmit a separate ultrasound signal for texture analysis images from the ultrasound signal for B-mode images. In other words, it is possible to generate texture analysis images by processing the echo signal for B-mode images. Alternatively, a separate ultrasound signal specifically for texture analysis images can be transmitted, and the corresponding echo signal can be processed to generate texture analysis images.

[0094] In a particular embodiment, a conversion function identification table similar to the conversion function identification table 350 for color Doppler mode shown in Figure 5 is provided for each mode. In a particular embodiment, the types of structures 203 registered in the conversion function identification table corresponding to each mode are different.

[0095] For example, the conversion function identification table 350 for shear wave elastography registers structures 203 such as the liver, mammary gland, prostate, and thyroid gland, and also registers their conversion functions, but does not register other structures 203 such as the gallbladder. When the shear wave elastography mode is activated and the structure detection unit 312 detects the thyroid gland as a structure 203, a bounding box 205 is applied to the structure 203 as described above, and the conversion function corresponding to the thyroid gland generates a thyroid ROI 210 from the thyroid gland's bounding box 205. The shear wave elastography image is displayed inside the thyroid gland ROI 210.

[0096] In contrast, a different process is performed for structures 203 that are not registered, such as the gallbladder. When the shear wave elastography mode is activated and the structure detection unit 312 detects the gallbladder as structure 203, a bounding box 205 is applied to the structure 203 as described above. The system displays on the display device 108 that the gallbladder, which is not subject to the shear wave elastography mode, has been detected, and prompts the operator to select a structure 203 that is subject to the shear wave elastography mode for imaging. In this case, there is no conversion function corresponding to the gallbladder, and the ROI 210 of the gallbladder is not generated from the bounding box 205 of the gallbladder. Therefore, a shear wave elastography image of the gallbladder is not displayed.

[0097] Thus, the conversion function identification table 350 is set up to correspond to each of the different image display modes. In other specific embodiments, two or more different image display modes share part or all of one conversion function identification table 350. In this way, in each of the different image display modes, it is easy to set up a set of structures for which an ROI 210 is to be set and another set of structures for which an ROI 210 is not to be set.

[0098] The ROI210 generated by applying the conversion coefficient has a high probability of being in the position and size desired by the operator. However, the ROI210 generated by applying the conversion coefficient may not be in the position and size desired by the operator. In that case, the operator is provided with a function to adjust the position of the ROI to the target area on the B-mode image using an input device such as a trackball or arrow keys. The vertical and horizontal size of the ROI210 can also be adjusted using the input device. The ROI definition unit 318 may correct or replace the conversion function corresponding to a structure to match the correction, taking into account the frequency of the operator's corrections (for example, if the same correction is made to the ROI210 of a particular structure 7 or more times out of 10).

[0099] The parameters applied by the ultrasonic signal control unit 308 for color Doppler imaging to the ROI 210 generated by applying a conversion coefficient can be set to have a high probability of being in the position and size desired by the operator. However, the parameters automatically set by the ultrasonic signal control unit 308 for color Doppler imaging may not be as desired by the operator. In that case, the operator can manually adjust the parameters for the ultrasonic signal for color Doppler imaging using the user interface 110. The ultrasonic signal control unit 308 for color Doppler imaging may modify the parameters applied to the corresponding structure 203 and ROI 210 combination, taking into account the frequency of operator modifications, and may change the automatically set parameters to be closer to the operator's preference.

[0100] Continuing the explanation with reference to Figure 4, in step 524, an ultrasonic signal for color Doppler imaging, with parameters set by the ultrasonic signal control unit 308 to correspond to the structure 203 and ROI 210, is emitted from the ultrasonic probe 102 toward the object 203, and a corresponding echo signal is received by the ultrasonic probe 102. The echo signal is processed by the color Doppler image processing unit 306 to generate a color Doppler image. The superimposed image generation unit 310 combines the B-mode image 201 with images of other image modes, such as the color mode image 211 formed inside the ROI 210, and displays them in the display area 200. Once the processing in step 524 is complete, the procedure moves to step 526.

[0101] In step 526, it is determined whether a predetermined time (e.g., 0.5 to 5.0 seconds, more preferably 0.7 to 2.0 seconds, even more preferably 1 second) or a predetermined number of frames of the B-mode image 201 have been drawn without an instruction to end the color Doppler mode being detected. The predetermined number of frames may be 1 to 50, more preferably 5 to 20, and even more preferably 10. If the determination is No, i.e., an instruction to end the color Doppler mode was given before the predetermined time or predetermined number of frames had elapsed, the process ends. Tracking of a specific structure 203 in target lock-on mode can be terminated by various operations, and the tracking of the next structure 203 can be moved. In certain embodiments, the operator can reset the tracking of the structure 203 by moving the ultrasonic probe 102 away from the object being inspected. Alternatively, tracking of the structure 203 in target lock-on mode can be moved to the next structure 203 by keyboard input (e.g., pressing a color button) or by voice instruction. The tracking of a structure 203 in target lock-on mode can also be transferred to the next structure 203 by user input, selecting a structure other than the currently being tracked structure from the list of structures 203 displayed on the display device.

[0102] If no instruction to end color Doppler mode is detected and a predetermined time or number of frames has elapsed, the structure 203 is detected (step 528). The detection of the structure 203 in step 528 is performed on the structure 203 registered in step 504. For example, if the structure 203 registered in step 504 is the gallbladder, the gallbladder is the target of detection even if the spleen is detected with a higher likelihood than the gallbladder, and its position information is used in subsequent steps.

[0103] Next, in step 529, it is determined whether or not a structure 203 is detected, that is, whether or not there is an object to generate or update the bounding box 205. In certain embodiments, if a structure 203 is not detected, a message is output to the operator informing them that the target structure 203 has not been detected (step 530). In certain embodiments, the message includes guidance on how to move the ultrasonic probe 102 so that the target structure 203 to be locked on is visualized. Based on this message, the operator can move the ultrasonic probe 102 to the appropriate position. In certain embodiments, even if a structure 203 is not detected, in step 524, an ultrasonic signal for color Doppler imaging with parameters set by the ultrasonic signal control unit 308 to correspond to the structure 203 and ROI 210 is emitted from the ultrasonic probe 102 toward the object 203, and a corresponding echo signal is received by the ultrasonic probe 102. The echo signal is processed by the color Doppler image processing unit 306 to generate a color Doppler image. In other embodiments, the process may proceed to step 526 after step 530. If the structure 203 is detected, the process proceeds to step 510 described above.

[0104] Figure 9 is a conceptual diagram illustrating the difference between target lock-on mode and target unlock mode. Images 412-416 in the upper row show how ROI210 is set up over time when target lock-on mode is selected. Images 422-426 in the lower row show how ROI210 is set up over time when target unlock mode is selected. Both figures are extracted portions of B-mode image 201 that include ROI210.

[0105] Figure 412 shows the state in target lock-on mode where the event detection unit 320 detects a predetermined event and sets the ROI 210 by applying a conversion function corresponding to the type of structure 203 to the bounding box 205 of the structure 203. As mentioned above, displaying the bounding box 205 is not mandatory. Figure 422 shows the state in target unlock mode where the event detection unit 320 detects a predetermined event and sets the ROI 210 by applying a conversion function corresponding to the type of structure 203 to the bounding box 205 of the structure 203 detected immediately before the event detection (the most recent structure 203 at the time of event detection) or the structure 203 detected immediately after the event detection.

[0106] In both images 412 and 422, the trained deep learning model 316 detected that the B-mode image 201 contained the pancreas 431 and the inferior vena cava (IVC) 432, and output that the pancreas 431 had a higher likelihood than the IVC 432. The bounding box 205 is displayed surrounding the pancreas 431, a transformation function for the pancreas is applied to the bounding box 205 to generate an ROI 210, which is displayed at the location of the pancreas 431. A color Doppler image is displayed within the ROI 210. At this point, there is no difference between the two images.

[0107] Next, due to the effects of respiration and other factors, the position depicted in the B-mode image 201 shifts, and in images 414 and 424, the gallbladder 433 is depicted in addition to the pancreas 431. In this example, the likelihood of the gallbladder 433 being identified is higher than that of the pancreas 431. Therefore, in target unlock mode, as shown in image 424, the gallbladder 433 is identified as the recognized structure 203, and a bounding box 205 surrounding the gallbladder 433 and a corresponding ROI 210 are displayed. In contrast, in target lock-on mode, as shown in image 414, the pancreas 431 remains the recognized structure 203, and the bounding box 205 and ROI 210 are positioned relative to the pancreas 431. In image 414, the pancreas 431 is depicted smaller than the pancreas 431 depicted in image 412, and a smaller bounding box 205 is also displayed. However, in certain embodiments, the ROI210 in image 414 can be the same size and position as the ROI210 in image 412. In this example, by not changing the position and size of ROI210, it is not necessary to change the color Doppler mode parameters corresponding to this ROI210, enabling uninterrupted rendering of the color Doppler mode image. In other embodiments, an ROI210 of the position and size corresponding to the small pancreas depicted in image 414 is set in image 414, the color Doppler mode parameters are reset, and optimal parameters are set for the new ROI210.

[0108] Due to the effects of respiration and other factors, the position depicted in the B-mode image 201 shifts further, and in images 416 and 426, the pancreas 431 disappears from the B-mode image 201, and the liver 434 is depicted. In this example, in target unlock mode, as shown in image 426, the liver 434 is recognized as the structure 203, and a bounding box 205 surrounding the liver 434 and a corresponding ROI 210 are displayed. In contrast, in target lock-on mode, as shown in image 416, the pancreas 431 remains the structure 203 to be recognized, and the ROI 210 is positioned at the location where the pancreas 431 existed before it disappeared. In embodiments where the bounding box 205 is not hidden in principle, the bounding box 205 can be hidden to indicate that there is no structure 203 being recognized. In other embodiments, the bounding box 205 is positioned at the location and shape where the pancreas 431 existed before it disappeared. The system repeatedly attempts to detect the pancreas 431. Thus, even when the deep learning model 316 detects multiple organs, or when the organ of interest temporarily disappears from the image, the color region of interest can be set more stably for the desired organ.

[0109] Tracking a specific structure 203 in target lock-on mode can be terminated by various operations, allowing the user to move on to tracking the next structure 203. In certain embodiments, the operator can reset tracking of a structure 203 by moving the ultrasonic probe 102 away from the object being inspected. Alternatively, tracking of a structure 203 in target lock-on mode can be moved to the next structure 203 by keyboard input (e.g., pressing a color button) or by voice command. Tracking of a structure 203 in target lock-on mode can also be moved to the next structure 203 by user input, selecting a structure other than the currently being tracked structure from a list of structures 203 displayed on the display device.

[0110] This specification discloses the subject matter, including best modes, and illustrates with examples to enable those skilled in the art to carry out the subject matter, including the manufacture and use of any apparatus or system, and the execution of incorporated methods. The patentable scope of the subject matter is defined by the claims and may include other examples conceivable to those skilled in the art. Such other examples are intended to be included in the claims if they have structural elements not different from the language of the claims, or if they include equivalent structural elements not substantially different from the language of the claims.

[0111] Further embodiments of the present invention can be provided by the embodiments described below. [Embodiment 1] An ultrasonic data processing device that generates an ultrasonic image displayed in the display area of ​​a display device, A first image generation unit generates a first ultrasonic image from a first echo signal generated in response to irradiating an object containing multiple types of structures with a first ultrasonic signal, A structure detection unit that analyzes the first ultrasonic image to detect a structure, A bounding box generation unit that generates a rectangular bounding box corresponding to the aforementioned structure, having sides parallel to each side of the rectangular display area, An ROI defining unit defines an ROI corresponding to the structure by applying a first transformation function corresponding to the structure to the information defining the bounding box, A second image generation unit generates a second ultrasound image from a second echo signal generated in response to irradiating the ROI of the target with a second ultrasound signal, Includes, The second ultrasound image is displayed in the display area as at least a part of the first ultrasound image. The second ultrasonic signal is projected along multiple sound lines, An ultrasonic data processing device, wherein the first conversion function converts information defining the bounding box such that the ROI is defined by a left boundary along one of the plurality of sound rays of the second ultrasonic signal and a right boundary along one of the plurality of sound rays of the second ultrasonic signal. [Embodiment 2] The ultrasonic data processing apparatus according to any of the preceding embodiments, wherein the first conversion function is configured to define ROIs of different areas and / or shapes depending on the type of structure. [Embodiment 3] The ultrasonic data processing apparatus according to any of the preceding embodiments, wherein the first conversion function is a function that makes the ROI a predetermined height or width with respect to at least one of the height and width of the bounding box. [Embodiment 4] The first transformation function described above is: The height of the ROI is set so as not to exceed a predetermined maximum height threshold. The height of the ROI is set so that it does not fall below a predetermined minimum height threshold. The width of the ROI is set so as not to exceed a predetermined maximum width threshold. An ultrasonic data processing apparatus according to any of the preceding embodiments, which defines the ROI such that the width of the ROI does not fall below a predetermined minimum width threshold. [Embodiment 5] A first image generation unit that generates a first ultrasound image from the first echo signal, A superimposed image generation unit generates a superimposed image by superimposing the second ultrasound image onto the first ultrasound image, Includes, The ultrasonic data processing apparatus according to claim 1, wherein the first ultrasonic image and the superimposed image are displayed in the display area. [Embodiment 6] An ultrasonic probe configured to emit a first ultrasonic signal and receive a first echo signal, and to emit a second ultrasonic signal and receive a second echo signal, The display device having the display area, Includes, The display device includes a display panel having pixels arranged in two dimensions with horizontal rows and vertical columns, The aforementioned ultrasonic probe is a convex-shaped ultrasonic probe. An ultrasonic data processing device according to any of the preceding embodiments, wherein the plurality of sound rays are arranged radially. [Embodiment 7] An ultrasonic probe configured to emit a first ultrasonic signal and receive a first echo signal, and to emit a second ultrasonic signal and receive a second echo signal, The display device having the display area, Includes, The display device includes a display panel having pixels arranged in two dimensions with horizontal rows and vertical columns, The aforementioned ultrasonic probe is a linear ultrasonic probe, The ultrasonic data processing apparatus according to any of the preceding embodiments, wherein the plurality of sound lines are parallel to the vertical sides of the rectangle of the display area. [Embodiment 8] The bounding box generation unit is configured to identify the structure and the bounding box by inputting at least a portion of the first ultrasonic data into a deep learning model. The ultrasonic data processing apparatus according to any of the preceding embodiments, wherein the deep learning model is a deep neural network trained using training data in which ultrasonic data is annotated with the type of structure and coordinate information of the bounding box corresponding to the structure. [Embodiment 9] The ultrasonic data processing apparatus according to any of the preceding embodiments, wherein the second ultrasonic signal is an ultrasonic signal of a different image display mode from the first ultrasonic signal. [Embodiment 10] An ultrasonic data processing apparatus according to any of the preceding embodiments, wherein the first ultrasonic signal is an ultrasonic signal for generating a B-mode image, and the second ultrasonic signal is an ultrasonic signal for displaying an ROI on the B-mode image and generating an image displayed within the ROI. [Embodiment 11] The ultrasonic data processing apparatus according to any of the preceding embodiments, wherein the subject is a human or a non-human mammal, and the structure is an organ, a part of an organ, a blood vessel, or a lesion. [Embodiment 12] A program that causes the processor of an ultrasonic data processing device to execute multiple steps, When the program is executed by the processor, an ultrasonic image is generated that is displayed in the display area of ​​the display device. The aforementioned steps are: The steps include: detecting structures by analyzing a first ultrasonic image generated from a first echo signal that occurs in response to irradiating an object containing multiple types of structures with a first ultrasonic signal; The steps include generating a rectangular bounding box corresponding to the aforementioned structure, having sides parallel to each side of the rectangular display area, The steps include defining an ROI corresponding to the structure by applying a first transformation function corresponding to the structure to the information defining the bounding box, The steps include generating a second ultrasound image from a second echo signal generated in response to irradiating the ROI of the target with a second ultrasound signal, Includes, The second ultrasound image is displayed in the display area as at least a part of the first ultrasound image. The second ultrasonic signal is projected along multiple sound lines, The first transformation function transforms the information defining the bounding box such that the ROI is defined by a left boundary along one of the plurality of sound rays of the second ultrasonic signal and a right boundary along the other of the plurality of sound rays of the second ultrasonic signal. program. [Embodiment 13] The program according to any of the preceding embodiments, wherein the first conversion function is configured to define ROIs of different areas and / or shapes depending on the type of structure. [Embodiment 14] The program according to any of the preceding embodiments, wherein the first conversion function is a function that makes the ROI a predetermined ratio of height or width to at least one of the height and width of the bounding box. [Embodiment 15] The first transformation function described above is: The height of the ROI is set so as not to exceed a predetermined maximum height threshold. The height of the ROI is set so that it does not fall below a predetermined minimum height threshold. The width of the ROI is set so as not to exceed a predetermined maximum width threshold. A program according to any of the preceding embodiments, which defines the ROI such that the width of the ROI does not fall below a predetermined minimum width threshold. [Embodiment 16] The ultrasonic data processing device is A first image generation unit that generates a first ultrasound image from the first echo signal, A superimposed image generation unit generates a superimposed image by superimposing the second ultrasound image onto the first ultrasound image, Includes, The program according to claim 15, wherein the first ultrasound image and the superimposed image are displayed in the display area. [Embodiment 17] The ultrasonic data processing device is An ultrasonic probe configured to emit a first ultrasonic signal and receive a first echo signal, and to emit a second ultrasonic signal and receive a second echo signal, The display device having the display area, Includes, The display device includes a display panel having pixels arranged in two dimensions with horizontal rows and vertical columns, The aforementioned ultrasonic probe is either a convex-type ultrasonic probe or a linear-type ultrasonic probe. If the ultrasonic probe is a convex-shaped ultrasonic probe, the plurality of sound lines are radially arranged. The program according to any of the preceding embodiments, wherein the ultrasonic probe is a linear ultrasonic probe, and the plurality of sound lines are parallel to the vertical sides of the rectangle of the display area. [Embodiment 18] The aforementioned steps are: The process includes the step of identifying the structure and the bounding box by inputting at least a portion of the first ultrasound image into a deep learning model, The program according to any prior embodiment, wherein the deep learning model is a deep neural network trained using training data in which ultrasonic data is annotated with the type of structure and coordinate information of the bounding box corresponding to the structure. [Embodiment 19] The program according to any of the preceding embodiments, wherein the second ultrasonic signal is an ultrasonic signal of a different image display mode from the first ultrasonic signal. [Embodiment 20] The program according to any of the preceding embodiments, wherein the first ultrasonic signal is an ultrasonic signal for generating a B-mode image, and the second ultrasonic signal is an ultrasonic signal for displaying an ROI on the B-mode image and generating an image displayed within the ROI. [Embodiment 21] The program according to any of the preceding embodiments, wherein the subject is a human or a non-human mammal, and the structure is an organ, a part of an organ, a blood vessel, or a lesion. [Embodiment 22] A non-temporary computer-readable medium for storing a program described in any of the preceding embodiments. [Embodiment 23] An ultrasonic data processing device that generates an ultrasonic image displayed in the display area of ​​a display device, A first image generation unit generates ultrasound data for B-mode imaging from echo signals generated in response to irradiating an object containing multiple types of structures with an ultrasound signal for B-mode imaging, A structure detection unit that analyzes ultrasonic data for the B-mode image to detect structures, A bounding box generation unit that generates a rectangular bounding box corresponding to the aforementioned structure, having sides parallel to each side of the rectangular display area, An ROI defining unit defines an ROI corresponding to the structure by applying a first transformation function corresponding to the structure to the information defining the bounding box, A second image generation unit analyzes the texture of the ultrasonic data for the B-mode image of the structure located at the ROI of the target, and generates an ultrasonic image for texture analysis. Includes, The ultrasonic image for texture analysis is displayed in the display area as at least a portion of the B-mode image. The ultrasonic signal for the B-mode image is emitted along multiple sound lines. An ultrasonic data processing device, wherein the first conversion function converts information defining the bounding box such that the ROI is defined by a left boundary along one of the plurality of sound rays of the second ultrasonic signal and a right boundary along one of the plurality of sound rays of the second ultrasonic signal.

[0112] The invention is not limited to this embodiment, and various modifications are possible without departing from the spirit of the invention. Furthermore, this specification uses examples to disclose the subject matter, including the best mode, and to enable those skilled in the art to carry out the subject matter, including the manufacture and use of any device or system, and the execution of the incorporated method. The patentable scope of the subject matter is defined by the claims and may include other examples that a person skilled in the art may conceive. Such other examples are intended to be included in the claims if they have structural elements that are not different from the language of the claims, or if they include equivalent structural elements that are substantially not different from the language of the claims. [Explanation of Symbols]

[0113] 100 Ultrasound diagnostic equipment 102 Ultrasound probe 102a Vibration element 103 Transmitting beamformer 104 Transmitter 105 Receiver 106 Receiving beamformer 107 Processor 108 Display device 109 memory 110 User Interface / Keyboard 111 Speakers 112 Programs 200 display area 201 B-mode image / First ultrasound image 203 Structures 205 Bounding Box 210 ROI 211 Color mode image / Second ultrasound image 251 The top-left vertex of the bounding box 252 Top right vertex of the bounding box 253 The bottom left vertex of the bounding box 254 The bottom right vertex of the bounding box 260 points 261 Sound line corresponding to the top left vertex of the bounding box 262 Sound line corresponding to the upper right vertex of the bounding box 263 The tone corresponding to the bottom left vertex of the bounding box 264 The tone wire corresponding to the bottom right vertex of the bounding box 265 Central tone line 266 angle 271 The tone corresponding to the left side of ROI 273 The tone corresponding to the right-hand side of ROI The top-left vertex of ROI 281 The upper right vertex of ROI 282 The bottom left vertex of ROI 283 284 ROI, bottom right vertex 300 Functional Blocks 302 B-mode image processing unit 304 Ultrasonic signal control unit for B-mode imaging 306 Color Doppler Image Processing Unit 308 Ultrasonic signal control unit for color Doppler imaging 310 Image generation unit / Superimposed image generation unit 312 Structure detection unit 314 Bounding box generation unit 316 Pre-trained Deep Learning Models 318 ROI Definition Section 320 Event Detection Unit 332 Input Interfaces 334 Output Interface 336 Communication Interfaces 350 Conversion Function Specific Table 412-416 Target Lock-On Mode Images 422-426 Target unlock mode image 431: Pancreas 432: IVC 433: Gallbladder 434: Liver

Claims

1. An ultrasonic data processing device that generates an ultrasonic image displayed in the display area of ​​a display device, A first image generation unit generates a first ultrasonic image from a first echo signal generated in response to irradiating an object containing multiple types of structures with a first ultrasonic signal, A structure detection unit that analyzes the first ultrasonic image to detect a structure, A bounding box generation unit that generates a rectangular bounding box corresponding to the aforementioned structure, having sides parallel to each side of the rectangular display area, An ROI defining unit defines an ROI corresponding to the structure by applying a first transformation function corresponding to the structure to the information defining the bounding box, A second image generation unit generates a second ultrasound image from a second echo signal generated in response to irradiating the target ROI with a second ultrasound signal, Includes, The second ultrasound image is superimposed on the first ultrasound image and displayed in the display area. The second ultrasonic signal is projected along multiple sound lines, The first conversion function converts the information defining the bounding box such that the ROI is defined by the left boundary along one of the plurality of sound rays of the second ultrasonic signal and the right boundary along the other of the plurality of sound rays of the second ultrasonic signal, The ultrasonic data processing device is configured such that the first conversion function defines the ROI of different areas and / or shapes depending on the type of structure detected.

2. The ultrasonic data processing apparatus according to claim 1, wherein the first conversion function is a function that makes the ROI a predetermined ratio of height or width to at least one of the height and width of the bounding box.

3. The first transformation function is, The height of the ROI shall not exceed a predetermined maximum height threshold. The height of the ROI does not fall below a predetermined minimum height threshold. The width of the ROI shall not exceed a predetermined maximum width threshold. The ultrasonic data processing apparatus according to claim 1, wherein the ROI is defined such that the width of the ROI does not fall below a predetermined minimum width threshold.

4. A first image generation unit that generates a first ultrasound image from the first echo signal, A superimposed image generation unit generates a superimposed image by superimposing the second ultrasound image onto the first ultrasound image, Includes, The ultrasonic data processing apparatus according to claim 1, wherein the first ultrasonic image and the superimposed image are displayed in the display area.

5. An ultrasonic probe configured to emit a first ultrasonic signal, receive a first echo signal, and emit a second ultrasonic signal, and receive a second echo signal, The display device having the display area, Includes, The display device includes a display panel having pixels arranged in two dimensions with horizontal rows and vertical columns, The aforementioned ultrasonic probe is a convex-shaped ultrasonic probe. The ultrasonic data processing apparatus according to claim 4, wherein the plurality of sound rays are arranged radially.

6. An ultrasonic probe configured to emit a first ultrasonic signal, receive a first echo signal, and emit a second ultrasonic signal, and receive a second echo signal, The display device having the display area, Includes, The display device includes a display panel having pixels arranged in two dimensions with horizontal rows and vertical columns, The aforementioned ultrasonic probe is a linear ultrasonic probe, The ultrasonic data processing apparatus according to claim 4, wherein the plurality of sound lines are parallel to the vertical sides of the rectangle of the display area.

7. The bounding box generation unit is configured to identify the structure and the bounding box by inputting at least a portion of the first ultrasound image into a deep learning model. The ultrasonic data processing apparatus according to claim 1, wherein the deep learning model is a deep neural network trained using training data in which ultrasonic data is annotated with the type of structure and coordinate information of a bounding box corresponding to the structure.

8. The ultrasonic data processing apparatus according to claim 1, wherein the second ultrasonic signal is an ultrasonic signal with a different image display mode from the first ultrasonic signal.

9. The ultrasonic data processing apparatus according to claim 8, wherein the first ultrasonic signal is an ultrasonic signal for generating a B-mode image, and the second ultrasonic signal is an ultrasonic signal for displaying an ROI on the B-mode image and generating an image displayed within the ROI.

10. The ultrasonic data processing apparatus according to any one of claims 1 to 9, wherein the subject is a human or a mammal other than a human, and the structure is an organ, a part of an organ, a blood vessel, or a lesion.

11. A program that causes the processor of an ultrasonic data processing device to execute multiple steps, When the program is executed by the processor, an ultrasonic image is generated that is displayed in the display area of ​​the display device. The aforementioned steps are: The steps include: detecting structures by analyzing a first ultrasonic image generated from a first echo signal that occurs in response to irradiating an object containing multiple types of structures with a first ultrasonic signal; The steps include generating a rectangular bounding box corresponding to the aforementioned structure, having sides parallel to each side of the rectangular display area, The steps include defining an ROI corresponding to the structure by applying a first transformation function corresponding to the structure to the information defining the bounding box, The steps include generating a second ultrasound image from a second echo signal generated in response to irradiating the target ROI with a second ultrasound signal, Includes, The second ultrasound image is superimposed on the first ultrasound image and displayed in the display area. The second ultrasonic signal is projected along multiple sound lines, The first conversion function converts the information defining the bounding box such that the ROI is defined by the left boundary along one of the plurality of sound rays of the second ultrasonic signal and the right boundary along the other of the plurality of sound rays of the second ultrasonic signal, The first conversion function is a program configured to define ROIs of different areas and / or shapes depending on the detected type of structure.

12. The program according to claim 11, wherein the first conversion function is a function that makes the ROI a predetermined ratio of height or width to at least one of the height and width of the bounding box.

13. The first transformation function is, The height of the ROI shall not exceed a predetermined maximum height threshold. The height of the ROI does not fall below a predetermined minimum height threshold. The width of the ROI shall not exceed a predetermined maximum width threshold. The program according to claim 11, which defines the ROI such that the width of the ROI does not fall below a predetermined minimum width threshold.

14. The ultrasonic data processing device is A first image generation unit that generates a first ultrasound image from the first echo signal, A superimposed image generation unit generates a superimposed image by superimposing the second ultrasound image onto the first ultrasound image, Includes, The program according to claim 13, wherein the first ultrasound image and the superimposed image are displayed in the display area.

15. The ultrasonic data processing device is An ultrasonic probe configured to emit a first ultrasonic signal, receive a first echo signal, and emit a second ultrasonic signal, and receive a second echo signal, The display device having the display area, Includes, The display device includes a display panel having pixels arranged in two dimensions with horizontal rows and vertical columns, If the ultrasonic probe is a convex-shaped ultrasonic probe, the plurality of sound lines are radial, The program according to claim 14, wherein, if the ultrasonic probe is a linear ultrasonic probe, the plurality of sound lines are parallel to the vertical sides of the rectangle of the display area.

16. The aforementioned steps are: The process includes the step of identifying the structure and the bounding box by inputting at least a portion of the first ultrasound image into a deep learning model, The program according to claim 11, wherein the deep learning model is a deep neural network trained using training data in which ultrasonic data is annotated with the type of structure and coordinate information of a bounding box corresponding to the structure.

17. The program according to claim 11, wherein the second ultrasonic signal is an ultrasonic signal of a different image display mode from the first ultrasonic signal.

18. The program according to claim 17, wherein the first ultrasonic signal is an ultrasonic signal for generating a B-mode image, and the second ultrasonic signal is an ultrasonic signal for displaying an ROI on the B-mode image and generating an image displayed within the ROI.

19. The program according to any one of claims 11 to 18, wherein the subject is a human or a non-human mammal, and the structure is an organ, a part of an organ, a blood vessel, or a lesion.

20. A non-temporary computer-readable medium for storing the program described in claim 19.

21. An ultrasonic data processing device that generates an ultrasonic image displayed in the display area of ​​a display device, A first image generation unit generates ultrasound data for B-mode imaging from echo signals generated in response to irradiating an object containing multiple types of structures with an ultrasound signal for B-mode imaging, A structure detection unit that analyzes ultrasonic data for the B-mode image to detect structures, A bounding box generation unit that generates a rectangular bounding box corresponding to the aforementioned structure, having sides parallel to each side of the rectangular display area, An ROI defining unit defines an ROI corresponding to the structure by applying a first transformation function corresponding to the structure to the information defining the bounding box, A second image generation unit performs texture analysis on the structure in the ultrasonic data for the B-mode image located at the position of the target ROI to generate an ultrasonic image for texture analysis, Includes, The ultrasonic image for texture analysis is superimposed on the B-mode image and displayed in the display area. The ultrasonic signal for the B-mode image is emitted along multiple sound lines. The first transformation function transforms the information defining the bounding box such that the ROI is defined by a left boundary along one of the plurality of sound rays of the ultrasonic signal for texture analysis and a right boundary along the other of the plurality of sound rays of the ultrasonic signal for texture analysis. The ultrasonic data processing device is configured such that the first conversion function defines the ROI of different areas and / or shapes depending on the type of structure detected.

Images