Ultrasonic imaging system and control method thereof

Abstract

The invention relates to an ultrasound imaging system and a control method thereof. Disclosed is an ultrasound imaging system including: a probe configured to transmit an ultrasound signal to a subject including a liver and receive an ultrasound echo signal reflected from the subject; a display configured to display the ultrasound image; an input interface configured to obtain a user input; a memory configured to store an artificial intelligence model; and at least one processor. The at least one processor is configured to: obtain ultrasound raw data by processing the ultrasound echo signal; generating a plurality of original image frames including different characteristic information by processing the ultrasonic original data; obtaining quantitative data about liver diseases from the artificial intelligence model by inputting the plurality of original image frames into the artificial intelligence model; and the ultrasound image is displayed on the display together with the quantitative data about the liver disease.

Description

Technical Field

[0001] This disclosure relates to an ultrasound imaging system and its control method that can use an artificial intelligence model to provide ultrasound images including various information. Background Technology

[0002] In the medical field, various medical imaging devices have recently been widely used to image and obtain information about human biological tissues for the purpose of early diagnosis of various diseases or surgeries. Representative examples of such medical imaging devices include ultrasound imaging devices, computed tomography (CT) devices, and magnetic resonance imaging (MRI) devices.

[0003] An ultrasound imaging device is a means of transmitting ultrasound signals generated by the transducer of a probe to an object and obtaining, non-invasively, at least one image of an internal region of the object (e.g., soft tissue or blood flow) by receiving information from signals reflected from the object. Ultrasound imaging devices can be used for medical purposes, such as observing the interior of an object, detecting foreign bodies, and measuring damage.

[0004] This ultrasound imaging device is widely used in conjunction with other imaging diagnostic devices because it is more stable than imaging devices that use X-rays, can display images in real time, and is safe because there is no radiation exposure.

[0005] Recently, various methods have emerged that utilize artificial intelligence models to process ultrasound images. Summary of the Invention

[0006] One aspect of this disclosure is to provide an ultrasound imaging system and its control method, which can use an artificial intelligence model to obtain quantitative data about liver diseases from raw ultrasound data and display the obtained data together with ultrasound images.

[0007] One aspect of this disclosure is to provide an ultrasound imaging system and its control method, which can generate raw images including various information by processing raw ultrasound data, and obtain quantitative data on liver diseases by inputting the generated raw images into an artificial intelligence model.

[0008] Other aspects of this disclosure will be set forth in part in the description which follows, and will be readily understood in part by way of description, or may be learned by practice of this disclosure.

[0009] One aspect of this disclosure provides an ultrasound imaging system comprising: a probe configured to transmit ultrasound signals to an object including a liver and to receive ultrasound echo signals reflected from the object; a display configured to display ultrasound images; an input interface configured to receive user input; a memory configured to store an artificial intelligence model; and at least one processor. The at least one processor may be configured to: obtain raw ultrasound data by processing the ultrasound echo signals; display a first ultrasound image generated by processing the raw ultrasound data on the display; obtain image frames of the first ultrasound image in response to a freeze command received through the input interface; generate a plurality of raw image frames corresponding to the obtained image frames and including different characteristic information by processing the raw ultrasound data; obtain quantitative data about a liver disease from the artificial intelligence model by inputting the plurality of raw image frames; and display a second ultrasound image including the quantitative data about the liver disease on the display.

[0010] The at least one processor may be configured to: generate a B-mode image as the first ultrasound image; and generate a first original image frame including attenuation information of the ultrasound echo signal and a second original image frame including scattering information of the ultrasound echo signal as the plurality of original image frames.

[0011] The at least one processor may also be configured to generate at least one of the following as the plurality of original image frames: a third original image frame including in-phase component information and quadrature component information of the original ultrasound data, and a fourth original image frame including spectral information of the original ultrasound data.

[0012] The at least one processor may be configured to: identify a first region of interest in the image frames of the first ultrasound image; determine a second region of interest corresponding to the first region of interest in each of the plurality of original image frames; extract local image frames corresponding to the second region of interest in each of the plurality of original image frames; and obtain quantitative data on the liver disease in the first region of interest by inputting the plurality of local image frames corresponding to each of the plurality of second regions of interest into the artificial intelligence model.

[0013] The at least one processor may be configured to: obtain first coordinate information of the first region of interest in the image frame of the first ultrasound image; convert the first coordinate information into second coordinate information in each of the plurality of original image frames; and determine the second region of interest based on the second coordinate information.

[0014] The at least one processor may be configured to obtain at least one of the fat fraction of the liver and the severity of hepatic steatosis as quantitative data regarding the liver disease.

[0015] The at least one processor may be configured to: obtain additional information about the object through the input interface, the additional information including at least one of subcutaneous fat thickness, body mass index (BMI), gender, age, and underlying diseases; and input the plurality of original image frames and the additional information about the object into the artificial intelligence model.

[0016] The at least one processor may be configured to display the quantitative data on the liver disease in at least one of a first region of the display that displays the first ultrasound image and the second ultrasound image, and a second region of the display that is separated from the first region.

[0017] The at least one processor may also be configured to display a heatmap on the display, the heatmap visualizing the distribution of the quantitative data regarding the liver disease.

[0018] The at least one processor may be configured to display the second ultrasound image on the display based on a command obtained through the input interface for activating an AI-based function.

[0019] Another aspect of this disclosure provides a control method for an ultrasound imaging system, the ultrasound imaging system including a probe, an input interface, a display, and at least one processor. The control method includes a control method executed by the at least one processor, wherein the control method may include: controlling the probe to send an ultrasound signal to an object including a liver and receiving ultrasound echo signals reflected from the object; obtaining raw ultrasound data by processing the ultrasound echo signals; displaying a first ultrasound image generated by processing the raw ultrasound data on the display; obtaining an image frame of the first ultrasound image in response to obtaining a freeze command through the input interface; generating a plurality of raw image frames corresponding to the obtained image frames and including different characteristic information by processing the raw ultrasound data; obtaining quantitative data about liver disease from the artificial intelligence model by inputting the plurality of raw image frames into the artificial intelligence model; and displaying a second ultrasound image including the quantitative data about the liver disease on the display.

[0020] The first ultrasound image may correspond to a B-mode image, and the step of generating the plurality of original image frames may include: generating a first original image frame including attenuation information of the ultrasound echo signal and a second original image frame including scattering information of the ultrasound echo signal.

[0021] The step of generating the plurality of original image frames may further include: generating at least one of a third original image frame including in-phase component information and quadrature component information of the original ultrasound data and a fourth original image frame including spectral information of the original ultrasound data as the plurality of original image frames.

[0022] The steps of obtaining the quantitative data on the liver disease may include: identifying a first region of interest in the image frames of the first ultrasound image; determining a second region of interest corresponding to the first region of interest in each of the plurality of original image frames; extracting local image frames corresponding to the second region of interest in each of the plurality of original image frames; and obtaining the quantitative data on the liver disease in the first region of interest by inputting a plurality of local image frames corresponding to each of the plurality of second regions of interest into the artificial intelligence model.

[0023] The step of determining the second region of interest may include: obtaining first coordinate information of the first region of interest in the image frame of the first ultrasound image; converting the first coordinate information into second coordinate information in each of the plurality of original image frames; and determining the second region of interest based on the second coordinate information.

[0024] The quantitative data regarding the liver disease may include at least one of the fat fraction in the liver and the severity of hepatic steatosis.

[0025] The control method may further include: obtaining additional information about the object through the input interface, the additional information including at least one of subcutaneous fat thickness, body mass index (BMI), gender, age, and underlying diseases; and inputting the plurality of original image frames and the additional information about the object into the artificial intelligence model.

[0026] The step of displaying the second ultrasound image may include: displaying the quantitative data on the liver disease in at least one of a first region of the display that displays the first ultrasound image and the second ultrasound image and a second region of the display that is separated from the first region.

[0027] The control method may further include: displaying a heatmap on the display, the heatmap visualizing the distribution of the quantitative data regarding the liver disease in the first ultrasound image.

[0028] The step of displaying the second ultrasound image can be performed based on a command obtained through the input interface to activate the artificial intelligence-based function. Attached Figure Description

[0029] These and / or other aspects of this disclosure will become clear and more readily understood from the following description of embodiments taken in conjunction with the accompanying drawings, in which: Figure 1A and Figure 1B This is a block diagram illustrating the components of an ultrasound imaging system according to various embodiments; Figure 2A , Figure 2B , Figure 2C and Figure 2D An ultrasound imaging apparatus according to various embodiments is shown; Figure 3 This is a flowchart illustrating a control method for an ultrasound imaging system according to an embodiment; Figure 4 This is a diagram illustrating the process of performing the artificial intelligence-based functions of the ultrasound imaging system according to an embodiment; Figure 5 An example is shown of displaying ultrasound images and quantitative data on liver disease on a display of an ultrasound imaging device; Figure 6 An example is shown of displaying ultrasound images and quantitative data on liver disease on a display of an ultrasound imaging device; Figure 7 Examples are shown of displaying ultrasound images and quantitative data on liver disease on a monitor of an ultrasound imaging device; and Figure 8 An example is shown of displaying ultrasound images and quantitative data on liver disease on a monitor of an ultrasound imaging device. Detailed Implementation

[0030] This disclosure will illustrate embodiments of the present disclosure to clarify the scope of the claims of the present disclosure and to enable those skilled in the art to practice the embodiments.

[0031] Throughout this specification, the same reference numerals refer to the same elements. This specification does not describe all components of the embodiments, and will not describe general content or repetition between embodiments within the art to which this disclosure pertains. As used in this specification, a “module” or “unit” may be implemented as one or more of software, hardware, or firmware, and according to embodiments, multiple “modules” or “units” may be implemented as a single element, or a single “module” or “unit” may include multiple elements.

[0032] Unless the relevant context clearly indicates otherwise, the singular form of the noun corresponding to an item may include a single item or multiple items.

[0033] In this disclosure, each of the phrases such as “A or B”, “at least one of A and B”, “at least one of A or B”, “A, B or C”, “at least one of A, B and C”, and “at least one of A, B or C” may include any one of the items listed together in the corresponding phrase or all possible combinations thereof.

[0034] The term “and / or” includes any combination of multiple related components or any one of multiple related components.

[0035] For example, expressions such as “A and / or B” or “at least one of A or B” can include all possible combinations of items listed together. For example, “A and / or B” or “at least one of A or B” can refer to (1) only A, (2) only B, or (3) all cases that include both A and B.

[0036] Terms such as “first,” “second,” “primary,” and “secondary” can be simply used to distinguish a given component from other corresponding components without limiting the corresponding components in any other way (e.g., importance or order).

[0037] The terms “front surface”, “rear surface”, “upper surface”, “lower surface”, “side surface”, “left side”, “right side”, “upper part”, “lower part”, etc., used in this disclosure are defined with reference to the accompanying drawings, and the shape and position of each component are not limited by these terms.

[0038] The terms “comprising,” “having,” etc., are intended to indicate the presence of the features, quantities, steps, operations, components, parts, or combinations thereof described in this disclosure, and do not exclude the presence or addition of one or more other features, quantities, steps, operations, components, parts, or combinations thereof.

[0039] When any component is referred to as being “connected” to another component, “joined” to another component, “supported” by another component, or “in contact” with another component, this includes cases where components are indirectly connected, joined, supported, or in contact with each other through a third component, as well as cases where they are directly connected, joined, supported, or in contact with each other.

[0040] When any component is said to be "on" or "above" another component, this includes not only cases where any component is in contact with another component, but also cases where the other component exists between the two components.

[0041] In this disclosure, the “object” being photographed may include a person, an animal, or a part thereof. For example, an object may include a part of a human body (organs, etc.) or a phantom.

[0042] In this disclosure, "ultrasound image" refers to an image of an object that has been generated or processed based on ultrasound signals (echo signals) sent to and reflected from the object.

[0043] In this disclosure, "visual indicators" may include various indicators such as letters, numbers, shapes (dots, lines, surfaces, three-dimensional structures), colors, animations, and visual effects.

[0044] In the following description, embodiments will be described in detail with reference to the accompanying drawings.

[0045] Figure 1A and Figure 1B This is a block diagram illustrating the components of an ultrasound imaging system according to various embodiments.

[0046] Reference Figure 1A and Figure 1B The ultrasound imaging system 100 may include a probe 20 and an ultrasound imaging device 40.

[0047] The ultrasound imaging device 40 can be implemented as either a pushcart model or a portable model. Portable ultrasound imaging devices may include, for example, smartphones, laptops, personal digital assistants (PDAs), tablet PCs, etc., including probes and applications, but are not limited to these. The ultrasound imaging device 40 can also be implemented with an integrated probe.

[0048] The probe 20 may include a wired probe that is connected to the ultrasound imaging device 40 via a wire to communicate with the ultrasound imaging device 40 via a wire, a wireless probe that is wirelessly connected to the ultrasound imaging device 40 to communicate with the ultrasound imaging device 40 via a wire, and / or a hybrid probe that is connected to the ultrasound imaging device 40 via a wire or wirelessly to communicate with the ultrasound imaging device 40 via a wire or wirelessly. The probe 20 may be referred to as an "ultrasound probe" because it transmits and receives ultrasound signals.

[0049] According to various embodiments, such as Figure 1A As shown, the ultrasound imaging device 40 may include an ultrasound transmitter / receiver module 110. For example... Figure 1B As shown, probe 20 may also include an ultrasound transmitter / receiver module 110. According to various embodiments, both the ultrasound imaging device 40 and probe 20 may also include the ultrasound transmitter / receiver module 110.

[0050] According to various embodiments, probe 20 may also include at least one or a combination of image processor 130, display 140, and input interface 170. The description of ultrasound transmitter / receiver module 110, image processor 130, display 140, and input interface 170 included in ultrasound imaging device 40 can also be applied to ultrasound transmitter / receiver module 110, image processor 130, display 140, and input interface 170 included in probe 20.

[0051] Figure 1A A block diagram illustrating the components of an ultrasound imaging system 100 in the case where probe 20 is a wired probe or a hybrid probe is shown. In the case where probe 20 is a wired probe or a hybrid probe, probe 20 may include cables and connectors capable of being connected to a connector of the ultrasound imaging device 40.

[0052] The probe 20 may include multiple transducers. The multiple transducers may be arranged in a predetermined configuration to be implemented as a transducer array. The transducer array may correspond to a one-dimensional (1D) array or a two-dimensional (2D) array. The multiple transducers may transmit ultrasonic signals to the object 10 in response to a transmission signal applied from the transmission module 113. The multiple transducers may form a received signal by receiving ultrasonic signals reflected from the object 10 (echo signals). The probe 20 may be implemented as integrated with the ultrasound imaging device 40, or as a separate type connected to the ultrasound imaging device 40 via wires. The ultrasound imaging device 40 may be connected to one or more probes 20, depending on the implementation type.

[0053] The probe 20 can be implemented as a two-dimensional probe. When the probe 20 is implemented as a two-dimensional probe, the multiple transducers included in the probe 20 can be arranged in two dimensions to form a two-dimensional transducer array. For example, the two-dimensional transducer array can have the form in which multiple subarrays including multiple transducers arranged in a first direction are arranged in a second direction different from the first direction.

[0054] When probe 20 is implemented as a two-dimensional probe, the ultrasonic transmitter / receiver module 110 may include at least one of an analog beamformer and a digital beamformer. Furthermore, the two-dimensional probe may include at least one or a combination of analog beamformers and digital beamformers, depending on the implementation type.

[0055] The processor 120 controls the transmitting module 113 to form a transmitting signal to be applied to each of the multiple transducers included in the probe 20, taking into account the position and focus of the transducers.

[0056] The processor 120 can control the receiving module 115, taking into account the positions and focal points of multiple transducers, to generate ultrasound data by performing analog-to-digital conversion on the received signals received from the probe 20 and summing the digitally converted received signals. The ultrasound data may include raw ultrasound data and / or ultrasound image data generated based on the raw ultrasound data. The raw ultrasound data may also be referred to as RF data.

[0057] When probe 20 is implemented as a two-dimensional probe, processor 120 can calculate a time delay value for digital beamforming for each of the plurality of subarrays included in the two-dimensional transducer array. Processor 120 can also calculate a time delay value for analog beamforming for each transducer included in one of the plurality of subarrays. Processor 120 can control the analog beamformer and the digital beamformer to form a transmission signal to be applied to each of the plurality of transducers based on the time delay values ​​for analog beamforming and digital beamforming. Processor 120 can also control the analog beamformer to sum the signals received from the plurality of transducers in each subarray based on the time delay values ​​for analog beamforming. Processor 120 can also control the ultrasound transmitter / receiver module 110 to perform analog-to-digital conversion on the summed signal for each subarray. Processor 120 can also control the digital beamformer to generate ultrasound data by summing the digitally converted signal based on the time delay values ​​for digital beamforming.

[0058] Image processor 130 can use the generated ultrasound data to generate and / or process ultrasound images. Processing ultrasound images by image processor 130 may include processing raw ultrasound data to generate raw image frames including various information and / or generating ultrasound images in various modes. Raw ultrasound data may also be referred to as RF data.

[0059] Ultrasound images can be provided in various modes. For example, modes used for ultrasound images may include amplitude mode (A-mode), brightness mode (B-mode), color Doppler mode, Doppler mode (D-mode), elastography mode (E-mode), motion mode (M-mode), and volume mode. Depending on the mode of the ultrasound image, the method of processing the raw ultrasound data can vary.

[0060] The display 140 can display the generated ultrasound images and various information processed in the ultrasound imaging device 40 or probe 20. The probe 20 or ultrasound imaging device 40 may include one or more displays 140, depending on the implementation type. The display 140 may also include a touch panel or touchscreen. The display 140 may also include a flexible display.

[0061] The probe 20 itself may also have a display 140.

[0062] The processor 120 can control the overall operation of the ultrasound imaging device 40 and the operation of its components. The processor 120 can execute or control various operations and / or functions of the ultrasound imaging device 40 by running programs or instructions stored in the memory 150. The processor 120 can also control the operation of the ultrasound imaging device 40 by receiving control signals from the input interface 170 or external devices.

[0063] The ultrasound imaging device 40 may include a communication module 160, and can be connected to and communicate with external devices (e.g., probes, servers, computing devices, medical devices, portable devices (smartphones, tablet PCs, wearable devices, etc.)) via the communication module 160.

[0064] The communication module 160 may include one or more components that enable communication with external devices. The communication module 160 may include at least one of, for example, a short-range communication module, a wired communication module, and a wireless communication module.

[0065] The communication module 160 can receive control signals or data from an external device. The processor 120 can control the operation of the ultrasound imaging device 40 based on the control signals received through the communication module 160. Furthermore, the processor 120 can send control signals to the external device through the communication module 160, thereby controlling the external device based on the sent control signals. The external device can operate based on the control signals received from the ultrasound imaging device 40, or it can process the data received from the ultrasound imaging device 40.

[0066] Programs or applications associated with the ultrasound imaging device 40 may be installed in an external device. These programs or applications installed in the external device may control the ultrasound imaging device 40 or operate based on control signals or data received from the ultrasound imaging device 40.

[0067] External devices can receive or download programs or applications related to the ultrasound imaging device 40 from the ultrasound imaging device 40, probe 20, or computing device 30 to install and run the programs or applications on the external device. The ultrasound imaging device 40 or probe 20 providing the programs or applications may include recording media storing instructions, commands, installation files, executable files, or related data for the corresponding programs or applications. External devices may also be sold together with the installed programs or applications.

[0068] The memory 150 can store various data or programs, input and output ultrasound data, ultrasound images, etc., used to drive and control the ultrasound imaging device 40.

[0069] Input interface 170 can receive user input for controlling ultrasound imaging device 40. For example, user input may include, but is not limited to, input from operation buttons, keypad, dial pad, mouse, trackball, micro switch, knob, etc., input from touchpad or touch screen, voice input, motion input, biometric information input (e.g., iris recognition, fingerprint recognition, etc.).

[0070] Figure 1B A control block diagram of an ultrasound imaging system 100 is shown, assuming probe 20 is a wireless probe or a hybrid probe. According to various embodiments, Figure 1BThe ultrasound imaging device 40 shown can be used as a reference. Figure 1A The described ultrasound imaging device 40 is a replacement. (Refer to...) Figure 1A The probe 20 described can be used as a reference. Figure 1B The described probe 20 is replaced.

[0071] The probe 20 may include a display 112, a transmitting module 113, a battery 114, a transducer 117, a charging module 116, a receiving module 115, an input interface 109, a processor 118, and a communication module 119. Figure 1B A probe 20 is shown, but is not limited to, including both a transmitting module 113 and a receiving module 115. The probe 20 may include only a portion of the configuration of the transmitting module 113 and the receiving module 115. A portion of the configuration of the transmitting module 113 and the receiving module 115 may be included in the ultrasound imaging device 40. Additionally, the probe 20 may also include an image processor 130.

[0072] Transducer 117 may include multiple transducers. The multiple transducers may be arranged in a predetermined configuration to be implemented as a transducer array. The transducer array may correspond to a one-dimensional (1D) array or a two-dimensional (2D) array. The multiple transducers may transmit ultrasonic signals to object 10 in response to a transmission signal applied from transmission module 113. The multiple transducers may also receive ultrasonic signals reflected from object 10 to form or generate an electrical received signal.

[0073] The charging module 116 can charge the battery 114. The charging module 116 can receive power from an external source. The charging module 116 can receive power wirelessly. The charging module 116 can also receive power via a wire. The charging module 116 can transfer the received power to the battery 114.

[0074] The processor 118 controls the transmitting module 113 to generate or form a transmitting signal to be applied to each of the multiple transducers, taking into account the position and focus of the multiple transducers.

[0075] The processor 118 controls the receiving module 115, taking into account the positions and focus of multiple transducers, to generate ultrasound data by performing analog-to-digital conversion on the received signals received from the transducers 117 and summing the digitally converted received signals. According to embodiments of this disclosure, when the probe 20 includes an image processor 130, the probe 20 can use the generated ultrasound data to generate an ultrasound image.

[0076] When probe 20 is implemented as a two-dimensional probe, processor 118 can calculate a time delay value for digital beamforming for each of the multiple subarrays included in the two-dimensional transducer array. Processor 118 can also calculate a time delay value for analog beamforming for each transducer included in one of the multiple subarrays. Processor 118 can control the analog beamformer and the digital beamformer to form a transmission signal to be applied to each of the multiple transducers based on the time delay values ​​for analog and digital beamforming. Processor 118 can also control the analog beamformer to sum the signals received from the multiple transducers for each subarray based on the time delay values ​​for analog beamforming. Processor 118 can also control the ultrasound transmitter / receiver module 110 to perform analog-to-digital conversion on the summed signal for each subarray. Processor 118 can also control the digital beamformer to generate ultrasound data by summing the digitally converted signal based on the time delay values ​​for digital beamforming.

[0077] The processor 118 can control the overall operation of the probe 20 and the operation of its components. The processor 118 can execute or control various operations or functions of the probe 20 by running programs or instructions stored in the memory 111. The processor 118 can also control the operation of the probe 20 by receiving control signals from the input interface 109 of the probe 20 or from an external device (e.g., ultrasound imaging equipment 40). The input interface 109 can receive user input for controlling the probe 20. For example, user input may include, but is not limited to, input via buttons, keypad, mouse, trackball, microswitch, knob, etc.; input via touchpad or touchscreen; voice input; motion input; biometric information input (e.g., iris recognition, fingerprint recognition, etc.).

[0078] The display 112 can display ultrasound images generated by the probe 20, ultrasound images generated by processing ultrasound data generated in the probe 20, ultrasound images received from the ultrasound imaging device 40, or various information processed in the ultrasound imaging system 100. The display 112 can also display status information of the probe 20. The status information of the probe 20 may include at least one of the following: probe 20 device information, probe 20 battery status information, probe 20 frequency band information, probe 20 output information, information on whether the probe 20 is malfunctioning, probe 20 setting information, and probe 20 temperature information.

[0079] The probe 20 may include one or more displays 112, depending on the implementation type. The display 112 may include a touch panel or a touch screen. The display 112 may also include a flexible display.

[0080] The communication module 119 can wirelessly transmit the generated ultrasound data or ultrasound images to the ultrasound imaging device 40 via a wireless network. The communication module 119 can also receive control signals and data from the ultrasound imaging device 40.

[0081] The ultrasound imaging device 40 can receive ultrasound data and / or ultrasound images from the probe 20.

[0082] When the probe 20 includes an image processor 130 capable of generating ultrasound images from ultrasound data, the probe 20 can send ultrasound data or ultrasound images generated by the image processor 130 to the ultrasound imaging device 40.

[0083] In the absence of an image processor 130 capable of generating ultrasound images from ultrasound data, probe 20 may transmit ultrasound data to ultrasound imaging device 40.

[0084] The ultrasound imaging device 40 may include a processor 120, an image processor 130, a display 140, a memory 150, a communication module 160, and an input interface 170.

[0085] The image processor 130 uses ultrasound data received from the probe 20 to generate and / or process ultrasound images.

[0086] Display 140 may display ultrasound images received from probe 20, ultrasound images generated by processing ultrasound data received from probe 20, and / or various information processed by ultrasound imaging system 100. Ultrasound imaging device 40 may include one or more displays 140 depending on the implementation type. Display 140 may also include a touch panel or touchscreen. Display 140 may also include a flexible display.

[0087] The processor 120 can control the overall operation of the ultrasound imaging device 40 and the operation of its components. The processor 120 can execute or control various operations or functions of the ultrasound imaging device 40 by running programs or applications stored in the memory 150. The processor 120 can also control the operation of the ultrasound imaging device 40 by receiving control signals from the input interface 170 or external devices.

[0088] The ultrasound imaging device 40 may include a communication module 160, and can be connected to and communicate with external devices (e.g., probe 20, computing devices, medical devices, portable devices (smartphones, tablet PCs, wearable devices, etc.)) via the communication module 160.

[0089] The communication module 160 may include one or more components that enable communication with external devices. The communication module 160 may include at least one of, for example, a short-range communication module, a wired communication module, and a wireless communication module.

[0090] The communication module 160 of the ultrasound imaging device 40 and the communication module 119 of the probe 20 can communicate using a network or short-range wireless communication method. For example, the communication module 160 of the ultrasound imaging device 40 and the communication module 119 of the probe 20 can communicate using any of the following: wireless LAN, Wi-Fi, Bluetooth, ZigBee, Wi-Fi Direct (WFD), Infrared Data Association (IrDA), Bluetooth Low Energy (BLE), Near Field Communication (NFC), Wireless Broadband Internet (WiBro), Global Microwave Access Interoperability (WiMAX), Shared Wireless Access Protocol (SWAP), Wireless Gigabit Alliance (WiGig), RF communication, and wireless data communication methods including 60 GHz millimeter wave (mm wave) short-range communication, etc.

[0091] Therefore, the communication module 160 of the ultrasound imaging device 40 and the communication module 119 of the probe 20 may include at least one of the following modules: wireless LAN communication module, Wi-Fi communication module, Bluetooth communication module, ZigBee communication module, Wi-Fi Direct (WFD) communication module, Infrared Data Association (IrDA) communication module, Bluetooth Low Energy (BLE) communication module, Near Field Communication (NFC) module, Wireless Broadband Internet (WiBro) communication module, Global Microwave Access Interoperability (WiMAX) communication module, Shared Wireless Access Protocol (SWAP) communication module, Wireless Gigabit Alliance (WiGig) communication module, RF communication module, and 60 GHz millimeter wave (mm wave) short-range communication module.

[0092] The probe 20 can transmit its device information (e.g., ID information) to the ultrasound imaging device 40 using a first communication method (e.g., BLE) and can wirelessly pair with the ultrasound imaging device 40. The probe 20 can also transmit ultrasound data and / or ultrasound images to the paired ultrasound imaging device 40. The device information of the probe 20 may include various information about the probe 20's serial number, model name, or battery status. The probe 20 can also transmit ultrasound data and / or ultrasound images to the ultrasound imaging device 40 paired via the first communication method using a second communication method (e.g., 60 GHz millimeter wave and Wi-Fi).

[0093] The ultrasound imaging device 40 can receive device information (e.g., ID information) of the probe 20 from the probe 20 using a first communication method (e.g., BLE) and can wirelessly pair with the probe 20. The ultrasound imaging device 40 can also send an activation signal to the paired probe 20 and receive ultrasound data and / or ultrasound images from the probe 20. In this case, the activation signal may include a signal for controlling the operation of the probe 20. The ultrasound imaging device 40 can also send an activation signal to the paired probe 20 and receive ultrasound data and / or ultrasound images from the probe 20 using a second communication method (e.g., 60 GHz millimeter wave and Wi-Fi).

[0094] The first communication method for pairing the probe 20 and the ultrasound imaging device 40 with each other may have a lower frequency band than the second communication method used by the probe 20 to transmit ultrasound data and / or ultrasound images to the ultrasound imaging device 40.

[0095] The display 140 of the ultrasound imaging device 40 may display a UI (user interface) indicating device information of the probe 20. For example, the display 140 may display identification information indicating the wireless probe 20, a pairing method indicating the method of pairing with the probe 20, the data communication status between the probe 20 and the ultrasound imaging device 40, a UI indicating the method of performing data communication with the ultrasound imaging device 40, and / or a UI indicating the battery status of the probe 20.

[0096] When the probe 20 includes a display 112, the display 112 of the probe 20 may display a UI indicating device information of the probe 20. For example, the display 112 may display identification information indicating that the wireless probe 20 is wireless, a pairing method indicating the method of pairing with the probe 20, the data communication status between the probe 20 and the ultrasound imaging device 40, a UI indicating the method of performing data communication with the ultrasound imaging device 40, and / or a UI indicating the battery status of the probe 20.

[0097] The communication module 160 can receive control signals or data from external devices. The processor 120 can control the operation of the ultrasound imaging device 40 in response to the control signals received through the communication module 160.

[0098] Furthermore, the processor 120 can send control signals to external devices via the communication module 160 to control the external devices according to the sent control signals. The external devices can operate according to the control signals received from the ultrasound imaging equipment 40, or process the data received from the ultrasound imaging equipment 40.

[0099] An external device may receive or download programs or applications related to the ultrasound imaging device 40 from the ultrasound imaging device 40 or the probe 20 to install and run the programs or applications on the external device. The ultrasound imaging device 40 or the probe 20 providing the programs or applications may include a recording medium storing instructions, commands, installation files, executable files, or related data of the programs or applications. The external device may be sold together with the installed programs or applications.

[0100] The memory 150 can store various data or programs, input and output ultrasound data, ultrasound images, etc., used to drive and control the ultrasound imaging device 40.

[0101] Figure 2A , Figure 2B , Figure 2C and Figure 2D An ultrasound imaging apparatus according to various embodiments is shown.

[0102] Reference Figure 2A and Figure 2B The ultrasound imaging devices 40a and 40b may include a main display 140a and a secondary display 140b. The main display 140a and the secondary display 140b may correspond to... Figure 1A and Figure 1B The display 140. At least one of the main display 140a and the secondary display 140b can be implemented as a touch screen. At least one of the main display 140a and the secondary display 140b can display ultrasound images or various information processed in the ultrasound imaging devices 40a and 40b.

[0103] At least one of the main display 140a and the secondary display 140b may be implemented as a touchscreen and provide a GUI (Graphical User Interface), allowing user input of data for controlling the ultrasound imaging devices 40a and 40b. For example, the main display 140a may display ultrasound images, and the secondary display 140b may display a control panel for controlling the display of ultrasound images in the form of a GUI. Data for controlling the display of images can be input into the secondary display 140b via the control panel displayed in the GUI.

[0104] For example, the GUI provided via the secondary display 140b may include a Time Gain Compensation (TGC) button, a Lateral Gain Compensation (LGC) button, a freeze button, a trackball, a micro switch, a knob, and / or an AI-based function button 173.

[0105] Ultrasonic imaging devices 40a and 40b can use input control data to control the display of ultrasound images on the main display 140a. Ultrasonic imaging devices 40a and 40b can also be connected to probe 20 via wires or wirelessly to send and receive ultrasound signals to and from the object.

[0106] Reference Figure 2B In addition to the main display 140a and the secondary display 140b, the ultrasound imaging device 40b may also include a control panel 165. The control panel 165 may include buttons, a trackball, microswitches, knobs, etc., and data for controlling the ultrasound imaging device 40b can be input from the user into the control panel 165.

[0107] For example, control panel 165 may include a TGC button 171, a freeze button 172, and / or an AI-based function button 173. The TGC button 171 is a button for setting the TGC value for each depth of the ultrasound image.

[0108] Additionally, the ultrasound imaging device 40b can receive a freeze command via the freeze button 172 while displaying an ultrasound image comprising multiple image frames. The ultrasound imaging device 40b can pause the display of the ultrasound image and display the image frame at the time the freeze command is received. The ultrasound imaging device 40b can capture the image frame at the time the freeze command is received. The ultrasound imaging device 40b can also store the image frame at the time the freeze command is received.

[0109] The AI-based function button 173 can be a button used to activate one of a variety of AI-based functions. AI-based functions can refer to functions that use an artificial intelligence model. For example, AI-based functions may include functions such as: disease diagnosis, image segmentation, image improvement, fetal analysis, body marker setting, parameter measurement, and / or obtaining quantitative data on liver diseases.

[0110] Various input devices, such as buttons, trackballs, microswitches, and knobs, included in the control panel 165, can be configured as a GUI on the main display 140a and / or the secondary display 140b. Ultrasonic imaging devices 40a and 40b can be connected to the probe 20 to send and receive ultrasonic signals to and from the object.

[0111] Ultrasonic imaging devices 40a and 40b may include various types of output interfaces (such as speakers, LEDs, and vibration devices). For example, ultrasonic imaging devices 40a and 40b can output various information in the form of graphics, sound, or vibration through the output interfaces. Ultrasonic imaging devices 40a and 40b can also output various notifications or data through the output interfaces.

[0112] Reference Figure 2C and Figure 2D The ultrasound imaging devices 40c and 40d can be implemented in a portable form. Portable ultrasound imaging devices 40c and 40d may include, for example, smartphones, laptops, PDAs, or tablet PCs including probes and applications, but are not limited to these.

[0113] The ultrasound imaging device 40c may include a main body 41. (See reference...) Figure 2C The probe 20 can be connected to one side of the body 41 via a wire. For this purpose, the body 41 may include a connection terminal to which the cable connected to the probe 20 can be attached and detached. The probe 20 may include a cable with connection terminals capable of connecting to the body 41.

[0114] Reference Figure 2D The probe 20 can be wirelessly connected to the ultrasound imaging device 40d. The main body 41 may include an input / output interface (e.g., a touch screen) 145. Ultrasound images, various information, and / or a GUI can be displayed on the input / output interface 145.

[0115] The ultrasound imaging device 40d and the probe 20 can establish communication or pairing using short-range wireless communication. For example, the ultrasound imaging device 40d and the probe 20 can communicate using Bluetooth, BLE, Wi-Fi, or Wi-Fi Direct.

[0116] Ultrasonic imaging devices 40c and 40d can run programs or applications related to probe 20 to control probe 20 and output information about probe 20. Ultrasonic imaging devices 40c and 40d can perform probe-related operations while communicating with a predetermined computing device 30. Probe 20 can register with ultrasonic imaging devices 40c and 40d, or with the predetermined computing device 30. Ultrasonic imaging devices 40c and 40d can communicate with the registered probe 20 and perform probe-related operations.

[0117] Ultrasonic imaging devices 40c and 40d may also include various types of output interfaces (such as speakers, LEDs, and vibration devices). For example, ultrasonic imaging devices 40c and 40d can output various information in the form of graphics, sound, or vibration through the output interfaces. Ultrasonic imaging devices 40c and 40d can also output various notifications or data through the output interfaces.

[0118] The ultrasound imaging devices 40a, 40b, 40c, and 40d according to various embodiments can use artificial intelligence (AI) models to process ultrasound images. The ultrasound imaging devices 40a, 40b, 40c, and 40d can also obtain various information from the ultrasound images. The AI ​​model can be stored in the ultrasound imaging devices 40a, 40b, 40c, and 40d.

[0119] For example, ultrasound imaging devices 40a, 40b, 40c, and 40d can use AI models to perform image processing (such as generating ultrasound images, correcting ultrasound images, improving image quality, encoding, and / or decoding). Ultrasound imaging devices 40a, 40b, 40c, and 40d can also use AI models to perform quantitative data acquisition, reference line definition, anatomical information acquisition, lesion information acquisition, surface extraction, boundary definition, length measurement, area measurement, volume measurement, and / or annotation creation for diseases in various tissues from raw image frames and / or ultrasound images in various modes.

[0120] AI models can be implemented using various artificial neural network models or deep neural network models. Furthermore, various machine learning algorithms or deep learning algorithms can be used to learn and create AI models. For example, AI models can be implemented using models such as convolutional neural networks (CNNs), recurrent neural networks (RNNs), generative adversarial networks (GANs), and long short-term memory (LSTM).

[0121] As described above, the ultrasound imaging system 100 may include at least one of the memories 111 and 150. For example, each of the probe 20 and the ultrasound imaging device 40 may include one or more memories. At least one of the memories 111 and 150 may include volatile memories (e.g., S-RAM and D-RAM) and / or non-volatile memories (e.g., ROM and EPROM).

[0122] The ultrasound imaging system 100 may include at least one of processors 118, 120, and 130. For example, each of the probe 20 and the ultrasound imaging device 40 may include one or more processors. At least one of the processors 118, 120, and 130 may include various processing circuits and / or multiple processors. At least one of the processors 118, 120, and 130 may be configured to perform various functions individually and / or collectively in a distributed manner.

[0123] The ultrasound imaging system 100 may include multiple processors configured to perform various functions individually and / or jointly, or may include an integrated processor configured to perform all the various functions. For example, the processor 120 and image processor 130 of the ultrasound imaging device 40 described above may be configured as separate processors or as an integrated processor.

[0124] At least one of memories 111 and 150 may store various algorithms, instructions, commands, data, and / or artificial intelligence models for controlling probe 20 and processing raw ultrasound data. At least one of memories 111 and 150 may store an artificial intelligence model learned to perform artificial intelligence-based functions. At least one of memories 111 and 150 may store additional information about the subject input through input interfaces 109 and 170 (including at least one of subcutaneous fat thickness, body mass index (BMI), sex, age, and underlying diseases).

[0125] At least one of processors 118, 120, and 130 may include one or more of the following: a central processing unit (CPU), a graphics processing unit (GPU), an accelerated processing unit (APU), an integrated many-core (MIC), a digital signal processor (DSP), a neural processing unit (NPU), a hardware accelerator, and a machine learning accelerator.

[0126] At least one of processors 118, 120, and 130 can control probe 20 and ultrasound imaging device 40. At least one of processors 118, 120, and 130 can control the operation of ultrasound imaging system 100 by running various algorithms, instructions, commands, data, and / or artificial intelligence models stored in at least one of memories 111 and 150.

[0127] At least one of processors 118, 120, and 130 can perform operational control of probe 20 and / or processing of raw ultrasound data obtained by probe 20. At least one of processors 118, 120, and 130 can use the raw ultrasound data to generate raw image frames and / or ultrasound images of various modes, including various information.

[0128] At least one of processors 118, 120, and 130 can input raw image frames and / or ultrasound images of various modes into a learned AI model and obtain results output from the AI ​​model. For example, at least one of processors 118, 120, and 130 can use the AI ​​model to perform quantitative data acquisition, reference line definition, anatomical information acquisition, lesion information acquisition, surface extraction, boundary definition, length measurement, area measurement, volume measurement, and / or annotation creation for diseases of various tissues from raw image frames and / or ultrasound images of various modes.

[0129] Figure 3 This is a flowchart illustrating a control method for an ultrasound imaging system according to an embodiment.

[0130] Reference Figure 3At least one of the processors 120 and 130 of the ultrasound imaging system 100 can obtain raw ultrasound data (701) by processing ultrasound echo signals received through the probe 20 and reflected from the object. At least one of the processors 120 and 130 can generate raw ultrasound data by performing analog-to-digital conversion on the received signals obtained from the probe 20, taking into account the positions and focal points of multiple transducers, and summing the digitally converted received signals.

[0131] At least one of processors 120 and 130 can display a first ultrasound image (702) generated by processing raw ultrasound data on display 140. At least one of processors 120 and 130 can generate first ultrasound images in various modes by processing raw ultrasound data. The first ultrasound image can be a video comprising multiple image frames. The first ultrasound image can be displayed on at least one of the main display 140a and the secondary display 140b of the ultrasound imaging device 40.

[0132] For example, at least one of processors 120 and 130 can generate ultrasound images in amplitude mode (A-mode), brightness mode (B-mode), color Doppler mode, Doppler mode (D-mode), elastography mode (E-mode), motion mode (M-mode), and / or volume mode by processing raw ultrasound data. The method of processing the raw ultrasound data can vary depending on the mode of the ultrasound image.

[0133] In an embodiment, the first ultrasound image may be a "B-mode image". At least one of processors 120 and 130 may perform the following processing to generate a B-mode image from raw ultrasound data (i.e., RF data): 1) Envelope detection, used to extract amplitude information from raw ultrasound data (i.e., RF data); 2) Logarithmic compression, used to compress the dynamic range of the signal that has undergone envelope detection; 3) Time gain compensation, used to compensate for signal attenuation based on the depth of the signal that has undergone logarithmic compression; 4) Spatial interpolation, used to fill the blank spaces between scan lines of the signal that has undergone time gain compensation; 5) Post-processing, used to remove noise, emphasize edges, and enhance contrast on the signal that has undergone spatial interpolation; and 6) Convert the post-processed signal into an 8-bit black and white image.

[0134] At least one of processors 120 and 130 may receive a freeze command via input interface 109 of probe 20 or input interface 170 of ultrasound imaging device 40. At least one of processors 120 and 130 may also receive a command (703) for activating artificial intelligence-based functions via input interface 109 of probe 20 or input interface 170 of ultrasound imaging device 40.

[0135] For example, the processor 120 of the ultrasound imaging device 40 can receive a command to activate the AI-based function via an AI-based function button 173 located on the control panel 165 or displayed on the monitor 140. The user can input the command to activate the AI-based function by pressing or touching the AI-based function button 173. In response to receiving the command to activate the AI-based function, the processor 120 can use an AI model to perform processing of quantitative data on liver disease provided via the monitor 140.

[0136] At least one of the processors 120 and 130 of the ultrasound imaging system 100 can acquire an image frame (704) of the first ultrasound image in response to a freeze command received via input interfaces 109 and 170. For example, the processors 120 and 130 of the ultrasound imaging device 40 can receive a freeze command via a freeze button 172 located on a control panel 165 or displayed on a display 140. The user can input a freeze command by pressing or touching the freeze button 172. The processors 120 and 130 can pause the display of the first ultrasound image in response to receiving a freeze command and display the image frame at the time point when the freeze command is received on the display 140. The processors 120 and 130 can capture the image frame at the time point when the freeze command is received. The processors 120 and 130 can also store the image frame at the time point when the freeze command is received.

[0137] At least one of the processors 120 and 130 of the ultrasound imaging system 100 can generate a plurality of raw image frames (705), which correspond to the image frames of a first ultrasound image obtained by processing raw ultrasound data and include different characteristic information.

[0138] The generation of the original image frame can be performed through a different process than that used to generate the first ultrasound image. The original image frame may not be displayed on the display 140, and the process for generating the original image frame may not include at least one of time gain compensation, spatial interpolation, noise removal, post-processing, and image conversion.

[0139] For example, at least one of processors 120 and 130 can generate a first raw image frame including attenuation information of the ultrasonic echo signal (RF signal) and a second raw image frame including scattering information of the ultrasonic echo signal as a plurality of raw image frames.

[0140] Furthermore, at least one of processors 120 and 130 can also generate at least one of a third raw image frame including in-phase component information and quadrature component information of the raw ultrasound data, and a fourth raw image frame including spectral information of the raw ultrasound data, as a plurality of raw image frames. At least one of processors 120 and 130 can generate a third raw image frame for independently visualizing amplitude and phase information by decomposing the raw ultrasound data into I (in-phase) and Q (quadrature) components. At least one of processors 120 and 130 can generate a fourth raw image frame for observing characteristics of a specific frequency band by applying a Fourier transform to the raw ultrasound data.

[0141] Multiple raw image frames are not limited to the illustrated raw image frames. In addition to the illustrated raw image frames, the ultrasound imaging system 100 can also generate other raw image frames that include other information. The characteristic information included in the raw image frames can vary according to user input. The user can select the desired characteristic information of the raw image frames through input interfaces 109 and 170.

[0142] At least one of the processors 120 and 130 of the ultrasound imaging system 100 can input multiple raw image frames into an artificial intelligence model and obtain quantitative data on liver disease from the artificial intelligence model (706). For example, at least one of the processors 120 and 130 can input all or a portion of the raw image frames into the artificial intelligence model. The quantitative data on liver disease may include at least one of the liver fat fraction and the severity of hepatic steatosis.

[0143] At least one of processors 120 and 130 can identify a first region of interest (ROI) in an image frame of a first ultrasound image and determine a second region of interest corresponding to the first ROI in each of a plurality of original image frames. For example, at least one of processors 120 and 130 can obtain first coordinate information of the first ROI in an image frame of the first ultrasound image and convert the first coordinate information into second coordinate information in each of the plurality of original image frames. At least one of processors 120 and 130 can determine the second ROI based on the second coordinate information. At least one of processors 120 and 130 can extract local image frames corresponding to the second ROI in each of the plurality of original image frames. At least one of processors 120 and 130 can obtain quantitative data on liver disease in the first ROI by inputting multiple local image frames corresponding to each of the plurality of second ROIs into an artificial intelligence model.

[0144] Additionally, at least one of processors 118, 120, and 130 can obtain additional information about the subject (including at least one of subcutaneous fat thickness, body mass index (BMI), sex, age, and underlying diseases) from memory 150 or input interfaces 109 and 170. At least one of processors 118, 120, and 130 can input multiple raw image frames and additional information about the subject into an artificial intelligence model to obtain quantitative data on liver disease.

[0145] At least one of the processors 118, 120, and 130 of the ultrasound imaging system 100 can display a first ultrasound image and quantitative data about liver disease together on displays 112 and 140 (707). For example, at least one of the processors 118, 120, and 130 can display a second ultrasound image including quantitative data about liver disease on displays 112 and 140. At least one of the processors 118, 120, and 130 can display the quantitative data about liver disease in at least one of a first region of display 140 displaying the first and second ultrasound images and a second region of display 112 and 140 divided from the first region. At least one of the processors 118, 120, and 130 can also display a heatmap on displays 112 and 140 visualizing the distribution of the quantitative data about liver disease.

[0146] Therefore, the disclosed ultrasound imaging system 100 can generate raw images containing various information by processing raw ultrasound data, and obtain quantitative data on liver diseases by inputting the generated raw images into an artificial intelligence model. The ultrasound imaging system 100 can also reduce the data processing load of the artificial intelligence model by using raw images generated through preprocessing of raw ultrasound data as input. Furthermore, the ultrasound imaging system 100 can even provide users with information that might be impossible to examine in various existing ultrasound imaging modalities by using raw ultrasound data and an artificial intelligence model.

[0147] Figure 4 This is a diagram illustrating the process of performing the artificial intelligence-based functions of the ultrasound imaging system according to an embodiment.

[0148] Reference Figure 4The ultrasound imaging device 40, constituting the ultrasound imaging system 100, can control the probe 20 to send ultrasound signals to an object including the liver (TL). The ultrasound imaging device 40 can obtain raw ultrasound data 810 by processing the ultrasound echo signals (i.e., RF signals) received by the probe 20 and reflected from the object. The raw ultrasound data 810 can also be referred to as RF data. The processor 120 of the ultrasound imaging device 40 can generate the raw ultrasound data 810 by performing analog-to-digital conversion on the RF signals and summing the digitally converted received signals, taking into account the positions and focal points of multiple transducers.

[0149] The image processor 130 of the ultrasound imaging device 40 can generate a first ultrasound image 820 in various modes by processing the raw ultrasound data 810. The processor 120 of the ultrasound imaging device 40 can display a first region of interest (ROI) 1 on the first ultrasound image 820. For example, the image processor 130 can generate ultrasound images in amplitude mode (A mode), brightness mode (B mode), color Doppler mode, Doppler mode (D mode), elastography mode (E mode), motion mode (M mode), and / or volume mode by processing the raw ultrasound data. The mode of the first ultrasound image 820 can vary according to user input obtained through input interfaces 109 and 170. The first ultrasound image 820 can be a "B mode image".

[0150] Processor 120 can set a first region of interest (ROI) 1 based on user input obtained through input interfaces 109 and 170. Processor 120 can also use a learning model for detecting regions of interest to set the first ROI 1. For example, a first ultrasound image 820 can be input to the learning model, and the first ROI 1 can be set in the first ultrasound image 820 based on the output of the learning model. The first ROI 1 can be set in the entire region of the liver TL, or it can be set in at least a portion of the liver TL.

[0151] The processor 120 of the ultrasound imaging device 40 can acquire image frames of the first ultrasound image 820 in response to receiving a freeze command via input interfaces 109 and 170. The processors 120 and 130 can pause the display of the first ultrasound image 820 in response to receiving the freeze command, and display the image frame at the time when the freeze command was received via the display 140.

[0152] The processor 120 of the ultrasound imaging device 40 can receive commands to activate AI-based functions via AI-based function buttons 173 located on the control panel 165 or displayed on the monitor 140. In response to receiving the command to activate the AI-based functions, the processor 120 can use an AI model to perform processing of quantitative data on liver disease provided via the monitor 140.

[0153] The image processor 130 of the ultrasound imaging device 40 can generate multiple raw image frames 831, 832, 833, and 834 that correspond to the image frames of the acquired first ultrasound image 820 and include different characteristic information. The processor 120 can control the image processor 130 to generate multiple raw image frames 831, 832, 833, and 834.

[0154] For example, the image processor 130 may generate a first original image frame 831 including attenuation information of the ultrasonic echo signal (RF signal) and a second original image frame 832 including scattering information of the ultrasonic echo signal, as a plurality of original image frames 831, 832, 833 and 834.

[0155] The first raw image frame 833 may correspond to a power spectral density map (PSD map) for observing the attenuation characteristics of the RF signal in the depth direction. The image processor 130 may perform a Fourier transform on the raw ultrasound data 810 in the depth direction, calculate the power value based on frequency and depth, and generate PSD maps for multiple frequency bands. Because tissues (e.g., the liver) absorb more ultrasound energy the more fat they contain, power attenuation occurs more rapidly with depth. Different attenuation patterns can be detected for each frequency band of the RF signal.

[0156] A second raw image frame 832 can be obtained through Nakagami imaging processing for analyzing the scattering characteristics of the tissue. The image processor 130 can set a specific region (e.g., a region of interest) by obtaining the envelope data of the raw ultrasound data 810, and can generate an image including scattering information by analyzing the intensity distribution of the RF signal in the specific region and calculating Nakagami parameters. The fat content of the tissue (e.g., the liver) can be estimated from the values ​​of the Nakagami parameters.

[0157] Additionally, the image processor 130 can generate at least one of a third raw image frame 833 including in-phase component information and quadrature component information of the raw ultrasound data 810, and a fourth raw image frame 834 including spectral information of the raw ultrasound data 810. The image processor 130 can generate the third raw image frame 833 for independently visualizing amplitude and phase information by decomposing the raw ultrasound data into I (in-phase) and Q (quadrature) components. The image processor 130 can generate the fourth raw image frame 834 for observing characteristics of a specific frequency band by applying a Fourier transform to the raw ultrasound data.

[0158] The generation of original image frames 831, 832, 833, and 834 can be performed through a different process than that used to generate the first ultrasound image 820. The original image frames may not be displayed on the display 140, and the process for generating the original image frames may not include at least one of time gain compensation, spatial interpolation, noise removal, post-processing, and image conversion.

[0159] Multiple raw image frames are not limited to the illustrated raw image frames. In addition to the illustrated raw image frames, the ultrasound imaging system 100 may also generate other raw image frames that include other information.

[0160] The processor 120 of the ultrasound imaging device 40 can input multiple raw image frames 831, 832, 833, and 834 into the artificial intelligence model 860. The processor 120 can input all raw image frames 831, 832, 833, and 834, or a portion of the raw image frames 831, 832, 833, and 834, into the artificial intelligence model 860.

[0161] For example, image processor 120 can identify a first region of interest (ROI) 1 in the image frames of the first ultrasound image 820, and determine a second region of interest ROI 2 corresponding to the first ROI 1 in each of the plurality of original image frames 831, 832, 833, and 834. Processor 120 can obtain first coordinate information of the first ROI 1 in the image frames of the first ultrasound image 820, and convert the first coordinate information into second coordinate information in each of the plurality of original image frames 831, 832, 833, and 834. Processor 120 can determine the second ROI 2 in each of the plurality of original image frames 831, 832, 833, and 834 based on the second coordinate information. Processor 120 can extract local image frames corresponding to the second ROI 2 in each of the plurality of original image frames 831, 832, 833, and 834. The processor 120 can input multiple local image frames corresponding to each of the multiple second regions of interest ROI2 into the artificial intelligence model 860.

[0162] The processor 120 can obtain quantitative data on liver disease in the first region of interest (ROI) 1 of the first ultrasound image 820 by inputting all the original image frames 831, 832, 833, and 834, or multiple local image frames corresponding to each of the multiple second regions of interest (ROIs) 2, into the artificial intelligence model 860. For example, the quantitative data on liver disease may include at least one of the fat fraction of the liver and the severity of hepatic steatosis.

[0163] Multiple raw image frames 831, 832, 833, and 834 may be displayed without the display 140. The processor 120 may perform other image processing to generate a first ultrasound image 820 and multiple raw image frames 831, 832, 833, and 834.

[0164] Multiple first regions of interest (ROIs) 1 can be set at different locations of the liver TL in the first ultrasound image 820. In this case, a number of second regions of interest ROIs 2 can be set at different locations in each of the multiple original image frames 831, 832, 833, and 834. In other words, as the location and / or number of first regions of interest ROIs 1 set in the first ultrasound image 820 changes, the location and / or number of second regions of interest ROIs 2 can vary in each of the original image frames 831, 832, 833, and 834.

[0165] Additionally, the processor 120 of the ultrasound imaging device 40 can obtain additional information 870 about the subject (including at least one of subcutaneous fat thickness, body mass index (BMI), sex, age, and underlying diseases) from the memory 150 or input interfaces 109 and 170. The processor 120 can input multiple raw image frames 831, 832, 833, and 834, along with the additional information 870 about the subject, into the artificial intelligence model 860 to obtain quantitative data about liver disease. When the additional information 870 about the subject is input into the artificial intelligence model 860, more accurate quantitative data about liver disease can be obtained.

[0166] The processor 120 of the ultrasound imaging device 40 can control the display 140 to display a final ultrasound image 880 including quantitative data on liver disease. The final ultrasound image 880 may be a still image corresponding to an image frame of the first ultrasound image 820 acquired in response to a freeze command. The final ultrasound image 880 may be referred to as a second ultrasound image.

[0167] Quantitative data on liver disease can be displayed at various locations within the final ultrasound image 880. For example, quantitative data on liver disease can be displayed within the first region of interest (ROI) 1. Alternatively, quantitative data on liver disease can be displayed in an information display area 890, which is a region within the final ultrasound image 880 that differs from the first region of interest (ROI) 1.

[0168] Figure 5 An example is shown of displaying ultrasound images and quantitative data on liver disease on a monitor of an ultrasound imaging device.

[0169] Reference Figure 5The processor 120 of the ultrasound imaging device 40 can set multiple first regions of interest (ROIs) at different locations in the liver TL in the first ultrasound image 820. For example, the processor 120 can set each of the upper region 910, the middle region 920, and the lower region 930 of the liver TL as a first ROI. The processor 120 can automatically set multiple ROIs at various locations in the liver TL based on user input or without user input, using a learning model for detecting ROIs.

[0170] The processor 120 can control the display 140 to display a final ultrasound image 900 including quantitative data on disease in the upper, middle, and lower regions 910, 920, and 930 of the liver TL. For example, the ultrasound imaging device 40 can display a final ultrasound image 900 including the fat fraction in each of the upper, middle, and lower regions 910, 920, and 930 of the liver TL. The final ultrasound image 900 may be a still image corresponding to an image frame of a first ultrasound image 820 acquired in response to a freeze command. The final ultrasound image 900 may be referred to as a second ultrasound image.

[0171] For example, processor 120 may determine multiple second regions of interest (ROIs) corresponding to the upper region 910, middle region 920, and lower region 930 of the liver TL in each of the multiple original image frames 831, 832, 833, and 834. For example, processor 120 may set the second ROIs at positions corresponding to each of the upper region 910, middle region 920, and lower region 930 of the liver TL in each of the first original image frame 831 and the second original image frame 832.

[0172] Processor 120 can extract local image frames corresponding to each of the upper region 910, middle region 920, and lower region 930 of the liver TL in each of the multiple raw image frames 831, 832, 833, and 834. Processor 120 can input the multiple local image frames into artificial intelligence model 860.

[0173] The artificial intelligence model 860 can output quantitative data on the disease in each of the upper, middle, and lower regions 910 and 920 of the liver TL. The processor 120 of the ultrasound imaging device 40 can control the display 140 to display the quantitative data on the disease in each of the upper, middle, and lower regions 930 of the liver TL.

[0174] For example, the fat fraction of the upper region 910, middle region 920, and lower region 930 of the liver TL within the final ultrasound image 900 can be displayed separately in the upper region 910, middle region 920, and lower region 930, respectively. Furthermore, the average fat fraction of the upper region 910, middle region 920, and lower region 930 of the liver TL can be displayed in the information display area 940 of the final ultrasound image 900.

[0175] In addition, the ultrasound imaging device 40 can display the severity of hepatic steatosis in each of the upper region 910, middle region 920 and lower region 930 of the liver TL.

[0176] Figure 6 An example is shown of displaying ultrasound images and quantitative data on liver disease on a monitor of an ultrasound imaging device.

[0177] Reference Figure 6 The processor 120 of the ultrasound imaging device 40 can set the entire region 1010 of the liver TL in the first ultrasound image 820 as the first region of interest. The processor 120 can set the entire region 1010 of the liver TL as the first region of interest based on user input or automatically without user input.

[0178] Processor 120 can determine a second region of interest corresponding to the entire region 1010 of the liver TL in each of the plurality of raw image frames 831, 832, 833, and 834. Processor 120 can extract local image frames corresponding to the entire region 1010 of the liver TL in each of the plurality of raw image frames 831, 832, 833, and 834. Processor 120 can input the plurality of local image frames into artificial intelligence model 860. Artificial intelligence model 860 can output quantitative data on the disease in the entire region 1010 of the liver TL.

[0179] Processor 120 can control display 140 to display a final ultrasound image 1000 including quantitative data on the disease in the entire region 1010 of the liver TL. The final ultrasound image 1000 may be a still image corresponding to an image frame of a first ultrasound image 820 acquired in response to a freeze command. The final ultrasound image 1000 may be referred to as a second ultrasound image. For example, ultrasound imaging device 40 may display the average fat fraction of the entire region 1010 of the liver TL and the average RF signal power in the depth direction along scan line 1020.

[0180] Within the final ultrasound image 1000, the average fat fraction of the entire region 1010 of the liver TL can be displayed in the first information display area 1030, and the average RF signal power can be displayed in the second information display area 1040.

[0181] In addition, the ultrasound imaging device 40 can display the severity of hepatic steatosis in the entire region 1010 of the liver TL.

[0182] Figure 7 An example is shown of displaying ultrasound images and quantitative data on liver disease on a monitor of an ultrasound imaging device.

[0183] Reference Figure 7 The processor 120 of the ultrasound imaging device 40 can control the display 140 to simultaneously display a first ultrasound image 820 and a final ultrasound image 880 including quantitative data on liver disease. The processor 120 can simultaneously display the first ultrasound image 820 and the final ultrasound image 880 in a first area 1110 of the display 140. The final ultrasound image 880 may be referred to as the second ultrasound image.

[0184] Additionally, the processor 120 can display quantitative data about liver disease in a second region 1120 of the display 140, separate from the first region 1110. The ultrasound imaging device 40 can store the final ultrasound image 880 and the quantitative data about liver disease. Even when the freeze command for displaying the first ultrasound image 820 is released, the previously obtained quantitative data about liver disease can continue to be displayed in the second region 1120 of the display 140.

[0185] Figure 8 An example is shown of displaying ultrasound images and quantitative data on liver disease on a monitor of an ultrasound imaging device.

[0186] Reference Figure 8 The processor 120 of the ultrasound imaging device 40 can control the display 140 to display a final ultrasound image 880 including quantitative data about liver disease and a heat map 1210 visualizing the distribution of the quantitative data about liver disease. For example, the processor 120 can simultaneously display the final ultrasound image 880 and the heat map 1210 in a first region 1110 of the display 140. The heat map 1210 can be displayed as an overlay on either the first ultrasound image 820 or the final ultrasound image 880.

[0187] The aspects of the screen provided by the display 140 of the ultrasound imaging device 40 are not limited to those illustrated. Ultrasound images and quantitative data on liver diseases can be presented on various screens according to the design.

[0188] As can be readily understood from the above, the disclosed ultrasound imaging system and its control method can utilize artificial intelligence models to obtain quantitative data on liver diseases from raw ultrasound data and display the obtained data together with ultrasound images.

[0189] The disclosed ultrasound imaging system and its control method can generate raw images containing various information by processing raw ultrasound data, and obtain quantitative data on liver diseases by inputting the generated raw images into an artificial intelligence model.

[0190] By using raw images generated through preprocessing of raw ultrasound data as input to an artificial intelligence model, the disclosed ultrasound imaging system and its control method can reduce the amount of data processing required by the artificial intelligence model and even provide users with information that cannot be examined in various existing ultrasound imaging modes.

[0191] Furthermore, the disclosed ultrasound imaging system and its control method minimize user intervention when using artificial intelligence models to obtain data, and provide the data along with ultrasound images familiar to the user. This ensures the consistency of the acquired data and improves user convenience.

[0192] The disclosed embodiments can be implemented in the form of a recording medium storing computer-executable instructions. The instructions can be stored as program code, and when executed by a processor, a program module can be created to perform the operations of the disclosed embodiments. The recording medium can be implemented as a computer-readable recording medium.

[0193] Computer-readable recording media include any type of recording medium in which computer-readable instructions are stored. For example, recording media may include read-only memory (ROM), random access memory (RAM), magnetic tape, magnetic disk, flash memory, optical data storage devices, etc.

[0194] Additionally, the device-readable recording medium may be provided in the form of a non-transitory storage medium. Here, "non-transitory storage medium" simply means that it is a tangible device and does not contain signals (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently in the storage medium and cases where data is temporarily stored. For example, a "non-transitory storage medium" may include a buffer that temporarily stores data.

[0195] According to embodiments, methods according to the various embodiments disclosed in this document may be included and set in a computer program product. The computer program product is a commodity and can be traded between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable recording medium (e.g., an optical disc read-only memory (CD-ROM)) or distributed online (e.g., downloaded or uploaded) between two user devices (e.g., smartphones) via an app store (e.g., the Play Store™). In the case of online distribution, at least a portion of the computer program product (e.g., a downloadable application) may be at least temporarily stored or temporarily created in a machine-readable recording medium (such as the memory of a manufacturer's server, app store server, and relay server).

[0196] The embodiments disclosed with reference to the accompanying drawings have been described above. Those skilled in the art will understand that various changes in form and detail may be made therein without departing from the spirit and scope of this disclosure as defined by the appended claims. The disclosed embodiments are illustrative and should not be construed as restrictive.

Claims

1. An ultrasound imaging system, comprising: The probe is configured to send ultrasound signals to an object including the liver and receive ultrasound echo signals reflected from the object; The display is configured to show ultrasound images; The input interface is configured to obtain user input. The memory is configured to store artificial intelligence models; and At least one processor, Wherein, the at least one processor is configured to: By processing the ultrasonic echo signal, raw ultrasonic data is obtained; A first ultrasound image generated by processing the raw ultrasound data is displayed on the monitor. In response to receiving a freeze command through the input interface, an image frame of the first ultrasound image is obtained; By processing the raw ultrasound data, multiple raw image frames are generated that correspond to the obtained image frames and include different characteristic information. By inputting the multiple raw image frames into the artificial intelligence model, quantitative data on liver disease is obtained from the artificial intelligence model; and A second ultrasound image, including the quantitative data on the liver disease, is displayed on the monitor.

2. The ultrasound imaging system as described in claim 1, wherein, The at least one processor is configured to: Generate a B-mode image as the first ultrasound image; and A first original image frame including attenuation information of the ultrasonic echo signal and a second original image frame including scattering information of the ultrasonic echo signal are generated as the plurality of original image frames.

3. The ultrasound imaging system as described in claim 2, wherein, The at least one processor is configured to: Further, at least one of the following is generated as the plurality of original image frames: a third original image frame including in-phase component information and quadrature component information of the original ultrasound data, and a fourth original image frame including spectral information of the original ultrasound data.

4. The ultrasound imaging system as described in claim 1, wherein, The at least one processor is configured to: Identify a first region of interest in the image frame of the first ultrasound image; In each of the plurality of original image frames, a second region of interest corresponding to the first region of interest is determined; Extract the local image frame corresponding to the second region of interest from each of the plurality of original image frames; and By inputting multiple local image frames corresponding to each of the multiple second regions of interest into the artificial intelligence model, quantitative data on the liver disease in the first region of interest are obtained.

5. The ultrasound imaging system as described in claim 4, wherein, The at least one processor is configured to: Obtain the first coordinate information of the first region of interest in the image frame of the first ultrasound image; The first coordinate information is converted into second coordinate information in each of the plurality of original image frames; and Based on the second coordinate information, the second region of interest is determined.

6. The ultrasound imaging system as claimed in claim 1, wherein, The at least one processor is configured to: Obtain at least one of the fat fraction of the liver and the severity of hepatic steatosis as quantitative data regarding the liver disease.

7. The ultrasound imaging system as claimed in claim 1, wherein, The at least one processor is configured to: Additional information about the object is obtained through the input interface, including at least one of subcutaneous fat thickness, body mass index (BMI), gender, age, and underlying medical conditions; and The multiple original image frames and the additional information about the object are input into the artificial intelligence model.

8. The ultrasound imaging system as claimed in claim 1, wherein, The at least one processor is configured to: The quantitative data regarding the liver disease are displayed in at least one of a first region of the display showing the first ultrasound image and the second ultrasound image, and a second region of the display separate from the first region.

9. The ultrasound imaging system as claimed in claim 1, wherein, The at least one processor is configured to: A heatmap is further displayed on the monitor, which visualizes the distribution of the quantitative data regarding the liver disease.

10. The ultrasound imaging system of claim 1, wherein, The at least one processor is configured to: The second ultrasound image is displayed on the monitor based on a command obtained through the input interface to activate the AI-based function.

11. A control method for an ultrasound imaging system, the ultrasound imaging system comprising a probe, an input interface, a display, and at least one processor, the control method comprising a control method executed by the at least one processor. in, The control method includes: The probe is controlled to send ultrasound signals to an object including the liver and to receive ultrasound echo signals reflected from the object; Raw ultrasound data is obtained by processing the ultrasound echo signal; A first ultrasound image generated by processing the raw ultrasound data is displayed on the monitor. In response to receiving a freeze command through the input interface, an image frame of the first ultrasound image is obtained; By processing the raw ultrasound data, multiple raw image frames are generated that correspond to the obtained image frames and include different characteristic information. By inputting the multiple raw image frames into an artificial intelligence model, quantitative data on liver disease are obtained from the artificial intelligence model; and A second ultrasound image, including the quantitative data on the liver disease, is displayed on the monitor.

12. The control method as described in claim 11, wherein, The first ultrasound image corresponds to a B-mode image, and The steps for generating the plurality of original image frames include: A first original image frame including attenuation information of the ultrasonic echo signal and a second original image frame including scattering information of the ultrasonic echo signal are generated.

13. The control method as described in claim 12, wherein, The step of generating the plurality of original image frames further includes: At least one of the following is generated as the plurality of original image frames: a third original image frame including in-phase component information and quadrature component information of the original ultrasound data, and a fourth original image frame including spectral information of the original ultrasound data.

14. The control method as described in claim 11, wherein, The steps for obtaining the quantitative data on the liver disease include: Identify a first region of interest in the image frame of the first ultrasound image; In each of the plurality of original image frames, a second region of interest corresponding to the first region of interest is determined; Extract the local image frame corresponding to the second region of interest from each of the plurality of original image frames; and By inputting multiple local image frames corresponding to each of the multiple second regions of interest into the artificial intelligence model, quantitative data on the liver disease in the first region of interest are obtained.

15. The control method as described in claim 14, wherein, The steps for determining the second region of interest include: Obtain the first coordinate information of the first region of interest in the image frame of the first ultrasound image; The first coordinate information is converted into second coordinate information in each of the plurality of original image frames; and Based on the second coordinate information, the second region of interest is determined.

16. The control method as described in claim 11, wherein, The quantitative data regarding the liver disease include at least one of the fat fraction of the liver and the severity of hepatic steatosis.

17. The control method of claim 11, further comprising: Additional information about the object is obtained through the input interface, including at least one of subcutaneous fat thickness, body mass index (BMI), gender, age, and underlying diseases; as well as The multiple original image frames and the additional information about the object are input into the artificial intelligence model.

18. The control method as described in claim 11, wherein, The steps for displaying the second ultrasound image include: The quantitative data regarding the liver disease are displayed in at least one of a first region of the display showing the first ultrasound image and the second ultrasound image, and a second region of the display separate from the first region.

19. The control method of claim 11, further comprising: A heatmap is displayed on the monitor, which visualizes the distribution of the quantitative data regarding the liver disease.

20. The control method as described in claim 11, wherein, Based on the command obtained through the input interface for activating the AI-based function, the step of displaying the second ultrasound image is executed.

Images