Ultrasound diagnosis apparatus and method of controlling same

Abstract

An ultrasound diagnosis apparatus according to one aspect of the invention disclosed herein includes: a display for displaying a wireless ultrasound image; a control panel for receiving a user's inputs for controlling the display of the ultrasound image; and a processor for performing a first operation and a second operation according to the user's inputs input through the control panel. The processor determines a response time on the basis of the number of frames present between ultrasound image frames displayed on the display when the user inputs related to the first operation and the second operation are received, acquires usage pattern information by inputting the response time to a machine learning model, performs response time correction of the ultrasound image on the basis of the usage pattern information, and outputs the ultrasound image frame with the corrected response time through the display.

Description

Ultrasonic diagnostic device and control method thereof

[0001] The disclosed invention relates to an ultrasonic diagnostic device for obtaining an ultrasonic image and a control method thereof.

[0002] In recent years, various medical imaging devices have been widely used in the medical field to obtain visual information about human tissues for the early diagnosis of various diseases or for surgical procedures. Representative examples of these medical imaging devices include ultrasound diagnostic devices, CT devices, and MRI devices.

[0003] An ultrasound imaging device is a device that non-invasively acquires at least one image of a part inside the object (e.g., soft tissue or blood flow) by irradiating an ultrasound signal generated from a transducer of a probe to the object and receiving information on the signal reflected from the object. In particular, ultrasound diagnostic devices are used for medical purposes such as observing the inside of an object, detecting foreign substances, and measuring injuries. These ultrasound diagnostic devices have the advantages of being highly stable compared to diagnostic devices that use X-rays, being able to display images in real time, and being safe because there is no radiation exposure, and are therefore widely used along with other imaging diagnostic devices.

[0004] While viewing an ultrasound image, a user can pause the image by pressing a button for an action, such as Freeze, Save, or Measure, at the desired frame for measurement. However, due to human reaction time—the time it takes for the body to react after perceiving an external stimulus (e.g., a visual stimulus)—there may be differences between the desired frame for measurement and the frame obtained by pressing the corresponding action button.

[0005] Therefore, there is a problem that the user must perform additional operations to obtain the desired frame.

[0006] One aspect of the disclosed invention provides an ultrasound diagnostic device and a control method thereof that learns a user's usage pattern through a machine learning model and automatically corrects the reaction time (human reaction time) when the user presses a button for a predetermined action based on the learned usage pattern information, thereby increasing user convenience and shortening examination time.

[0007] An ultrasound diagnostic device according to one aspect of the disclosed invention comprises: a display for displaying an ultrasound image; a control panel for receiving a user's input for controlling the display of the ultrasound image; and a processor for performing a first operation and a second operation according to a user's input inputted through the control panel; wherein the processor determines a response time based on the number of frames existing between ultrasound image frames displayed on the display when user inputs regarding the first operation and the second operation are received, inputs the response time into a machine learning model to obtain usage pattern information, and performs response time correction of the ultrasound image based on the usage pattern information, thereby outputting an ultrasound image frame with the response time corrected through the display.

[0008] A method for controlling an ultrasound diagnostic device, comprising: a display for displaying an ultrasound image according to one aspect of the disclosed invention; and a control panel for receiving a user's input for controlling the display of the ultrasound image, the method comprising: receiving a user input regarding a series of operations including a first operation and a second operation through the control panel; determining a response time based on the number of frames existing between ultrasound image frames displayed on the display when the user input regarding the first operation and the second operation is received; inputting the response time into a machine learning model to obtain usage pattern information; performing a response time correction of the ultrasound image based on the usage pattern information; and outputting an ultrasound image frame with the response time corrected through the display.

[0009] According to one aspect of the disclosed invention, the user can obtain a desired frame without performing a separate operation, thereby increasing user convenience.

[0010] In addition, according to one aspect of the disclosed invention, the user can obtain a desired frame without performing a separate operation, so the examination time is shortened and the examination flow is optimized, so that the examinee can also receive a quick and accurate examination.

[0011] Additionally, according to one aspect of the disclosed invention, it is possible to obtain accurate video frames by compensating for user-specific customized reaction times.

[0012] However, the effects that can be achieved by the ultrasonic diagnostic device and the control method thereof of the present disclosure are not limited to those mentioned above, and other effects not mentioned will be clearly understood by a person having ordinary skill in the art to which the present invention pertains from the description below.

[0013] Figure 1 illustrates a control block diagram of an ultrasound imaging system (100) when the probe (20) is a wired probe or a hybrid probe.

[0014] Figure 2 illustrates a control block diagram of an ultrasound imaging system (100) when the probe (20) is a wireless probe or hybrid probe.

[0015] FIG. 3 and FIG. 6 are drawings showing an ultrasound imaging system (100) according to one embodiment.

[0016] FIG. 7 is a drawing for explaining buttons constituting a control panel (165) of an ultrasonic diagnostic device (40) for receiving user input according to one embodiment.

[0017] FIG. 8 is a diagram illustrating an example of an input interface (170) displayed on an auxiliary display (122) based on input received from a user regarding a target action according to one embodiment.

[0018] FIG. 9 is a drawing showing an example of an ultrasound image displayed on the main display (121) based on input received from a user regarding a target action.

[0019] FIG. 10 is a diagram illustrating an example of an input interface (170) displayed on an auxiliary display (122) based on input received from a user regarding a caliper operation according to one embodiment.

[0020] FIG. 11 is a drawing showing an example of an ultrasound image displayed on the main display (121) based on input received from a user regarding a caliper operation.

[0021] FIG. 12 is a drawing showing an example of a screen displayed on the main display (121) based on input received from a user regarding an annotation operation.

[0022] FIG. 13 is an example of a screen displayed on a display (140) after rewinding an ultrasound image by operating a trackball (167) when the ultrasound image frame at the time when a user input regarding a first action is received is not the ultrasound image frame desired by the user, according to one embodiment.

[0023] FIG. 14 is an example of a screen displayed on a display (140) at a time when a user input for a second operation is received after the ultrasound image frame at the time when the user input is received is not the ultrasound image frame desired by the user, after the trackball (167) is operated to rewind the ultrasound image, according to one embodiment.

[0024] Figure 15 is a diagram for explaining obtaining user usage pattern information through a machine learning model.

[0025] FIG. 16 is a flowchart illustrating automatic correction of an ultrasound image based on acquired usage pattern information according to one embodiment.

[0026] FIG. 17 is a flowchart illustrating automatic correction of an ultrasound image based on acquired usage pattern information according to another embodiment.

[0027]

[0028] This disclosure clarifies the scope of the claims of the present disclosure and explains the principles of the embodiments of the present disclosure and discloses embodiments thereof so that those skilled in the art can practice the embodiments of the present disclosure. The embodiments of the present disclosure may be implemented in various forms.

[0029] Throughout the specification, the same reference numerals denote the same components. This specification does not describe all elements of the embodiments, and any content that is general in the technical field to which the present invention pertains or that overlaps between the embodiments is omitted. The term 'module' or 'unit' used in the specification may be implemented by one or a combination of two or more of software, hardware, or firmware, and depending on the embodiments, multiple 'modules' or 'units' may be implemented as a single element, or a single 'module' or 'unit' may include multiple elements.

[0030] The singular form of a noun corresponding to an item may include one or more of said items, unless the relevant context clearly indicates otherwise.

[0031] In this disclosure, each of the phrases "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" may include any one of the items listed together in the corresponding phrase, or all possible combinations thereof.

[0032] The term "and / or" includes any combination of a plurality of related described elements or any one of a plurality of related described elements.

[0033] Terms such as "first," "second," or "first" or "second" may be used simply to distinguish one component from another and do not qualify the components in any other respect (e.g., importance or order).

[0034] In addition, terms such as 'front', 'rear', 'top', 'bottom', 'side', 'left', 'right', 'upper', and 'lower' used in the present disclosure are defined based on the drawings, and the shape and position of each component are not limited by these terms.

[0035] Terms such as "include" or "have" are intended to specify the presence of a feature, number, step, operation, component, part or combination thereof described in the present disclosure, but do not preclude the presence or addition of one or more other features, numbers, steps, operations, components, parts or combinations thereof.

[0036] When a component is said to be “connected,” “coupled,” “supported,” or “in contact with” another component, this includes not only cases where the components are directly connected, coupled, supported, or in contact, but also cases where the components are indirectly connected, coupled, supported, or in contact through a third component.

[0037] When we say that a component is "on" another component, this includes not only cases where the component is in contact with the other component, but also cases where there is another component between the two components.

[0038] Hereinafter, an ultrasonic device according to various embodiments will be specifically described with reference to the attached drawings. In the description with reference to the attached drawings, identical or corresponding components are assigned similar drawing numbers, and redundant descriptions thereof may be omitted.

[0039] In the present disclosure, the image may include a medical image acquired by a medical imaging device such as a magnetic resonance imaging (MRI) device, a computed tomography (CT) device, an ultrasound imaging device, or an X-ray imaging device.

[0040] In the present disclosure, an 'object' refers to a subject to be photographed, and may include a human, an animal, or a part thereof. For example, the object may include a part of the body (such as an organ or system) or a phantom.

[0041] Throughout this disclosure, the term “ultrasonic image” means an image of an object that is processed based on an ultrasonic signal transmitted to the object and reflected from the object.

[0042] Hereinafter, embodiments are described in detail with reference to the drawings.

[0043] Referring to FIGS. 1 and 2, the ultrasonic imaging system (100) may include a probe (20) and an ultrasonic diagnostic device (40).

[0044] The ultrasonic diagnostic device (40) may be implemented in a cart-type as well as a portable type. Examples of portable ultrasonic imaging devices include, but are not limited to, a smart phone, laptop computer, PDA, tablet PC, etc., which include a probe and an application.

[0045] The probe (20) may include a wired probe that is connected to the ultrasonic diagnostic device (40) by wire and communicates with the ultrasonic diagnostic device (40) by wire, a wireless probe that is connected wirelessly to the ultrasonic diagnostic device (40) and communicates wirelessly with the ultrasonic diagnostic device (40), and / or a hybrid probe that is connected wired or wirelessly to the ultrasonic diagnostic device (40) and communicates wired or wirelessly with the ultrasonic diagnostic device (40).

[0046] According to various embodiments, as illustrated in FIG. 1, the ultrasonic diagnostic device (40) may include an ultrasonic transceiver module (110), and as illustrated in FIG. 2, the probe (20) may include an ultrasonic transceiver module (110). According to various embodiments, it is also possible for both the ultrasonic diagnostic device (40) and the probe (20) to include an ultrasonic transceiver module (110).

[0047] According to various embodiments, the probe (20) may further include an image processor (130), a display (140), and / or an input interface (170).

[0048] Accordingly, the description regarding the ultrasound transmission / reception module (110), the image processor (130), the display (140), and / or the input interface (170) included in the ultrasound diagnostic device (40) may also be applied to the ultrasound transmission / reception module (110), the image processor (130), the display (140), and / or the input interface (170) included in the probe (20).

[0049] Figure 1 illustrates a control block diagram of an ultrasound imaging system (100) when the probe (20) is a wired probe or a hybrid probe.

[0050] The probe (20) may include a plurality of transducers. The plurality of transducers may transmit an ultrasonic signal to the target object (10) according to a transmission signal applied from the transmission module (113). The plurality of transducers may receive an ultrasonic signal (echo signal) reflected from the target object (10) and form a reception signal. In addition, the probe (20) may be implemented as an integral part with the ultrasonic diagnostic device (40), or may be implemented as a separate part connected to the ultrasonic diagnostic device (40) by a wire. In addition, the ultrasonic diagnostic device (40) may be connected to one or a plurality of probes (20) depending on the implementation form.

[0051] If the probe (20) is a wired probe or a hybrid probe, it may include a cable and connector that can be connected to the connector of the ultrasonic diagnostic device (40).

[0052] The probe (20) according to one embodiment may be implemented as a two-dimensional probe. When the probe (20) is implemented as a two-dimensional probe, a plurality of transducers included in the probe (20) may be arranged in two dimensions to form a two-dimensional transducer array.

[0053] For example, a two-dimensional transducer array may be in the form of a plurality of sub-arrays including a plurality of transducers arranged in a first direction in a second direction different from the first direction.

[0054] Additionally, when the probe (20) according to one embodiment is implemented as a two-dimensional probe, the ultrasound transmission / reception module (110) may include an analog beamformer and a digital beamformer. Alternatively, the two-dimensional probe may include one of the analog beamformer and the digital beamformer, or both, depending on the implementation form.

[0055] The processor (120) controls the transmission module (113) to form a transmission signal to be applied to each transducer (115) by considering the positions and focus points of the plurality of transducers included in the probe (20).

[0056] The processor (120) can control the receiving module (115) to generate ultrasonic data by converting an analog-to-digital reception signal received from the probe (20) and adding the digitally converted reception signals by taking into account the positions and focus points of a plurality of transducers.

[0057] When the probe (20) is implemented as a two-dimensional probe, the processor (120) can calculate a time delay value for digital beamforming for each sub-array of a plurality of sub-arrays included in the two-dimensional transducer array. In addition, the processor (120) can calculate a time delay value for analog beamforming for each transducer included in any one of the plurality of sub-arrays. The processor (120) can control the analog beamformer and the digital beamformer to form a transmission signal to be applied to each of the plurality of transducers according to the time delay value for analog beamforming and the time delay value for digital beamforming. In addition, the processor (120) can control the analog beamformer to add up signals received from the plurality of transducers for each sub-array according to the time delay value for analog beamforming. In addition, the processor (120) can control the ultrasound transmission / reception module (110) to convert the signal added up for each sub-array into analog-to-digital. Additionally, the processor (120) can control the digital beamformer to generate ultrasound data by adding digitally converted signals according to a time delay value for digital beamforming.

[0058] The image processor (130) uses the generated ultrasound data to create an ultrasound image.

[0059] The display (140) can display the generated ultrasound image and various information processed by the ultrasound diagnostic device (40) and / or the probe (20). The probe (20) and / or the ultrasound diagnostic device (40) may include one or more displays (140) depending on the implementation type. In addition, the display (140) may include a touch panel or a touch screen.

[0060] The processor (120) can control the overall operation of the ultrasonic diagnostic device (40) and the signal flow between internal components of the ultrasonic diagnostic device (40). The processor (120) can perform or control various operations or functions of the ultrasonic diagnostic device (40) by executing programs or instructions stored in the memory (150). In addition, the processor (120) can receive a control signal from an input interface (170) or an external device and control the operation of the ultrasonic diagnostic device (40).

[0061] The ultrasonic diagnostic device (40) includes a communication module (160) and can be connected to an external device (e.g., a probe (20), a server, a medical device, a portable device (smartphone, tablet PC, wearable device, etc.)) through the communication module (160).

[0062] The communication module (160) may include one or more components that enable communication with an external device, and may include, for example, at least one of a short-range communication module, a wired communication module, and a wireless communication module.

[0063] The communication module (160) can also receive control signals and data from an external device and transmit the received control signals to the processor (120) so that the processor (120) can control the ultrasonic diagnostic device (40) according to the received control signals.

[0064] Alternatively, it is also possible to control the external device according to the control signal of the processor by having the processor (120) transmit a control signal to the external device through the communication module (160).

[0065] For example, the external device can process data from the external device according to control signals from the processor received through the communication module.

[0066] An external device may be installed with a program capable of controlling the ultrasonic diagnostic device (40), and this program may include commands for performing part or all of the operations of the processor (120).

[0067] The program may be pre-installed on an external device, or the user of the external device may download and install the program from a server providing the application. The server providing the application may include a storage medium containing the program.

[0068] The memory (150) can store various data or programs for driving and controlling the ultrasonic diagnostic device (40), input / output ultrasonic data, ultrasonic images, etc.

[0069] The input interface (170) can receive user input for controlling the ultrasonic diagnostic device (40). For example, the user input may include, but is not limited to, input for operating a button, keypad, mouse, trackball, jog switch, knob, etc., input for touching a touchpad or touch screen, voice input, motion input, biometric information input (e.g., iris recognition, fingerprint recognition, etc.), etc.

[0070] FIG. 2 illustrates a control block diagram of an ultrasound imaging system (100) when the probe (20) is a wireless probe or a hybrid probe.

[0071] According to various embodiments, the ultrasonic diagnostic device (40) illustrated in FIG. 2 may be replaced with the ultrasonic diagnostic device (40) described with reference to FIG. 1.

[0072] According to various embodiments, the probe (20) illustrated in FIG. 1 may of course be replaced with the probe (20) to be described with reference to FIG. 2.

[0073] The probe (20) may include a transmitting module (113), a battery (114), a transducer (115), a charging module (116), a receiving module (117), a processor (118), and a communication module (119). In FIG. 1B, the probe (20) is illustrated as including both the transmitting module (113) and the receiving module (117), but depending on the implementation form, the probe (20) may include only a part of the configuration of the transmitting module (113) and the receiving module (117), and a part of the configuration of the transmitting module (113) and the receiving module (117) may be included in the ultrasonic diagnostic device (40). Alternatively, the probe (20) may further include an image processor (130).

[0074] The transducer (115) may include a plurality of transducers. The plurality of transducers may transmit ultrasonic signals to the target object (10) according to a transmission signal applied from the transmission module (113). The plurality of transducers may receive ultrasonic signals reflected from the target object (10) and form a reception signal.

[0075] The charging module (116) can charge the battery (114). The charging module (116) can receive power from an external source. The charging module (116) can receive power wirelessly. However, this is not limited to this, and the charging module (116) can also receive power through a wire. The charging module (116) can transmit the received power to the battery (114).

[0076] The processor (118) controls the transmission module (113) to form a transmission signal to be applied to each of the plurality of transducers by considering the positions and focus points of the plurality of transducers.

[0077] The processor (118) controls the receiving module (117) to generate ultrasound data by converting an analog-to-digital reception signal received from a transducer (115) and adding the digitally converted reception signals by taking into account the positions and focal points of a plurality of transducers. Alternatively, when the probe (20) includes an image processor (130), the generated ultrasound data can be used to generate an ultrasound image.

[0078] When the probe (20) is implemented as a two-dimensional probe, the processor (118) can calculate a time delay value for digital beamforming for each sub-array of a plurality of sub-arrays included in the two-dimensional transducer array. In addition, the processor (118) can calculate a time delay value for analog beamforming for each transducer included in any one of the plurality of sub-arrays. The processor (118) can control the analog beamformer and the digital beamformer to form a transmission signal to be applied to each of the plurality of transducers according to the time delay value for analog beamforming and the time delay value for digital beamforming. In addition, the processor (118) can control the analog beamformer to add up signals received from the plurality of transducers for each sub-array according to the time delay value for analog beamforming. In addition, the processor (118) can control the ultrasound transmission / reception module (110) to convert the signal added for each sub-array into analog-to-digital. Additionally, the processor (118) can control the digital beamformer to generate ultrasound data by adding digitally converted signals according to a time delay value for digital beamforming.

[0079] The processor (118) can control the overall operation of the probe (20) and the signal flow between the internal components of the probe (20). The processor (118) can perform or control various operations or functions of the probe (20) by executing programs or instructions stored in the memory (111). In addition, the processor (118) can receive a control signal from the input interface (170) of the probe (20) or an external device (e.g., an ultrasonic diagnostic device (40)) and control the operation of the probe (20).

[0080] The communication module (119) can wirelessly transmit generated ultrasound data or ultrasound images to an ultrasound diagnostic device (40) via a wireless network. In addition, the communication module (119) can receive control signals and data from the ultrasound diagnostic device (40).

[0081] The ultrasonic diagnostic device (40) can receive ultrasonic data or ultrasonic images from the probe (20).

[0082] In one embodiment, when the probe (20) includes an image processor (130) capable of generating an ultrasound image using ultrasound data, the probe (20) can transmit the ultrasound data and / or the ultrasound image generated by the image processor (130) to the ultrasound diagnostic device (40).

[0083] In one embodiment, when the probe (20) does not include an image processor (130) capable of generating an ultrasound image using ultrasound data, the probe (20) may transmit the ultrasound data to the ultrasound diagnostic device (40). The ultrasound data may include ultrasound raw data, and the ultrasound image may mean ultrasound image data.

[0084] The ultrasonic diagnostic device (40) may include a processor (120), an image processor (130), a display (140), a memory (150), a communication module (160), and an input interface (170).

[0085] The image processor (130) generates an ultrasound image using ultrasound data received from the probe (20).

[0086] The display (140) can display an ultrasound image received from the probe (20), an ultrasound image generated by processing ultrasound data received from the probe (20), and various information processed in the ultrasound imaging system (100). The ultrasound diagnostic device (40) may include one or more displays (140) depending on the implementation form. In addition, the display (140) may include a touch panel or a touch screen.

[0087] The processor (120) can control the overall operation of the ultrasonic diagnostic device (40) and the signal flow between internal components of the ultrasonic diagnostic device (40). The processor (120) can execute a program or app stored in the memory (150) to perform or control various operations or functions of the ultrasonic diagnostic device (40). In addition, the processor (120) can receive a control signal from an input interface (170) or an external device to control the operation of the ultrasonic diagnostic device (40).

[0088] The ultrasonic diagnostic device (40) includes a communication module (160) and can be connected to an external device (e.g., a probe (20), a server, a medical device, a portable device (smartphone, tablet PC, wearable device, etc.)) through the communication module (160).

[0089] The communication module (160) may include one or more components that enable communication with an external device, and may include, for example, at least one of a short-range communication module, a wired communication module, and a wireless communication module.

[0090] The communication module (160) of the ultrasonic diagnostic device (40) and the communication module (119) of the probe (20) may communicate using a network or a short-range wireless communication method. For example, the communication module (160) of the ultrasonic diagnostic device (40) and the communication module (119) of the probe (20) may communicate using any one of wireless data communication methods including wireless LAN, Wi-Fi, Bluetooth, zigbee, WFD (Wi-Fi Direct), infrared communication (IrDA, infrared Data Association), BLE (Bluetooth LowEnergy), NFC (Near Field Communication), Wibro (Wireless Broadband Internet, Wibro), WiMAX (World Interoperability for Microwave Access, WiMAX), SWAP (Shared Wireless Access Protocol), WiGig (Wireless Gigabit Alliance, WiGig), RF communication, 60GHz millimeter wave (mm Wave) short-range communication, etc.

[0091] To this end, the communication module (160) of the ultrasonic diagnostic device (40) and the communication module (119) of the probe (20) may include at least one of a wireless LAN communication module, a Wi-Fi communication module, a Bluetooth communication module, a zigbee communication module, a WFD (Wi-Fi Direct) communication module, an infrared communication (IrDA, infrared Data Association) module, a BLE (Bluetooth LowEnergy) communication module, an NFC (Near Field Communication) communication module, a Wibro (Wireless Broadband Internet, Wibro) communication module, a WiMAX (World Interoperability for Microwave Access, WiMAX) communication module, a SWAP (Shared Wireless Access Protocol) communication module, a WiGig (Wireless Gigabit Alliance, WiGig) communication module, an RF communication module, and a 60GHz millimeter wave (mm Wave) short-range communication module.

[0092] In one embodiment, the probe (20) may transmit device information (e.g., ID information) of the probe (20) using a first communication method (e.g., BLE), may be wirelessly paired with an ultrasonic diagnostic device (40), and may transmit ultrasound data and / or ultrasound images to the paired ultrasonic diagnostic device (40).

[0093] The device information of the probe (20) may include various information related to the serial number, model name, battery status, etc. of the probe (20).

[0094] The ultrasonic diagnostic device (40) may receive device information (e.g., ID information) of the probe (20) from the probe (20) using a first communication method (e.g., BLE), may be wirelessly paired with the probe (20), may transmit an activation signal to the paired probe (20), and may receive ultrasonic data and / or ultrasonic images from the probe (20). At this time, the activation signal may include a signal for controlling the operation of the probe (20).

[0095] In one embodiment, the probe (20) can transmit device information (e.g., ID information) of the probe (20) using a first communication method (e.g., BLE), be wirelessly paired with an ultrasonic diagnostic device (40), and transmit ultrasound data and / or ultrasound images to the ultrasonic diagnostic device (40) paired by the first communication method using a second communication method (e.g., 60 GHz millimeter wave, Wi-Fi).

[0096] The ultrasonic diagnostic device (40) can receive device information (e.g., ID information) of the probe (20) from the probe (20) using a first communication method (e.g., BLE), can be wirelessly paired with the probe (20), can transmit an activation signal to the paired probe (20), and can receive ultrasonic data and / or ultrasonic images from the probe (20) using a second communication method (e.g., 60 GHz millimeter wave, Wi-Fi).

[0097] According to various embodiments, the first communication method used to pair the probe (20) and the ultrasonic diagnostic device (40) with each other may have a lower frequency band than the frequency band of the second communication method used to transmit ultrasonic data and / or ultrasonic images from the probe (20) to the ultrasonic diagnostic device (40).

[0098] The display (140) of the ultrasonic diagnostic device (40) can display UIs indicating device information of the probe (20). For example, the display (140) can display UIs indicating identification information of the wireless probe (20), a pairing method indicating a pairing method with the probe (20), a data communication status between the probe (20) and the ultrasonic diagnostic device (40), a method of performing data communication with the ultrasonic diagnostic device (40), and a battery status of the probe (20).

[0099] When the probe (20) includes a display (140), the display (140) of the probe (20) can display UIs indicating device information of the probe (20). For example, the display (140) can display UIs indicating identification information of the wireless probe (20), a pairing method indicating a pairing method with the probe (20), a data communication status between the probe (20) and the ultrasonic diagnostic device (40), a method of performing data communication with the ultrasonic diagnostic device (40), and a battery status of the probe (20).

[0100] It is also possible for the communication module (160) to receive control signals and data from an external device and transmit the received control signals to the processor (120) so that the processor (120) controls the ultrasonic diagnostic device (40) according to the received control signals.

[0101] Alternatively, it is also possible to control the external device according to the control signal of the processor (120) by having the processor (120) transmit a control signal to the external device through the communication module (160).

[0102] For example, the external device can process data of the external device according to a control signal of the processor (120) received through the communication module.

[0103] An external device may be installed with a program capable of controlling the ultrasonic diagnostic device (40), and this program may include commands for performing part or all of the operations of the processor (120).

[0104] The program may be pre-installed on an external device, or the user of the external device may download and install the program from a server providing the application. The server providing the application may include a storage medium containing the program.

[0105] The memory (150) can store various data or programs for driving and controlling the ultrasonic diagnostic device (40), input / output ultrasonic data, ultrasonic images, etc.

[0106] An example of an ultrasound imaging system (100) according to one embodiment of the present disclosure is described below with reference to FIGS. 3, 4, 5, and 6.

[0107] FIGS. 3, 4, 5, and 6 are drawings showing an ultrasonic imaging device according to one embodiment.

[0108] Referring to FIGS. 3 and 4, the ultrasound imaging device (40a, 40b) may include a main display (121; 140) and a sub-display (122; 140). At least one of the main display (121) and the sub-display (122) may be implemented as a touch screen. At least one of the main display (121) or the sub-display (122) may display an ultrasound image or various information processed in the ultrasound imaging device (40a, 40b). In addition, at least one of the main display (121) or the sub-display (122) may be implemented as a touch screen and may provide a GUI, thereby receiving data for controlling the ultrasound imaging device (40a, 40b) from a user. For example, the main display (121) may display an ultrasound image, and the sub-display (122) may display a control panel (for example, the control panel (165) of FIG. 4) for controlling the display of the ultrasound image in the form of a GUI. The sub-display (122) can receive data for controlling the display of images through a control panel displayed in GUI format. For example, a TGC (Time Gain Compensation) button, a Freeze button, a trackball, a jog switch, a knob, etc. can be provided as a GUI on the sub-display (122).

[0109] The ultrasonic imaging device (40a, 40b) can control the display of the ultrasonic image displayed on the main display (121) using the input control data. In addition, the ultrasonic imaging device (40a, 40b) can be connected to the probe (20) by wire or wirelessly to transmit and receive ultrasonic signals to and from the target object.

[0110] Referring to FIG. 4, the ultrasonic imaging device (40b) may further include a control panel (165) in addition to the main display (121) and the sub-display (122). The control panel (165) may include buttons, a trackball, a jog switch, a knob, etc., and may receive data for controlling the ultrasonic imaging device (40b) from a user. For example, the control panel (165) may include a TGC (Time Gain Compensation) button (171), a Freeze button (172), etc. The TGC button (171) is a button for setting a TGC value according to the depth of the ultrasonic image. In addition, when the ultrasonic imaging device (40b) detects an input of the Freeze button (172) while scanning an ultrasonic image, the ultrasonic imaging device (40b) may maintain a state in which a frame image at the corresponding point in time is displayed.

[0111] Meanwhile, buttons, trackballs, jog switches, knobs, etc. included in the control panel (165) may be provided as a GUI on the main display (121) or sub-display (122). In addition, the ultrasonic imaging device (40a, 40b) may be connected to the probe (20) to transmit and receive ultrasonic signals to the target object.

[0112] Referring to FIGS. 5 and 6, the ultrasonic imaging device (40c) may also be implemented in a portable form. Examples of portable ultrasonic imaging devices (40c) include, but are not limited to, a smart phone, laptop computer, PDA, tablet PC, etc., including a probe and an application.

[0113] The ultrasonic imaging device (40c) may include a main body (41). Referring to FIG. 2c, a probe (20) may be connected to one side of the main body (41) by a wire. To this end, the main body (41) may include a detachable connection terminal for a cable connected to the probe (20), and the probe (20) may include a detachable connection terminal for a cable connected to the main body (40).

[0114] Referring to Fig. 6, the probe (20) can be wirelessly connected to an ultrasonic diagnostic device (40). The main body (41) can include an input / output interface (e.g., a touch screen) (145; 140, 170). The input / output interface (145) can display an ultrasonic image, various information processed in the ultrasonic imaging device, a GUI, etc.

[0115] Additionally, an ultrasound image may be displayed on the input / output interface (145). The ultrasound imaging device (40c) may correct the ultrasound image displayed on the input / output interface (145) using AI. The ultrasound imaging device (40c) may provide an alarm that notifies information about a lesion among the ultrasound images displayed on the input / output interface (145) using various audiovisual tools such as graphics, sound, and vibration using AI.

[0116] The ultrasonic imaging device (40c) can output a control panel displayed in GUI format through the input / output interface (145).

[0117] The ultrasonic imaging device (40d) and the probe (20) can establish communication or be paired using short-range wireless communication. For example, the ultrasonic imaging device (40d) and the probe (20) can communicate using Bluetooth, BLE, Wi-Fi, or Wi-Fi Direct.

[0118] The ultrasonic imaging device (40c, 40d) can execute a program or application related to the probe (20), control the probe (20), and output information related to the probe (20). The ultrasonic imaging device (40c, 40d) can communicate with a predetermined server and perform operations related to the probe (20). The probe (20) can be registered with the ultrasonic imaging device (40c, 40d) or registered with a predetermined server. The ultrasonic imaging device (40c, 40d) can communicate with the registered probe (20) and perform operations related to the probe (20).

[0119] Additionally, the ultrasonic imaging device (40c, 40d) may include various types of input / output interfaces, such as speakers, LEDs, and vibration devices. For example, the ultrasonic imaging device (40c, 40d) may output various information in the form of graphics, sounds, or vibrations through the input / output interface. Additionally, the ultrasonic imaging device (40c, 40d) may output various notifications or data through the input / output interface.

[0120] According to one embodiment of the present disclosure, an ultrasound imaging device (40a, 40b, 40c, or 40d) may process an ultrasound image or obtain additional information from an ultrasound image using an artificial intelligence (AI) model. According to one embodiment of the present disclosure, an ultrasound imaging device (40a, 40b, 40c, or 40d) may generate an ultrasound image or perform processing such as correction, image quality improvement, encoding, or decoding on an ultrasound image using an AI model. In addition, according to one embodiment of the present disclosure, an ultrasound imaging device (40a, 40b, 40c, or 40d) may perform processing such as baseline definition, anatomical information acquisition, lesion information acquisition, surface extraction, boundary definition, length measurement, area measurement, volume measurement, or annotation generation from an ultrasound image using an AI model.

[0121] The AI ​​model may be installed on the ultrasound imaging device (40a, 40b, 40c, or 40d) or on a server.

[0122] AI models can be implemented using various artificial neural networks or deep neural networks. Furthermore, AI models can be trained and generated using various machine learning or deep learning algorithms. For example, AI models can be implemented using models such as convolutional neural networks (CNNs), recurrent neural networks (RNNs), generative adversarial networks (GANs), and long short-term memory (LSTMs).

[0123] FIG. 7 is a drawing for explaining buttons constituting a control panel (165) of an ultrasonic diagnostic device (40) for receiving user input according to one embodiment.

[0124] According to one embodiment, the control panel (165) of the ultrasound diagnostic device (40) may include a plurality of buttons for receiving input from a user to control the display of ultrasound images.

[0125] At this time, the button is for receiving user input and may be implemented in the form of a push button, key pad, mouse, trackball, jog switch, knob, etc. that can be operated by the user, or may be implemented in the form of a touch pad or touch screen that recognizes the user's touch. In addition, as described above, the button may be implemented in the form of a GUI on the main display (121) or sub-display (122).

[0126] According to one embodiment, the control panel (165) may include a button for a freeze operation (hereinafter referred to as a “freeze button”) (172).

[0127] A freeze action may include an action of maintaining a state in which a frame at a specific point in time among an ultrasound image composed of multiple frames is displayed.

[0128] A user may press a button (172) for a freeze operation to maintain a state in which a frame image at a specific point in time is displayed while an ultrasound image is being output. Accordingly, the processor (120) may receive a user input for the freeze operation and control the display (140) to maintain a state in which an ultrasound image frame at the point in time when the button (172) for the freeze operation is pressed is output.

[0129] According to one embodiment, the control panel (165) may include a button (173) for a Measure operation or a Caliper operation (hereinafter referred to as a “measure button”). In the present disclosure, a method is disclosed in which a user presses the measure button (173) of the control panel (165) and then selects one of the Measure operation or the Caliper operation through the input interface (170), but this should not be construed as being limited thereto, and a button for the Measure operation and a button for the Caliper operation may be separately provided on the control panel (165).

[0130] The measuring operation may include an operation of measuring various parameters, such as length, area, volume, etc., of an object displayed in a frame at a specific time point among an ultrasound image composed of multiple frames, and / or an operation of displaying at least one indicator (e.g., a guide line, text, etc.) related to the measurement. In this case, the object displayed in the frame at a specific time point may be a biological tissue of a human body that is known in advance, such as a fetal head diameter or a fetal heart size.

[0131] A user can operate a control panel (165) and / or an input interface (170) to measure various parameters, such as length, area, and volume, of an object displayed in a frame at a specific point in time while an ultrasound image is being output.

[0132] Accordingly, the processor (120) can receive a user input for a measure operation from the control panel (165) and / or the input interface (170), maintain a state in which an ultrasound image frame at the time when the user input for the measure operation is received is output, and control the display (140) to display at least one indicator for parameter measurement.

[0133] A caliper operation may include an operation of measuring various parameters, such as length, area, volume, etc., of a structure or lesion displayed in a frame at a specific point in time among an ultrasound image composed of multiple frames, and / or an operation of displaying at least one indicator (e.g., a guide line, text, etc.) related to the measurement. In this case, the specific structure or lesion displayed in the frame at a specific point in time may be an unknown biological tissue that is not known in advance.

[0134] The user can operate the control panel (165) and / or the input interface (170) to measure various parameters, such as the length, area, and volume of a structure or lesion displayed in a frame at a specific point in time while the ultrasound image is being output.

[0135] Accordingly, the processor (120) can receive user input for the user's caliper operation from the control panel (165) and / or the input interface (170), maintain a state in which the ultrasound image frame at the time the user input is received is output, and control the display (140) to display at least one indicator for parameter measurement.

[0136] According to one embodiment, the control panel (165) may include a button for annotation operation. The button for annotation operation may include at least one of a text button (174) for displaying input text, an arrow button (175) for displaying an arrow, or a body marker button (not shown) for displaying a body marker.

[0137] The annotation action may include at least one of an action of displaying user-entered text to describe or identify a structure displayed in a frame at a specific point in time among an ultrasound image composed of multiple frames, an action of displaying an arrow in a location and direction desired by the user, or an action of displaying a body marker to visually indicate a specific part that is the object of the current diagnosis.

[0138] Referring to FIG. 7, the Text button (174) and the Arrow button (175) are shown as being provided separately on the control panel (165), but are not limited thereto.

[0139] That is, one button for annotation operation is provided on the control panel (165), and when the user presses the button for annotation operation, it can be implemented in a form that guides the user to select one of the operations of displaying text, arrows, or body markers through the input interface (170).

[0140] The processor (120) can receive user input for annotation operation from the control panel (165) and / or the input interface (170), maintain a state in which an ultrasound image frame at the time when the user input for the annotation operation is received is output, and control the display (140) to display at least one of text, an arrow, or a body marker on the ultrasound image frame.

[0141] According to one embodiment, the control panel (165) may include a trackball (176) for manipulating the ultrasound image display.

[0142] The user can zoom in, zoom out, or rotate the ultrasound image displayed on the display (120) by rotating or moving the trackball (176). In addition, the user can also adjust the depth, size, direction, etc. of the output ultrasound image by manipulating the trackball (176).

[0143] The user can manipulate the trackball (176) to adjust the speed at which multiple ultrasound image frames are displayed or to rewind the frames to obtain an image frame in which a desired object is displayed among the multiple ultrasound image frames. In other words, the user can use the trackball (176) to adjust the image playback speed or rewind the image to manipulate the ultrasound image in more detail to check a specific part or find a desired section.

[0144] According to one embodiment, the control panel (165) may include a Save button (177) for saving ultrasound image frames.

[0145] After pressing a button for controlling the display of an ultrasound image, a user can press a save button (177) to save an ultrasound image frame after the action corresponding to the button has been performed. The button for controlling the display of an ultrasound image may include a freeze button, a measure button, a caliper button, or an annotation button.

[0146] According to various embodiments, the control panel (165) may further include other buttons for controlling the operation of the ultrasonic diagnostic device (40) or controlling the display of ultrasonic images output to the display (120) in addition to the buttons described above. In addition, according to various embodiments, the control panel (165) may be implemented in a form in which some of the buttons described above are omitted.

[0147] Above, the multiple buttons provided on the control panel (165) to receive user input have been described. Below, the operations performed according to the user input entered through each button will be described with reference to FIGS. 8 to 14.

[0148] FIG. 8 is a diagram illustrating an example of an input interface (170) displayed on an auxiliary display (122) based on input received from a user regarding a target action according to one embodiment.

[0149] FIG. 9 is a drawing showing an example of an ultrasound image displayed on the main display (121) based on input received from a user regarding a target action.

[0150] According to one embodiment, the processor (120) may control the display (140) based on receiving user input regarding a master operation. In this case, the display (140) may include a main display (121) and an auxiliary display (122).

[0151] According to one embodiment, the processor (120) may provide an input interface (170a) by displaying a plurality of buttons in a GUI format on the auxiliary display (122) so that the user can receive input regarding various measurement items based on the user's selection of a measure operation on the input interface (170) after pressing the measurement button on the control panel (165).

[0152] At this time, multiple buttons can be displayed in stages to allow input regarding various measurement items.

[0153] Referring to FIG. 8, the processor (120) may display a plurality of buttons (8a) to allow the user to select a primary measurement item, such as fetal biometry or amniotic fluid index (AFL), performed to evaluate the health and development of the fetus.

[0154] The processor (120) may display buttons (8b) for a plurality of secondary measurement items related to a primary measurement (e.g., fetal biometry) selected by the user. For example, the buttons (8b) for secondary measurement items may be buttons for measuring circumferences, sizes, etc. of previously known body parts, such as biparietal diameter (BPD), head circumference (HC), and abdominal circumference (AC).

[0155] According to one embodiment, the processor (120) may display an ultrasound image (140a) including an indicator regarding a measurement item on the main display (121) based on the selection of a measurement item through the input interface (170a) after the user selects a measurement action.

[0156] Referring to FIG. 9, a plurality of indicators (9a) corresponding to measurement items selected by the user can be displayed on the ultrasound image displayed on the main display (121).

[0157] For example, a user may include a graphic indicator (9b) that conveys information about the part where a measurement is being performed using visual elements such as a shape including two points and a connecting line connecting the points, so that multiple indicators (9a) can indicate the object of measurement, and / or a text indicator (9c) that conveys the measurement value numerically.

[0158] In various embodiments, if a separate measure button is provided on the control panel (165), the processor (120) may display the same input interface (170a) and ultrasound image (140a) based on the user pressing the measure button.

[0159] FIG. 10 is a diagram illustrating an example of an input interface (170) displayed on an auxiliary display (122) based on input received from a user regarding a caliper operation according to one embodiment.

[0160] FIG. 11 is a drawing showing an example of an ultrasound image displayed on the main display (121) based on input received from a user regarding a caliper operation.

[0161] According to one embodiment, the processor (120) may control the display (140) based on receiving user input regarding the caliper operation. In this case, the display (140) may include a main display (121) and an auxiliary display (122).

[0162] According to one embodiment, the processor (120) may provide an input interface (170b) by displaying a plurality of buttons in a GUI format on the auxiliary display (122) so that the user can receive input regarding various measurement items based on the user's selection of a caliper operation on the input interface (170) after pressing the measurement button on the control panel (165).

[0163] Referring to FIG. 10, the processor (120) may display an input interface (170b) including a plurality of buttons (10a) for a measurement method for measuring the length, diameter, area, etc. of an unknown structure or lesion displayed in an ultrasound image.

[0164] For example, the plurality of buttons (10a) may include at least one of a Distance button for measuring the distance between any two points selected by the user, a Trace button for drawing a line along a structure displayed in an ultrasound image, an Ellipse button for drawing a circle or an ellipse, a Spline button for drawing a curve for measuring along a curved path or boundary of a structure, or a Stenosis button for measuring the degree of a lesion or stenosis by checking the diameter of a specific area.

[0165] According to one embodiment, the processor (120) may display an ultrasound image (140b) including an indicator regarding the measurement method on the main display (121) based on the user's selection of a caliper operation and the selection of the measurement method through the input interface (170b).

[0166] Referring to FIG. 11, a plurality of indicators (11a) corresponding to a measurement method selected by a user can be displayed on an ultrasound image displayed on the main display (121).

[0167] For example, if the user selects the Distance button, the plurality of indicators (11a) may include a graphic indicator (11b) that conveys information about the portion where the measurement is to be performed using visual elements such as a shape including two points selected by the user and a connecting line connecting the points, and / or a text indicator (11c) that conveys the measurement value numerically.

[0168] In various embodiments, if a separate caliper button is provided on the control panel (165), the processor (120) may display the same input interface (170b) and ultrasound image (140b) based on the user pressing the caliper button.

[0169] FIG. 12 is a drawing showing an example of a screen displayed on the main display (121) based on input received from a user regarding an annotation operation.

[0170] According to one embodiment, the processor (120) can control the display (140) based on receiving user input regarding an annotation operation.

[0171] According to one embodiment, the processor (120) may display an ultrasound image (140b) including an indicator corresponding to an annotation selected by the user on the main display (121) based on the annotation action selected by the user.

[0172] Referring to FIG. 12, a plurality of indicators (12a, 12b, 12c, 12d, 13, 14) corresponding to annotations selected by the user can be displayed on an ultrasound image (140c) displayed on the main display (121).

[0173] For example, when a user selects a body marker button (not shown), the processor (120) can control the main display (121) to display a body marker (13) corresponding to the object displayed in the ultrasound image frame at the time the user pressed the button at the location where the object is displayed.

[0174] Additionally, when the user selects the text button (174), the processor (120) can control the main display (121) so that the text (12a, 12b, 12c, 12d) entered by the user is displayed together with the ultrasound image in the ultrasound image frame at the time the user presses the button.

[0175] When the user selects the arrow button (175), the processor (120) can control the main display (121) to display an arrow (14) along with the ultrasound image in the ultrasound image frame at the time the user pressed the button.

[0176] At this time, the user can also manipulate the trackball (176) to display an arrow or text at a desired location.

[0177] At this time, when the user inputs the control panel (165) or input interface (170) for the aforementioned actions during the inspection (e.g., measuring action, caliper action, annotation action), the user inputs after the desired image frame has passed due to the human reaction time. Accordingly, the user must rewind the ultrasound image by operating the trackball (167) to obtain the desired image frame, which will be described below with reference to FIGS. 13 and 14.

[0178] FIG. 13 is an example of a screen displayed on a display (140) after rewinding an ultrasound image by operating a trackball (167) when the ultrasound image frame at the time when a user input regarding a first action is received is not the ultrasound image frame desired by the user, according to one embodiment.

[0179] FIG. 14 is an example of a screen displayed on a display (140) at a time when a user input for a second operation is received after the ultrasound image frame at the time when the user input is received is not the ultrasound image frame desired by the user, after the trackball (167) is operated to rewind the ultrasound image, according to one embodiment.

[0180] According to one embodiment, when the processor (120) receives a user input regarding a measure operation, a caliper operation, or an annotation operation, the processor (120) may control the display (140) to maintain a state of displaying the ultrasound image frame at the time the user input was received.

[0181] The processor (120) can control the display (140) to display an image frame indicator (300) including information about the number of currently played image frames, along with the ultrasound image frame at the time the user input was received.

[0182] The video frame indicator (300) may include information about the currently playing frame and information about the total number of frames.

[0183] Information about the currently playing frame and the total number of frames can be implemented as numbers (e.g., 77 / 77) or as a bar-shaped graphic.

[0184] For example, referring to FIG. 13, if the image frame indicator (300) includes the '77 / 77' indication, it may indicate that the total number of frames of the ultrasound image is 77, and that the 77th image frame among them is being played.

[0185] Input regarding the mag action, caliper action, or annotation action is performed by the user, and a time difference corresponding to the reaction time may occur between the time the user recognizes the desired ultrasound image frame and the time the user performs the input regarding the mag action, caliper action, or annotation action due to the human reaction time.

[0186] In one embodiment, if the ultrasound image frame displayed according to the user input regarding the motion is not the desired ultrasound image frame, the user can manipulate the display (140) trackball (167) to rewind the ultrasound image so that the desired ultrasound image frame is displayed.

[0187] At this time, the processor (120) can control the display (140) to display an image frame indicator (300) including information about the number of image frames currently played as the user operates the trackball (167), along with the ultrasound image frames played as the user operates the trackball (167).

[0188] The video frame indicator (300) may include information about the currently playing frame and the total number of frames after the user rewinds.

[0189] Information about the currently playing frame and the total number of frames, as before the user rewinds, can be implemented as numbers (e.g., 72 / 77) or as a bar-shaped graphic.

[0190] For example, referring to FIG. 14, if the image frame indicator (300) includes the '72 / 77' display, it can indicate that the total number of frames of the ultrasound image is 77 and that the user has rewinded and the 72nd image frame is being played.

[0191] In one embodiment, the user may press a button corresponding to a second action to perform a second action to control the display of the video frame if the currently playing frame is the video frame desired by the user after rewinding.

[0192] For example, if the user rewinds and the currently playing frame is the video frame the user wanted, the user can press the same action button as before operating the trackball (167) to perform the same action as the first action on that video frame.

[0193] Additionally, the user can press the save button to save the video frame that is currently playing after rewinding if it is the video frame that the user wanted.

[0194] In other words, the second action may be the same action as the first action, or it may be an action to save the currently playing frame after rewinding.

[0195] Hereinafter, referring to FIG. 16, a description will be given of determining a response time based on information about the currently playing video frame before and after rewinding by the processor (120) and obtaining information on the user's usage pattern.

[0196] Figure 15 is a diagram for explaining obtaining user usage pattern information through a machine learning model.

[0197] In one embodiment, the control panel (165) may receive input from a user regarding a series of actions. The series of actions may sequentially include an input regarding a first action (S1), a trackball operation (S2), and an input regarding a second action (S3).

[0198] The input (S1) regarding the first operation may be an input regarding any one of an operation for setting a measure, an operation for setting a caliper, or an annotation, which is a predetermined operation for controlling the display of an ultrasound image displayed on a display (e.g., main display (121)) (140).

[0199] Trackball operation (S2) is an operation for changing an ultrasound image frame displayed on the display (140) to a desired ultrasound image frame based on an input regarding the first operation by the user operating the trackball (167), and may include a trackball operation for a rewind operation.

[0200] The input (S2) regarding the second operation is for controlling the display of the ultrasound image frame displayed on the display (140) after the ultrasound image frame displayed on the display (140) is changed to a desired ultrasound image frame through the operation of the trackball (167), and may be an input regarding the same operation as the first operation or an input regarding a storage operation. Since the second operation is not an operation for changing the ultrasound image frame displayed on the display (140), the ultrasound image frame displayed on the display (140) when the processor (120) receives the input regarding the second operation is the same as the ultrasound image frame displayed on the display (140) after the operation of the trackball (167) is completed.

[0201] According to one embodiment, the processor (120) may receive a series of user inputs from a control panel (165).

[0202] The processor (120) can determine the user's reaction time based on a series of user inputs received.

[0203] The processor (120) can calculate the number of ultrasound image frames that exist between the ultrasound image frames displayed when user input regarding the first action and the second action is received.

[0204] That is, the processor (120) can determine the response time through the following equation 1 based on the number of ultrasound image frames existing between the ultrasound image frames displayed when user input regarding the first and second actions is received, i.e., the change in the number of frames between the first and second actions.

[0205] [Formula 1]

[0206] 1 / Frame rate (Hz) at the time of the first movement x Change in frame count between the first and second movements = Response time (sec)

[0207] For example, referring to FIGS. 13 and 14, the ultrasound image frame being played at the time when the user input regarding the first action is received is the 77th frame, and the ultrasound image frame being played at the time when the user input regarding the second action is received is the 72nd frame. Therefore, the number of ultrasound image frames existing between the ultrasound image frames displayed when the user input regarding the first action and the second action is received, i.e., the change in the number of frames between the first action and the second action is 5 frames.

[0208] If the frame rate of the ultrasound image shown in Fig. 13 is 30 frames per second (fps), the response time can be determined to be approximately 0.17 seconds.

[0209] That is, the processor (120) can determine the user's reaction time with respect to the first action, which can be processed by a machine learning model stored in the memory (150).

[0210] In one embodiment, when the processor (120) obtains multiple data regarding the response time for each operation, it can input the multiple data into a machine learning model (121) to obtain user usage pattern information.

[0211] In one embodiment, the machine learning model may be a machine learning model for feature extraction that extracts features of data when multiple data regarding reaction times for each acquired action are input, and outputs processed data including the extracted features.

[0212] Features of data may include elements that a machine learning model extracts from the data to perform classification or prediction.

[0213] In one embodiment, the machine learning model may be a machine learning model for acquiring user usage pattern information, which outputs processing data including data on user patterns when a plurality of response time data for each action is input, when the user uses the ultrasound diagnostic device.

[0214] For example, the processor (120) may include a first machine learning model for processing a reaction time for a maker action, a second machine learning model for processing a reaction time for a caliper action, a third machine learning model for processing a reaction time for a text action, a fourth machine learning model for processing a reaction time for an arrow action, and / or a fifth machine learning model for processing a reaction time for a body marker action.

[0215] The processor (120) can obtain first processing data based on processing of response time data regarding the measurement operation.

[0216] The first processing data may include feature data extracted from response time data regarding the seller's actions and / or data regarding usage pattern information extracted from response time data regarding the seller's actions. The first processing data may have a smaller capacity than the first data.

[0217] The processor (120) can obtain second processing data based on processing of response time data regarding caliper operation.

[0218] The second processing data may include feature data extracted from response time data regarding caliper operation and / or data regarding usage pattern information extracted from response time data regarding caliper operation. The capacity of the second processing data may be smaller than that of the second data.

[0219] The processor (120) can obtain third processing data based on processing of response time data regarding text motion.

[0220] The third processing data may include feature data extracted from response time data regarding text actions and / or data regarding usage pattern information extracted from response time data regarding text actions. The volume of the third processing data may be smaller than that of the third data.

[0221] The processor (120) can obtain fourth processing data based on processing the response time data regarding the arrow operation.

[0222] The fourth processing data may include feature data extracted from response time data regarding arrow motion and / or data regarding usage pattern information extracted from response time data regarding arrow motion. The capacity of the fourth processing data may be smaller than that of the fourth data.

[0223] The processor (120) can obtain fifth processing data based on processing of reaction time data regarding body marker operation.

[0224] The fifth processing data may include feature data extracted from response time data regarding body marker operations and / or data regarding usage pattern information extracted from response time data regarding body marker operations. The capacity of the fifth processing data may be smaller than that of the fifth data.

[0225] The processor (120) can input the first processing data, the second processing data, the third processing data, the fourth processing data, and / or the fifth processing data into a machine learning model stored in the memory (150) to obtain usage pattern information related to the response time for each user action.

[0226] The usage pattern information may include at least one of response time information regarding a measure operation, response time information regarding a caliper operation, or response time information regarding an annotation operation.

[0227] At this time, the reaction time information regarding the annotation action may include at least one of reaction time information regarding the text action, reaction time information regarding the arrow action, or reaction time information regarding the body marker.

[0228] The machine learning model stored in the memory (150) may be a machine learning model for determining user usage pattern information, which outputs information related to the reaction time according to each user action when first processing data, second processing data, third processing data, fourth processing data, and / or fifth processing data are input.

[0229] The processor (120) can input the first processing data, the second processing data, the third processing data, the fourth processing data, and / or the fifth processing data into the machine learning model and store the acquired usage pattern information in the memory (150).

[0230] In one embodiment, the processor (120) can perform response time correction of an ultrasound image based on usage pattern information accumulated and stored in the memory (150).

[0231] The processor (120) can control the display (140) to display an ultrasound image frame with a corrected response time corresponding to the first action based on the user input regarding the first action being received.

[0232] For example, the processor (120) can control the display (140) to automatically rewind and display the number of ultrasound image frames corresponding to the reaction time when a user input regarding a first action is received based on the stored usage pattern information.

[0233] Accordingly, the user can obtain an ultrasound image frame at a desired point in time without a separate operation such as operating a trackball (167).

[0234] In this embodiment, the application stored in memory (150) may include a machine learning model. The machine learning model included in the application may be updated by an external server.

[0235] The processor (120) can obtain the aforementioned usage pattern information for each user account logged into the application. Accordingly, accurate pattern information for each user can be obtained.

[0236] FIG. 16 is a flowchart illustrating automatic correction of an ultrasound image based on acquired usage pattern information according to one embodiment.

[0237] In one embodiment, the processor (120) can control the display (140) to display an ultrasound image (1601). An ultrasound image obtained by scanning an object through the probe (20) can be reproduced on the display (140) by the processor (120).

[0238] The processor (120) can determine whether a user input is received (1602) regarding a series of operations for controlling the display of an ultrasound image displayed on the display (140) via the control panel (165) and / or the input interface (170).

[0239] A series of actions may include, in sequence, an input (S1) regarding a first action, a trackball operation (S2), and an input (S3) regarding a second action, as described with reference to FIG. 15.

[0240] At this time, the first action may be one of the actions of setting a measure, setting a caliper, or setting an annotation on the ultrasound image.

[0241] The second action may be the same action as the first action, or it may be a Save action that saves the current frame.

[0242] The processor (120) can determine a reaction time for a first action when a series of user inputs are received (1603).

[0243] At this time, the processor (120) can calculate the number of ultrasound image frames existing between the ultrasound image frames displayed when the user inputs regarding the first action and the second action are received. In addition, the processor (120) can determine the response time as a value obtained by multiplying the number of ultrasound image frames existing between the ultrasound image frames displayed when the user inputs regarding the first action and the second action are received by the inverse of the frame rate at the time of the first action.

[0244] The processor (120) can input the reaction time for the determined first action into the machine learning model (1604). At this time, the processor (120) can obtain multiple reaction time data for each action by repeating steps 1602 and 1603, and input this as an input value into the machine learning model.

[0245] The processor (120) can sufficiently acquire multiple response time data for each action, process the data through a machine learning model, and determine whether or not the user's pattern information has been acquired based on the data (1605). At this time, the user's pattern information may include information about the response time when the user inputs a control command for an action of setting a measure, information about the response time when the user inputs a control command for an action of setting a caliper, or information about the response time when the user inputs a control command for an action of setting an annotation.

[0246] At this time, information about the reaction time when the user inputs a control command regarding an action of setting an annotation may include at least one of reaction time information regarding an action of displaying text, reaction time information regarding an action of displaying an arrow, or reaction time information regarding an action of displaying a body marker.

[0247] Additionally, the processor (120) may obtain usage pattern information for each user account logged into the application stored in the memory (150).

[0248] If the processor (120) fails to acquire user pattern information (No of 1605), it determines that it has not sufficiently acquired multiple response time data for each action, and can acquire multiple response time data for each action by repeating steps 1602 to 1603.

[0249] When the processor (120) obtains user pattern information (example of 1605), it can determine whether a user input regarding a predetermined action has been received (1606).

[0250] When a user input regarding a predetermined action is received (example of 1606), the processor (120) can perform response time correction of an ultrasound image regarding a predetermined action based on the user input (1607).

[0251] For example, correcting the response time of an ultrasound image by the processor (120) may include rewinding the ultrasound image by the response time of an action corresponding to a predetermined action based on learned user pattern information.

[0252] Accordingly, the processor (120) can control the display (140) to output an ultrasound image with a corrected response time (1608).

[0253] At this time, the ultrasound image with the response time corrected may include ultrasound image frames with the number of ultrasound image frames corresponding to the response time rewound.

[0254] Accordingly, the user can obtain ultrasound image frames without separate operation and without the influence of human reaction time, thereby increasing inspection efficiency.

[0255] In order to more accurately estimate and correct the user's reaction time, it is necessary to determine whether the user's operation of the trackball (167) between the first and second movements was due to the inability to obtain the desired ultrasound image frame due to the reaction time, or whether it was an intentional act for detailed examination. If the user's rewind action through the operation of the trackball (167) is to rewind the image to observe the discovered lesion in more detail, the reaction time according to the series of user inputs can be excluded from the input values ​​of the machine learning model. This will be described below with reference to FIG. 18.

[0256] FIG. 17 is a flowchart illustrating automatic correction of an ultrasound image based on acquired usage pattern information according to another embodiment.

[0257] In another embodiment, the processor (120) can control the display (140) to display an ultrasound image (1701). An ultrasound image obtained by scanning an object through the probe (20) can be reproduced on the display (140) by the processor (120).

[0258] The processor (120) can determine whether a user input is received (1702) regarding a series of operations for controlling the display of an ultrasound image displayed on the display (140) via the control panel (165) and / or the input interface (170).

[0259] A series of actions may include, in sequence, an input (S1) regarding a first action, a trackball operation (S2), and an input (S3) regarding a second action, as described with reference to FIG. 16.

[0260] At this time, the first action may be one of the actions of setting a measure, setting a caliper, or setting an annotation on the ultrasound image.

[0261] The second action may be the same action as the first action, or it may be a Save action that saves the current frame.

[0262] The processor (120) can determine a reaction time for a first action when a series of user inputs are received (1703).

[0263] At this time, the processor (120) can calculate the number of ultrasound image frames existing between the ultrasound image frames displayed when the user inputs regarding the first action and the second action are received. In addition, the processor (120) can determine the response time as a value obtained by multiplying the number of ultrasound image frames existing between the ultrasound image frames displayed when the user inputs regarding the first action and the second action are received by the inverse of the frame rate at the time of the first action.

[0264] The processor (120) can determine whether the reaction time for the determined first action is less than a preset time before inputting the reaction time into the machine learning model (1704).

[0265] At this time, the preset time can be set by the user or automatically set to a time derived through machine learning.

[0266] If the determined response time is greater than or equal to a preset time (No of 1704), the processor (120) may exclude data regarding the determined response time from the learning data without inputting it into the machine learning model for obtaining user usage pattern information (1705).

[0267] If the reaction time is longer than the preset time, the rewinding of the ultrasound image frame corresponding to the reaction time is likely to be an intentional manipulation rather than a reaction time compensation.

[0268] For example, if the preset time is 1 second, the processor (120) may exclude from the learning data a reaction time of 1 second or longer for the first determined action, assuming that it is an intentional manipulation action rather than a reaction time compensation action.

[0269] If the determined response time is less than a preset time (e.g., 1704), the processor (120) may input the response time for the determined first action into the machine learning model (1706). At this time, the processor (120) may obtain multiple response time data for each action by repeating steps 1702 to 1705, and input these data as input values ​​into the machine learning model. At this time, data not intended for response time correction is excluded from the input values, thereby enabling more accurate usage pattern information to be extracted.

[0270] The processor (120) can sufficiently acquire multiple response time data regarding each action, process the data through a machine learning model, and determine whether or not the user's pattern information has been acquired based on the data (1707). At this time, the user's pattern information may include information regarding the response time when the user inputs a control command regarding an action of setting a measure, information regarding the response time when the user inputs a control command regarding an action of setting a caliper, or information regarding the response time when the user inputs a control command regarding an action of setting an annotation.

[0271] At this time, information about the reaction time when the user inputs a control command regarding an action of setting an annotation may include at least one of reaction time information regarding an action of displaying text, reaction time information regarding an action of displaying an arrow, or reaction time information regarding an action of displaying a body marker.

[0272] If the processor (120) fails to acquire user pattern information (No of 1707), it determines that it has not sufficiently acquired multiple response time data for each action, and can acquire multiple response time data for each action by repeating steps 1702 to 1705.

[0273] When the processor (120) obtains user pattern information (example of 1707), it can determine whether a user input regarding a predetermined action has been received (1708).

[0274] When a user input regarding a predetermined action is received (example of 1708), the processor (120) can perform response time correction of an ultrasound image regarding a predetermined action based on the user input (1709).

[0275] For example, correcting the response time of an ultrasound image by the processor (120) may include rewinding the ultrasound image by the response time of an action corresponding to a predetermined action based on learned user pattern information.

[0276] Accordingly, the processor (120) can control the display (140) to output an ultrasound image with a corrected response time (1710).

[0277] At this time, the ultrasound image with the response time corrected may include ultrasound image frames with the number of ultrasound image frames corresponding to the response time rewound.

[0278] According to another embodiment, through a process of refining data input to a machine learning model, more accurate user pattern information can be extracted, thereby improving the efficiency of user inspection.

[0279] According to one embodiment, an ultrasound diagnostic device includes a display (140) for displaying an ultrasound image; a control panel (165) for receiving a user's input for controlling the display of the ultrasound image; and a processor (120) for performing a first operation and a second operation according to a user's input inputted through the control panel; wherein the processor (120) determines a response time based on the number of frames existing between ultrasound image frames displayed on the display (140) when user inputs regarding the first operation and the second operation are received, inputs the response time into a machine learning model to obtain usage pattern information, and performs response time correction of the ultrasound image based on the usage pattern information, thereby outputting an ultrasound image frame with the response time corrected through the display (140).

[0280] The above first operation may include a measure operation, a caliper operation, or an annotation operation.

[0281] The above annotation operation may include an operation of displaying at least one of text, body marker, or arrow on the ultrasound image.

[0282] The second operation may be the same operation as the first operation.

[0283] The above processor (120) can determine the reaction time as the product of the number of frames existing between the ultrasound image frames displayed when the user input regarding the first operation and the second operation is received and the inverse of the frame rate at the time when the user input regarding the first operation is received.

[0284] The processor (120) may input the reaction time into a machine learning model based on the reaction time being less than a preset time.

[0285] The user's usage pattern information may include information about a reaction time when the user inputs a control command for an action to set a measure, information about a reaction time when the user inputs a control command for an action to set a caliper, or information about a reaction time when the user inputs a control command for an action to set an annotation.

[0286] The above ultrasound diagnostic device (40) may further include a memory (150) that stores an application including the machine learning model.

[0287] The above processor (120) can obtain user usage pattern information for each user account logged into the application.

[0288] In one embodiment, a method for controlling an ultrasound diagnostic device (40) including a display (140) for displaying an ultrasound image and a control panel (165) for receiving a user's input for controlling the display of the ultrasound image may include receiving a user input regarding a series of operations including a first operation and a second operation through the control panel (165), determining a response time based on the number of frames existing between ultrasound image frames displayed on the display (140) when the user input regarding the first operation and the second operation is received, inputting the response time into a machine learning model to obtain usage pattern information, performing a response time correction of the ultrasound image based on the usage pattern information, and outputting an ultrasound image frame having the response time corrected through the display (140).

[0289] The above first operation may include a measure operation, a caliper operation, or an annotation operation.

[0290] The above annotation operation may include an operation of displaying at least one of text, body marker, or arrow on the ultrasound image.

[0291] The second operation may be the same operation as the first operation.

[0292] The second operation may include a save operation that saves the current frame of the ultrasound image.

[0293] Determining the above reaction time may include determining the reaction time as a value obtained by multiplying the number of ultrasound image frames present between the ultrasound image frames displayed when the user input regarding the first action and the second action is received by the reciprocal of the frame rate at the time when the user input regarding the first action is received.

[0294] Obtaining usage pattern information by inputting the above reaction time into a machine learning model can input the reaction time into the machine learning model based on the reaction time being less than a preset time.

[0295] The user's usage pattern information may include information about a reaction time when the user inputs a control command for an action to set a measure, information about a reaction time when the user inputs a control command for an action to set a caliper, or information about a reaction time when the user inputs a control command for an action to set an annotation.

[0296] The above ultrasound diagnostic device (40) may further include a memory (150) that stores an application including the machine learning model.

[0297] Obtaining usage pattern information by inputting the above reaction time into a machine learning model may include obtaining the user usage pattern information for each user account logged into the application.

Claims

1. A display that displays ultrasound images; A control panel for receiving user input to control the display of the above ultrasound image; and A processor that performs a first action and a second action according to a user's input entered through the control panel; The above processor, Determine the response time based on the number of frames existing between the ultrasound image frames displayed on the display when the user input regarding the first action and the second action is received, The above reaction time is input into a machine learning model to obtain usage pattern information, An ultrasonic diagnostic device that performs response time correction of the ultrasonic image based on the above usage pattern information and outputs an ultrasonic image frame with the response time corrected through the display.

2. In paragraph 1, The above first operation is, An ultrasonic diagnostic device comprising a measure operation, a caliper operation, or an annotation operation.

3. In paragraph 1, The above annotation action is, An ultrasound diagnostic device comprising an action of displaying at least one of text, body markers, or arrows on the ultrasound image.

4. In paragraph 1, The second operation above is, An ultrasonic diagnostic device having the same operation as the first operation above.

5. In paragraph 1, The second operation above is, An ultrasound diagnostic device including a save operation for saving the current frame of the ultrasound image.

6. In paragraph 1, The above processor, An ultrasonic diagnostic device that determines the response time as a value obtained by multiplying the number of ultrasound image frames present between the ultrasound image frames displayed when the user input regarding the first operation and the second operation is received by the reciprocal of the frame rate at the time when the user input regarding the first operation is received.

7. In paragraph 1, The above processor, An ultrasonic diagnostic device that inputs the reaction time into a machine learning model based on the reaction time being less than a preset time.

8. In paragraph 7, The above user's usage pattern information is: An ultrasonic diagnostic device comprising information about a reaction time when the user inputs a control command regarding an action of setting a measure, information about a reaction time when the user inputs a control command regarding an action of setting a caliper, or information about a reaction time when the user inputs a control command regarding an action of setting an annotation.

9. In paragraph 1, The above ultrasonic diagnostic device, An ultrasound diagnostic device further comprising a memory for storing an application including the machine learning model.

10. In paragraph 9, The above processor, An ultrasonic diagnostic device that obtains user usage pattern information for each user account logged into the above application.

11. A method for controlling an ultrasonic diagnostic device, comprising: a display for displaying an ultrasonic image; and a control panel for receiving a user's input for controlling the display of the ultrasonic image. Receiving user input regarding a series of actions including a first action and a second action through the above control panel, Determine the response time based on the number of frames existing between the ultrasound image frames displayed on the display when the user input regarding the first action and the second action is received, The above reaction time is input into a machine learning model to obtain usage pattern information, A control method of an ultrasonic diagnostic device, comprising: performing response time correction of the ultrasonic image based on the usage pattern information; and outputting an ultrasonic image frame with the response time corrected through the display.

12. In paragraph 11, The above first operation is, A method for controlling an ultrasonic diagnostic device including a measure operation, a caliper operation, or an annotation operation.

13. In paragraph 11, The action of setting the above annotation is: A method for controlling an ultrasound diagnostic device, comprising an action of displaying at least one of text, body markers or arrows on the ultrasound image.

14. In paragraph 11, The second operation above is, A method for controlling an ultrasonic diagnostic device having the same operation as the first operation above.

15. In paragraph 11, The second operation above is, A control method of an ultrasonic diagnostic device including a save operation for saving the current frame of the ultrasound image.

Images