Method for displaying ultrasound images, ultrasound diagnostic apparatus, and computer program product

Abstract

A method for displaying an ultrasound image is disclosed, the method comprising the following steps: identifying a lesion area included in the ultrasound image; diagnosing the lesion area to obtain a diagnosis result; generating a diagnostic image by displaying a first area in the lesion area of ​​the ultrasound image, the first area being at least one area on which a lesion is diagnosed; and displaying a user interface screen including the diagnostic image and the diagnosis result.

Description

Technical Field

[0001] The disclosed embodiments relate to a method for displaying an ultrasound image, an ultrasound diagnostic apparatus, and a computer program product.

[0002] In particular, the disclosed embodiments relate to a method for displaying an ultrasound image including a lesion, an ultrasound diagnostic apparatus, and a computer program product. Background Art

[0003] The ultrasonic diagnostic apparatus transmits an ultrasonic signal generated by a transducer of a probe to a subject and receives information of the signal reflected from the subject to obtain at least one image of a portion (eg, soft tissue or blood) inside the subject.

[0004] Recently, due to the development of image processing technologies such as computer-aided detection and diagnosis (CAD) systems, machine learning, etc., an ultrasonic diagnostic apparatus automatically analyzes an acquired ultrasonic image using a computer to detect an abnormal area as a portion where an abnormality occurs in an object or to generate an analysis result. Specifically, the ultrasonic diagnostic apparatus may output a user interface screen including a portion where an abnormality occurs in an object and a diagnostic result of the analysis (e.g., whether the detected portion is a malignant tumor). Therefore, a user (e.g., a doctor, etc.) may confirm a diagnostic result corresponding to an abnormal portion detected in a medical image (e.g., an ultrasonic image). Summary of the invention

[0005] Technical issues

[0006] The disclosed embodiments and their effects are intended to provide a method for displaying ultrasound images, an ultrasound diagnostic apparatus, and a computer program product, which provide information related to the diagnosis results to allow the user to more accurately diagnose the patient's disease when providing the diagnosis results by automatically analyzing the ultrasound images.

[0007] Technical Solution

[0008] According to the disclosed embodiment, the method for displaying an ultrasound image includes: identifying a lesion area included in the ultrasound image; diagnosing the lesion area to obtain a diagnosis result; displaying a first area in the lesion area in the lesion area of ​​the ultrasound image and generating a diagnostic image, wherein the first area is at least one area serving as a basis for diagnosing the lesion; and displaying a user interface screen including the diagnostic image and the diagnostic result.

[0009] Beneficial Effects

[0010] By using the method of displaying an ultrasound image, an ultrasound diagnostic apparatus, and a computer program product according to the disclosed embodiments, information related to the diagnosis result is provided, so that the user can diagnose the patient's disease more accurately.

[0011] Specifically, using the method of displaying an ultrasound image, the ultrasound diagnostic apparatus, and the computer program product according to the disclosed embodiments, a user can identify a portion of an ultrasound image that is a basis for a diagnosis result, so that the user can accurately determine a disease of a subject.

[0012] Specifically, using the method for displaying ultrasound images, ultrasound diagnostic apparatus and computer program product according to the disclosed embodiments, information related to the diagnosis result obtained based on computer-aided detection and diagnosis (CAD) (specifically, a portion of the ultrasound image that serves as the basis for the diagnosis result) and / or additional information for describing the diagnosis result is provided to the user, thereby helping the user to diagnose the disease through the ultrasound image. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The present disclosure can be easily understood by combination of the following detailed description and the accompanying drawings, in which the reference numerals refer to structural elements.

[0014] Figure 1 is a block diagram showing the configuration of an ultrasonic diagnostic apparatus according to the embodiment.

[0015] Figure 2 is a block diagram showing a configuration of an ultrasonic diagnostic apparatus according to another embodiment.

[0016] Figure 3a is a diagram showing the appearance of an ultrasonic diagnostic apparatus according to the embodiment.

[0017] Figure 3b is another diagram showing the appearance of the ultrasonic diagnostic apparatus according to the embodiment.

[0018] Figure 3c is another diagram showing the appearance of the ultrasonic diagnostic apparatus according to the embodiment.

[0019] Figure 4 is a block diagram showing a configuration of an ultrasonic diagnostic apparatus according to another embodiment.

[0020] Figure 5 is a block diagram showing a configuration of an ultrasonic diagnostic apparatus according to another embodiment.

[0021] Figure 6 is a diagram showing an ultrasonic image acquired in the embodiment.

[0022] Figure 7 is a diagram illustrating a user interface screen including a diagnosis image provided in an embodiment.

[0023] Figure 8 is a diagram illustrating a user interface screen including another diagnosis image provided in the embodiment.

[0024] Fig. 9 is another diagram showing an ultrasonic image acquired in the embodiment.

[0025] Fig.10 is a diagram illustrating a user interface screen including another diagnosis image provided in the embodiment.

[0026] Fig.11 is a diagram illustrating a user interface screen including another diagnosis image provided in the embodiment.

[0027] Fig.12 is another diagram illustrating a user interface screen including another diagnosis image provided in the embodiment.

[0028] Fig.13 is a diagram illustrating an example of a user interface screen provided in the embodiment.

[0029] Fig.14 is a diagram illustrating another example of a user interface screen provided in the embodiment.

[0030] Fig.15 is a diagram illustrating another example of a user interface screen provided in the embodiment.

[0031] Fig.16 is a diagram illustrating another example of a user interface screen provided in the embodiment.

[0032] Fig.17 is a diagram illustrating another example of a user interface screen provided in the embodiment.

[0033] Fig.18 is a diagram illustrating another example of a user interface screen provided in the embodiment.

[0034] Fig.19 is a diagram illustrating another example of a user interface screen provided in the embodiment.

[0035] Fig. 20 is a diagram illustrating another example of a user interface screen provided in the embodiment.

[0036] Fig.21 is a diagram illustrating another example of a user interface screen provided in the embodiment.

[0037] Fig. 22 is a diagram illustrating another example of a user interface screen provided in the embodiment.

[0038] Fig.23 is a diagram illustrating another example of a user interface screen provided in the embodiment.

[0039] Fig.24 is a diagram illustrating another example of a user interface screen provided in the embodiment.

[0040] Fig.25 is a diagram illustrating a diagnosis image output in response to a user input.

[0041] Fig.26 is another diagram illustrating a diagnostic image output in response to a user input.

[0042] Fig. 27 is another diagram illustrating a diagnostic image output in response to a user input.

[0043] Fig.28 is a diagram illustrating a user interface screen including a plurality of diagnosis images.

[0044] Fig.29 is a diagram for describing a neural network for obtaining analysis results of an ultrasound image.

[0045] Fig.30 is a flowchart illustrating a method of displaying an ultrasound image according to an embodiment.

[0046] Fig.31 is a diagram showing an actual implementation example of a user interface screen according to an embodiment.

[0047] Fig.32 is another diagram showing an actual implementation example of a user interface screen according to an embodiment. DETAILED DESCRIPTION

[0048] Best Practices

[0049] A method for displaying an ultrasound image according to a disclosed embodiment includes: identifying a lesion area included in the ultrasound image; diagnosing the lesion area to obtain a diagnosis result; displaying a first area in the lesion area in the lesion area of ​​the ultrasound image and generating a diagnostic image, wherein the first area is at least one area serving as a basis for diagnosing the lesion; and displaying a user interface screen including the diagnostic image and the diagnostic result.

[0050] In addition, when there are a plurality of features indicating characteristics of the lesion in the lesion region, the first region may be a region indicating at least one of the plurality of features.

[0051] In addition, the first region may be a region serving as a basis for determining at least one of the shape of the lesion, the orientation of the lesion, the edge of the lesion, the echogenicity with respect to the lesion region, and the rear of the lesion region.

[0052] Additionally, the diagnosis result may include information indicating whether the lesion is benign or malignant.

[0053] In addition, generating the diagnostic image may include: when there are multiple features indicating the lesion area, using at least one of different colors, marks, symbols, transparency, echo, dashed line type and dashed line thickness to display at least one area in the lesion area, so as to distinguish different features from each other among the multiple features.

[0054] In addition, the plurality of features may include at least one of a shape of the lesion, an orientation of the lesion, a marginal form of the lesion, an echogenicity for the lesion region, and a posterior feature of the lesion region as a basis for diagnosing the lesion.

[0055] In addition, generating the diagnostic image may include: when a feature of the lesion area appearing in a second area included in the lesion area changes and thus a diagnosis result changes, displaying the second area in the ultrasound image and generating the diagnostic image.

[0056] In addition, generating the diagnostic image may further include displaying the first region to be distinguished from the second region and generating the diagnostic image.

[0057] Additionally, displaying the user interface screen may include displaying the user interface screen including a list including a plurality of items corresponding to a plurality of features indicating the diagnosis image, the diagnosis result, and the lesion.

[0058] In addition, generating the diagnostic image may further include: in response to a user input for selecting any one of the multiple items, displaying at least one partial area indicating a feature corresponding to the selected item as the first area in the ultrasound image, and generating the diagnostic image.

[0059] In addition, in the list, the plurality of items may include information specifically indicating features corresponding to the plurality of items, and each of the plurality of items may be displayed to match at least one partial region corresponding thereto.

[0060] In addition, displaying the user interface screen may further include displaying the first feature included in the list to be distinguished from other features when the first feature included in the list changes and thus the diagnosis result changes.

[0061] In addition, generating the diagnostic image may include: displaying diagnostic information, which is information about a feature of the lesion region appearing in the first region, in the ultrasound image and generating the diagnostic image.

[0062] In addition, obtaining the diagnostic information may include: using at least one of computer-aided detection and diagnosis (CAD) technology and artificial intelligence (AI) technology to analyze the ultrasound image, and obtaining at least one of the lesion area, the diagnostic result, and the diagnostic information corresponding to the diagnostic result.

[0063] In addition, identifying the lesion area may include identifying the outline of the lesion area, and the method of displaying an ultrasound image may further include: when there are multiple outlines as a result of the identification, displaying the multiple outlines in the ultrasound image and generating multiple sub-images; and outputting a user interface screen including the multiple sub-images.

[0064] An ultrasound diagnostic device may include: a display; a memory configured to store at least one instruction; and a processor configured to execute at least one of the at least one instruction to identify a lesion area included in an ultrasound image, diagnose the lesion area to obtain a diagnosis result, display a first area in the lesion area of ​​the ultrasound image to generate a diagnostic image, and control a user interface screen including the diagnostic image and the diagnostic result to be displayed through the display, the first area being at least one area serving as a basis for diagnosing a lesion in the lesion area.

[0065] A computer program product may include: a computer program including commands that can be executed by a computer and configured to be stored in a recording medium to execute a method for displaying an ultrasound image on a computer, the computer program may include: identifying a lesion area included in an ultrasound image; diagnosing the lesion area to obtain a diagnosis result; displaying a first area in the lesion area of ​​the ultrasound image and generating a diagnostic image, the first area being at least one area serving as a basis for diagnosing the lesion; and displaying a user interface screen including the diagnostic image and the diagnostic result.

[0066] Mode for Carrying Out the Invention

[0067] This specification describes the principles of the present disclosure and discloses embodiments so that the scope of the present disclosure can be clarified and those skilled in the art to which the present disclosure belongs can implement the present disclosure. The disclosed embodiments can be implemented in various forms.

[0068] Throughout this specification, the same reference numerals represent the same components. This specification does not describe all components of the embodiment, and the general description in the technical field to which the present disclosure belongs and the repeated description between the embodiments will be omitted. The terms "component" or "part" used herein can be implemented in software or hardware, and according to the embodiment, multiple "components" or "parts" can be implemented as one unit or one element, or one "component" or "part" can include multiple units or elements. Hereinafter, the operating principle and embodiments of the present disclosure will be described with reference to the accompanying drawings.

[0069] In this specification, an “image” may include a medical image obtained by a medical imaging apparatus such as a magnetic resonance imaging (MRI) apparatus, a computed tomography (CT) apparatus, an ultrasonic imaging apparatus, and an X-ray imaging apparatus.

[0070] In this specification, the "object" is to be photographed and may include a person, an animal, or a part thereof. For example, the object may include a part (organ) of the human body, a phantom, and the like.

[0071] Throughout the specification, an 'ultrasound image' refers to an image of an object, which is processed based on an ultrasound signal transmitted to and reflected from the object.

[0072] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings.

[0073] Figure 1 is a block diagram showing the configuration of an ultrasonic diagnostic apparatus according to the embodiment.

[0074] Figure 1 1 is a block diagram showing a configuration of an ultrasonic diagnostic apparatus 100 according to an embodiment. The ultrasonic diagnostic apparatus 100 may include a probe 20, an ultrasonic transceiver 110, a controller 120, an image processing unit 130, a display unit 140, a storage unit 150, a communication unit 160, and an input unit 170.

[0075] The ultrasonic diagnostic apparatus 100 may be implemented as a portable type as well as a cart type. Examples of the portable ultrasonic diagnostic apparatus may include a smartphone, a laptop computer, a personal digital assistant (PDA), a tablet personal computer (PC), etc. (including a probe and an application), but the present disclosure is not limited thereto.

[0076] The probe 20 may include a plurality of transducers. The plurality of transducers may transmit an ultrasonic signal to the object 10 according to a transmission signal applied from the transmission unit 113. The plurality of transducers may receive the ultrasonic signal reflected from the object 10 to form a reception signal. In addition, the probe 20 may be implemented as one body with the ultrasonic diagnostic apparatus 100, or may be implemented as a separate type in which the probe 20 is connected to the ultrasonic diagnostic apparatus 100 in a wired or wireless manner. In addition, according to the implementation form, the ultrasonic diagnostic apparatus 100 may include one or more probes 20.

[0077] The controller 120 controls the transmission unit 113 to form a transmission signal to be applied to each of the plurality of transducers in consideration of the positions and focal points of the plurality of transducers included in the probe 20 .

[0078] The controller 120 controls the receiving unit 115 to convert reception signals received from the probe 20 in an analog-to-digital conversion manner in consideration of the positions and focal points of the plurality of transducers, and sums the digitally converted reception signals, thereby generating ultrasound data.

[0079] The image processing unit 130 generates an ultrasound image using the ultrasound data generated by the ultrasound receiving unit 115 .

[0080] The display unit 140 may display a generated ultrasound image and various information processed by the ultrasound diagnostic apparatus 100. According to an implementation form, the ultrasound diagnostic apparatus 100 may include one or more display units 140. In addition, the display unit 140 may be implemented as a touch screen combined with a touch panel.

[0081] The controller 120 may control the overall operation of the ultrasonic diagnostic apparatus 100 and the signal flow between the internal components of the ultrasonic diagnostic apparatus 100. The controller 120 may include a memory storing a program or data for performing the functions of the ultrasonic diagnostic apparatus 100 and a processor for processing the program or data. In addition, the controller 120 may control the operation of the ultrasonic diagnostic apparatus 100 by receiving a control signal from the input unit 170 or an external device.

[0082] The ultrasonic diagnostic apparatus 100 may include a communication unit 160 , and may be connected to an external device (eg, a server, a medical device, a portable device (smart phone, tablet PC, wearable device, etc.)) through the communication unit 160 .

[0083] The communication unit 160 may include one or more components that enable communication with an external device, and may include, for example, at least one of a short-range communication module, a wired communication module, and a wireless communication module.

[0084] The communication unit 160 may receive a control signal and data from an external device and transmit the received control signal to the controller 120 , so that the controller 120 may control the ultrasound diagnostic apparatus 100 in response to the received control signal.

[0085] Alternatively, the controller 120 may transmit a control signal to the external device through the communication unit 160 , so that the external device may be controlled in response to the control signal of the controller 120 .

[0086] For example, the external device may process data of the external device in response to a control signal of the controller received through the communication unit.

[0087] A program capable of controlling the ultrasonic diagnostic apparatus 100 may be installed in an external device, and the program may include instructions for executing some or all operations of the controller 120 .

[0088] The program may be pre-installed in the external device, or may be installed by a user of the external device by downloading the program from a server providing the application. The server providing the application may include a recording medium storing the corresponding program.

[0089] The storage unit 150 may store various types of data or programs for driving and controlling the ultrasonic diagnostic apparatus 100 , input / output ultrasonic data, acquired ultrasonic images, and the like.

[0090] The input unit 170 may receive a user input for controlling the ultrasonic diagnostic apparatus 100. For example, the user input may include an input for manipulating a button, a keyboard, a mouse, a trackball, a micro switch, a knob, etc., an input for touching a touch pad or a touch screen, a voice input, a motion input, and a biometric information input (e.g., iris recognition or fingerprint recognition), but the present disclosure is not limited thereto.

[0091] Figure 2 is a block diagram showing a configuration of an ultrasonic diagnostic apparatus according to another embodiment.

[0092] Reference Figure 2 , the ultrasonic diagnostic apparatus 100 may include a wireless probe 20 and a main body 40. Figure 2 The same reference numerals are used to indicate Figure 1 Therefore, in the description Figure 2 The ultrasonic diagnostic apparatus 100 shown in FIG. Figure 1 In addition, since the receiving unit 115 can be formed by a plurality of transducers, Figure 1 The "receiving unit 115" shown in Figure 2 Shown as “transducer 115 ” in FIG.

[0093] The wireless probe 20 may include a transmitting unit 113, a transducer 115, a receiving unit 117, a controller 118, and a communication unit 119. Figure 2 113 and the receiving unit 117, but according to the implementation form, the wireless probe 20 may only include some components of the transmitting unit 113 and the receiving unit 117, and some components of the transmitting unit 113 and the receiving unit 117 may be included in the body 40. Optionally, the wireless probe 20 may further include an image processing unit 130.

[0094] The transducer 115 may include a plurality of transducers. The plurality of transducers may transmit an ultrasonic signal to the object 10 according to a transmission signal applied from the transmission unit 113. The plurality of transducers may receive the ultrasonic signal reflected from the object 10 to form a reception signal.

[0095] The controller 118 controls the transmission unit 113 to form a transmission signal to be applied to each of the plurality of transducers in consideration of the positions and focal points of the plurality of transducers.

[0096] Considering the positions and focal points of the multiple transducers, the controller 118 controls the receiving unit 117 to convert the received signals received from the transducer 115 in an analog-to-digital conversion manner, and sums the digitally converted received signals to generate ultrasound data. Optionally, when the wireless probe 20 includes an image processing unit 130, the generated ultrasound data can be used to generate an ultrasound image.

[0097] The communication unit 119 may wirelessly transmit the generated ultrasound data or ultrasound image to the subject 40 through the wireless network 30. In addition, the communication unit 119 may receive a control signal and data from the subject 40.

[0098] In addition, according to the implementation form, the ultrasound diagnostic apparatus 100 may include one or more wireless probes 20 .

[0099] The main body 40 may receive ultrasound data or an ultrasound image from the wireless probe 20. The main body 40 may include a controller 120, an image processing unit 130, a display unit 140, a storage unit 150, a communication unit 160, and an input unit 170.

[0100] Figure 3a is a diagram showing the appearance of an ultrasonic diagnostic apparatus according to the embodiment.

[0101] Figure 3b is another diagram showing the appearance of the ultrasonic diagnostic apparatus according to the embodiment.

[0102] Figure 3c is another diagram showing the appearance of the ultrasonic diagnostic apparatus according to the embodiment.

[0103] Reference Figure 3a and Figure 3b , the ultrasonic diagnostic apparatuses 100a and 100b may include a main display unit 121 and a sub-display unit 122, respectively. One of the main display unit 121 and the sub-display unit 122 may be implemented as a touch screen. The main display unit 121 and the sub-display unit 122 may display an ultrasonic image or various information processed by the ultrasonic diagnostic apparatuses 100a and 100b. In addition, the main display unit 121 and the sub-display unit 122 may be implemented as a touch screen, and a graphical user interface (GUI) is provided to receive data for controlling the ultrasonic diagnostic apparatuses 100a and 100b from a user. For example, the main display unit 121 may display an ultrasonic image, and the sub-display unit 122 may display a control panel for controlling the ultrasonic image in the form of a GUI. The sub-display unit 122 may receive data for controlling the display of an image through the control panel displayed in the form of a GUI. The ultrasonic diagnostic apparatuses 100a and 100b may control the display of the ultrasonic image displayed on the main display unit 121 using the input control data.

[0104] Reference Figure 3b In addition to the main display unit 121 and the sub-display unit 122, the ultrasonic diagnostic device 100b may further include a control panel 165. The control panel 165 may include buttons, a trackball, a micro switch, a knob, etc., and may receive data for controlling the ultrasonic diagnostic device 100b from a user. For example, the control panel 165 may include a time gain compensation (TGC) button 171, a freeze button 172, etc. The TGC button 171 is a button for setting a TGC value for each depth of an ultrasonic image. In addition, when an input of the freeze button 172 is detected while scanning an ultrasonic image, the ultrasonic diagnostic device 100b may remain in a state of displaying a frame image at a corresponding time point.

[0105] Meanwhile, an input of a button, a track ball, a micro switch, a knob, etc. included in the control panel 165 may be provided to a GUI in the main display unit 121 or the sub display unit 122 .

[0106] Reference Figure 3c , the ultrasonic diagnostic apparatus 100c may be implemented as a portable type. Examples of the portable ultrasonic diagnostic apparatus 100c may include a smart phone, a laptop computer, a PDA, a tablet PC, etc. (including a probe and an application), but the present disclosure is not limited thereto.

[0107] The ultrasonic diagnostic apparatus 100c may include a probe 20 and a body 40, and the probe 20 may be connected to one side of the body 40 in a wired or wireless manner. The body 40 may include a touch screen 145. The touch screen 145 may display an ultrasonic image, various information processed by the ultrasonic diagnostic apparatus, a GUI, and the like.

[0108] According to the method for displaying an ultrasonic image, an ultrasonic diagnostic device, and a computer program product of the disclosed embodiment, a diagnostic result is generated by analyzing the ultrasonic image to determine whether an abnormality occurs in the object, and the information corresponding to the diagnostic result appearing in the ultrasonic image is provided to the user, so that the user can accurately diagnose the patient's disease by referring to the diagnostic result to increase the convenience of the user. Here, the "user" may be a doctor who diagnoses the patient's disease, an ultrasound physician who performs an ultrasonic scan on the patient's object, and the like. In addition, "information corresponding to the diagnostic result" refers to information corresponding to the diagnostic result displayed on the ultrasonic image, information serving as the basis for the diagnostic result, a local area in the ultrasonic image serving as the basis for the diagnostic result, and a local area in the ultrasonic image that affects the diagnostic result. In the following, for convenience, "information corresponding to the diagnostic result" will be referred to as "diagnostic information."

[0109] As described above, a method of displaying an ultrasound image, an ultrasound diagnostic apparatus, and a computer program product capable of increasing the convenience of a user who automatically receives a diagnosis result will be described in detail with reference to the accompanying drawings.

[0110] In the disclosed embodiments, the ultrasound diagnostic device may be an electronic device capable of acquiring, processing and / or displaying ultrasound images. Specifically, the ultrasound diagnostic device may refer to an electronic device capable of performing the following operations: i) identifying a specific portion (e.g., a portion where a lesion occurs) in an ultrasound image; ii) analyzing the ultrasound image to obtain a diagnosis result or diagnostic information; or iii) processing, generating, modifying, updating or displaying a partial image, an entire image or information for diagnosis based on the ultrasound image.

[0111] Specifically, if Figure 3a to Figure 3c As shown, the ultrasonic diagnostic device according to the disclosed embodiment can be implemented as a cart-type device as well as a portable device. Examples of portable ultrasonic diagnostic devices may include a picture archiving and communication system (PACS) viewer, a smart phone, a laptop computer, a PDA, a tablet PC, etc., but the present disclosure is not limited thereto.

[0112] Figure 4 FIG. 1 is a block diagram showing a configuration of an ultrasonic diagnostic apparatus according to another embodiment. Figure 4 In the embodiment, the memory 410, the processor 420 and the display 430 may correspond to Figure 1 The storage unit 150, the controller 120 and the display unit 140 are shown, so the description of the storage unit 150 and the display unit 140 will be omitted here. Figure 1 Description of duplicate description.

[0113] Reference Figure 4 , the ultrasonic diagnostic apparatus 400 according to the disclosed embodiment includes a memory 410 , a processor 420 , and a display 430 .

[0114] The memory 410 stores at least one instruction.

[0115] The display 430 displays a predetermined screen.

[0116] The processor 420 may execute at least one of at least one instruction stored in the memory 410 to perform a predetermined operation, or may control other components in the ultrasound diagnostic apparatus 400 to perform a predetermined operation.

[0117] Specifically, the processor 420 executes at least one of at least one instruction to identify a lesion area included in an ultrasound image, diagnose the lesion area to obtain a diagnostic result, display a first area on the lesion area of ​​the ultrasound image to generate a diagnostic image, and control a user interface screen including the diagnostic image and the diagnostic result to be displayed through the display, the first area being at least one area that serves as a basis for diagnosing a lesion in the lesion area.

[0118] Here, lesions refer to changes in a living body due to a disease. Specifically, lesions may refer to all forms in which an object does not have a healthy tissue form or a situation in which an object has a state different from a healthy state. Specifically, lesions may be generally classified into benign and malignant. In addition, benign lesions may correspond to tumors, and malignant lesions may correspond to cancer.

[0119] In addition, in the disclosed embodiments, the "lesion region" may be a concept including not only the internal region of the lesion formed by the outline of the lesion but also the external region adjacent to the outline of the lesion. That is, the lesion region may mean not only the region where the lesion itself is imaged in the ultrasound image but also the region where the features of the lesion are imaged.

[0120] In addition, when there is a lesion in the object, the diagnostic information may be information indicating at least one of the identified lesion area, the size of the lesion, the location of the lesion, and the characteristics of the lesion (e.g., metastasis probability, malignancy risk, etc.). In addition, the diagnostic information may include standard terms for the disease defined by a society or association. For example, the diagnostic information may include information about a plurality of items included in a dictionary that is a standard term for ultrasound examination defined by the American College of Radiology (ACR).

[0121] In addition, the diagnosis result may include information indicating whether the identified lesion is malignant or benign, or include the specific name of the lesion. As another example, the diagnosis result may include information indicating whether the identified lesion is a papule, a nodule, a tumor, or a cancer. As another example, the diagnosis result may include information indicating the specific name of the lesion (e.g., a breast tumor, breast cancer, a thyroid tumor, a thyroid nodule, a thyroid cancer, etc.).

[0122] Hereinafter, for the convenience of description, a case where information indicating whether the diagnosis result is malignant or benign is included will be described and illustrated as an example. For example, the diagnosis result may be expressed as "likely benign", "benign", "likely malignant", "malignant", etc.

[0123] In the disclosed embodiments, the first region is at least a portion of a lesion region that is a basis for diagnosing a lesion, and may refer to at least a portion of an ultrasound image that is a basis for diagnosing a lesion. For example, when the lesion in the subject is a malignant lesion, the diagnosis result may be "malignant". In this case, the first region may be a feature in the ultrasound image that is a basis for determining the corresponding lesion as malignant, or an area where a feature in the ultrasound image is imaged. Figures 6 to 11 The first region is described in detail.

[0124] In addition, in the disclosed embodiments, "at least one of a, b and c" may have the meaning of including all of the following cases: only a, only b, only c, a and b, a and c, b and c, and a, b and c.

[0125] Specifically, the display 430 displays a predetermined screen under the control of the processor 420. The display 430 may display a medical image (e.g., an ultrasound image), a user interface screen, information about a patient, historical information about a patient's disease, image processing information, etc. Here, the image processing information may include an intermediate product or a final product generated by processing the ultrasound image by the processor 420. In addition, "image processing" may refer to an operation of processing, generating, modifying, and / or updating an ultrasound image.

[0126] For example, the display 430 may display a user interface screen including at least one of an ultrasound image, a diagnostic image corresponding to the ultrasound image, a diagnosis result of a lesion recognized based on the ultrasound image, and information for describing the diagnosis result.

[0127] Specifically, the memory 410 may store at least one program required for the operation of the ultrasonic diagnostic apparatus 400 or at least one instruction required for executing the at least one program. In addition, the memory 410 may include at least one processor for performing the above-mentioned operations.

[0128] In addition, the memory 410 may store at least one of an ultrasound image, information related to the ultrasound image, information about a patient, information about an object, and information about a test object. In addition, the memory 410 may store at least one of information and an image generated by the processor 420. In addition, the memory 410 may store at least one of an image, data, and information received from an external electronic device (not shown).

[0129] The processor 420 may generally control the operation of the ultrasonic diagnostic apparatus 400. Specifically, the processor 420 may execute at least one instruction to perform a predetermined operation or control a predetermined operation to be performed. Here, the instruction executed by the processor 420 may be at least one of the at least one instructions stored in the memory 410.

[0130] In addition, the processor 420 may be implemented in the form of one processor or in the form of a combination of multiple processors.

[0131] In addition, the processor 420 may include an internal memory (not shown) and at least one processor (not shown) for executing at least one stored program. Specifically, the internal memory (not shown) of the processor 420 may store one or more instructions. In addition, at least one processor (not shown) included in the processor 420 may execute at least one of the one or more instructions stored in the internal memory (not shown) of the processor 420 to perform a predetermined operation.

[0132] Specifically, the processor 420 may include a random access memory (RAM) (not shown), a read-only memory (ROM) (not shown), and at least one processor (not shown), the random access memory (RAM) being used as a storage area for storing signals or data input from the outside of the ultrasonic diagnostic device 400 or as a storage area corresponding to various tasks performed by the ultrasonic diagnostic device 400, and the read-only memory (ROM) storing a control program and / or a plurality of instructions for controlling the ultrasonic diagnostic device 400. The processor (not shown) may include a graphics processor (graphics processing unit, not shown) for graphics processing corresponding to video data. The processor (not shown) may be implemented as a system on chip (SoC) in which a core (not shown) and a GPU (not shown) are integrated. The processor (not shown) may include a single core, a dual core, a triple core, a quad core, and a multi-core.

[0133] For ease of description, in the disclosed embodiments, a case where the processor 420 executes instructions stored in the memory 410 to perform or control a predetermined operation will be described as an example.

[0134] Figure 5 FIG. 1 is a block diagram showing a configuration of an ultrasonic diagnostic apparatus according to another embodiment. Figure 5 The same reference numerals are used to indicate Figure 4 Therefore, in describing Figure 5 The ultrasonic diagnostic apparatus 500 shown in FIG. Figure 4 Description in Duplicate description.

[0135] When compared with the ultrasonic diagnostic apparatus 400, the ultrasonic diagnostic apparatus 500 may further include at least one of a data acquisition unit 440, an image processing unit 450, a user interface 460, and a communication unit 470. The data acquisition unit 440, the image processing unit 450, the user interface 460, and the communication unit 470 may correspond to the respective Figure 1 The ultrasonic transceiver 110, the image processing unit 130, the input unit 170 and the communication unit 160 are provided. Therefore, when describing the ultrasonic diagnostic apparatus 500, the description related to the ultrasonic diagnostic apparatus 500 will be omitted. Figure 1 Description in Duplicate description.

[0136] The data acquisition unit 440 may acquire ultrasound data. Specifically, the data acquisition unit 440 may receive an ultrasound echo signal reflected from an object through the probe 20 under the control of the processor 420 .

[0137] Specifically, the data acquisition unit 440 may directly acquire raw data for generating an ultrasound image under the control of the processor 420. In this case, the data acquisition unit 440 may correspond to the ultrasound transceiver 110. Here, the raw data may refer to raw data corresponding to an ultrasound echo signal acquired by being reflected from an object, or refer to data before processing.

[0138] Then, the processor 420 may control to generate an ultrasonic image corresponding to the ultrasonic echo signal received through the data acquiring unit 440. Specifically, the processor 420 may control the image processing unit 450 to generate an ultrasonic image using the ultrasonic echo signal.

[0139] In addition, the data acquisition unit 440 may receive an ultrasound image from an external electronic device (not shown) such as an external ultrasound diagnostic device (not shown) under the control of the processor 420. Specifically, the data acquisition unit 440 may be connected to the external electronic device (not shown) through a wired / wireless communication network, and may receive an ultrasound image transmitted from the external electronic device (not shown) or raw data for generating an ultrasound image under the control of the processor 420. When the data acquisition unit 440 acquires the raw data, the processor 420 may control the image processing unit 450 to generate an ultrasound image using the raw data received through the data acquisition unit 440.

[0140] When the data acquisition unit 440 receives an ultrasound image or raw data from an external electronic device (not shown), the data acquisition unit 440 may include a communication unit 470. That is, the communication unit 470 may be the data acquisition unit 440 itself, and the communication unit 470 may be implemented in the form of being included in the data acquisition unit 440.

[0141] The image processing unit 450 may perform at least one of generating an ultrasound image and processing the ultrasound image. In addition, the image processing unit 450 may be implemented in the form of being included in the processor 420. For example, when the processor 420 includes a plurality of processors, any one of the plurality of processors included in the processor 420 may be an image processor for image processing.

[0142] The user interface 460 may receive predetermined data or a predetermined command from a user. The user interface 460 may correspond to Figure 1 The user interface 460 may be formed as a touch screen or a touch pad integrally formed with the display 430. As another example, the user interface 460 may include a user input device such as a button, a keyboard, a mouse, a trackball, a micro switch, and a knob. In addition, the user interface 460 may include a microphone, a motion detection sensor, and a biometric information detection sensor for receiving voice input, motion input, and biometric information input (e.g., iris recognition, fingerprint recognition, etc.).

[0143] Therefore, the user interface 460 can receive input from manipulation buttons, keyboards, mice, trackballs, micro switches and knobs, input from touching a touchpad or touch screen, voice input, motion input and / or biometric information input (e.g., iris recognition, fingerprint recognition, etc.).

[0144] In the disclosed embodiment, the processor 420 may identify the lesion area based on the user input received through the user interface 460. For example, the ultrasound diagnostic apparatus 500 may display an ultrasound image through the display 430. The user may view the ultrasound image and input an input for selecting a predetermined position or a predetermined area corresponding to a portion of the suspected lesion in the ultrasound image through the user interface 460. Then, based on the received user input, the processor 420 may analyze the ultrasound image around or near the predetermined position or predetermined area selected by the user input, or analyze the ultrasound image within a predetermined range based on the selected position or predetermined area. Therefore, according to the analysis result, the outline of the lesion and the lesion area formed by the outline of the lesion can be identified.

[0145] In addition, in the disclosed embodiment, the processor 420 can automatically detect the lesion area included in the ultrasound image through a computer-based image processing technology. In the above example, the user can view the ultrasound image and input an input for selecting a predetermined position or a predetermined area corresponding to a portion of the suspected lesion in the ultrasound image through the user interface 460. Then, the processor 420 can use a computer-based automatic diagnosis technology to analyze at least a portion of the ultrasound image corresponding to the position or area selected by the user input. In addition, as a result of the analysis, the processor 420 can accurately extract the edge of the lesion or the lesion area formed by the edge of the lesion.

[0146] Here, computer-based image processing technology may include machine learning-based diagnosis technology, etc.

[0147] Here, machine learning can be performed by a CAD system that determines and detects whether an abnormality or lesion occurs in an object through computer operation, statistical machine learning based on data, or an artificial intelligence system that performs machine learning according to artificial intelligence technology. In the disclosed embodiment, the processor 420 can use computer-based image processing technology to analyze the ultrasound image and obtain desired information (e.g., lesion area, information about the lesion, diagnosis result, etc.) as the analysis result.

[0148] Here, unlike the existing rule-based intelligent system, the artificial intelligence (AI) system is a system in which the machine learns and determines and generates the results expected by the user. AI technology includes machine learning (deep learning) and element technology using machine learning. Machine learning is an algorithm technology that classifies / learns the features of input data by itself, and element technology is a technology that utilizes machine learning algorithms such as deep learning and includes technical fields such as language understanding, visual understanding, deduction / prediction, knowledge expression, and motion control.

[0149] In the disclosed embodiments, the processor 420 may use machine learning, which is the above-mentioned computer-based image processing technology, to identify lesion areas, obtain diagnosis results, obtain diagnostic information, and / or establish criteria for identifying lesions.

[0150] Specifically, the image processing by AI technology can be performed by an operation based on a neural network. Specifically, an operation based on a neural network (such as a deep neural network (DNN)) can be used. In addition, the operation of the DNN may include an operation of a convolutional neural network (CNN). Fig.29 The operation of processing ultrasound images by AI technology is described in detail.

[0151] In the following, reference will be made to Figure 6 to Figure 32 The operations performed in the ultrasonic diagnostic apparatus 100, 400, or 500 according to the disclosed embodiment are described in detail. In addition, for ease of description, the following will use Figure 4 The operation of the disclosed embodiment is described using the ultrasonic diagnostic apparatus 400 as an example.

[0152] Figure 6 is a diagram showing an ultrasonic image acquired in the embodiment.

[0153] Reference Figure 6 , shows an ultrasound image 600 acquired by the ultrasound diagnostic device 400. Specifically, as an example, Figure 6An ultrasound image 600 corresponding to a portion of a breast acquired by a breast ultrasound scan is shown.

[0154] The processor 420 identifies the lesion region 610 included in the ultrasound image 600. Specifically, the processor 420 may analyze the ultrasound image 600 using an automatic diagnosis technology such as a CAD technology or an AI technology. In addition, as an analysis result, the processor 420 may automatically detect the lesion region 610 included in the ultrasound image 600. Hereinafter, the operations of identifying the lesion region 610 and analyzing the lesion region 610 to obtain at least one of a diagnosis result and diagnosis information may be performed using the CAD technology.

[0155] For example, the user may view the ultrasound image 600 and select a region of a suspected lesion or a position or a predetermined region in a region adjacent to the suspected portion in the ultrasound image 600 through the user interface 460. Specifically, the user may point to, touch, or click a position 611 in the lesion region 610 through the user interface 460. Then, the processor 420 may analyze the region around the position or region selected by the user input using the CAD technology or the AI ​​technology to accurately detect the lesion region 610.

[0156] In addition, the processor 420 may use CAD technology or AI technology to analyze the lesion area 610 to obtain a diagnosis result.

[0157] In addition, in order to allow the user to easily identify the detected lesion area 610, the processor 420 may process the outline 615 of the lesion area 610 to generate a diagnostic image (not shown) so as to allow the outline 615 of the lesion area 610 to be clearly displayed. For example, in order to allow the user to easily identify the outline 615 of the lesion area 610, the diagnostic image (not shown) may be an image generated by marking a line of a predetermined color on the outline 615 of the lesion area 610 in the ultrasound image 600. In addition, there may be various processes that allow the user to easily identify the lesion area 610. As another example, the diagnostic image (not shown) may be generated by a process of adjusting the transparency of the lesion area 610, marking the lesion area 610 with a specific color or pattern, or marking the outline 615 of the lesion area 610 with an outline line such as a dotted line or a solid line.

[0158] Then, the processor 420 displays the first region on the lesion region 610 of the ultrasound image 600 to generate a diagnostic image (not shown). Here, the first region is at least a portion of the lesion region as a basis for diagnosing the lesion, and may refer to at least a portion of the ultrasound image as a basis for diagnosing the lesion.

[0159] When a lesion is distinguished as malignant or benign, the features of the lesion classified as malignant are different from the features of the lesion classified as benign. When a lesion is distinguished as malignant or benign, a standard for distinguishing the lesion may be set based on at least one of a CAD system, an AI system, and a user setting. There are various types of lesions, and the lesions may be distinguished according to various standards. Therefore, the lesion discrimination standard may be optimized and set by experiments or machine learning.

[0160] Specifically, the lesion region 610 is the basis for analyzing the lesion, and the diagnosis result can be determined based on the periphery of the lesion region 610 and the contents contained in its internal area. For example, the lesion can be identified as a specific lesion based on the result of analyzing at least one of the shape of the lesion region, the orientation of the lesion, the edge of the lesion, the echo characteristics presented in the lesion region, and the posterior characteristics of the lesion region.

[0161] When diagnosing a lesion using CAD technology, etc., it is important to accurately extract the lesion region. In addition, the lesion region can be accurately extracted only when the image accuracy of the lesion region 610 (which is the region where the lesion is imaged in the ultrasound image 600) and the image accuracy of the surrounding area of ​​the lesion region 610 are high.

[0162] However, depending on the specific shape of the lesion, the quality of the ultrasound image, the product specifications of the ultrasound diagnostic apparatus performing the ultrasound scan, and the settings applied during the ultrasound scan of the ultrasound diagnostic apparatus, it may occur that it is difficult to accurately extract the lesion region.

[0163] In particular, when the outline of the identified lesion area changes, the shape of the lesion area, the orientation of the lesion, the edge characteristics of the lesion, etc. may be changed. Therefore, the diagnosis result of the lesion may also change. Therefore, when the lesion area is not accurately extracted and the diagnosis result is not accurately derived, the reliability of the diagnosis result may be reduced.

[0164] Existing ultrasonic diagnostic apparatuses that provide diagnostic results using CAD systems and the like only provide diagnostic results without describing the process of deriving the diagnostic results. Therefore, even when there is a situation where an erroneous diagnostic result is provided or a situation where the conclusion is unclear when deriving the diagnostic result, the user cannot recognize such an error. Therefore, when the user trusts the erroneous diagnostic result to finally derive the diagnostic result, the patient's disease may be misdiagnosed.

[0165] In addition, providing an erroneous diagnosis result does not particularly help the user to diagnose the object and may confuse the user when diagnosing the object.

[0166] In an embodiment of the present disclosure, in an ultrasonic diagnostic device 400 that identifies a lesion area by analyzing an ultrasonic image and provides a diagnostic result of the lesion, a diagnostic image in which an area (specifically, a first area) serving as a basis for the diagnostic result is displayed is provided to a user, thereby allowing the user to more trust the diagnostic result and more easily diagnose the object by referring to the diagnostic result and the first area.

[0167] For example, when the shape of the lesion area has an irregular shape or a variable shape, the edge of the lesion area becomes a spicule pattern, and the echo characteristics have a high echo characteristic with a low level mark, the breast lesion is more likely to be diagnosed as malignant. Figure 6 The lesion region 610 in the ultrasound image 600 shown has the above-mentioned characteristics of a malignant lesion, and thus the processor 420 may generate a diagnosis result such as “malignant” or “possibly malignant” as an analysis result of the lesion region 610 in the ultrasound image 600 .

[0168] In the above example, the basis for diagnosing the lesion included in the ultrasound image 600 as malignant may be that the shape of the lesion region 610 has an irregular shape rather than an oval or elliptical shape, and there is a burr pattern in the edge of the lesion region. Therefore, the shape of the outline 615 of the lesion region 610 and the edge regions 620 and 630 having the burr pattern in the respective lesion regions 610 may be the basis of the diagnosis result (specifically, the lesion is malignant).

[0169] Therefore, in the disclosed embodiment, since a diagnostic image is provided to the user in which an area (e.g., 620, 615 or 630 as an area showing a contour) that serves as a basis for diagnosing the lesion area 610 as malignant is displayed, the user can quickly and easily diagnose the object.

[0170] In an embodiment of the present disclosure, the first region of at least one region serving as a basis for diagnosing a lesion may be displayed as being distinguished from other regions in the diagnostic image. Specifically, the processor 420 may use at least one of a specific color, a symbol, transparency, an echo, a mark, and a line with a specific pattern to distinguish and display the first region and other regions in the diagnostic image.

[0171] Figure 7 is a diagram showing a user interface screen including a diagnostic image provided in an embodiment. Specifically, Figure 7 700 is shown as a user interface screen output from the ultrasonic diagnostic apparatus 400. Figure 7 of Figure 8 , Fig.10 , Fig.21 as well as Figure 23 to Figure 28, a lesion region (eg, 720) included in a diagnostic image (eg, 710) is simplified and schematically shown. Here, the outline of the lesion region 720 may correspond equivalently to Figure 6 The contour 615 of the lesion area.

[0172] Reference Figure 7 , the user interface screen 700 may include a diagnosis image 710 and a partial screen 750 indicating a diagnosis result. Hereinafter, for convenience of description, the partial screen 750 indicating a diagnosis result included in the user interface screen 700 will be referred to as a "diagnosis result 750".

[0173] Reference Figure 7 , the portion that is the basis for determining the lesion imaged in the diagnostic image 710 as “malignant” may be referred to as a portion having a burr pattern of the edge of the lesion region 720. Therefore, the diagnostic image 710 may be an image in which the region 730 having the burr pattern is displayed as the first region in the edge of the lesion region 720.

[0174] For example, the processor 420 may display a first area (eg, 730) to be distinguished from other areas. Figure 7 , a case where the first area is displayed using area indicator lines is shown as an example. Therefore, the user can view the diagnostic image 710 (which is an ultrasound image including a lesion) to easily identify the features of the lesion imaged in the first area (e.g., 730) (specifically, the edge with the burr pattern). Therefore, the user can quickly and easily determine whether the diagnosis result (e.g., "malignant") provided by the ultrasound diagnostic device is correct.

[0175] In addition, Figure 7 , a case where one region is displayed as the first region in the diagnostic image 710 is shown as an example. However, in the disclosed embodiment, the first region may exist as a plurality of regions in the diagnostic image 710. That is, in the disclosed embodiment, the "first region" merely refers to a region that is a basis for diagnosing a lesion, rather than referring to the number of regions being one.

[0176] In addition, when there are multiple features indicating the characteristics of the lesion in the lesion area (e.g., 610 or 701), the first area may be an area indicating at least one of the multiple features. Here, the multiple features are features of the lesion for distinguishing the lesion type, and include the shape of the lesion area, the orientation of the lesion, the edge of the lesion, the echo features appearing in the lesion area, and the rear features of the lesion area.

[0177] Specifically, a plurality of the above-described features may correspond to a plurality of items included in a dictionary, which is a standard terminology for ultrasound examination defined by the ACR.

[0178] Therefore, the first area may be an area indicating at least one feature among a plurality of the above-mentioned features.

[0179] Specifically, in Figure 6 In the example, the first areas 620 and 630 displayed may be areas indicating the edges of lesions among the above-mentioned features. Figure 6 In the figure, the displayed contour line 615 may be an area indicating the shape of the lesion area among the above-mentioned features.

[0180] In the disclosed embodiment, in the diagnosis image 710 , at least one region indicating any one of a plurality of the above-mentioned features may be displayed as a first region.

[0181] In addition, in the diagnostic image 710, multiple areas indicating two or more of the multiple above-mentioned features may be displayed as the first area. Here, when multiple areas indicating multiple features are displayed in the diagnostic image 710, the multiple areas may be displayed in the diagnostic image 710 to allow different features to be distinguished and displayed. Specifically, when there are multiple features indicating a lesion area, the processor 420 may display at least one of different colors, marks, symbols, transparency, echoes, dotted line types, and dotted line thicknesses to display at least one area in the lesion area, thereby distinguishing different features from each other among the multiple features. In addition, the processor 420 may control the display of the diagnostic image in which the at least one area is displayed.

[0182] For example, when Figure 6 When the ultrasound image 600 shown in is generated as a diagnostic image, regions 620 and 630 which are regions indicating edge features of a lesion may be marked with red indicator lines, and a contour line 615 indicating a shape feature of the lesion may be marked with a blue indicator line.

[0183] Figure 8 is a diagram showing a user interface screen including another diagnostic image provided in an embodiment. Specifically, Figure 8 FIG. 8 shows a user interface screen 800 output from the ultrasonic diagnostic apparatus 400. Figure 8 In the user interface screen 800 shown, the same reference numerals are used to indicate Figure 7 The components in the user interface screen 700 are the same components as shown.

[0184] In the disclosed embodiment, the processor 420 may display diagnostic information in the ultrasound image, thereby generating a diagnostic image 810 , the diagnostic information being information about features (eg, edge features of the lesion region) of a lesion region appearing in the first region (eg, 730 ).

[0185] Here, the diagnosis information may be information indicating features of a lesion region appearing in the first region or information serving as a basis for deriving a diagnosis result.

[0186] For example, since the feature of the lesion area appearing in the first area 730 becomes the edge feature of the lesion area, the diagnosis information 835 may be "edge: burr" (which is information specifically indicating the edge feature). In addition, the diagnosis information may be "malignant determination basis: edge-burr" (which is information serving as a basis for deriving a malignant determination as a diagnosis result).

[0187] In addition, the diagnosis information may be displayed in a region corresponding to the first region in the diagnosis image (eg, a region adjacent to the first region that does not interfere with the user's observation of the first region).

[0188] In the above-described example, the processor 420 may generate the diagnostic image 810 by displaying “edge: burr” or “malignant determination basis: edge-burr” as the diagnostic information 835 in the region adjacent to the first region 730 .

[0189] Fig. 9 is another diagram showing an ultrasonic image acquired in the embodiment. Fig. 9 , shows an ultrasound image 900 acquired by the ultrasound diagnostic device 400. Specifically, Fig. 9 An ultrasound image 900 corresponding to a breast portion acquired by breast ultrasound scanning is shown as an example.

[0190] In the case of breast lesions, the shape of the lesion area is oval or elliptical, the edge of the lesion area becomes a smooth pattern, and when the echo characteristics have high echo characteristics, the breast lesion is more likely to be diagnosed as benign. Fig. 9 The lesion region 910 in the ultrasound image 900 shown has the above-mentioned features of a benign lesion, and thus the processor 420 may generate a diagnosis result such as “benign” or “likely benign” as an analysis result of the lesion region 910 in the ultrasound image 900 .

[0191] In the above example, the basis for diagnosing the lesion included in the ultrasound image 900 as benign may be that the shape of the lesion region 910 is an oval or ellipse and the edge of the lesion region has a smooth pattern. Therefore, the shape 920 of the lesion region 910 and the edge region 930 having a smooth pattern in the lesion region 910 may be at least some of the regions in the ultrasound image 900 that are the basis for the diagnosis result (specifically, that the lesion is benign). Here, the line 920 indicating the shape of the lesion region 910 may be displayed to overlap with the outline 915 of the lesion region 910, or may be displayed in a region adjacent to the lesion region 910.

[0192] In the disclosed embodiment, a diagnostic image in which regions (e.g., 920 and 930) serving as a basis for diagnosing the lesion region 910 as benign are displayed is provided to the user, thereby allowing the user to have greater trust in the diagnosis result and to more easily diagnose the object by referring to the diagnosis result and the first region.

[0193] Fig.10 is a diagram showing a user interface screen including another diagnostic image provided in an embodiment. Specifically, Fig.10 1000 is shown as a user interface screen output from the ultrasonic diagnostic apparatus 400. Figure 8 and Fig. 9 similar, Fig.10 The lesion region 1020 included in the diagnostic image 1010 is schematically shown. Here, the outline of the lesion region 1020 may equally correspond to Fig. 9 The contour of the lesion area 915.

[0194] Reference Fig.10 , the user interface screen 1000 may include a diagnosis image 1010 and a diagnosis result 1050 .

[0195] The portion that is the basis for determining the lesion imaged in the diagnostic image 1010 as “benign” may be a portion where the edge of the lesion region 1020 has a smooth pattern. Therefore, the diagnostic image 1010 may be an image of a first region in which a region 1030 having a smooth pattern is displayed as an edge of the lesion region 1020. The user can view the diagnostic image 1010 (which is an ultrasound image including the lesion) to easily identify the features of the lesion imaged in the first region (e.g., 1030) (specifically, the edge having a smooth pattern). Therefore, the user can quickly and easily determine whether the diagnostic result (e.g., “benign”) provided by the ultrasound diagnostic apparatus is correct.

[0196] Fig.11 is a diagram showing a user interface screen including another diagnostic image provided in an embodiment. Specifically, Fig.11 FIG. 1 shows a user interface screen 1100 output from the ultrasonic diagnostic apparatus 400. Fig.11 In the user interface screen 1100 shown, the same reference numerals are used to indicate Fig.10 The components in the user interface screen 1000 are the same components as shown.

[0197] In the disclosed embodiment, the processor 420 may display diagnostic information 1135 in the ultrasound image to generate the diagnostic image 1110 , wherein the diagnostic information 1135 is information about features of a lesion region (eg, edge features of the lesion region) appearing in the first region (eg, 1030 ).

[0198] For example, since the features of the lesion area appearing in the first area 1030 become the edge features of the lesion area, the diagnostic information 1135 may be “edge: smooth” (which is information specifically indicating the edge features). In addition, the diagnostic information may be “benign determination basis: edge-smooth” (which is information serving as a basis for deriving a benign determination as a diagnostic result).

[0199] Fig.12 is another diagram illustrating a user interface screen including another diagnosis image provided in the embodiment.

[0200] In the disclosed embodiment, as a result of the analysis of the lesion, the processor 420 may determine whether the diagnosis result has an accuracy greater than or equal to a limit value, and when there is no diagnosis result with an accuracy greater than or equal to the limit value, the processor 420 may derive at least two diagnosis results.

[0201] For example, when the diagnosis result obtained as a result of the analysis of the lesion has an accuracy of 90% or more, the processor 420 may output a diagnosis result. Figure 6 When the lesion 610 shown is determined to be malignant, the processor 420 may determine the diagnosis result of the lesion 610 as “malignant”.

[0202] On the contrary, when the diagnosis result obtained as the analysis result of the lesion has an accuracy less than a limit value (e.g., 90%), the processor 420 may output a plurality of diagnosis results. For example, the diagnosis result 1050 for the lesion 1220 may be displayed in the user interface screen 1200 together with the diagnosis accuracy of each of the plurality of diagnosis results. In addition, when outputting a plurality of diagnosis results, the processor 420 may output the plurality of diagnosis results in the order of the accuracy of the plurality of diagnosis results.

[0203] An example will be described in which the probability that the lesion 1220 is determined to be benign is 70% and the probability that the lesion 1220 is determined to be malignant is 30% as the analysis result of the lesion region 1220 included in the ultrasound image. In addition, in the example, the processor 420 sets the limit value corresponding to the accuracy of the diagnosis result to 90%. In this case, the diagnosis result 1050 for the lesion 1220 may be displayed as "benign: 30% and malignant: 70%".

[0204] In addition, the diagnostic image 1210 may be an image in which both the first region (eg, 1030) serving as a basis for benign determination and the first region (eg, 730) serving as a basis for malignant determination are displayed. Fig. 9 and Fig.11 , the diagnosis image 1210 may be an image in which a plurality of pieces of diagnosis information (eg, 835 and 1135 ) corresponding to the first region are additionally displayed.

[0205] Fig.13 is a diagram illustrating an example of a user interface screen provided in the embodiment.

[0206] In the disclosed embodiment, the processor 420 may generate a user interface screen 1300 including a diagnosis image 1310, a diagnosis result 1330, and a list 1302 indicating features of a lesion area. In addition, the display 430 may display the user interface screen 1300 under the control of the processor 420.

[0207] In addition, the diagnosis image 1310 and the diagnosis result 1330 may be formed as one partial screen 1301. Specifically, the processor 420 may display the diagnosis result 1330 as overlapping in a partial area of ​​the diagnosis image 1310, thereby generating the diagnosis image 1310. Fig.13 In FIG. 1 , the diagnosis result 1330 is classified into benign and malignant.

[0208] In addition, list 1302 includes items corresponding to features indicating lesions. Here, each of the multiple features is a feature of the lesion for distinguishing the type of lesion, and includes the shape of the lesion area, the orientation of the lesion, the edge of the lesion, the echo feature appearing in the lesion area, and the rear feature of the lesion area.

[0209] Specifically, a plurality of the above-mentioned features may correspond to a plurality of items included in a dictionary, which is a standard terminology for ultrasound examination defined by ACR. In this case, the list 1302 may include a plurality of items 1360 included in the dictionary.

[0210] The accompanying drawings described and shown below (including Fig.13 ) in which a list 1302 showing and describing features indicating a lesion area is formed as an example of a dictionary.

[0211] In an embodiment of the present disclosure, when there are a plurality of partial regions corresponding to a plurality of features, the processor 420 may generate a diagnostic image in which the plurality of partial regions are displayed to be distinguished from each other in order to distinguish the plurality of features from each other.

[0212] Alternatively, processor 420 may generate diagnosis image 1320 such that a first region corresponding to an item selected by a user from among plurality of items 1360 included in list 1302 is included in diagnosis image 1320 .

[0213] Specifically, in response to a user input for selecting any one of the plurality of items included in the list 1302, the processor 420 may display at least one partial region indicating a feature corresponding to the selected item as a first region in the ultrasound image, thereby generating a diagnostic image. The processor 420 may then update, generate, and display the user interface screen 1300 to reflect the generated diagnostic image.

[0214] When the ultrasound diagnostic apparatus 400 displays the user interface screen 1300, the user may view the user interface screen 1300 and may want to identify a partial area on the ultrasound image where any one of the multiple features that are the basis of the diagnosis result 1330 appears. In this case, the user may select any one of the multiple features that are the basis of the diagnosis result 1330 through the user interface 460. Specifically, the user may select an item corresponding to any one of the multiple features included in the list 1302. Accordingly, the user interface 460 may receive a user input for selecting one of the items included in the list 1302. Then, based on the user input received through the user interface 460, the processor 420 may identify the feature corresponding to the selected item and generate a diagnostic image to allow a region (specifically, a first region) indicating the identified feature to be displayed in the ultrasound image.

[0215] That is, based on the user input received through the user interface 460, the ultrasound diagnostic apparatus 400 may output a user interface screen including a diagnostic image generated by displaying an area (specifically, a first area) indicating a feature in the ultrasound image corresponding to the selected item. Here, the area indicating a predetermined feature (specifically, the first area as a partial area in the ultrasound image) may mean an area that serves as a basis for determining or determining the predetermined feature. For example, when the shape feature of the lesion area is determined to be an ellipse, the first area may be at least a partial contour area in the lesion area having a curvature corresponding to the ellipse.

[0216] As described above, when a diagnostic image in which the first area indicates a feature selected based on user input is provided, the user can easily identify a partial area of ​​the ultrasound image showing the feature to be examined. Therefore, the convenience and ease with which the user can diagnose a lesion can be increased.

[0217] Described below Figures 14 to 19 The same reference numerals are used to indicate Fig.13 The same components as in .

[0218] Fig.14 is a diagram illustrating another example of a user interface screen provided in the embodiment.

[0219] Reference Fig.14, user interface screen 1400 may correspond to user interface screen 1300 .

[0220] A user input of a user viewing the user interface screen 1300 and selecting the “shape” item included in the list 1302 may be input to the ultrasonic diagnostic apparatus 400. Then, the processor 420 may identify a partial area 1410 indicating the “shape” feature selected in response to receiving the user input and control to output the user interface screen 1400 including the diagnostic image 1401, wherein the identified partial area 1410 is displayed as the first area. In addition, although Fig.14 2 shows, as an example, a case where the partial region serving as a basis for determining the predetermined feature is one partial region, but it is obvious that there may be a plurality of partial regions serving as a basis for determining the predetermined feature.

[0221] In addition, in the user interface screen 1400, the selected item 1411 in the list 1302 may be displayed as distinguished from the unselected items (e.g., the orientation item, the edge item, the echo item, and the posterior item). For example, the processor 420 may generate a diagnostic image 1401 in which the selected item 1411 in the list 1302 is displayed as distinguished from the unselected items (e.g., the orientation item, the edge item, the echo item, and the posterior item) using at least one of different colors, symbols, transparency, echoes, marks, and lines of a specific pattern.

[0222] In addition, the processor 420 may generate a diagnostic image in which diagnostic information (e.g., 835 or 1135) specifically indicating the features of the selected item is displayed in the region corresponding to the partial region 1410. For example, when the shape feature of the lesion region appearing in the partial region 1410 is “ellipse”, the processor 420 may generate a diagnostic image (not shown) in which diagnostic information written as “shape—ellipse” is displayed in the region corresponding to the partial region 1410.

[0223] Fig.15 is a diagram showing another example of a user interface screen provided in the embodiment. Fig.15 , user interface screen 1500 may correspond to user interface screen 1300 .

[0224] A user input of the user viewing the user interface screen 1500 and selecting the “orientation” item 1511 included in the list 1302 may be input to the ultrasonic diagnostic apparatus 400. Then, the processor 420 may identify a partial area 1510 indicating the “orientation” feature selected in response to receiving the user input and control to output the user interface screen 1500 including the diagnostic image 1501, wherein the identified partial area 1510 is displayed as the first area.

[0225] In addition, the processor 420 may generate a diagnostic image in which diagnostic information (e.g., 835 or 1135) specifically indicating the characteristics of the selected item is displayed in the region corresponding to the partial region 1510. For example, when the orientation characteristic of the lesion region appearing in the partial region 1510 is “parallel”, the processor 420 may generate a diagnostic image (not shown) in which diagnostic information written as “orientation-parallel” is displayed in the region corresponding to the partial region 1510.

[0226] Fig.16 is a diagram showing another example of a user interface screen provided in the embodiment. Fig.16 , user interface screen 1600 may correspond to user interface screen 1300 .

[0227] A user input in which the user views the user interface screen 1600 and selects the “edge” item 1611 included in the list 1302 may be input to the ultrasonic diagnostic apparatus 400. Then, the processor 420 may identify partial areas 1610 and 1620 indicating the “edge” feature selected in response to receiving the user input and control to output the user interface screen 1600 including the diagnostic image 1601, wherein the identified partial areas 1610 and 1620 are displayed as the first area.

[0228] In addition, the processor 420 may generate a diagnostic image in which diagnostic information (e.g., 835 or 1135) specifically indicating the characteristics of the selected item is displayed in the area corresponding to the partial areas 1610 and 1620. For example, when the edge feature of the lesion area appearing in the partial areas 1610 and 1620 is "smooth", the processor 420 may generate a diagnostic image (not shown) in which diagnostic information written as "edge-smooth" is displayed in the area corresponding to the partial areas 1610 and 1620.

[0229] Fig.17 is a diagram showing another example of a user interface screen provided in the embodiment. Fig.17 , user interface screen 1700 may correspond to user interface screen 1300 .

[0230] A user input in which the user views the user interface screen 1700 and selects the “echo” item 1711 included in the list 1302 may be input to the ultrasonic diagnostic apparatus 400. Then, the processor 420 may identify partial areas 1710 and 1720 indicating the “echo” feature selected in response to receiving the user input and control to output the user interface screen 1700 including the diagnostic image 1701, wherein the identified partial areas 1710 and 1720 are displayed as the first area.

[0231] In addition, the processor 420 may generate a diagnostic image in which diagnostic information (e.g., 835 or 1135) specifically indicating the characteristics of the selected item is displayed in the area corresponding to the partial areas 1710 and 1720. For example, when the echo characteristic of the lesion area appearing in the partial areas 1710 and 1720 is "hyperechoic", the processor 420 may generate a diagnostic image (not shown) in which diagnostic information written as "echo-hyperechoic" is displayed in the area corresponding to the partial areas 1710 and 1720.

[0232] Fig.18 is a diagram showing another example of a user interface screen provided in the embodiment. Fig.18 , user interface screen 1800 may correspond to user interface screen 1300 .

[0233] A user input that the user views the user interface screen 1800 and selects the “posterior” item 1811 included in the list 1302 may be input to the ultrasonic diagnostic apparatus 400. Then, the processor 420 may identify a partial area 1810 indicating the “posterior” feature selected in response to receiving the user input and control to output the user interface screen 1800 including the diagnostic image 1801, wherein the identified partial area 1810 is displayed as the first area.

[0234] For example, when the object is an abdomen including a gallbladder, a lesion area is identified, and an outer area of ​​the identified lesion area (specifically, a rear area of ​​the lesion area) is imaged very darkly within the ultrasound image, and it can be determined that the stone exists as a lesion in the gallbladder. In the above example, when the "rear" feature is selected, the processor 420 can generate a diagnostic image in which a very dark imaging area of ​​the outer area of ​​the identified lesion area is displayed as a first area.

[0235] In addition, the processor 420 may generate a diagnostic image in which diagnostic information (e.g., 835 or 1135) specifically indicating the features of the selected item is displayed in the region corresponding to the partial region 1810. For example, when the posterior feature of the lesion region appearing in the partial region 1810 is “no posterior finding”, the processor 420 may generate a diagnostic image (not shown) in which diagnostic information written as “no posterior finding” is displayed in the region corresponding to the partial region 1810.

[0236] Fig.19 is a diagram showing another example of a user interface screen provided in the embodiment. Fig.19 The same reference numerals are used to indicate Figures 13 to 18 The same components as in .

[0237] In the disclosed embodiment, the processor 420 may generate and display a user interface screen such that a plurality of diagnostic images corresponding to a plurality of features are sequentially displayed, each diagnostic image indicating a lesion result.

[0238] Specifically, the processor 420 may generate and display the diagnostic image so that the features corresponding to the items included in the list 1302 are sequentially displayed. For example, the processor 420 may control the ultrasound diagnostic apparatus 400 to sequentially display Fig.14 The diagnostic image 1401 shown, Fig.15 The diagnostic image 1501 shown, Fig.16 The diagnostic image 1601 shown, Fig.17 Diagnostic image 1701 and Fig.18 Diagnostic image 1801 is shown.

[0239] In addition, the display order among diagnostic image 1401, diagnostic image 1501, diagnostic image 1601, diagnostic image 1701 and diagnostic image 1801 may be determined based on user settings, proprietary settings of processor 420, importance among multiple features, or impact of multiple features on the diagnostic results.

[0240] Fig. 20 is a diagram showing another example of a user interface screen provided in the embodiment. Fig. 20 , the user interface screen 2000 may correspond to Fig.13 The user interface screen 1300 is shown. Fig.13 As described, the user may select at least two items from the plurality of items corresponding to the plurality of features included in the list 1302. Therefore, the processor 420 may control to generate and display a user interface screen 2000 including a diagnosis image 2010 and a diagnosis result 2050, in which a partial area indicating the plurality of features is displayed.

[0241] For example, when the edge item and the shape item included in the list 1302 are selected by user input, the processor 420 may be controlled to generate and display the diagnostic image 2010, in which the partial area 2040 serving as a basis for determining the edge feature is displayed as being distinguished from the partial area 2030 serving as a basis for determining the shape feature. In addition, the processor 420 may generate the diagnostic image 2010, in which the diagnostic information (e.g., 2041 or 2031) is displayed as overlapping with the area corresponding to the partial area (e.g., 2040 or 2030). In addition, in the diagnostic image 2010, the diagnostic information (e.g., 2041 or 2031) is displayed as overlapping with the area corresponding to the partial area (e.g., 2040 or 2030). Fig. 20 , as an example, a case where the partial region 2040 and the partial region 2030 are distinguished by different dotted line patterns is shown.

[0242] In addition, when the user does not select a specific item (or feature), the processor 420 may generate a diagnostic image (e.g., 2010) based on user settings, proprietary settings of the processor 420, importance between multiple features, or influence of multiple features on the diagnosis result, and display a partial area indicating two or more features of the multiple features in the diagnostic image. Specifically, the processor 420 may display the partial area in the diagnostic image using at least one of a specific color, symbol, transparency, echo, mark, and a line with a specific pattern to distinguish the multiple features from each other.

[0243] For example, the processor 420 displays a partial area indicating shape features of the lesion area using red dotted lines, displays a partial area indicating orientation features of the lesion using blue dotted lines, displays a partial area indicating edge features of the lesion using green dotted lines, displays a partial area indicating echo features appearing in the lesion area using yellow dotted lines, and displays a partial area indicating posterior features of the lesion area using purple dotted lines, thereby enabling the generation of a diagnostic image in which different features are distinguished from each other.

[0244] Fig.21 is a diagram showing another example of a user interface screen provided in the embodiment. Fig.21 The same reference numerals are used to indicate Fig. 20 The same components as in .

[0245] In the disclosed embodiment, in the list 2102, multiple items may include information specifically indicating features corresponding to the multiple items. For example, the shape item may include information of "irregularity" (which is a feature specifically describing the shape feature), and the edge item may include information of "burr".

[0246] In addition, the processor 420 may control each of the plurality of items included in the list 2102 to be displayed to match at least one partial region corresponding thereto. Specifically, the processor 420 may display each of the plurality of items to match at least one partial region corresponding thereto using at least one of different colors, marks, symbols, transparency, echo, dashed line types, and dashed line thicknesses.

[0247] For example, as an example, the case where the partial image 2030 is a partial image as a basis for determining shape features and the partial image 2040 is a partial image as a basis for determining edge features will be described. In this case, the processor 420 may control the partial image 2030 and the shape items in the list 2102 to be displayed with at least one of the same color, the same mark, the same symbol, the same transparency, the same echo, the same type of dotted line, and the same thickness of dotted line. In addition, the processor 420 may control the partial image 2040 and the edge items in the list 2102 to be displayed with at least one of the same color, the same mark, the same symbol, the same transparency, the same echo, the same type of dotted line, and the same thickness of dotted line. In this case, the shape items and the edge items may be displayed using at least one of the colors, marks, symbols, transparency, echoes, dotted line types, and dotted line thicknesses that are distinguished from each other.

[0248] In addition, the diagnosis result 2050 may be determined and displayed as malignant based on the fact that the shape of the lesion area is irregular and the edge of the lesion area has burrs.

[0249] Fig. 22 is a diagram illustrating another example of a user interface screen provided in the embodiment.

[0250] Reference Fig. 22 , the processor 420 may generate a user interface screen 2200 including a diagnosis image 2210 , a diagnosis result 2250 , and a list 2202 indicating features of a lesion area.

[0251] In the disclosed embodiment, when the predetermined features included in the list 2202 are changed and thus the diagnosis result is changed, the processor 420 may distinguish the predetermined features included in the list 2202 from other features.

[0252] When the features of the partial region included in the lesion region change, there may be a case where the diagnosis result changes. That is, there may be a case where there are features that can change the diagnosis result by affecting the determination of the diagnosis result (e.g., benign or malignant). Hereinafter, the features that can change the diagnosis result are referred to as "main features". In addition, the partial region indicating the main features in the ultrasound image is separately referred to as the second region.

[0253] For example, the processor 420 may acquire a contour area of ​​the lesion and a contour of the lesion, and acquire at least one of a shape of the lesion, an orientation of the lesion, a shape of an edge of the lesion, an echo for the lesion area, and a posterior feature of the lesion area based on the acquired contour of the lesion.

[0254] Here, there may be a situation where there is an area where the outline of the lesion is unclear. For example, there may be a situation where the outline of the lesion in a partial area (e.g., 2230) within the lesion area 2220 is unclear. In this case, in the partial area 2230, the processor 420 may obtain a first outline as the outline of the lesion, or obtain a second outline as the outline of the lesion. In this case, the shape of the lesion may be determined as an ellipse based on the first outline, and the shape of the lesion may be determined as irregular based on the second outline. That is, when the outline of the lesion in the partial area (e.g., 2230) changes, and when the feature that serves as the basis for the diagnosis result changes, the processor 420 may control the display of the diagnostic image 2201, and display the partial area (e.g., 2230) in the diagnostic image 2201.

[0255] In addition, the processor 420 may generate a diagnostic image by displaying a portion where the contour of the lesion is unclear in the ultrasound image. Here, the "portion where the contour of the lesion is unclear" may be a portion where the accuracy of the extracted contour has a value less than or equal to a limit value. Specifically, the processor 420 may display a contour extracted from a portion (e.g., 2230) where the contour of the lesion is an unclear area to distinguish it from a contour of a portion that is not unclear (e.g., a portion outside 2230), thereby generating and displaying a diagnostic image 2201.

[0256] Reference Fig. 22 , the processor 420 may extract the lesion region 2220 having the outline shown. Here, the extracted portion whose outline is unclear may be displayed as a partial region 2230. Then, as shown in FIG. Fig. 22 As shown, the processor 420 may control the display of the diagnostic image 2201 , and display the partial area 2230 in the diagnostic image 2201 .

[0257] Specifically, as an analysis result of the lesion area 2220 having the outline shown, the processor 420 may determine the shape feature appearing in the partial area 2230 as irregular. However, when the shape feature appearing in the partial area 2230 changes to an ellipse, there may be a case where the diagnosis result 2250 changes from malignant to benign. Specifically, when the outline in the partial area 2230 changes, the shape feature may change to an ellipse, and when the shape feature changes to an ellipse, the diagnosis result 2250 may change from malignant to benign. In this case, the shape feature may be the main feature described above.

[0258] In the disclosed embodiment, the processor 420 may add an indication (e.g., a mark 2275) for distinguishing the main feature appearing in the above-mentioned unclear area (e.g., 2230) from other features in the list 2202, thereby controlling the generation and display of the list 2202. In addition, in the item corresponding to the main feature in the list 2202, the processor 420 may add an indication (e.g., a mark 2272) for distinguishing the currently acquired shape feature 2271 from other features (e.g., an ellipse).

[0259] In addition, in the disclosed embodiment, when the contour of the lesion area extracted by the processor 420 includes an unclear portion (e.g., a portion where the accuracy of the contour is less than or equal to the limit value) (e.g., 2230), the processor 420 may control the display of an icon 2232, which is used to perform manual correction or contour re-extraction of the area corresponding to the partial area 2230. For example, the icon 2232 may be represented as a mark "c" indicating correction. Here, manual correction may mean an operation in which a user manually corrects the unclear contour portion. Specifically, when a user input for selecting the icon 2232 is received, the processor 420 may control to output a user interface screen including an editing window (not shown) for correcting the contour within the partial area 2230. As another example, when the icon 2232 is an icon for re-extracting the contour, and when a user input for selecting the icon 2232 is received, the processor 420 may perform an automatic diagnosis operation such as a CAD operation to re-extract the contour and generate and display a diagnostic image of the lesion area formed by the re-extracted contour. In addition, the icon 2232 may be displayed in the partial region 2230 which is the region requiring contour correction, or at a position adjacent to the partial region 2230 .

[0260] In addition, in the disclosed embodiment, when the feature (i.e., main feature) appearing in at least a portion of the region (e.g., 2230) included in the lesion region changes and thus the diagnosis result changes, the processor 420 may display at least a portion of the region (e.g., 2230) in the ultrasound image, thereby generating a diagnostic image 2201. Hereinafter, for ease of description, the above-mentioned "at least a portion of the region (e.g., 2230)" will be referred to as a "second region". In addition, the processor 420 may allow the user interface screen 2200 to include at least one of a symbol, a mark, and information indicating that the diagnosis result may change. Specifically, the processor 420 may control to display at least one of a symbol, a mark, and information indicating that the diagnosis result changes when the feature appearing in the second region 2230 changes in at least one of the diagnostic image 2201, the diagnostic result 2250, and the list 2202.

[0261] Reference Fig. 22As an example, the processor 420 may detect the partial region 2230 as a basis for determining the shape feature of the lesion region 2220. As a result of the analysis of the lesion region 2220, the processor 420 may determine the shape feature appearing in the partial region 2230 as irregular. However, when the shape feature appearing in the partial region 2230 changes to an ellipse, there may be a case where the diagnosis result 2250 changes from malignant to benign. That is, the shape feature may be a main feature.

[0262] In this case, processor 420 may display an indication (e.g., mark 2275) in list 2202 for distinguishing the main feature from other features in list 2202. In addition, in the item corresponding to the main feature in list 2202, processor 420 may add an indication (e.g., mark 2272) for distinguishing the currently determined shape feature 2271 from other features (e.g., ellipse).

[0263] In addition, when the diagnosis result 2250 is displayed, information indicating that the diagnosis result may change may be displayed.

[0264] In addition, when there are multiple main features among the multiple features that are the basis for diagnosing the lesion, the processor 420 may display multiple second regions in the diagnostic image so as to distinguish the multiple main features from each other. For example, when the main features are shape features and edge features, the processor 420 may generate and display a diagnostic image in which the second region corresponding to the shape feature is displayed to be distinguished from the second region corresponding to the edge feature.

[0265] In addition, when determining whether there is a main feature, the processor 420 may consider whether a specific determination of a predetermined feature has an accuracy greater than or equal to a limit value. Specifically, in the shape feature, an example will be described in which the accuracy is 70% and the limit value is 90% when the shape feature is determined to be an ellipse. When the accuracy of determining that the shape feature is an ellipse is 90%, since the accuracy has a value greater than and equal to the limit value, the processor 420 may not determine the shape feature as a main feature.

[0266] However, when the accuracy of determining that the shape feature is an ellipse is 70%, since the accuracy has a value less than the limit value, the processor 420 may determine the shape feature as a main feature. Therefore, when there is a predetermined feature determined as a main feature, the processor 420 may control the user interface screen 2200 to include at least one of a second area (e.g., 2230) indicating the main feature, an item (e.g., 2270) including detailed information about the main feature, and an indication (e.g., a mark (2231 or 2275)) indicating that the main feature exists.

[0267] Fig.23is a diagram illustrating another example of a user interface screen provided in the embodiment.

[0268] Fig.24 is a diagram illustrating another example of a user interface screen provided in the embodiment.

[0269] exist Fig.23 and Fig.24 In FIG. 2 , a lesion region (eg, 2220) included in a diagnostic image (eg, 2201) is simplified and schematically shown. Fig.23 and Fig.24 The same reference numerals are used to indicate Fig. 22 The same components as in .

[0270] Reference Fig.23 , when compared with the user interface screen 2200, the user interface screen 2300 may further include a sub-screen 2303 including detailed information 2380 corresponding to the main features.

[0271] Here, the detailed information 2380 may include information indicating the determination accuracy (or diagnosis probability) of the main feature. Specifically, an example will be described in which the diagnosis probability of determining the shape feature as an ellipse is 20% (or 0.2), the diagnosis probability of determining the shape feature as a circle is 70% (or 0.7), and the diagnosis probability of determining the shape feature as irregular is 10% (or 0.1). Then, the processor 420 may control the generation and display of information 2380 indicating the determination accuracy of the main feature on the user interface screen 2300. Specifically, the information 2380 may include a diagnosis probability 2381 to be determined as an ellipse, a diagnosis probability 2382 to be determined as a circle, and a diagnosis probability 2383 to be determined as irregular. In addition, the diagnosis probability 2381 to be determined as an ellipse, the diagnosis probability 2382 to be determined as a circle, and the diagnosis probability 2383 to be determined as irregular can be displayed in the order of increasing probability values.

[0272] Reference Fig.24 , the information 2480 included in the user interface screen 2400 may correspond to Fig.23 Here, information 2480 includes the same content as information 2380, and optionally, may be represented in a different format. Fig.24 In the user interface screen 2400 shown in , a diagnosis probability 2381 to be determined as an ellipse, a diagnosis probability 2382 to be determined as a circle, and a diagnosis probability 2383 to be determined as irregular may be represented in a graphical form 2485 .

[0273] In the disclosed embodiment, when the ultrasound diagnostic device 400 outputs an ultrasound image including a lesion area in response to user input selecting a location or a predetermined partial area of ​​the lesion area, the processor 420 may be controlled to generate or display a diagnostic image in which information about characteristics of the lesion appearing in the selected location or partial area or information serving as a basis for the diagnostic result is displayed as overlapping with the ultrasound image.

[0274] Fig.25 is a diagram illustrating a diagnosis image output in response to a user input.

[0275] Fig.26 is another diagram illustrating a diagnostic image output in response to a user input.

[0276] Fig. 27 is another diagram illustrating a diagnostic image output in response to a user input.

[0277] exist Figure 25 to Figure 27 In the present invention, the same reference numerals are used to indicate the same components. Figure 25 to Figure 27 The arrow mark (e.g., 2530) shown in FIG. 1 indicates the position or partial area selected or pointed to by the user input. Figure 25 to Figure 27 In FIG. 2 , the ultrasound image (eg, 2501) includes a lesion area 2510. In addition, in Figure 25 to Figure 27 , the ultrasonic image 2501, the ultrasonic image 2601, and the ultrasonic image 2701 may equally correspond to each other.

[0278] Reference Fig.25 , the ultrasound diagnostic apparatus 400 may display an ultrasound image 2501 including a lesion area 2510. In addition, the user may select a position or area in the ultrasound image 2501 that the user particularly wants to view. Here, the selection operation may be performed by manipulating (e.g., clicking or double-clicking) a selection portion such as a cursor or pointer included in the user interface 460 to select a predetermined position or area. In addition, when a cursor or pointer for selection is located on the ultrasound image 2501, the processor 420 may recognize that the corresponding position is selected. Figure 25 to Figure 27 , the position of the selection portion for selection is shown as an arrow (e.g., 2530, 2630, or 2730).

[0279] Reference Fig.25, when a predetermined position P1 on the lesion area 2510 included in the ultrasound image 2501 is selected, the processor 420 may control to generate and display a diagnostic image 2502 in which at least one of a partial area 2550 corresponding to the position P1 selected in response to the selection input and information 2551 on a feature appearing in the partial area is displayed. For example, when the partial area 2550 corresponding to the selected position P1 is an area indicating a shape feature from among a plurality of features for deriving a diagnostic result, information 2551 such as “shape: ellipse” may be displayed in the diagnostic image 2502.

[0280] Reference Fig.26 , when a predetermined position P2 on the lesion area 2510 included in the ultrasound image 2601 is selected, the processor 420 may control to generate and display a diagnostic image 2602 in which at least one of a partial area 2650 corresponding to the position P2 selected in response to the selection input and information 2651 on a feature appearing in the partial area is displayed. For example, when the partial area 2650 corresponding to the selected position P2 is an area indicating a posterior feature from among a plurality of features for deriving a diagnosis result, information 2651 such as “no posterior” may be displayed in the diagnostic image 2602.

[0281] Reference Fig. 27 , when a predetermined position P3 on the lesion area 2510 included in the ultrasound image 2701 is selected, the processor 420 may control to generate and display a diagnostic image 2702 in which at least one of a partial area 2750 corresponding to the position P3 selected in response to the selection input and information 2751 about features appearing in the partial area is displayed. For example, when the partial area 2750 corresponding to the selected position P3 is an area indicating an echo feature for deriving a diagnosis result among a plurality of features, information 2751 such as “echo mode: hypoechoic” may be displayed in the diagnostic image 2702.

[0282] In addition, even when there is no user input, the processor 420 can move the contour line of the lesion area 2510 included in the ultrasound image (e.g., 2501) in a clockwise or counterclockwise direction and control to generate and display a diagnostic image (e.g., 2502), in which at least one of the area corresponding to the position where the contour line is moved and information about features appearing in the area (e.g., 2551) is displayed.

[0283] In addition, the processor 420 may divide the contour of the lesion area 2510 included in the ultrasound image (e.g., 2501) into a plurality of segments, and control to generate and display a diagnostic image in which there is at least one of an area corresponding to the divided segments and information about a feature appearing in the area. In addition, the plurality of segments distinguished from each other may be displayed using at least one of different colors, marks, symbols, transparency, echo, dotted line type, and dotted line thickness.

[0284] Fig.28 is a diagram illustrating a user interface screen including a plurality of diagnosis images.

[0285] In the disclosed embodiment, in order to identify the lesion area, the processor 420 may use at least one of the CAD technology and the AI ​​technology to extract the contour of the lesion. In addition, when there are multiple extracted contours of the lesion, the processor 420 may control to generate and display a user interface screen including multiple ultrasound images, in which the multiple extracted contours of the lesion are displayed.

[0286] In addition, when there are multiple extracted contours of the lesion and each of the multiple extracted contours corresponds to a different diagnosis result, the processor 420 may control to generate and display a user interface screen including multiple ultrasound images showing the multiple extracted contours of the lesion.

[0287] Reference Fig.28 , the user interface screen 2800 may include a plurality of ultrasound images 2810, 2820, 2830, 2840, 2850, and 2860. Here, each of the plurality of ultrasound images 2810, 2820, 2830, 2840, 2850, and 2860 included in the user interface screen 2800 may be an ultrasound image in which a different profile is displayed.

[0288] In addition, each of the plurality of ultrasonic images 2810, 2820, 2830, 2840, 2850, and 2860 may be arranged in the order of the accuracy of the extracted contours. For example, the ultrasonic image 2810 displaying the contour 2801 having the highest contour accuracy may be arranged at the first position, and the ultrasonic image 2820 displaying the contour 2821 having the second highest contour accuracy may be arranged at the second position. In addition, the ultrasonic image 2830 displaying the contour 2831 having the third highest contour accuracy may be arranged at the third position, and the ultrasonic image 2840 displaying the contour 2841 having the fourth highest contour accuracy may be arranged at the fourth position.

[0289] In addition, in each of the plurality of ultrasound images 2820 , 2830 , 2840 , 2850 , and 2860 , a contour 2801 having the highest contour accuracy may be shown as a comparison target.

[0290] In addition, a diagnosis result derived based on the extracted contour may be displayed in each of the plurality of ultrasound images 2810, 2820, 2830, 2840, 2850, and 2860 ( Fig.28 not shown).

[0291] In addition, when each of the plurality of ultrasound images 2810 , 2820 , 2830 , 2840 , 2850 , and 2860 is selected on the user interface screen 2800 , a diagnosis image including diagnosis information corresponding to a diagnosis result may be enlarged and displayed.

[0292] Fig.29 is a diagram for describing a neural network for obtaining analysis results of an ultrasound image.

[0293] In the disclosed embodiment, the processor 420 may use at least one of the CAD technology and the AI ​​technology to analyze the lesion and obtain at least one of the lesion area, the diagnosis result, and the diagnosis information corresponding to the diagnosis result.

[0294] Hereinafter, an operation of acquiring at least one of a lesion region, a diagnosis result, and diagnosis information corresponding to the diagnosis result using the AI ​​technology will be described.

[0295] AI technology is a technology for obtaining a desired result by performing operations through a neural network to analyze an input image or input data. Here, the neural network can train training data (e.g., a plurality of different ultrasound images) to optimize and set weight values ​​in the neural network. Then, the neural network learns the input data by itself through the neural network with the optimized weight values, thereby outputting the desired result.

[0296] Specifically, the neural network may be a DNN. In addition, the operation of the DNN may include the operation of a CNN. Specifically, the data recognition model may be implemented by an exemplary neural network, and the implemented data recognition model may be trained using training data. In addition, the learned data recognition model may be used to analyze and classify data to be input (e.g., an ultrasound image including a lesion area), and a specific area (e.g., a contour area of ​​a lesion) or a specific area image (e.g., an image in the first area corresponding to a partial area serving as a basis for deriving a diagnosis result) included in the ultrasound image may be analyzed and / or classified.

[0297] In an embodiment of the present disclosure, the processor 420 may perform operations through a neural network to obtain at least one of a lesion area, a diagnosis result, and diagnosis information corresponding to the diagnosis result from an ultrasound image.

[0298] Optionally, the above-mentioned operation through the neural network may be performed by the image processing unit 450 .

[0299] Optionally, the above-mentioned operation through the neural network can be performed through an external server (not shown), and the operation results through the neural network (for example, at least one of the lesion area, the diagnosis result, and the diagnosis information corresponding to the diagnosis result) can be received through the communication unit 470.

[0300] The processor 420 may perform operations through a neural network (eg, DNN 2920) including an input layer, a hidden layer, and an output layer. Fig.29 In FIG. 1 , a case where a hidden layer included in a neural network is formed as a DNN formed of multiple stages is shown as an example.

[0301] Reference Fig.29 , DNN 2920 includes an input layer 2930, a hidden layer 2940, and an output layer 2950. Fig.29 , a DNN 2920 is shown as an example which analyzes information included in an ultrasound image as input data and outputs desired output data 2970. Here, the output data 2970 may include at least one of a lesion area, a diagnosis result, and diagnosis information corresponding to the diagnosis result.

[0302] The multiple layers forming the DNN 2920 may include multiple nodes (e.g., 2931) for receiving data. In addition, as shown in the figure, two adjacent layers are connected by multiple edges (e.g., 2936). Each node has a corresponding weight value. Therefore, the DNN 2920 can obtain output data by calculating the input signal and the weight value based on the value obtained by performing the convolution operation, for example.

[0303] Here, DNN 2920 may be formed as a CNN neural network that performs CNN operations.

[0304] Reference Fig.29 In the example shown, the input layer 2930 receives an ultrasound image 2910 including a lesion region 2902 .

[0305] exist Fig.29 In FIG. 2 , a case where the hidden layer 2940 is formed of three layers is shown as an example. The depth of the hidden layer 2940 may vary according to the n-order specification and / or design specification of the neural network used.

[0306] Reference Fig.29, DNN 2920 includes a first layer (Layer1) formed between the input layer 2930 and the first hidden layer (HIDDEN LAYER1), a second layer (Layer2) formed between the first hidden layer (HIDDEN LAYER1) and the second hidden layer (HIDDEN LAYER2), a third layer (Layer3) formed between the second hidden layer (HIDDEN LAYER2) and the third hidden layer (HIDDEN LAYER3), and a fourth layer (Layer4) formed between the third hidden layer (HIDDEN LAYER3) and the output layer 2950.

[0307] A plurality of nodes included in the input layer 2930 of the DNN 2920 receives a plurality of pieces of data corresponding to the ultrasound image 2910. Here, the plurality of pieces of data may be a plurality of partial images generated by dividing the ultrasound image 2910 by performing a filtering process.

[0308] In addition, the output layer 2950 may output output data 2970 as an analysis result of the ultrasound image 2910 acquired through operations in a plurality of layers included in the hidden layer 2940 .

[0309] Specifically, when DNN 2920 is formed by CNN and when the correlation between information included in an image is local, CNN may introduce the concept of a filter that illuminates only a specific area and perform a convolution operation on multiple pieces of information in the filter, thereby accurately extracting information about the features of the image in the filter.

[0310] Specifically, the convolution layer and the pooling layer are alternately arranged in the hidden layer 2940 present in the CNN-based neural network 2920, and the depth of the filter of each layer increases from left to right. In addition, the last level of the CNN-based neural network 2920 can be formed as a fully connected layer. Here, the convolution layer is a data layer generated according to the convolution operation, and the pooling layer is a layer for reducing the number of data or the size of the data by operations such as subsampling or pooling. Multiple data (e.g., feature maps) indicating the features of the input image are generated by passing through the convolution layer and the pooling layer. Specifically, the image features of the ultrasound image 2910 are generated by the operation of the hidden layer 2940, and the lesion area, the diagnosis result for the lesion, and the diagnostic information corresponding to the diagnosis result can be obtained based on the image features.

[0311] In addition, when a plurality of pieces of data generated by passing through a convolution layer and a pooling layer are processed by a hidden layer formed as a fully connected layer, desired output data 2970 may be output.

[0312] In addition, in order to improve the accuracy of output data output by the DNN 2920, training may be performed in a direction from the output layer 2950 to the input layer 2930, and in order to improve the accuracy of the output data, weight values ​​of nodes (e.g., 2931) forming the DNN 2920 may be corrected. Therefore, before the ultrasound image 2910 is input, the DNN 2920 may learn a plurality of ultrasound images including a plurality of different lesions to correct the weight value of each node in the orientation of the lesion area included in the ultrasound image or in the orientation where the diagnosis result is accurately detected.

[0313] In addition, DNN 2920 can perform operations through a neural network to obtain a standard for detecting a diagnosis result. For example, a standard indicating a feature of a malignant lesion or a standard indicating a feature value of a benign lesion can be obtained.

[0314] Fig.30 is a flowchart showing a method for displaying an ultrasound image according to an embodiment. The method 3000 for displaying an ultrasound image according to the disclosed embodiment may be performed by referring to Figures 1 to 29 The method 3000 of displaying an ultrasonic image may be performed by the ultrasonic diagnostic apparatus 100, 400, or 500 of the disclosed embodiment described. Therefore, each operation of the method 3000 of displaying an ultrasonic image may be performed by each configuration of the ultrasonic diagnostic apparatus 100, 400, or 500, and the method 3000 of displaying an ultrasonic image may include the same configuration features as those in the above-mentioned ultrasonic diagnostic apparatus 100, 400, or 500. That is, Fig.30 It may be a diagram showing operations performed by the ultrasound diagnostic apparatus 100, 400, or 500 according to the disclosed embodiment. Therefore, when describing the method 3000 for displaying an ultrasound image, the description related to the method 3000 will be omitted. Figures 1 to 29 Description of duplicate description.

[0315] The method 3000 of displaying an ultrasound image identifies a lesion region included in the ultrasound image ( S3010 ). Operation S3010 may be performed by the processor 420 .

[0316] The lesion region identified in operation S3010 is diagnosed to obtain a diagnosis result ( S3020 ). Operation S3020 may be performed by the processor 420 .

[0317] Then, in the lesion region, a first region of at least one region as a basis for diagnosing the lesion is displayed in the lesion region of the ultrasound image to generate a diagnostic image ( S3030 ). Operation S3030 may be performed by the processor 420 .

[0318] Then, a user interface screen including the diagnosis image and the diagnosis result is displayed ( S3040 ). Operation S3040 may be performed by the processor 420 .

[0319] Fig.31 is a diagram showing an actual implementation example of a user interface screen according to an embodiment.

[0320] Fig.32 is another diagram showing an actual implementation example of a user interface screen according to an embodiment.

[0321] Specifically, in Fig.31 and Fig.32 , an actual user interface screen output from an ultrasonic diagnostic apparatus implemented according to the disclosed embodiment is shown.

[0322] Reference Fig.31 , the user interface screen 3100 may include a diagnostic image 3110 generated by marking the extracted contour area in the ultrasound image, a diagnostic result 3120, and a list 3130 indicating features of the lesion area. Here, the diagnostic image 3110, the diagnostic result 3120, and the list 3130 indicating features of the lesion area are respectively Fig.13 The illustrated diagnosis image 1310 , diagnosis result 1330 , and list 1302 indicating features of the lesion area correspond identically, and thus a detailed description thereof will be omitted here.

[0323] In the disclosed embodiment, due to selection of at least one item in the list 3130 or setting of the processor 420 , the first region may be displayed in the diagnostic image 3110 .

[0324] Reference Fig.32 , when compared to the user interface screen 3100 , the user interface screen 3200 may further include ultrasound scanning information 3240 for the patient or object.

[0325] In addition, the disclosed embodiments may be implemented in the form of a computer-readable recording medium storing computer-executable commands and data. The commands may be stored in the form of program codes, and when the commands are executed by a processor, a predetermined program module may be generated to perform a predetermined operation. In addition, when the commands are executed by a processor, the commands may perform the predetermined operations of the disclosed embodiments.

[0326] Computer readable media may include separate program instructions, data files, data structures, etc., or a combination thereof. The program commands recorded in the computer readable medium may be specifically designed and configured for the embodiment, or may be available to a technician in the field of computer software. Examples of computer readable recording media include magnetic media (such as hard disks, floppy disks, and magnetic tapes), optical recording media (such as compact disk read-only memories (CD-ROMs) and digital versatile disks (DVDs)), magneto-optical media (such as optical magnetic disks), and hardware devices (such as ROMs, RAMs, flash memories, etc.) that are specifically configured to store and execute program commands. Examples of program commands include machine language codes generated by a compiler and high-level language codes that can be executed by a computer using an interpreter, etc.

[0327] In addition, the method for displaying an ultrasound image according to the above-mentioned embodiment can be implemented as a computer program product, which includes a recording medium storing a program, wherein the program performs the following operations: obtaining sentences composed of multiple languages; and using multiple language translation models to obtain vector values ​​corresponding to words included in the sentences in the multiple languages, converting the obtained vector values ​​into vector values ​​corresponding to the target language, and obtaining the sentence composed of the target language based on the converted vector values.

Claims

1. A method for displaying an ultrasound image, comprising: identifying a lesion area included in the ultrasound image by an ultrasound diagnostic device; Acquiring a diagnosis result based on the lesion area by using the ultrasonic diagnostic device; generating a diagnostic image including a first region in the lesion region in the ultrasound image, the first region being at least one region serving as a basis for diagnosing the lesion; displaying a user interface screen including the diagnosis image and the diagnosis result; as well as when a feature of the lesion region appearing in a second region included in the lesion region changes and thus the diagnosis result changes, displaying the diagnostic image including the second region in the lesion region, Wherein, displaying the diagnostic image including the second region in the lesion region includes displaying visual information indicating that the diagnostic result will change when a feature of the lesion in the second region changes.

2. The method according to claim 1, wherein: When there are a plurality of features indicating characteristics of the lesion in the lesion region, the first region is a region indicating at least one of the plurality of features.

3. The method according to claim 1, wherein: The first region is a region serving as a basis for determining at least one feature among a shape of the lesion, an orientation of the lesion, an edge of the lesion, an echogenicity with respect to the lesion region, and a rear portion of the lesion region.

4. The method according to claim 1, wherein: The diagnosis result includes information indicating whether the lesion is benign or malignant.

5. The method according to claim 1, wherein: Generating the diagnostic image comprises: When there are a plurality of features indicating the lesion area, displaying at least one of the lesion areas using at least one of different colors, marks, symbols, transparencies, echogenicity, dashed line types, and dashed line thicknesses to distinguish different features from each other among the plurality of features; and The plurality of features include at least one of a shape of the lesion, a position of the lesion, a marginal form of the lesion, an echogenicity for the lesion region, and a posterior feature of the lesion region as a basis for diagnosing the lesion.

6. The method according to claim 1, wherein: Generating the diagnostic image further includes displaying the first region to be distinguished from the second region and generating the diagnostic image.

7. The method according to claim 1, wherein: Displaying the user interface screen includes displaying the user interface screen including a list including a plurality of items corresponding to a plurality of features indicating the diagnosis image, the diagnosis result, and the lesion.

8. The method according to claim 7, wherein: Generating the diagnostic image includes: in response to a user input for selecting any one of the plurality of items, displaying at least one partial region indicating a feature corresponding to the selected item as the first region in the ultrasound image, and generating the diagnostic image.

9. The method according to claim 7, wherein: Displaying the user interface screen further includes displaying the first feature included in the list to be distinguished from other features when the first feature included in the list changes and thus the diagnosis result changes.

10. The method according to claim 1, wherein: Generating the diagnostic image includes displaying diagnostic information in the ultrasound image and generating the diagnostic image, the diagnostic information being information on features of a lesion region appearing in the first region.

11. The method according to claim 10, wherein: Acquiring the diagnostic information includes: using at least one of computer-aided detection and diagnosis technology and artificial intelligence technology to analyze the ultrasound image, and acquiring at least one of the lesion area, the diagnostic result, and diagnostic information corresponding to the diagnostic result.

12. The method according to claim 1, wherein: Identifying the lesion region includes identifying a contour of the lesion region, and wherein the method further comprises: When a plurality of the contours exist as a result of the recognition, displaying the plurality of contours in the ultrasound image and generating a plurality of sub-images; and A user interface screen including the plurality of sub-images is output.

13. An ultrasonic diagnostic device, comprising: monitor; a memory configured to store at least one instruction; as well as A processor configured to execute at least one of the at least one instructions to identify a lesion area included in an ultrasound image, obtain a diagnosis result based on the lesion area, generate a diagnostic image including a first area in the lesion area of ​​the ultrasound image, control a user interface screen including the diagnostic image and the diagnostic result to be displayed through the display, and when a feature of the lesion area appearing in a second area included in the lesion area changes and thus the diagnosis result changes, display the diagnostic image including the second area in the lesion area, and display visual information indicating that the diagnosis result will change when a feature of the lesion in the second area changes, wherein the first area is at least one area that serves as a basis for diagnosing the lesion in the lesion area.

14. A computer program product comprising: A computer program including commands executable by a computer and configured to be stored in a recording medium to execute a method of displaying an ultrasound image on a computer, Wherein, the computer program comprises: identifying a lesion region included in an ultrasound image; Acquire a diagnosis result based on the lesion area; generating, in the lesion region of the ultrasound image, a diagnostic image including a first region in the lesion region, the first region being at least one region as a basis for diagnosing the lesion; displaying a user interface screen including the diagnosis image and the diagnosis result; and when a feature of the lesion region appearing in a second region included in the lesion region changes and thus the diagnosis result changes, displaying the diagnostic image including the second region in the lesion region, Wherein, displaying the diagnostic image including the second region in the lesion region includes displaying visual information indicating that the diagnostic result will change when a feature of the lesion in the second region changes.

Images