Ultrasound diagnostic device and control method for ultrasound diagnostic device

Abstract

To provide an ultrasonic diagnostic device which allows a user to accurately diagnose a lesion part and a control method of the ultrasonic diagnostic device.SOLUTION: An ultrasonic diagnostic device comprises: an image feature calculation unit (25) which calculates a multidimensional image feature in each of a plurality of ultrasonic images by analyzing the plurality of ultrasonic images obtained by imaging a lesion part of a subject; a score calculation unit (26) calculates a score of the lesion part in each of the plurality of ultrasonic images on the basis of the multidimensional image feature; a determination unit (27) which performs determination of the lesion part on the basis of the score calculated for each of the plurality of ultrasonic images; an extraction unit (28) which extracts the ultrasonic image that best expresses the determination result by the determination unit from the plurality of ultrasonic images as a basis image; a monitor (23); and a display control unit (22) which displays the determination result by the determination unit and the basis image extracted by the extraction unit on the monitor (23).SELECTED DRAWING: Figure 1

Description

Technical Field

[0005]

[0001] The present invention relates to an ultrasonic diagnostic apparatus for determining a lesion part and a control method of the ultrasonic diagnostic apparatus.

Background Art

[0002] Conventionally, in the medical field, an ultrasonic diagnostic apparatus using ultrasonic images has been put into practical use. Generally, an ultrasonic diagnostic apparatus includes an ultrasonic probe incorporating a vibrator array and a device body connected to the ultrasonic probe, transmits an ultrasonic beam from the ultrasonic probe toward a subject, receives an ultrasonic echo from the subject with the ultrasonic probe, and generates an ultrasonic image by electrically processing the received signal.

[0003] A user such as a doctor diagnoses, for example, a lesion part of a subject by checking the ultrasonic image generated in this way. At this time, the user usually selects an ultrasonic image appropriate for diagnosis by checking a plurality of ultrasonic images taken in an ultrasonic examination. In order to reduce the labor of the user for selecting an ultrasonic image in this way, the technique disclosed in Patent Document 1 has been developed.

[0004] Patent Document 1 discloses a device that automatically detects candidates for lesion parts by analyzing ultrasonic images and reduces the number of ultrasonic images by merging a series of ultrasonic images in which candidates for lesion parts are not detected by image processing. The user can select an ultrasonic image suitable for diagnosis by checking a plurality of ultrasonic images in which candidates for lesion parts are detected and remain after merging.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Summary of the Invention

[0006] However, even if the device automatically detects potential lesions as described in Patent Document 1, users may not trust the detection results and may select ultrasound images that are unsuitable for diagnosis, such as those that do not show the characteristic shape of the lesion. This could lead to a decrease in the accuracy of lesion diagnosis.

[0007] This invention was made to solve the problems of the past, and aims to provide an ultrasound diagnostic device and a control method for the ultrasound diagnostic device that enable users to accurately diagnose lesions. [Means for solving the problem]

[0008] The above objective can be achieved with the following configuration. [1] An ultrasound diagnostic device that determines a lesion based on multiple ultrasound images taken of the lesion of a subject, An image feature calculation unit calculates multidimensional image features in each of multiple ultrasound images by analyzing multiple ultrasound images, A scoring unit that calculates a score for each of multiple ultrasound images based on multidimensional image features, A determination unit that determines the lesion based on scores calculated for each of multiple ultrasound images, An extraction unit extracts the ultrasound image that best represents the judgment result from multiple ultrasound images as the basis image, Monitor and, A display control unit that displays the judgment result from the judgment unit and the evidence image extracted by the extraction unit on a monitor. An ultrasound diagnostic device equipped with the following features. [2] The ultrasound diagnostic device described in [1], wherein multiple ultrasound images constitute a video of the lesion. [3] The ultrasound diagnostic apparatus described in [1] or [2], wherein each of the multiple ultrasound images is an image of the whole or a part of the lesion. [4] The ultrasound diagnostic apparatus described in [2], wherein the multiple ultrasound images are images obtained by downsampling, interpolation, or synthesis from images that make up a video of the lesion. [5] The extraction part forms a border around the lesion in the evidence image. The display control unit is an ultrasound diagnostic apparatus according to any one of [1] to [4] that superimposes a bounding box onto the reference image. [6] The extraction unit forms a heat map in the evidence image in which the contribution rate of the portion that contributes to the calculation of the score by the score calculation unit is represented by the intensity of the color or the difference in color. The display control unit is an ultrasound diagnostic apparatus according to any one of [1] to [4] that displays a heat map on a monitor. [7] The ultrasound diagnostic apparatus according to any one of [1] to [6], wherein the score calculation unit calculates a score using a machine learning model that has learned the multidimensional image features of ultrasound images. [8] The judgment unit makes a judgment based on the score of the ultrasound image in which the lesion is captured most large among multiple ultrasound images, as described in any of [1] to [7]. [9] The ultrasound diagnostic apparatus according to any one of [1] to [7], wherein the determination unit makes a determination based on a majority vote, maximum value, median, or mode of the scores of multiple ultrasound images.

[10] The ultrasound diagnostic apparatus according to any one of [1] to [7], wherein the determination unit removes outliers from the scores of multiple ultrasound images and makes a determination based on the majority vote, maximum value, median, or mode of the scores of the multiple ultrasound images from which the outliers have been removed.

[11] A control method for an ultrasound diagnostic device that determines a lesion based on multiple ultrasound images taken of the lesion of a subject, By analyzing multiple ultrasound images, multidimensional image features are calculated for each of the multiple ultrasound images. Based on multidimensional image features, the score for the lesion in each of the multiple ultrasound images is calculated. The lesion is identified based on the score calculated for each of the multiple ultrasound images. From multiple ultrasound images, the ultrasound image that best represents the diagnosis of the lesion is extracted as the supporting image. Display the determination result and the basis image on the monitor Control method for an ultrasonic diagnostic apparatus

Effect of the Invention

[0009] According to the present invention, an ultrasonic diagnostic apparatus includes an image feature calculation unit that calculates multi-dimensional image features in each of a plurality of ultrasonic images by analyzing the plurality of ultrasonic images, a score calculation unit that calculates a score of a lesion part in each of the plurality of ultrasonic images based on the multi-dimensional image features, a determination unit that determines a lesion part based on the scores calculated for each of the plurality of ultrasonic images, an extraction unit that extracts, as a basis image, an ultrasonic image that best represents the determination result by the determination unit from the plurality of ultrasonic images, a monitor, and a display control unit that displays the determination result by the determination unit and the basis image extracted by the extraction unit on the monitor. Therefore, a user can accurately diagnose a lesion part

Brief Description of the Drawings

[0010] [Figure 1] It is a block diagram showing the configuration of an ultrasonic diagnostic apparatus according to Embodiment 1 of the present invention [Figure 2] It is a block diagram showing the internal configuration of a transmission / reception circuit in Embodiment 1 [Figure 3] It is a block diagram showing the internal configuration of an image generation unit in Embodiment 1 [Figure 4] It is a diagram schematically showing a lesion part having an oval shape [Figure 5] It is a diagram schematically showing a lesion part having a polygonal shape [Figure 6] It is a diagram schematically showing a lesion part having a lobulated shape [Figure 7] It is a diagram schematically showing a lesion part having an irregular shape [Figure 8] It is a diagram showing an example of display of a determination result regarding a lesion part [Figure 9] It is a diagram showing another example of display of a determination result regarding a lesion part [Figure 10]It is a diagram showing an example of display of a score regarding the malignancy of a lesion part. [Figure 11] It is a diagram showing an example of display of a score regarding the category of a lesion part. [Figure 12] It is a diagram showing an example of a basis image displayed on a monitor. [Figure 13] It is a diagram showing an example of an ultrasonic image in which a lesion part is highlighted. [Figure 14] It is a diagram showing an example of an ultrasonic image on which a heat map regarding the score of a lesion part is superimposed. [Figure 15] It is a flowchart showing the operation of the ultrasonic diagnostic apparatus of Embodiment 1. [Figure 16] It is a block diagram showing the configuration of the ultrasonic diagnostic apparatus of Embodiment 2.

Embodiments for Carrying Out the Invention

[0011] Hereinafter, embodiments of this invention will be described based on the accompanying drawings. The description of the constituent elements described below is made based on typical embodiments of the present invention, but the present invention is not limited to such embodiments. In this specification, a numerical range represented by "~" means a range including the numerical values described before and after "~" as the lower limit value and the upper limit value. In this specification, "identical" and "the same" shall include the error ranges generally acceptable in the technical field.

[0012] Embodiment 1 FIG. 1 shows the configuration of the ultrasonic diagnostic apparatus according to Embodiment 1 of the present invention. The ultrasonic diagnostic apparatus includes an ultrasonic probe 1 and a device main body 2. The ultrasonic probe 1 and the device main body 2 are wired-connected to each other via a cable not shown. ]

[0013] The ultrasonic probe 1 has a transducer array 11 and a transmission / reception circuit 12 connected to the transducer array 11.

[0014] [[ID=*45]] The main unit 2 has an image generation unit 21 connected to the transmitting / receiving circuit 12 of the ultrasonic probe 1. A display control unit 22 and a monitor 23 are sequentially connected to the image generation unit 21, and an image memory 24 is connected to the image generation unit 21. An image feature calculation unit 25, a score calculation unit 26, a determination unit 27, and an extraction unit 28 are sequentially connected to the image memory 24. The determination unit 27 is connected to the display control unit 22. The extraction unit 28 is connected to the display control unit 22 and the image memory 24. In addition, a main unit control unit 29 is connected to the transmitting / receiving circuit 12, the image generation unit 21, the display control unit 22, the image memory 24, the image feature calculation unit 25, the score calculation unit 26, the determination unit 27, and the extraction unit 28. An input device 30 is also connected to the main unit control unit 29.

[0015] Furthermore, the processor 31 for the main body of the device 2 is composed of an image generation unit 21, a display control unit 22, an image feature calculation unit 25, a score calculation unit 26, a determination unit 27, an extraction unit 28, and a main body control unit 29.

[0016] The transducer array 11 of the ultrasonic probe 1 has a plurality of ultrasonic transducers arranged in one or two dimensions. Each of these transducers transmits ultrasound according to a drive signal supplied from the transmitting / receiving circuit 12 and receives reflected waves from the subject to output an analog received signal. Each transducer is constructed by forming electrodes at both ends of a piezoelectric body made of, for example, a piezoelectric ceramic represented by PZT (Lead Zirconate Titanate), a polymer piezoelectric element represented by PVDF (Poly Vinylidene Di Fluoride), or a piezoelectric single crystal represented by PMN-PT (Lead Magnesium Niobate-Lead Titanate).

[0017] The transmitting / receiving circuit 12 transmits ultrasonic waves from the transducer array 11 and generates a sound line signal based on the received signal acquired by the transducer array 11, under the control of the main unit control 29. As shown in Figure 2, the transmitting / receiving circuit 12 has a pulser 41 connected to the transducer array 11, and an amplifier 42, an AD (Analog Digital) converter 43, and a beamformer 44 connected sequentially in series to the transducer array 11.

[0018] The pulser 41 includes, for example, multiple pulse generators, and based on a transmission delay pattern selected according to a control signal from the main unit control 29, it supplies each drive signal to the multiple transducers of the transducer array 11, adjusting the delay amount, so that the ultrasonic waves transmitted from the multiple transducers form an ultrasonic beam. In this way, when a pulsed or continuous wave voltage is applied to the electrodes of the transducers of the transducer array 11, the piezoelectric material expands and contracts, generating pulsed or continuous wave ultrasonic waves from each transducer, and an ultrasonic beam is formed from the combined wave of these ultrasonic waves.

[0019] The transmitted ultrasonic beam is reflected by an object, such as a part of the subject, and the ultrasonic echo propagates toward the transducer array 11 of the ultrasonic probe 1. The ultrasonic echo propagating toward the transducer array 11 is received by each of the transducers that make up the transducer array 11. At this time, each transducer that makes up the transducer array 11 expands and contracts upon receiving the propagating ultrasonic echo, generating a received signal which is an electrical signal, and outputs these received signals to the amplification unit 42.

[0020] The amplification unit 42 amplifies the signals input from each transducer constituting the transducer array 11 and transmits the amplified signals to the AD conversion unit 43. The AD conversion unit 43 converts the signals transmitted from the amplification unit 42 into digital received data and transmits this received data to the beamformer 44. The beamformer 44 performs so-called receive focus processing by adding each received data converted by the AD conversion unit 43 with a corresponding delay, according to the sound velocity or sound velocity distribution set based on the reception delay pattern selected according to the control signal from the main unit control unit 29. Through this receive focus processing, each received data converted by the AD conversion unit 43 is phase-corrected and added together, and a sound ray signal with a focused ultrasonic echo is obtained.

[0021] As shown in Figure 3, the image generation unit 21 of the main body 2 of the device has a configuration in which a signal processing unit 45, a DSC (Digital Scan Converter) 46, and an image processing unit 47 are connected in series in sequence. The signal processing unit 45 applies distance-dependent attenuation correction to the sound line signal transmitted from the transmitting / receiving circuit 12 of the ultrasonic probe 1 according to the depth of the ultrasonic reflection position, and then performs envelope detection processing to generate an ultrasonic image signal (B-mode image signal), which is tomographic image information about the tissue within the subject.

[0022] The DSC46 converts the ultrasonic image signal generated by the signal processing unit 45 into an image signal that follows the scanning method of a normal television signal (raster conversion). The image processing unit 47 performs various necessary image processing, such as grayscale processing, on the ultrasound image signal input from the DSC 46, and then outputs a signal representing the ultrasound image to the display control unit 22 and the image memory 24. The signal representing the ultrasound image generated in this way by the image generation unit 21 will simply be called the ultrasound image.

[0023] The image memory 24 is a memory that stores ultrasound images generated by the image generation unit 21 under the control of the main unit control unit 29. For example, the image memory 24 can store multiple ultrasound images that constitute a video, generated by the image generation unit 21 by continuously capturing regions containing the same lesion of the subject. Each of the multiple ultrasound images may be an image of the entire lesion, or an image of a part of the lesion.

[0024] Image memory 24 can be recorded using recording media such as flash memory, HDD (Hard Disc Drive), SSD (Solid State Drive), FD (Flexible Disc), MO (Magneto-Optical Disc), MT (Magnetic Tape), RAM (Random Access Memory), CD (Compact Disc), DVD (Digital Versatile Disc), SD card (Secure Digital card), or USB memory (Universal Serial Bus memory).

[0025] The image feature calculation unit 25 calculates multidimensional image features for each of the multiple ultrasound images stored in the image memory 24 by analyzing multiple ultrasound images of the lesion area of ​​the subject. The image feature calculation unit 25 can input multiple ultrasound images into an image recognition model, such as a neural network in machine learning, and calculate the intermediate numerical data output for each of the multiple ultrasound images as multidimensional image features. This intermediate numerical data is sometimes commonly called a feature in the field of machine learning.

[0026] Furthermore, the image feature calculation unit 25 can also calculate, for example, the area ratio of low-luminance regions having a luminance lower than a predetermined luminance to the entire area of ​​the ultrasound image, and the contrast ratio between the edges of low-luminance regions and the surrounding areas, as multidimensional image features. The image feature calculation unit 25 can calculate the area ratio of low-luminance regions to the entire area of ​​the ultrasound image and the contrast ratio between the edges of low-luminance regions and the surrounding areas using an image recognition model, or it can calculate them using known image analysis methods without using an image recognition model.

[0027] The score calculation unit 26 calculates a score for each of the multiple ultrasound images based on the multidimensional image features calculated by the image feature calculation unit 25. The score calculation unit 26 can calculate a score for a lesion from multidimensional image features, for example, by using a machine learning model (e.g., a neural network and a deep learning model) that has been pre-trained with training data on the relationships between numerous multidimensional image features obtained from numerous typical ultrasound images in which many lesions are captured, and information about the lesions such as their pathological findings, shape, and diagnostic findings.

[0028] The score calculation unit 26 can calculate multiple types of scores, including a benign / malignant score, a category score, a shape score, a border score, a halo score, a border fracture score, and a breast tissue location score. The benign / malignant score is a score that represents the degree of benignity (benign score) or malignancy (malignant score) of the lesion of a subject from which an ultrasound image has been taken.

[0029] Category scores represent the so-called screening or diagnostic categories of lesions, as defined by organizations such as JABTS (the Japanese Association of Breast and Thyroid Sonology) or ACR (American College of Radiology). For example, JABTS defines screening categories 1-5 as follows: Examination Category 1: No abnormal findings Screening Category 2: Findings present, but no further examination required. Screening Category 3: Benign, but malignancy cannot be ruled out. Screening Category 4: Suspected malignancy Screening Category 5: Malignant

[0030] The shape score is a score related to the shape of the lesion. For example, the shape score is higher the more "constrictions" and "corners" the lesion has. Generally, the shape of the lesion M is classified into, for example, circular or elliptical (no constrictions, no corners) as shown in Figure 4, polygonal (no constrictions, with corners) as shown in Figure 5, lobulated (with constrictions, no corners) as shown in Figure 6, and irregular (with constrictions, with corners) as shown in Figure 7, depending on whether the shape, including the hyperechoic portion of the boundary in the ultrasound image, has "constrictions" and "corners".

[0031] The score calculation unit 26 can, for example, calculate the number of constrictions in the lesion M as a score.

[0032] The boundary score is a score related to the smoothness of the boundary portion of the lesion M. For example, a rougher boundary portion has a higher value, and a smoother boundary portion has a lower value.

[0033] The halo score indicates whether or not a so-called halo, a hyperechoic zone, is present at the boundary of the lesion M. The halo score will output either a "+" score indicating the presence of a halo or a "-" score indicating the absence of a halo.

[0034] The boundary fracture score indicates whether or not the boundary between the breast tissue and its surrounding tissues is fractured by the lesion M. The score calculation unit 26 can, for example, extract the boundary of the tissue surrounding a certain area of ​​the lesion M and determine whether or not the extracted boundary is fractured by the boundary of the lesion M. As an example of the boundary fracture score, either a "+" score indicating that the boundary is fractured or a "-" score indicating that the boundary is not fractured will be output.

[0035] The breast site score indicates the location of the lesion M within the breast tissue. For example, the breast site score can indicate whether the lesion M is located on the ductal side or the lobular side of the breast tissue.

[0036] The determination unit 27 determines the lesion M based on the scores calculated by the score calculation unit 26 for multiple ultrasound images. For example, the determination unit 27 can determine whether the shape of the lesion M is circular, elliptical, polygonal, lobulated, or irregular based on the shape score calculated by the score calculation unit 26. The determination unit 27 can also determine whether the boundary of the lesion M in the ultrasound image is smooth, rough, or indistinct based on the boundary score calculated by the score calculation unit 26.

[0037] Furthermore, the determination unit 27 can also determine the histological type of lesion M based on, for example, the halo score, border fracture score, and breast tissue location score calculated by the score calculation unit 26. For example, lesions occurring in the breast are generally known to be classified into several histological types, such as DCIS (Ductal Carcinoma In Situ), IDC (Invasive Ductal Carcinoma), and ILC (Invasive Lobular Carcinoma). The determination unit 27 can determine, for example, whether lesion M is invasive or non-invasive based on the halo score and border fracture score, and whether lesion M is located on the ductal side or the lobular side based on the breast tissue location score, and finally determine the histological type of lesion M by combining these determination results.

[0038] Here, the determination unit 27 can make a determination based on the score of the ultrasound image in which the lesion M is largest among multiple ultrasound images. In this process, the determination unit 27 calculates the maximum diameter of the lesion M in multiple ultrasound images by analyzing the multiple ultrasound images, and can identify the ultrasound image in which the lesion M is largest based on the calculated maximum diameter value.

[0039] Furthermore, the determination unit 27 can also make a determination for each of the multiple types of scores based on a majority vote of the scores of multiple ultrasound images, for example, the shape score having the most frequently calculated value among multiple shape scores, and the boundary score having the most frequently calculated value among multiple boundary scores.

[0040] Furthermore, the determination unit 27 can also make a determination for each of the multiple types of scores based on the maximum value of the scores of multiple ultrasound images, for example, the maximum shape score among multiple shape scores, and the maximum boundary score among multiple boundary scores.

[0041] Furthermore, the determination unit 27 can also make a determination for each of the multiple types of scores based on the median value of the scores of multiple ultrasound images, for example, the shape score that is the median of the multiple shape scores, or the mode, for example, the value that was obtained most frequently among the multiple shape scores.

[0042] Furthermore, the determination unit 27 can also remove outliers for each of the multiple types of scores and make a determination based on the majority vote, maximum value, median, or mode of the multiple scores from which the outliers have been removed. For example, the determination unit 27 may have a certain threshold for the scores and can exclude scores that exceed the threshold as outliers.

[0043] Furthermore, the determination unit 27 identifies a score corresponding to the determination result and outputs the information of the ultrasound image from which that score was calculated to the extraction unit 28. For example, if the determination unit 27 determines that the boundary of the lesion M in the ultrasound image is rough, it can identify the boundary score that is determined to be rough from among multiple boundary scores and output the information of the ultrasound image from which that boundary score was calculated to the extraction unit 28.

[0044] Furthermore, the determination unit 27 can output to the display control unit 22 representative values ​​of scores that are easy for the user to understand, such as the benign / malignant score of the lesion M, the category score of the lesion M, and the number of constrictions in the lesion M, from the scores calculated by the score calculation unit 26. In this case, the determination unit 27 can output, for example, the score of the ultrasound image in which the lesion M is largest, the majority vote of multiple ultrasound images, or the maximum value of the scores of multiple ultrasound images as representative values, in the same way as when determining the lesion M.

[0045] Furthermore, the determination unit 27 sends the determination result regarding the lesion M to the display control unit 22.

[0046] The extraction unit 28 receives the determination result of the lesion M from the determination unit 27 and extracts at least one ultrasound image from among the multiple ultrasound images stored in the image memory 24 that best represents the determination result by the determination unit 27 as the basis image. For example, if the determination unit 27 determines that the boundary of the lesion M in the ultrasound image is rough, the extraction unit 28 can extract the ultrasound image with the highest boundary score among the at least one boundary score determined to be rough, which is sent from the determination unit 27, as the basis image for the determination that the boundary of the lesion M is rough.

[0047] Furthermore, if the determination unit 27 determines, for example, that the boundary of the lesion M in the ultrasound image is rough, the extraction unit 28 can select all ultrasound images with a boundary score higher than a predetermined boundary score threshold, calculate the similarity between the selected ultrasound images, and group together multiple ultrasound images whose calculated similarity is lower than a certain value, i.e., multiple similar ultrasound images, and extract the ultrasound image with the maximum or median boundary score in each group as the supporting image. The extraction unit 28 can extract multiple supporting images if, for example, multiple groups of similar ultrasound images are created.

[0048] The display control unit 22 performs predetermined processing on the ultrasound image generated by the image generation unit 21, the representative value of the score output by the determination unit 27, the determination result from the determination unit 27, and at least one evidence image U extracted by the extraction unit 28, and displays them on the monitor 23.

[0049] The display control unit 22 can display the findings feature A1, as shown in Figure 8, on the monitor 23 as a result of the determination in the determination unit 27. In the example in Figure 8, findings feature A1 includes the items "shape," "boundary line," "internal echo," "halo," "boundary line fracture," "backward echo," "aspect ratio," and "calcification."

[0050] The "Shape" item indicates whether the shape of the lesion M is circular, elliptical, polygonal, lobulated, or irregular. The "Boundary" item indicates whether the boundary of the lesion M is smooth, rough, or indistinct. The "Internal Echo" item is divided into "Level" and "Homogeneity" items. The "Level" item indicates whether the echo level, which represents the brightness within the lesion M in the ultrasound image, is "None," "Very Low," "Low," "Iso," or "High." The closer the echo level is to "None," the lower the brightness within the lesion M in the ultrasound image, and the closer the echo level is to "High," the higher the brightness within the lesion M in the ultrasound image. The "Homogeneity" item indicates whether the inside of the lesion M in the ultrasound image is homogeneous or heterogeneous.

[0051] The "halo" column indicates whether or not a halo, a hyperechoic band, is present at the boundary of lesion M. A "+" indicates the presence of a halo, and a "-" indicates the absence of a halo. The "boundary fracture" column indicates whether or not the boundary of lesion M is fractured. A "+" indicates a fractured boundary, and a "-" indicates that the boundary is not fractured.

[0052] The "Posterior Echo" item indicates whether the echo level in the deeper region of lesion M is enhanced, unchanged, attenuated, or absent compared to the echo level in the shallower region of lesion M. The "Aspect Ratio" item indicates whether the depth dimension of lesion M in the ultrasound image is larger or smaller than the dimension in the direction perpendicular to the depth direction. The "Calcification" item indicates whether the calcification is fine or coarse, if any calcified areas are present in lesion M.

[0053] In the example shown in Figure 8, finding feature A1 indicates that the "shape" is irregular, the "boundary" is unclear, the "level" of the "internal echo" is equal, the "homogeneity" is heterogeneous, the "halo" is "+" (present), the "boundary fracture" is "+" (the boundary is fractured), the "posterior echo" is weak, the "aspect ratio" is small, and there is no "calcification".

[0054] Furthermore, the display control unit 22 can display the estimated tissue type A2, as shown in Figure 9, on the monitor 23 as a result of the determination in the determination unit 27. In the example in Figure 9, the estimated tissue type A2 indicates the items "DCIS," "IDC," "ILC," and "other," and the probability that the lesion M captured in multiple ultrasound images has a tissue type corresponding to the items "DCIS," "IDC," "ILC," and "other."

[0055] Furthermore, the display control unit 22 can display a malignancy grade A3, as shown in Figure 10, on the monitor 23 as a representative value of the score output by the determination unit 27. In the example in Figure 10, the display of malignancy grade A3 includes a numerical value (○○%) representing the malignancy of the lesion M and a meter that visually represents that value.

[0056] Furthermore, the display control unit 22 can display a representative value of the score output by the determination unit 27, such as the examination category A4 shown in Figure 11, on the monitor 23. In the example in Figure 11, the display of examination category A4 indicates whether the examination category of lesion M is 1 to 5. In Figure 11, it is shown that the examination category of lesion M is 4.

[0057] Here, for example, if a user selects one of the judgment results of the judgment unit 27 displayed on the monitor 23 via the input device 30, the display control unit 22 displays on the monitor 23 the evidence image U extracted by the extraction unit 28 that best represents the selected judgment result, as shown in Figure 12.

[0058] For example, if the determination unit 27 determines that the boundary of the lesion M is unclear, and the user selects the "unclear" area in the "boundary line" item of the finding feature A1, the display control unit 22 can display on the monitor 23 at least one evidence image U extracted by the extraction unit 28 that best represents the unclear boundary of the lesion M.

[0059] When the display control unit 22 displays multiple evidence images U that best represent the judgment result of the judgment unit 27 on the monitor 23, it can display the multiple evidence images U in a so-called scrolling display, and can also sequentially switch between them according to the user's instructions via the input device 30. In this case, the display control unit 22 assigns numbers sequentially to the evidence images U, from the oldest generated to the most recently generated, and can display the evidence images U and their numbers together. By checking the numbers, the user can distinguish and recognize the multiple ultrasound images from one another.

[0060] In this way, the judgment result of the judgment unit 27 and the supporting image U corresponding to that judgment result are displayed together. Therefore, by checking the supporting image U, the user can understand the basis of the judgment result, and thus easily and accurately select an ultrasound image suitable for the diagnosis of the subject, and accurately diagnose the subject.

[0061] Furthermore, as shown in Figure 13, the extraction unit 28 can detect lesions M by performing image analysis on the reference image U and form a bounding box L surrounding the detected lesions M. In this case, the display control unit 22 can superimpose the bounding box L formed by the extraction unit 28 onto the reference image U. At this time, the display control unit 22 can also color the area inside the bounding box L. By highlighting the lesions M in the reference image U in this way, the user can easily grasp the location and shape of the lesions M in the reference image U.

[0062] Furthermore, as shown in Figure 14, the extraction unit 28 can form a heatmap HM in the evidence image U in which the contribution rate of the portion that contributes to the score calculation by the score calculation unit 26 is represented by the intensity of color or the difference in color. The portion that contributes to the score calculation is the region that was focused on in the score calculation. For example, when a shape score or boundary score is calculated, the portion that contributed to the judgment in the boundary of the lesion M and / or its vicinity is highlighted. For example, if the boundary is determined to be unclear, the unclear portion of the boundary is highlighted. Also, for echoes inside the lesion M, for example, when a score related to homogeneity is calculated, the portion that contributed to the judgment inside the lesion M, for example, the heterogeneous portion, is highlighted. By checking the heatmap HM, the user can understand the region on the evidence image U related to the score and understand the basis for the judgment result of the judgment unit 27 in more detail.

[0063] The monitor 23, under the control of the display control unit 22, displays the ultrasound image generated by the image generation unit 21, the representative value of the score output by the determination unit 27, the determination result from the determination unit 27, and at least one evidence image U extracted by the extraction unit 28, and has a display device such as an LCD (Liquid Crystal Display) or an organic EL display (Organic Electroluminescence Display).

[0064] The main unit control unit 29 controls each part of the main unit 2 and the transmitting / receiving circuit 12 of the ultrasonic probe 1 based on a control program or the like that is stored in advance.

[0065] The input device 30 is for the user to perform input operations and consists of devices such as a keyboard, mouse, trackball, touchpad, and touch sensor placed on top of the monitor 23.

[0066] The processor 31, which includes an image generation unit 21, a display control unit 22, an image feature calculation unit 25, a score calculation unit 26, a determination unit 27, an extraction unit 28, and a main unit control unit 29, is composed of a CPU (Central Processing Unit) and a control program for causing the CPU to perform various processes. However, it may also be composed of an FPGA (Field Programmable Gate Array), a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), a GPU (Graphics Processing Unit), or other ICs (Integrated Circuits), or a combination thereof.

[0067] Furthermore, the image generation unit 21, display control unit 22, image feature calculation unit 25, score calculation unit 26, determination unit 27, extraction unit 28, and main unit control unit 29 of the processor 31 can be partially or entirely integrated into a single CPU or the like.

[0068] Next, the operation of the ultrasound diagnostic apparatus of Embodiment 1 of the present invention will be explained using the flowchart shown in Figure 15.

[0069] First, in step S1, with the ultrasound probe 1 in contact with the subject's body surface by the user, multiple ultrasound images of the subject's lesion M are acquired. When ultrasound images are acquired, the transmitting / receiving circuit 12 generates an acoustic signal by performing a so-called receive focus process under the control of the main unit control 29. The acoustic signal generated by the transmitting / receiving circuit 12 is sent to the image generation unit 21. The image generation unit 21 generates an ultrasound image using the acoustic signal sent from the transmitting / receiving circuit 12. This process is repeated to acquire multiple consecutive ultrasound images. The acquired ultrasound images are sent to the image memory 24 and stored in the image memory 24.

[0070] Next, in step S2, the image feature calculation unit 25 calculates multidimensional image features for each of the multiple ultrasound images generated in step S1 and stored in the image memory 24 by analyzing the multiple ultrasound images. The image feature calculation unit 25 can, for example, input multiple ultrasound images into an image recognition model such as a neural network in machine learning, and calculate the intermediate numerical data output for each of the multiple ultrasound images as multidimensional image features. The image feature calculation unit 25 can also calculate, for example, the area ratio of low-luminance regions having a luminance lower than a predetermined luminance to the entire area of ​​the ultrasound image, the contrast ratio between the edge of the low-luminance region and the surrounding area, etc., as multidimensional image features.

[0071] In step S3, the score calculation unit 26 calculates a score for the lesion M of the subject based on the multidimensional image features in the multiple ultrasound images calculated in step S2. The score calculation unit 26 can calculate multiple types of scores for the lesion M, such as a benign / malignant score, a category score, a shape score, and a boundary score. The score calculation unit 26 can also calculate a score from the multidimensional image features using, for example, a machine learning model that has learned the relationship between the multidimensional image features and these scores.

[0072] In step S4, the determination unit 27 determines the lesion M based on the scores calculated for each of the multiple ultrasound images in step S3. The determination unit 27 can output the shape of the lesion M, the smoothness of the boundary of the lesion M, the tissue type of the lesion M, etc., as determination results. The determination unit 27 also sends the ultrasound image information corresponding to each determination result to the extraction unit 28.

[0073] In step S5, the extraction unit 28 extracts at least one ultrasound image that best represents the determination result from the multiple ultrasound images stored in the image memory 24 in step S1, based on the ultrasound image information corresponding to the determination result sent from the determination unit 27 in step S4, and uses it as the basis image U.

[0074] Finally, in step S6, the display control unit 22 displays the findings characteristic A5, which is the judgment result in step S4, and at least one evidence image U that best represents any of the items of findings characteristic A5 on the monitor 23, for example as shown in Figure 12. At this time, if the user selects a judgment result indicated by an item in findings characteristic A5 via the input device 30, for example, the display control unit 22 can display at least one evidence image U that best represents the selected judgment result on the monitor 23. By checking the evidence image U, the user can understand the basis for the selected judgment result in detail.

[0075] Once the process in step S6 is completed, the operation of the ultrasound diagnostic device shown in the flowchart in Figure 15 is finished.

[0076] As described above, according to the ultrasound diagnostic apparatus of Embodiment 1 of the present invention, the score calculation unit 26 calculates a score for the lesion M based on the multidimensional image features calculated by the image feature calculation unit 25, the determination unit 27 makes a determination on the lesion M based on the score calculated by the score calculation unit 26, and the extraction unit 28 extracts the ultrasound image that best represents the determination result by the determination unit 27 from among a plurality of ultrasound images as the basis image U. The determination result by the determination unit 27 and the basis image U extracted by the extraction unit 28 are displayed on the monitor 23, so that the user can understand the basis of the selected determination result in detail and make an accurate diagnosis of the lesion M of the subject.

[0077] Although the ultrasound probe 1 and the main unit 2 are described as being connected to each other by a wire, they can also be connected to each other wirelessly. Furthermore, the main unit 2 of the device may be a stationary type, a portable type, or a handheld type consisting of a smartphone or tablet computer, etc. Thus, the type of equipment that makes up the main unit 2 is not particularly limited.

[0078] Furthermore, although the transmitting and receiving circuit 12 is provided in the ultrasonic probe 1, it may also be provided in the main body of the device 2 instead. Furthermore, although the image generation unit 21 is provided in the main body of the device 2, it may also be provided in the ultrasonic probe 1 instead.

[0079] Furthermore, although the ultrasound diagnostic apparatus of Embodiment 1 is equipped with an ultrasound probe 1, it is not required to be equipped with an ultrasound probe 1. In this case, for example, the main body of the apparatus 2 (not shown) may be equipped with an image input unit instead of an image generation unit 21, which is connected to an external ultrasound diagnostic apparatus, server device, or storage medium, and to which multiple ultrasound images are input. Multiple ultrasound images input to the image input unit from an external device (not shown) are stored in the image memory 24. Processing is performed on the multiple ultrasound images stored in this manner by the image feature calculation unit 25, score calculation unit 26, determination unit 27, extraction unit 28, and display control unit 22.

[0080] Embodiment 2 Figure 16 shows an ultrasound diagnostic apparatus of Embodiment 2. This ultrasound diagnostic apparatus is equipped with an apparatus body 2A in place of the apparatus body 2 shown in Figure 1. The apparatus body 2A is equipped with an image thinning unit 51 in addition to the apparatus body 2 shown in Figure 1, and an apparatus body control unit 29A in place of the apparatus body 29. Furthermore, in the apparatus body 2A, a processor 31A is configured by an image generation unit 21, a display control unit 22, an image feature calculation unit 25, a score calculation unit 26, a determination unit 27, an extraction unit 28, and an apparatus body control unit 29A.

[0081] The image thinning unit 51 thins out multiple ultrasound images stored in the image memory 24. For example, the image thinning unit 51 can thin out ultrasound images from multiple ultrasound images at regular intervals. In addition, the image thinning unit 51 can perform image analysis on multiple ultrasound images to detect lesions M in each of the multiple ultrasound images, and thin out ultrasound images in which lesions M could not be detected.

[0082] In this way, the image thinning unit 51 performs the thinning process, and based on the remaining ultrasound images, the image feature calculation unit 25 calculates multidimensional image features. The score calculation unit 26 calculates a score for the lesion M based on the calculated multidimensional image features, and the determination unit 27 determines the lesion M based on the calculated score for the lesion M. The extraction unit 28 extracts the evidence image U that best represents the determination result by the determination unit 27 from the ultrasound images remaining after the thinning process by the image thinning unit 51. The display control unit 22 displays the determination result by the determination unit 27 and the evidence image U extracted by the extraction unit 28 on the monitor 23.

[0083] Thus, even when multiple ultrasound images are filtered by the image filtering unit 51, the display control unit 22 displays the judgment result from the judgment unit 27 and the evidence image U extracted by the extraction unit 28 on the monitor 23. This allows the user to understand the basis for the selected judgment result in detail and accurately diagnose the lesion M of the subject.

[0084] Although not shown in the diagram, the main unit 2A of the device may also be equipped with an image interpolation unit that interpolates multiple ultrasound images stored in the image memory 24 instead of the image thinning unit 51. In this case, the number of ultrasound images used to calculate multidimensional image features by the image feature calculation unit 25 and the number of ultrasound images to be extracted by the extraction unit 28 will increase compared to the number of ultrasound images stored in the image memory 24. However, since the display control unit 22 displays the judgment result by the judgment unit 27 and the evidence image U extracted by the extraction unit 28 on the monitor 23, the user can understand the basis of the selected judgment result in detail and accurately diagnose the lesion M of the subject.

[0085] Although not shown in the diagram, the main unit 2A of the device may also include an image synthesis unit instead of an image thinning unit 51, which synthesizes multiple ultrasound images stored in the image memory 24 to generate multiple composite images. The image synthesis unit can generate composite images with improved image quality by synthesizing, for example, temporally adjacent and similar ultrasound images. The image feature calculation unit 25 calculates multidimensional image features from each of the multiple composite images by performing image analysis on the multiple composite images. The extraction unit 28 extracts the composite image that best represents the judgment result by the judgment unit 27 from the multiple composite images as the basis image U. In this way, even when multiple ultrasound images are synthesized, the display control unit 22 displays the judgment result by the judgment unit 27 and the basis image U extracted by the extraction unit 28 on the monitor 23, so that the user can understand the basis of the selected judgment result in detail and accurately diagnose the lesion M of the subject.

[0086] Alternatively, instead of the main unit 2A having an image thinning unit 51, an image interpolation unit, or an image synthesis unit, multiple thinned ultrasound images, multiple interpolated ultrasound images, or multiple synthesized ultrasound images can be stored in the image memory 24, and these ultrasound images can be used for processing. [Explanation of Symbols]

[0087] 1 Ultrasound probe, 2,2A Main unit, 11 Transducer array, 12 Transmit / receive circuit, 21 Image generation unit, 22 Display control unit, 23 Monitor, 24 Image memory, 25 Image feature calculation unit, 26 Score calculation unit, 27 Judgment unit, 28 Extraction unit, 29,29A Main unit control unit, 30 Input device, 31,31A Processor, 41 Pulsar, 42 Amplifier unit, 43 AD conversion unit, 44 Beamformer, 45 Signal processing unit, 46 DSC, 47 Image processing unit, 51 Image thinning unit, A1 Finding features, A2 Estimated histological type, A3 Malignancy grade, A4 Screening category, HM Heatmap, L Borderline, M Lesion area, U Evidence image.

Claims

1. An ultrasound diagnostic device that determines a lesion based on multiple ultrasound images taken of the lesion of a subject, An image feature calculation unit calculates multidimensional image features in each of the plurality of ultrasound images by analyzing the plurality of ultrasound images, A score calculation unit that calculates multiple types of scores for the lesion in each of the multiple ultrasound images based on the multidimensional image features, A determination unit that stores in advance multiple findings items corresponding to multiple types of scores calculated for each of the multiple ultrasound images, multiple candidate features for each of the multiple findings items, classifies each feature of the multiple findings items into one of the multiple candidates based on the multiple types of scores, and outputs the classified features for each of the multiple findings items as multiple determination results for the lesion. An extraction unit extracts multiple different evidence images from the multiple ultrasound images for the multiple determination results from the determination unit, Monitor and, The display control unit displays the plurality of determination results by the determination unit and the evidence images extracted by the extraction unit and corresponding to each of the plurality of determination results on the monitor. Equipped with, An ultrasound diagnostic device in which each of the multiple supporting images is the ultrasound image that best represents each of the multiple judgment results.

2. The ultrasound diagnostic apparatus according to claim 1, wherein the plurality of ultrasound images are images that constitute a video of the lesion.

3. The ultrasound diagnostic apparatus according to claim 1 or 2, wherein each of the plurality of ultrasound images is an image of the whole or a part of the lesion.

4. The ultrasound diagnostic apparatus according to claim 2, wherein the plurality of ultrasound images are images obtained by downsampling, interpolating, or combining images from images constituting a video of the lesion.

5. The extraction unit forms a bounding line around the lesion in the evidence image. The ultrasound diagnostic apparatus according to claim 1 or 2, wherein the display control unit superimposes the outline onto the reference image.

6. The extraction unit forms a heatmap in the reference image in which the contribution rate of the portion that contributes to the calculation of the score by the score calculation unit is represented by the intensity of color or the difference in color. The ultrasonic diagnostic apparatus according to claim 1 or 2, wherein the display control unit displays the heat map on the monitor.

7. The ultrasound diagnostic apparatus according to claim 1 or 2, wherein the score calculation unit calculates the score using a machine learning model that has learned the multidimensional image features of the ultrasound image.

8. The ultrasound diagnostic apparatus according to claim 1 or 2, wherein the determination unit outputs the plurality of determination results based on the plurality of scores for the ultrasound image in which the lesion is captured most prominently among the plurality of ultrasound images.

9. The ultrasound diagnostic apparatus according to claim 1 or 2, wherein the determination unit outputs a plurality of determination results based on the majority vote, maximum value, median, or mode of each of the plurality of types of scores of the plurality of ultrasound images.

10. The ultrasound diagnostic apparatus according to claim 1 or 2, wherein the determination unit removes outliers from the plurality of types of scores of the plurality of ultrasound images and outputs the plurality of determination results based on the majority vote, maximum value, median, or mode of the plurality of types of scores of the plurality of ultrasound images from which the outliers have been removed.

11. A control method for an ultrasound diagnostic device that determines a lesion based on multiple ultrasound images taken of the lesion of a subject, By analyzing the plurality of ultrasound images, multidimensional image features are calculated for each of the plurality of ultrasound images. Based on the multidimensional image features, multiple types of scores for the lesion in each of the multiple ultrasound images are calculated. Multiple findings items corresponding to the multiple types of scores calculated for each of the multiple ultrasound images, and multiple candidate features for each of the multiple findings items are stored in advance, and each feature of the multiple findings items is classified into one of the multiple candidates based on the multiple types of scores, and the classified features for each of the multiple findings items are output as multiple judgment results for the lesion. For the aforementioned multiple judgment results, multiple evidence images that are different from each other are extracted from the aforementioned multiple ultrasound images. The plurality of judgment results and the supporting images corresponding to each of the plurality of judgment results are displayed on the monitor. Each of the multiple supporting images is the ultrasound image that best represents each of the multiple judgment results. A method for controlling an ultrasound diagnostic device.

Images