Ultrasound diagnostic device and ultrasound image processing method

Abstract

To display a mark for notifying a user of detection of a lesion part together with inspection support information different from the mark to support inspection of a lesion part overall.SOLUTION: An ultrasonic diagnostic apparatus displays a mark 48 for specifying a lesion part 50 on a tomographic image 42. The ultrasonic diagnostic apparatus displays a two-dimensional map 44 and a graph 46 irrespective of whether the mark 48 is displayed or not. The two-dimensional map 44 indicates spatial probability distribution of a lesion part. The graph 46 indicates temporal probability distribution of a lesion part.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present disclosure relates to ultrasound diagnostic devices and ultrasound image processing methods, and more particularly to generating and providing information to assist in ultrasound examinations. [Background technology]

[0002] In an ultrasound examination using an ultrasound diagnostic device, an examiner observes an ultrasound image while simultaneously operating a probe (ultrasound probe). If a lesion (or more precisely, a lesion candidate) such as a mass appears in the ultrasound image, the lesion is examined in detail.

[0003] Some ultrasound diagnostic devices have a function of automatically detecting a lesion that appears in an ultrasound image and notifying the user of the detected lesion. This function is called CADe (Computer Aided Detection). CADe is a technology that belongs to or is related to CAD (Computer Aided Diagnosis). When CADe is executed, the lesion probability (lesion score, lesion similarity) is generally calculated, and a figure (mark) surrounding the lesion is displayed when the lesion probability exceeds a threshold. If the lesion probability is smaller than the threshold, the mark is not displayed. By controlling the presence or absence of the mark display using the threshold in this way, the occurrence of a situation in which marks are frequently displayed on an ultrasound image, or the occurrence of a situation in which many marks are displayed on an ultrasound image, is avoided.

[0004] Patent Document 1 discloses an ultrasound diagnostic device equipped with an object detection function. Patent Document 1 does not disclose a technique for displaying a spatial distribution of lesion probability or a temporal change in lesion probability as a dynamic image. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2020-28680 A Summary of the Invention [Problem to be solved by the invention]

[0006] When a mark notifying a lesion is displayed on an ultrasound image, the examiner can recognize the fact that a lesion has been detected, but cannot recognize the spatial distribution of the lesion probability or the change in the lesion probability over time through the mark. If the lesion probability does not exceed a threshold, the mark is not displayed, and even if something that appears to be a lesion is actually detected, no information about it is provided to the examiner. It is desirable to display more detailed information about the lesion as an auxiliary display in addition to the display of the mark notifying the lesion.

[0007] An object of the present disclosure is to support an examiner in probe operation and image observation. That is, an object of the present disclosure is to display marks together with examination support information separate from the marks. Alternatively, an object of the present disclosure is to provide the examiner with auxiliary information for evaluating a lesion. [Means for solving the problem]

[0008] The ultrasound diagnostic device according to the present disclosure is characterized in including: a detection unit that executes a detection process to detect a lesion in an ultrasound image; a mark generation unit that generates a mark indicating the lesion based on output information of the detection unit; a reference information generation unit that generates, based on the output information, reference information representing at least one of a spatial lesion probability distribution and a temporal change in lesion probability as examination support information separate from the mark; and a display that displays the ultrasound image, the mark, and the reference information in real time.

[0009] The ultrasound image processing method according to the present disclosure is characterized in that it includes the steps of: executing a detection process to detect a lesion in an ultrasound image; generating a mark indicating the lesion based on output information representing a result of the detection process; generating reference information representing at least one of a spatial lesion probability distribution and a temporal change in lesion probability as examination support information separate from the mark based on the output information; and displaying the ultrasound image, the mark, and the reference information in real time.

[0010] The program according to the present disclosure is a program executed on an information processing device, and is characterized in that it includes a function of executing a detection process to detect a lesion in an ultrasound image, a function of generating a mark indicating the lesion based on output information representing a result of executing the detection process, a function of generating reference information representing at least one of a spatial lesion probability distribution and a temporal change in lesion probability as examination support information separate from the mark based on the output information, and a function of displaying the ultrasound image, the mark, and the reference information in real time. Effect of the Invention

[0011] According to the present disclosure, it is possible to support an examiner in probe operation and image observation. That is, according to the present disclosure, it is possible to display marks together with examination support information separate from the marks. Alternatively, according to the present disclosure, it is possible to provide the examiner with auxiliary information for evaluating a lesion. [Brief description of the drawings]

[0012] [Figure 1] 1 is a block diagram showing an ultrasound diagnostic apparatus according to an embodiment. [Diagram 2] FIG. 13 is a diagram showing an example of a display image. [Diagram 3] FIG. 2 is a conceptual diagram illustrating a detection process and some processes associated therewith. [Figure 4] FIG. 11 is a diagram illustrating a first example of a map generation method. [Diagram 5] FIG. 11 is a diagram illustrating a second example of a map generation method. [Figure 6] FIG. 13 is a diagram for explaining color conversion. [Figure 7] FIG. 13 is a diagram showing a modified example of a graph. [Figure 8] FIG. 13 is a diagram showing another example of a display image. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] Hereinafter, an embodiment will be described with reference to the drawings.

[0014] (1) Overview of the embodiment An ultrasound diagnostic apparatus according to an embodiment has a detection unit, a mark generation unit, a reference information generation unit, and a display. The detection unit executes a detection process to detect a lesion in an ultrasound image. The mark generation unit generates a mark notifying a lesion based on output information from the detection unit. The reference information generation unit generates reference information representing at least one of a spatial lesion probability distribution and a temporal change in lesion probability as examination support information separate from the mark based on the output information. The display displays the ultrasound image, the mark, and the reference information in real time.

[0015] According to the above configuration, in addition to the mark, the reference information is displayed as examination assistance information. In the embodiment, the reference information is displayed regardless of whether the mark is displayed or not. Through observation of the reference image, the examiner can obtain more detailed information about the current examination target (current ultrasound diagnosis target). This can assist in the examination of the lesion or verify the validity of the mark display. Furthermore, it becomes possible to evaluate the contents of the ultrasound image even in a situation where the mark is not displayed.

[0016] In an embodiment, the ultrasound image is a moving image showing the current examination object. A mark is displayed when a lesion satisfying a certain condition is detected. The reference information is a reference image showing the reaction of the detector to the current examination object, which is a continuously displayed moving image. In this way, the ultrasound image, the mark and the reference information are displayed in real time during the ultrasound examination.

[0017] When a large amount of information or a variety of information is displayed on an ultrasound image, it becomes difficult to observe the ultrasound image. On the other hand, in order to comprehensively or multifacetedly evaluate the current examination object, it is necessary to provide more detailed information other than the marks. Therefore, in the embodiment, the marks are displayed on the ultrasound image, and the reference information is displayed near the ultrasound image.

[0018] In an embodiment, the reference information generating unit includes a map generating unit that generates a two-dimensional map representing a spatial probability distribution of a lesion as the reference information. The lesion can be evaluated in detail through observation of the two-dimensional map. In addition, it is also possible to evaluate a site that is not the target of notification.

[0019] In an embodiment, when the output information includes information that satisfies a mark display condition and information that does not satisfy the mark display condition, the mark generation unit generates a mark based on the information that satisfies the mark display condition. The map generation unit generates a two-dimensional map based on the information that satisfies the mark display condition and the information that does not satisfy the mark display condition. Typically, marks are displayed intermittently. Meanwhile, two-dimensional maps are displayed continuously.

[0020] In an embodiment, the reference information generating unit includes a graph generating unit configured to generate a graph representing a change in lesion probability over time as the reference information. By referring to the graph, the detection frequency of a lesion can be recognized, or a lesion can be easily searched for. A plurality of graphs corresponding to a plurality of objects or a plurality of classes may be displayed.

[0021] In an embodiment, the graph generator identifies a representative value of the lesion probability in each frame, and generates a graph based on the identified representative values ​​from the multiple frames. For example, the representative value for each frame is a maximum value. The frames are received frames or displayed frames.

[0022] In an embodiment, the mark generation unit generates a mark when the output information satisfies a mark display condition. The mark display condition includes a condition that requires that the lesion probability identified by the output information exceeds a threshold. The graph includes a display element indicating the threshold. The graph generation unit changes the display element according to a change in the threshold. With this configuration, the threshold can be set while referring to multiple representative values.

[0023] In an embodiment, the reference information generator includes a map generator and a graph generator. The map generator generates a two-dimensional map representing a spatial distribution of lesion probability. The graph generator generates a graph representing a change in lesion probability over time. The reference information includes the two-dimensional map and the graph. This configuration can comprehensively or multifacetedly support the examiner's probe operation and image observation.

[0024] An ultrasound image processing method according to an embodiment includes a detection step, a mark generation step, a reference information generation step, and a display step. In the detection step, a detection process is executed to detect a lesion in an ultrasound image. In the mark generation step, a mark notifying the lesion is generated based on output information representing a result of the execution of the detection process. In the reference information generation step, reference information representing at least one of a spatial lesion probability distribution and a temporal change in lesion probability is generated as examination support information separate from the mark based on the output information. In the display step, the ultrasound image, the mark, and the reference information are displayed in real time.

[0025] The above-mentioned ultrasound image processing method is realized as a software function or a hardware function. A program for executing the above-mentioned ultrasound image processing method may be installed in an information processing device via a network or a portable storage medium. The concept of the information processing device includes an ultrasound diagnostic device, a medical system, a computer, and the like. The information processing device includes a non-transitory storage medium.

[0026] (2) Details of the embodiment An ultrasound diagnostic device according to an embodiment is shown in Fig. 1. The ultrasound diagnostic device is a medical device for performing ultrasound examinations on a subject (living body) in a medical institution or the like. As described in detail below, the ultrasound diagnostic device according to the embodiment has a CADe function, that is, a function of automatically detecting a lesion included in a tomographic image and notifying the lesion by a mark.

[0027] In Fig. 1, a probe 10 is a device that transmits ultrasonic waves into a living body and receives reflected waves from the living body. More specifically, the probe 10 is composed of a probe head, a cable, and a connector. The probe head is held by an examiner. The probe head is the main part of the probe 10, and hereinafter the probe head will be simply referred to as the probe 10.

[0028] A transducer element array consisting of a plurality of transducer elements is provided within the probe 10. An ultrasonic beam 11 is formed by the transducer element array, and the ultrasonic beam 11 is electronically scanned. Known electronic scanning methods include an electronic linear scanning method and an electronic sector scanning method. A scan plane 12 is formed by electronically scanning the ultrasonic beam 11. The scan plane 12 is repeatedly formed by repeatedly electronically scanning the ultrasonic beam 11. A two-dimensional transducer element array may be provided within the probe 10, and volume data may be acquired from within a living body.

[0029] During transmission, the transceiver 13 supplies multiple transmission signals in parallel to the transducer array. This forms a transmission beam. During reception, the transceiver 13 processes multiple reception signals output in parallel from the transducer array. This processing includes A / D conversion, phasing addition (delay addition), etc.

[0030] As a result of processing multiple received signals, receive beam data is generated. Multiple receive beam data aligned in the electronic scanning direction constitute receive frame data. Multiple receive frame data aligned on the time axis constitute a receive frame data sequence. Each receive beam data is composed of multiple echo data aligned in the depth direction.

[0031] The image forming unit 14 is a module that generates a display frame data string based on the received frame data string. The image forming unit 14 has a coordinate conversion function, a pixel interpolation function, a frame rate adjustment function, and the like. Specifically, the image forming unit 14 is configured by a digital scan converter (DSC). Each display frame data corresponds to a tomographic image as a still image, and the display frame data string corresponds to a tomographic image as a moving image. An ultrasound image other than a tomographic image may be formed. In the embodiment, the display frame data string is sent to the display processing unit 18 and also to the lesion detection unit 22.

[0032] The received frame data sequence may be temporarily stored in memory 16. Similarly, the displayed frame data sequence may be temporarily stored in memory 20. The memories 16 and 20 each function as a cine memory having a ring buffer structure. In a freeze state (a state in which transmission and reception are stopped) after real-time operation, the frame data sequences stored in the memories 16 and 20 are read out.

[0033] The lesion detection unit 22 is configured with a machine learning object detector. In detail, the lesion detection unit 22 includes an object detection network (object detection model) such as CNN, and the object detection network has a plurality of layer parts arranged in series. The plurality of layer parts execute a plurality of detection processes in stages. Each layer part includes one or more convolution layers, and also includes one or more pooling layers as necessary. A series of detection processes is configured by the plurality of detection processes. Note that R-CNN (Regional CNN), SSD (Single Shot MultiBox Detector), M2Det, YOLO, etc. are known as object detection networks.

[0034] When one tomographic image is input to the lesion detection unit 22, output information reflecting the feature amount of the tomographic image is generated. The output information represents the detection result of the lesion. The output information is sent to the post-processing unit 24 and the map generation unit 30.

[0035] In the post-processing unit 24, maximum value determination processing and threshold processing are executed for each display frame. The output information is information representing the spatial distribution of lesion probability. That is, the output information includes multiple lesion probabilities (lesion scores) arranged two-dimensionally. In maximum value determination processing, the maximum value among the multiple lesion probabilities is determined. In threshold processing, the determined maximum value is compared with a threshold. If the maximum value exceeds the threshold, it is determined whether to display a mark indicating lesion detection. If the maximum value is below the threshold, no mark is displayed.

[0036] In the post-processing unit 24, an exclusion process may be applied. For example, in the exclusion process, a lesion area related to a false detection may be rejected. The maximum value determined in the maximum value determination process is sent to the graph generating unit 28. The maximum value after the threshold process is sent to the mark generating unit 26.

[0037] When the mark display condition is met (i.e., when the maximum value exceeds the threshold value), the mark generation unit 26 generates a rectangular frame surrounding the lesion as a mark based on the cell information corresponding to the maximum value. The mark is a graphic element. An image representing the mark is sent to the display processing unit 18. The mark may be generated continuously, and the mark may be actually drawn when it is determined that the lesion should be displayed. In any case, when the mark display condition is met, the mark appears on the screen.

[0038] The map generating unit 30 generates a two-dimensional map (two-dimensional color map) showing a spatial probability distribution of a lesion for each display frame based on the output information. A two-dimensional map as a moving image is composed of a plurality of two-dimensional maps corresponding to a plurality of display frames arranged in chronological order. Information showing the two-dimensional map as a moving image is sent to the display processing unit 18. The two-dimensional map is a moving image corresponding to a tomographic image as a moving image, and is a first reference image that supports the ultrasound examination.

[0039] The graph generating unit 28 generates a graph based on the multiple maximum values ​​arranged in chronological order. The graph is a moving image showing the temporal change in the probability of a lesion, and is a second reference image that supports the ultrasound examination. The multiple maximum values ​​are multiple representative values ​​representing the multiple two-dimensional maps. Multiple representative values ​​other than the multiple maximum values ​​may also be used. For example, multiple average values ​​may be used. Data showing the generated graph is sent to the display processing unit 18.

[0040] The graph generated by the graph generating unit 28 includes a line indicating the threshold value in addition to a plurality of bars indicating a plurality of maximum values. Information indicating the currently set threshold value is sent to the post-processing unit 24 and the graph generating unit 28 from a control unit 34 described later.

[0041] The display processing unit 18 has an image synthesis function, a color calculation function, and the like. A display image to be displayed on the screen of the display 32 is generated by the display processing unit 18. The display image includes a tomographic image as an ultrasonic image, a mark superimposed on the tomographic image, a two-dimensional map as a first reference image, and a graph as a second reference image. The tomographic image, the mark, the two-dimensional map, and the graph each correspond to a moving image that displays information on the current inspection target in real time. In the frozen state, the tomographic image, the mark, the two-dimensional map, and the graph may be displayed as a moving image or a still image.

[0042] The control unit 34 is configured with a processor (e.g., a CPU) that executes a program. The control unit 34 controls the operation of each component shown in Fig. 1. The image forming unit 14, the display processing unit 18, the lesion detection unit 22, the post-processing unit 24, the mark generating unit 26, the graph generating unit 28, and the map generating unit 30 may each be configured with a processor, or may be realized as a function performed by, for example, a CPU.

[0043] The operation panel 36 is an input device having a plurality of switches, a plurality of knobs, a keyboard, a trackball, etc. The above threshold value is set using the operation panel 36. The display 32 is configured by, for example, a liquid crystal display or an organic EL display device.

[0044] An example of a display image is shown in Fig. 2. The display image 40 is displayed on the screen of a display device, and includes a tomographic image 42, a two-dimensional map 44, a graph 46, and the like.

[0045] The tomographic image 42 is a black and white image, and a mark 48 is displayed as a color figure superimposed on the tomographic image 42. In the illustrated example, the horizontal axis of the tomographic image 42 corresponds to the electronic scanning direction, and the vertical axis corresponds to the depth direction. A lesion (cross-sectional image of a tumor mass) 50 is currently included in the tomographic image 42, and the mark 48 surrounds the lesion 50. Any figure may be used as the mark 48, but the shape and size of the mark 48 are determined so as not to interfere with observation of the lesion. The content of the tomographic image 42 changes significantly depending on changes in the position and attitude of the probe. A two-dimensional blood flow image may be synthesized and displayed on the tomographic image 42.

[0046] The two-dimensional map 44 shows a probability distribution of a lesion in a two-dimensional space. The two-dimensional map 44 has a coordinate system corresponding to the coordinate system of the tomographic image 42. In the two-dimensional map 44, the horizontal axis corresponds to the electronic scanning direction, and the vertical axis corresponds to the depth direction. The two-dimensional map 44 shows the same scanning plane as the tomographic image 42. The size of the two-dimensional map 44 is smaller than the size of the tomographic image 42, and the two-dimensional map 44 corresponds to a reduced image of the tomographic image 42. In the two-dimensional map 44, a change in the probability of a lesion is expressed by a change in hue. Reference numeral 51 indicates a hue-changing portion corresponding to a lesion 50 in the tomographic image 42. The two-dimensional map 44 is a moving image, and its contents change dynamically with changes in the position and posture of the probe.

[0047] In addition, a depth scale 56 is provided along the vertical axis in the tomographic image 42. Correspondingly, a depth scale 58 is also provided along the vertical axis in the two-dimensional map 44.

[0048] In the illustrated example, the two-dimensional map 44 is displayed on the right side of the tomographic image 42, but it may be displayed on the left side, upper side, or lower side of the tomographic image 42. In consideration of the convenience of simultaneous observation of the tomographic image 42 and the two-dimensional map 44, it is desirable to display the two-dimensional map near the tomographic image 42 without overlapping it on the same screen.

[0049] A color bar 52 is displayed near the two-dimensional map 44. The color bar 52 is a color sample that represents a change in hue corresponding to a change in the lesion probability. In the illustrated example, the color bar 52 is displayed so that its longitudinal direction is parallel to the horizontal direction. The left end of the color bar 52 corresponds to a lesion probability of 0.0, and its right end corresponds to a lesion probability of 1.0. A line 55 indicating the currently set threshold value is displayed on the color bar 52. The color bar 52 may be displayed so that its longitudinal direction is parallel to the vertical direction. When the threshold value is changed, the line 55 moves horizontally accordingly.

[0050] Graph 46 shows the change in lesion probability over time. In graph 46, the horizontal axis corresponds to the time axis, and the vertical axis shows the lesion probability. Graph 46 is composed of bars 60 that are specified for each display frame, and the height of the bars 60 shows the lesion probability. Specifically, the height of each bar 60 shows the maximum value of the lesion probability in the display frame. Bars may be displayed at regular time intervals, or the maximum value smoothed in the time axis direction may be represented by a curve. Indicator 62 shows the current time. The display position of indicator 62 is fixed, but the display position may be scanned horizontally.

[0051] In addition to the multiple bars 60, the graph 46 includes a threshold line 64. The threshold line 64 indicates the threshold value set at each time. An indicator 66 indicates the current threshold value. When the threshold value is changed using the operation panel, the indicator 66 moves up and down in conjunction with the change. It is also possible to operate the indicator 66 itself, and by moving it up and down, the threshold value can be changed.

[0052] In the illustrated display example, a body mark 68 is displayed below the two-dimensional map 44, and a probe mark 70 is also displayed. In the embodiment, the diagnostic target tissue is a breast, and the body mark 68 is a schematic representation of the breast.

[0053] By displaying the two-dimensional map 44 and the graph 46 in real time together with the tomographic image on which the mark 48 is overlaid, the examiner can comprehensively or multifacetedly evaluate the lesion. For example, by observing the two-dimensional map 44, it is possible to recognize a detailed probability distribution of the lesion notified by the mark 48, or to recognize the presence of other lesions that do not satisfy the mark display conditions. In addition, by observing the graph 46, it is possible to recognize the frequency of lesion detection, which can be used for probe operation. It is also easy to recognize the appearance of a lesion that is not the subject of notification.

[0054] 3 shows a schematic diagram of a lesion detection process and several related processes. In the lesion detection unit 22, a detection process consisting of a plurality of detection steps is executed. The plurality of detection steps are executed in a plurality of layer parts arranged in chronological order in FIG. 3. The plurality of layer parts correspond to a trained network.

[0055] Specifically, a plurality of layer sections including layer sections L1 to L5 are shown in Fig. 3. Each layer section includes one or more convolution layers, and may also include one or more pooling layers as necessary.

[0056] The layer section L1 is an input layer section, to which a tomographic image 76 is input as an input image. In the illustrated configuration example, the multiple layer sections include M output layer sections. M output maps 78-1 to 78-M are output from the M output layer sections. M is an integer equal to or greater than 1, and may be 2 or 3, for example. The M output maps 78-1 to 78-M constitute the output information 77. The M output maps 78-1 to 78-M exhibit different responses depending on the lesion size. The output map 78-1 responds strongly to a lesion having a large size, and therefore is easy to represent the detection result of a lesion having a large size. The output map 78-M responds strongly to a lesion having a small size, and therefore is easy to represent the detection result of a lesion having a small size. Each output map 78-1 to 78-M is constituted by a plurality of cells 79-1 to 79-M arranged two-dimensionally. Each of the cells 79-1 to 79-M has cell information associated with specific coordinates.

[0057] An example of cell information is shown in the upper right of Fig. 3. The illustrated cell information 80 includes, for example, an X coordinate 82 and a Y coordinate 84 that specify the center coordinates of the detected lesion, information 86 indicating the width W of the detected lesion, information 88 indicating the height (vertical width) H of the detected lesion, an object score (OS) 90 indicating the probability that the lesion belongs to a particular class, etc. For each cell, n pieces of cell information corresponding to n classes may be generated.

[0058] The post-processing unit 24 includes a maximum value determination unit 72 and a threshold processing unit 74. In the maximum value determination unit 72, the maximum object score is identified as the maximum value from among the output information 77 (i.e., the multiple output maps 78-1 to 78-M) for each display frame. In the threshold processing unit 74, the identified maximum value is compared with a threshold value. If the maximum value exceeds the threshold value, the mark generating unit 26 generates a mark. If the maximum value does not reach the threshold value, no mark is generated. Note that instead of determining the maximum value, a process of determining the top K object scores may be executed. In that case, up to K marks are simultaneously generated based on the top K object scores. K is an integer equal to or greater than 2. However, the maximum value is referred to when generating a graph.

[0059] The graph generating unit 28 generates a graph by expressing the maximum value for each display frame with a bar, and repeating this process. Instead of a graph showing multiple maximum values ​​in chronological order, a graph showing multiple average values ​​in chronological order may be generated. A moving average process or the like may be applied to the graph showing the maximum values. The generated graph may also include a bar showing a maximum value that does not satisfy the mark display condition.

[0060] In the example shown in FIG. 3, output information 77 (i.e., a plurality of output maps 78-1 to 78-M) is sent directly to map generating unit 30. In an embodiment, map generating unit 30 generates a two-dimensional map based on output information 77. Specifically, the object score, i.e., the lesion probability, extracted from each cell is converted into a color. The converted color is mapped into a two-dimensional space. The plurality of output maps 78-1 to 78-M have different sizes, but share a common coordinate system. Each color is mapped to the common coordinate system. A portion of output information 77 may be the mapping target.

[0061] When the output information 77 includes information that satisfies the mark display conditions and information that does not satisfy the mark display conditions, the mark generation unit 26 generates a mark based only on the information that satisfies the mark display conditions (maximum value condition and threshold value condition). On the other hand, the map generation unit 30 generates a two-dimensional map based on both the information that satisfies the mark display conditions and the information that does not satisfy the mark display conditions. The graph generation unit 28 generates a graph based only on the information that satisfies the maximum value condition.

[0062] When generating the multiple output maps 78-1 to 78-M, feature maps 92 and 94 extracted midway through the detection process may be used.

[0063] A first example of a generated two-dimensional map is shown in Fig. 4. In the illustrated example, lesions are detected at two coordinates 102 and 104 in intermediate information 100. In reality, lesions are detected at, for example, tens or hundreds of coordinates within one display frame.

[0064] The lesion probability at coordinate 102 is (0.8) and the lesion probability at coordinate 104 is (0.4). The size of the lesion detected at coordinate 102 is shown at 102A and the size of the lesion detected at coordinate 104 is shown at 104A.

[0065] In the first generation example, the lesion probability is converted into a color code. For example, the lesion probability (0.8) at the coordinate 102 is converted into a color code (204), and the lesion probability (0.4) at the coordinate 104 is converted into a color code (120). Reference numeral 106 denotes information after conversion. A two-dimensional map 108 is generated by mapping colors (R value, G value, B value) corresponding to each color code into a two-dimensional space. In the process of generating the two-dimensional map 108, a reduction process and an enlargement process may be performed. An interpolation process or a smoothing process may be applied after the color mapping.

[0066] 5 shows a second example of generating a two-dimensional map. The intermediate information 100 already described is processed as follows. A local region 102B having a certain extent is defined based on a size 102A specified by coordinates 102, and similarly, a local region 104B having a certain extent is defined based on a size 104A specified by coordinates 104. The local regions 102B and 104B can be generated, for example, by reducing the sizes 102A and 104A.

[0067] The lesion probability (0.8) at coordinate 102 is converted into a color code (204), and that color code is assigned to the entire local region 102B. Similarly, the lesion probability (0.4) at coordinate 104 is converted into a color code (120), and that color code is assigned to the entire local region 104B. Based on the information 110 thus configured, a two-dimensional map 112 is generated by mapping the colors (R value, G value, B value) corresponding to each color code into a two-dimensional space.

[0068] In the second generation example, the sizes 102A and 104A may be used as they are as the local regions 102B and 104B.

[0069] FIG. 6 shows an example of color conversion. Reference numeral 114 denotes a color conversion function set. Specifically, the color conversion function set 114 is composed of a B conversion function 118B, a G conversion function 118G, and an R conversion function 118R. The horizontal axis indicates a color index, which corresponds to the lesion score. The vertical axis indicates relative luminance. FIG. 6 shows a color bar 116 expressing the color defined by each color index. The color conversion function set 114 shown in FIG. 6 is merely an example, and various color conversion function sets, i.e., various color bars, can be adopted. It is also possible to express the lesion score in grayscale.

[0070] FIG. 7 shows a modified example of the graph. The graph 120 is composed of multiple bars indicating the probability of a lesion. An index 122 indicates the current time. A step occurs in the threshold line 126. The step indicates a change in the threshold. A portion 126B older than the step indicates a relatively high threshold, and a portion 126A newer than the step indicates a relatively low threshold. The current threshold can be changed by moving the index 124 up or down. The change in threshold changes the shape of the threshold line 126.

[0071] For example, a tomographic image, a two-dimensional map, and a graph may be generated and displayed based on a frame data sequence stored in the cine memory. In this case, these images may be displayed not as moving images, but as still images associated with a specific time point in the past. In this case, the specific time point in the past may be specified by element 128. Element 128 is a mark that moves along the horizontal axis of graph 120. For example, a specific time point in the past at which a lesion was detected may be specified by element 128.

[0072] Another example of a display image is shown in Fig. 8. Display image 40A includes a tomographic image 42, a two-dimensional map 44A, graphs 46A and 46B, etc. Tomographic image 42 includes a lesion 50, and a mark 48 informing the same is superimposed and displayed. In this other example, two types of lesions (e.g., a mass and a non-mass) are detected, and a lesion probability, i.e., a lesion score, is calculated for each lesion.

[0073] In the two-dimensional map 44A, for example, the first lesion probability is represented by the brightness of a bluish hue, and the second lesion probability is represented by the brightness of a reddish hue. The relationship between the first lesion probability and the brightness of the bluish hue is represented by a first color bar 52A, and the relationship between the second lesion probability and the brightness of the reddish hue is represented by a second color bar 52B. Lines 55A and 55B indicating thresholds are displayed on the respective color bars 52A and 52B.

[0074] A first two-dimensional map showing the probability distribution of the first lesion and a second two-dimensional map showing the probability distribution of the second lesion may be generated and displayed separately. On the tomographic image 42, the first lesion and the second lesion are indistinguishably identified by the same mark 48, but a different marker may be displayed for each type of lesion.

[0075] The first graph 46A shows the change over time of the first lesion probability (maximum value). The second graph 46B shows the change over time of the second lesion probability (maximum value). The first graph 46A includes a line 64A as a display element indicating the threshold value, and the second graph 46B includes a line 64B as a display element indicating the threshold value.

[0076] According to the above embodiment, in addition to marks, a two-dimensional map and a graph are displayed as examination support information. The two-dimensional map and the graph are displayed regardless of whether or not marks are displayed. Thus, the examiner can obtain more detailed information about the current examination target by observing the displayed information. This can assist in the detailed examination of the lesion notified through the mark display, or can verify the validity of the mark display. Also, it becomes possible to evaluate the current examination target in a situation where marks are not displayed. [Explanation of symbols]

[0077] 10 probe, 14 image formation unit, 22 lesion detection unit, 24 post-processing unit, 26 mark generation unit, 28 graph generation unit, 30 map generation unit, 42 tomographic image, 44 two-dimensional map, 46 graph, 48 mark, 52 color bar.

Claims

1. A detection unit that executes a detection process to detect a lesion in an ultrasound image; a mark generating unit that generates a mark notifying the affected area based on output information from the detection unit; a map generating unit that generates, based on the output information, a two-dimensional map that represents a spatial probability distribution of a lesion on a scanning plane represented by the ultrasound image, as examination support information separate from the marks; a display for displaying a display image including the ultrasound image, the mark, and the two-dimensional map; Including, the ultrasound image, the mark, and the two-dimensional map are each a moving image displayed in real time; When the output information satisfies a mark display condition, the mark is displayed on the ultrasound image; The two-dimensional map is displayed at a position not overlapping with the ultrasound image, regardless of whether the output information satisfies the mark display condition. the two-dimensional map is a reduced image having a size smaller than the size of the ultrasound image; 1. An ultrasonic diagnostic apparatus comprising:

2. 2. The ultrasonic diagnostic apparatus according to claim 1, the two-dimensional map is a color map that represents a lesion probability in color; A color bar is displayed adjacent to the two-dimensional map, the color bar representing a change in color corresponding to a change in the probability of a lesion; the color bar includes a line indicating a mark display determination threshold value used in threshold processing of the output information; 1. An ultrasonic diagnostic apparatus comprising:

3. 3. The ultrasonic diagnostic apparatus according to claim 2, The line moves on the color bar in response to the change in the threshold value.

1. An ultrasonic diagnostic apparatus comprising:

4. 2. The ultrasonic diagnostic apparatus according to claim 1, When the output information includes information that satisfies a mark display condition and information that does not satisfy the mark display condition, the mark generation unit generates the mark based on the information that satisfies the mark display condition, and the map generation unit generates the two-dimensional map based on the information that satisfies the mark display condition and the information that does not satisfy the mark display condition.

1. An ultrasonic diagnostic apparatus comprising:

5. A detection unit that performs a detection process to detect a lesion in an ultrasound image; a mark generating unit that generates a mark notifying the affected area based on output information from the detection unit; a reference information generating unit that generates reference information representing a change in probability of a lesion over time as examination support information separate from the mark based on the output information; a display that displays the ultrasound image, the mark, and the reference information in real time; Including, the reference information generating unit includes a graph generating unit configured to generate, as the reference information, a graph representing the change in the lesion probability over time; 1. An ultrasonic diagnostic apparatus comprising:

6. 6. The ultrasonic diagnostic apparatus according to claim 5, The graph generation unit A representative value of the lesion probability in each frame is determined; generating the graph based on a plurality of representative values ​​identified from a plurality of frames; 1. An ultrasonic diagnostic apparatus comprising:

7. 7. The ultrasonic diagnostic apparatus according to claim 6, the representative value for each frame is a maximum value; 1. An ultrasonic diagnostic apparatus comprising:

8. 6. The ultrasonic diagnostic apparatus according to claim 5, the mark generating unit generates the mark when the output information satisfies a mark display condition; the mark display condition includes a condition that requires that the lesion probability specified by the output information exceeds a threshold; the graph includes a display element indicating the threshold value; 1. An ultrasonic diagnostic apparatus comprising:

9. 9. The ultrasonic diagnostic apparatus according to claim 8, The graph generating unit changes the display element in accordance with a change in the threshold value.

1. An ultrasonic diagnostic apparatus comprising:

10. A detection unit that executes a detection process to detect a lesion in an ultrasound image; a mark generating unit that generates a mark notifying the affected area based on output information from the detection unit; a reference information generating unit that generates reference information representing a spatial lesion probability distribution and a temporal change in lesion probability as examination support information other than the marks based on the output information; a display that displays the ultrasound image, the mark, and the reference information in real time; Including, The reference information generating unit a map generator for generating a two-dimensional map representing the spatial lesion probability distribution; a graph generator for generating a graph representing the change in lesion probability over time; Including, the reference information includes the two-dimensional map and the graph; 1. An ultrasonic diagnostic apparatus comprising:

11. performing a detection process for detecting a lesion in an ultrasound image; generating a mark indicating the lesion based on output information representing a result of the detection process; generating a graph representing a change in lesion probability over time as examination support information separate from the marks based on the output information; displaying the ultrasound image, the marks and the graph in real time; 13. An ultrasound image processing method comprising:

12. A program executed in an information processing device, A function of executing a detection process for detecting a lesion in an ultrasound image; a function of generating a mark for notifying the affected area based on output information representing a result of the detection process; a function of generating a graph showing a change in probability of a lesion over time as examination support information separate from the marks based on the output information; displaying the ultrasound image, the mark and the graph in real time; A program comprising:

Images