Ultrasonic diagnostic apparatus and ultrasonic image processing method

Abstract

An ultrasonic diagnostic apparatus and an ultrasonic image processing method are provided. A marker (48) that specifies a lesion portion (50) is displayed on a tomographic image (42). A two-dimensional graph (44) and a graph (46) are displayed regardless of display of the marker (48). The two-dimensional graph (44) represents a spatial distribution of a lesion probability. The graph (46) represents a temporal change in the lesion probability.

Description

Technical Field

[0001] This disclosure relates to ultrasound diagnostic apparatus and ultrasound image processing methods, and particularly to the generation and provision of information to assist ultrasound examinations. Background Technology

[0002] In ultrasound examinations using ultrasound diagnostic equipment, the examiner observes the ultrasound image while simultaneously operating the probe (ultrasound probe). If a lesion (more accurately, a potential lesion) is detected in the ultrasound image, the lesion is examined in detail.

[0003] Among ultrasound diagnostic devices, there are those capable of automatically detecting lesions appearing in ultrasound images and notifying the detected lesions. This function is called CADe (Computer Aided Detection). CADe is a technology belonging to or related to CAD (Computer Aided Detection and Diagnosis). When CADe is executed, the probability of a lesion is generally calculated (lesion score, lesion similarity, etc.). If the lesion probability exceeds a threshold, a graphic (marker) surrounding the lesion is displayed. If the lesion probability is less than the threshold, no mark is displayed. By using a threshold to control the presence or absence of marks, the situation of frequently displaying marks on the ultrasound image, or the situation of displaying a large number of marks on the ultrasound image, is avoided.

[0004] Document 1 (JP Patent Application Publication No. 2020-28680) discloses an ultrasound diagnostic device with object detection function. However, Document 1 does not describe a technique for displaying the spatial distribution of lesion probability and the temporal change of lesion probability as dynamic images.

[0005] When a marker indicating a lesion is displayed on an ultrasound image, the examiner can identify the detection of a lesion, but cannot discern the spatial distribution or temporal variation of the lesion probability from that marker. If the lesion probability does not exceed a threshold, the marker is not displayed; in fact, even if a lesion-like area is detected, no information about it can be provided to the examiner. It is desirable to display more detailed information about the lesion, separate from the marker indicating the lesion. Summary of the Invention

[0006] The purpose of this disclosure is to assist the examiner in probe operation and image observation. That is, the purpose of this disclosure is to display examination auxiliary information, different from that shown on the markings, together with the markings. Alternatively, the purpose of this disclosure is to provide the examiner with auxiliary information for evaluating lesions.

[0007] The ultrasound diagnostic apparatus disclosed herein is characterized by comprising: a detection unit that performs detection processing to detect lesions in an ultrasound image; a marker generation unit that generates a marker to notify the lesion based on output information from the detection unit; a reference information generation unit that generates reference information based on the output information, representing at least one of a spatial distribution of the probability of the lesion and a temporal change in the probability of the lesion, as examination auxiliary information different from the marker; and a display that displays the ultrasound image, the marker, and the reference information in real time.

[0008] The ultrasound image processing method disclosed herein is characterized by comprising the following steps: performing detection processing to detect lesions in an ultrasound image; generating a flag notifying the lesion based on output information characterizing the execution result of the detection processing; generating reference information characterizing at least one of the spatial distribution of the probability of the lesion and the temporal change of the probability of the lesion based on the output information, as examination auxiliary information different from the flag; and displaying the ultrasound image, the flag, and the reference information in real time. Attached Figure Description

[0009] Figure 1 This is a block diagram illustrating the ultrasonic diagnostic apparatus involved in the implementation.

[0010] Figure 2 This is a diagram representing an example of an image being displayed.

[0011] Figure 3 This is a conceptual diagram representing the detection process and several related processes.

[0012] Figure 4 This is the first example of a graph generation method.

[0013] Figure 5 This is the second example of a graph generation method.

[0014] Figure 6 It is a diagram used to illustrate color changes.

[0015] Figure 7 This is a diagram representing a variation of a chart.

[0016] Figure 8 This is a diagram representing other examples of displayed images. Detailed Implementation

[0017] The following description of the implementation method is based on the accompanying drawings.

[0018] (1) Overview of the implementation method

[0019] The ultrasound diagnostic apparatus according to the embodiment includes a detection unit, a marker generation unit, a reference information generation unit, and a display. The detection unit performs detection processing to detect lesions in an ultrasound image. The marker generation unit generates a marker to notify the lesion based on the output information of the detection unit. The reference information generation unit generates reference information, based on the output information, representing at least one of the spatial distribution of the probability of the lesion and the temporal change of the probability of the lesion, as examination auxiliary information different from the marker. The ultrasound image, the marker, and the reference information are displayed on the display in real time.

[0020] Based on the above structure, reference information is displayed in addition to markers as supplementary examination information. In this embodiment, the reference information is displayed regardless of whether markers are displayed. The examiner can obtain more detailed information about the current examination object (the current ultrasound diagnostic object) by observing the reference image. This can assist in the precise examination of lesions or verify the appropriateness of the marker display. Furthermore, the content of the ultrasound image can be evaluated even when markers are not displayed.

[0021] In this implementation, the ultrasound image is a dynamic image displaying the object being examined. A marker is displayed at the time point when a lesion meeting certain conditions is detected. The reference information is a continuously displayed dynamic image representing the detection unit's response to the object being examined. Thus, during ultrasound examination, the ultrasound image, markers, and reference information are displayed in real time.

[0022] Displaying a large amount of information or diverse information on an ultrasound image can hinder its observation. On the other hand, it is desirable to provide more detailed information beyond the markers, based on a comprehensive or multi-faceted evaluation of the object being examined. Therefore, in this embodiment, markers are displayed on the ultrasound image, and reference information is displayed nearby.

[0023] In this implementation, the reference information generation unit includes a graph generation unit that generates a two-dimensional graph representing the spatial distribution of the probability of lesions, which serves as reference information. Lesions can be evaluated in detail by observing the two-dimensional graph. Furthermore, areas not yet subject to notification can also be evaluated.

[0024] In this implementation, if the output information includes first information that satisfies the flag display conditions and second information that does not satisfy the flag display conditions, the flag generation unit generates a flag based on the first information, and the image generation unit generates a two-dimensional image based on the first and second information. Typically, the flag is displayed intermittently. On the other hand, the two-dimensional image is displayed continuously.

[0025] In this implementation, the reference information generation unit includes a chart generation unit that generates charts representing the time-varying probabilities of lesions as reference information. These reference charts allow for the identification of lesion detection frequencies and facilitate the search for lesions. Multiple charts corresponding to multiple objects or multiple classes can also be displayed.

[0026] In this implementation, the chart generation unit determines a representative value for the probability of lesions within each frame, and generates a chart based on multiple representative values ​​determined from multiple frames. For example, the representative value for each frame is the maximum value. A frame is either a received frame or a displayed frame.

[0027] In this implementation, the flag generation unit generates a flag when the output information meets the flag display conditions. The flag display conditions include a requirement that the probability of a lesion determined by the output information exceeds a threshold. The graph includes display elements representing the threshold. The graph generation unit changes the display elements as the threshold changes. With this structure, the threshold can be set while referencing multiple representative values.

[0028] In this implementation, the reference information generation unit includes a graph generation unit and a chart generation unit. The graph generation unit generates a two-dimensional graph representing the spatial distribution of the probability of lesions. The chart generation unit generates a chart representing the temporal change of the probability of lesions. The reference information includes both two-dimensional graphs and charts. Based on this structure, it can comprehensively or in multiple ways assist the examiner in probe operation and image observation.

[0029] The ultrasound image processing method according to the embodiment includes a detection step, a marker generation step, a reference information generation step, and a display step. In the detection step, detection processing is performed to detect lesions in the ultrasound image. In the marker generation step, a marker notifying the lesion is generated based on output information characterizing the execution result of the detection processing. In the reference information generation step, reference information characterizing at least one of the spatial distribution of the lesion probability and the temporal change of the lesion probability is generated based on the output information, serving as examination auxiliary information different from the marker. In the display step, the ultrasound image, the marker, and the reference information are displayed in real time.

[0030] The aforementioned ultrasound image processing method is implemented as a software function or a hardware function. The program executing the aforementioned ultrasound image processing method can be installed on an information processing device via a network or a removable storage medium. The concept of an information processing device includes ultrasound diagnostic devices, medical systems, computers, etc. The information processing device possesses a non-transitory storage medium.

[0031] (2) Details of the implementation method

[0032] exist Figure 1The image shows an ultrasound diagnostic apparatus according to an embodiment. An ultrasound diagnostic apparatus is a medical device used to perform ultrasound examinations on a subject (biological body) in a medical institution or similar setting. As detailed below, the ultrasound diagnostic apparatus according to the embodiment has a CADe function, that is, it has the function of automatically detecting lesions contained in a tomographic image and notifying the lesions by displaying a marker.

[0033] exist Figure 1 In this design, probe 10 is a device that transmits ultrasound waves into a living organism and receives reflected waves from within the organism. More specifically, probe 10 consists of a probe head, a cable, and a connector. The probe head is held by the examiner. The probe head is the main part of probe 10, and will be referred to as probe 10 below.

[0034] The probe 10 contains an array of vibrating elements, including multiple transducers. An ultrasonic beam 11 is formed by the array of transducers, and this ultrasonic beam 11 is electronically scanned. Known electronic scanning methods include linear electronic scanning and sector electronic scanning. A scanning surface 12 is formed by electronically scanning the ultrasonic beam 11. The scanning surface 12 is repeatedly formed by repeatedly electronically scanning the ultrasonic beam 11. Alternatively, a two-dimensional array of transducers can be provided within the probe 10 to obtain volumetric data from within a biological organism.

[0035] During transmission, the transceiver unit 13 supplies multiple transmission signals in parallel to the vibrating element array. This forms a transmission beam. During reception, the transceiver unit 13 processes the multiple received signals output in parallel from the vibrating element array. This processing includes A / D conversion, phase modulation addition (delayed addition), etc.

[0036] As a result of processing multiple received signals, received beam data is generated. Received frame data is composed of multiple received beam data arranged in the electronic scanning direction. Received frame data strings are composed of multiple received frame data arranged on the time axis. Each received beam data consists of multiple echo data arranged in the depth direction.

[0037] The image forming unit 14 is a module that generates a display frame data string based on the received frame data string. The image forming unit 14 has coordinate transformation, pixel interpolation, and frame rate adjustment functions. Specifically, the image forming unit 14 is composed of a digital scan converter (DSC). Each display frame data string corresponds to a tomographic image as a still image, and the display frame data string corresponds to a tomographic image as a moving image. Ultrasound images other than tomographic images can also be formed. In this embodiment, the display frame data string is sent to the display processing unit 18 and also to the lesion detection unit 22.

[0038] The received frame data string can be temporarily stored in memory 16. Similarly, the displayed frame data string can be temporarily stored in memory 20. Memory 16 and memory 20 function as image memories with a circular buffer structure. In the frozen state (transmission and reception stopped state) after real-time operation, the frame data strings stored in memory 16 and memory 20 are read out.

[0039] The lesion detection unit 22 is composed of a machine learning-based object detector. Specifically, the lesion detection unit 22 has an object detection network (object detection model) such as a CNN, which has multiple layers arranged in series. Multiple layers perform multiple detection steps in stages. Each layer contains one or more convolutional layers, and additionally, one or more pooling layers as needed. The multiple detection steps constitute a series of detection processes. Furthermore, as object detection networks, R-CNN (Regional CNN), SSD (Single Shot MultiBox Detector), M2Det, YOLO, etc., are known.

[0040] If a tomographic image is input to the lesion detection unit 22, output information reflecting the feature quantities of that tomographic image is generated. The output information characterizes the detection result of the lesion. The output information is sent to the post-processing unit 24 and the image generation unit 30. As shown by reference numeral 23, the received frame data string can be sent to the lesion detection unit 22.

[0041] In the post-processing unit 24, maximum value determination processing and threshold processing are performed for each display frame. The output information is information characterizing the spatial distribution of lesion probabilities. That is, the output information contains multiple lesion probabilities (lesion scores) arranged in a two-dimensional manner. In the maximum value determination processing, the maximum value among the multiple lesion probabilities is determined. In the threshold processing, the determined maximum value is compared with a threshold. If the maximum value exceeds the threshold, it is determined whether to display a flag indicating lesion detection. If the maximum value is lower than the threshold, the flag is not displayed.

[0042] In the post-processing unit 24, exclusion processing can also be used. For example, in exclusion processing, lesions involved in false detections can be discarded. The maximum value determined in the maximum value determination processing is sent to the chart generation unit 28. The maximum value after threshold processing is sent to the flag generation unit 26.

[0043] When the sign display conditions are met (i.e., when the maximum value exceeds a threshold), the sign generation unit 26 generates a rectangular frame surrounding the lesion as a sign based on the cell information corresponding to the maximum value. The sign is a graphic element. The image representing the sign is sent to the display processing unit 18. Sign generation can also be performed frequently so that the sign is actually drawn at the point in time when the display of the lesion is determined. In any case, the sign appears on the screen when the sign display conditions are met.

[0044] The image generation unit 30 generates a two-dimensional image (two-dimensional color image) representing the spatial distribution of the probability of lesions for each display frame based on the output information. The two-dimensional image, serving as a dynamic image, is constructed from multiple two-dimensional images corresponding to multiple display frames arranged in a time sequence. Information representing the two-dimensional image as a dynamic image is sent to the display processing unit 18. This two-dimensional image is a dynamic image corresponding to a tomographic image as a dynamic image, and serves as a first reference image for assisting ultrasound examination.

[0045] The chart generation unit 28 generates a chart based on multiple maximum values ​​arranged in time series. This chart is a dynamic image representing the time-varying probability of lesions and serves as a second reference image for auxiliary ultrasound examination. The multiple maximum values ​​are representative values ​​representing multiple two-dimensional graphs. Multiple representative values ​​other than the multiple maximum values ​​can also be used. For example, multiple average values ​​can also be used. The data representing the generated chart is sent to the display processing unit 18.

[0046] Furthermore, the chart generated by the chart generation unit 28 includes not only bars representing multiple maximum values, but also lines representing threshold values. The control unit 34 (described later) sends information representing the currently set threshold value to the post-processing unit 24 and the chart generation unit 28.

[0047] The display processing unit 18 has image synthesis and color calculation functions. It generates a display image to be displayed on the screen of the monitor 32. The display image includes a tomographic image (as an ultrasound image), markers superimposed on the tomographic image, a two-dimensional image serving as a first reference image, and a graph serving as a second reference image. The tomographic image, markers, two-dimensional image, and graph each correspond to a dynamic image displaying information related to the currently examined object in real time. Furthermore, in a frozen state, the tomographic image, markers, two-dimensional image, and graph can be displayed as either dynamic or static images.

[0048] The control unit 34 is composed of a processor (e.g., CPU) that executes programs. The system is controlled by the control unit 34. Figure 1The operation of each component shown. The image forming unit 14, display processing unit 18, lesion detection unit 22, post-processing unit 24, mark generation unit 26, chart generation unit 28 and chart generation unit 30 can each be composed of a processor, and they can be implemented, for example, as functions performed by a CPU.

[0049] The operation panel 36 is an input device with multiple switches, multiple knobs, a keyboard, a trackball, etc. The operation panel 36 is used to set the aforementioned thresholds. The display 32 is, for example, a liquid crystal display or an organic EL display device.

[0050] exist Figure 2 An example of an image display is shown. Image 40 is displayed on the screen of a monitor and includes tomographic images 42, two-dimensional images 44, charts 46, etc.

[0051] The tomographic image 42 is a black and white image, overlaid with markers 48 as color graphics. In the illustrated example, the horizontal axis of the tomographic image 42 corresponds to the direction of electronic scanning, and the vertical axis corresponds to the depth direction. Within the tomographic image 42, at the current time point, a lesion (specifically, a cross-sectional image of a tumor) 50 is contained, and the markers 48 surround this lesion 50. Any graphic can be used as the markers 48, but the shape and size of the markers 48 are determined so as not to obstruct the observation of the lesion. The content of the tomographic image 42 varies significantly depending on changes in the position and orientation of the probe. Two-dimensional blood flow images can also be synthesized and displayed on the tomographic image 42.

[0052] Two-dimensional figure 44 represents the probability distribution of lesions in two-dimensional space. Figure 44 has a coordinate system corresponding to that of the tomographic image 42. In Figure 44, the horizontal axis corresponds to the electronic scanning direction, and the vertical axis corresponds to the depth direction. Figure 44 represents the same scanning plane as shown in the tomographic image 42. The size of Figure 44 is smaller than that of the tomographic image 42; Figure 44 is equivalent to a scaled-down image of the tomographic image 42. In Figure 44, spatial variations in lesion probability are represented by changes in hue. Reference numeral 51 indicates the hue variation corresponding to the lesion 50 in the tomographic image 42. Figure 44 is a dynamic image; its content changes dynamically with changes in the probe's position and orientation.

[0053] Additionally, in the tomographic image 42, a depth scale 56 is set along the vertical axis. Correspondingly, in the two-dimensional image 44, a depth scale 58 is also set along the vertical axis.

[0054] In the illustrated example, the 2D plot 44 is displayed to the right of the tomographic image 42, but it can also be displayed to the left, above, or below the tomographic image. For convenient simultaneous viewing of both the tomographic image 42 and the 2D plot 44, it is desirable to display the 2D plot nearby of the tomographic image 42 within the same frame, without overlapping it.

[0055] A color band 52 is displayed near the two-dimensional graph 44. Color band 52 is a color sample representing the change in hue corresponding to the change in the probability of a lesion. In the illustrated example, color band 52 is displayed with its long side parallel to the horizontal direction. The left end of color band 52 corresponds to a lesion probability of 0.0, and its right end corresponds to a lesion probability of 1.0 (refer to reference numeral 54). A line 55 representing the currently set threshold is displayed on color band 52. Alternatively, color band 52 can be displayed with its long side parallel to the vertical direction. If the threshold is changed, line 55 moves horizontally in conjunction with it.

[0056] Chart 46 illustrates the temporal variation of lesion probability. In Chart 46, the horizontal axis corresponds to the time axis, and the vertical axis represents the lesion probability. Chart 46 consists of bars 60 generated for each display frame, with the height of each bar 60 representing the lesion probability. Specifically, the height of each bar 60 represents the maximum lesion probability within the display frame. The bars can be displayed at regular time intervals, or the maximum value can be represented as a curve smoothed along the time axis. Indicator 62 indicates the current time point. The display position of indicator 62 is fixed, but its display position can also be scanned horizontally.

[0057] In Figure 46, in addition to multiple bars 60, there is also a threshold line 64. The threshold line 64 represents the threshold set at each time point. The indicator 66 represents the current threshold. When the threshold is changed using the operation panel, the indicator 66 moves vertically in conjunction with it. The indicator 66 itself can also be operated to change the threshold by moving it vertically.

[0058] In the illustrated example, body marker 68 is shown on the lower side of two-dimensional diagram 44, and probe marker 70 is also shown. In this embodiment, the tissue to be diagnosed is the breast, and body marker 68 schematically represents the breast.

[0059] By displaying two-dimensional graphs 44 and 46 in real time along with the tomographic image containing marker 48, the examiner can comprehensively or multifacetedly evaluate the lesion. For example, by observing the two-dimensional graph 44, a detailed probability distribution of the lesions indicated by marker 48 can be identified, or the presence of other lesions that do not meet the marker display criteria can be identified. Furthermore, by observing the 46 graph, the frequency of lesion detection can be identified and applied to probe operation. The presence of lesions that are not the subject of notification is also easily identified.

[0060] exist Figure 3 The diagram illustrates the detection and processing of a lesion and several related processes. In the lesion detection unit 22, a detection process comprising multiple detection steps is performed. These multiple detection steps... Figure 3 The process is executed in multiple layers arranged in chronological order. These multiple layers are equivalent to completing the learning of the network.

[0061] Specifically, in Figure 3 The diagram shows multiple layers, L1 to L5. Each layer contains one or more convolutional layers, and additionally, one or more pooling layers as needed.

[0062] Layer L1 is the input layer, to which the tomographic image 76 is input as the input image. In the illustrated structural example, M output layers are included among the multiple layers. M output images 78-1 to 78-M are output from the M output layers. M is an integer greater than or equal to 1, for example, M can be 2 or 3. Output information 77 is constituted by the M output images 78-1 to 78-M. The M output images 78-1 to 78-M exhibit different responses depending on the size of the lesion. Output image 78-1 responds strongly to lesions with large sizes, thus easily representing the detection results of lesions with large sizes. Output image 78-M responds strongly to lesions with small sizes, thus easily representing the detection results of lesions with small sizes. Each output image 78-1 to 78-M is composed of multiple units 79-1 to 79-M arranged in two dimensions. Each unit 79-1 to 79-M has unit information corresponding to specific coordinates.

[0063] exist Figure 3 The upper right corner shows an example of cell information. The illustrated cell information 80 includes, for example, the X coordinate 82 and Y coordinate 84 determining the center coordinates of the detected lesion, information 86 indicating the width W of the detected lesion, information 88 indicating the height (width) H of the detected lesion, and an object score (OS) 90 indicating the probability that the lesion conforms to a specific classification, etc. N cell information corresponding to n classifications can be generated for each of these cell elements.

[0064] The post-processing unit 24 includes a maximum value determination unit 72 and a threshold processing unit 74. In the maximum value determination unit 72, for each display frame, the largest object score is determined from the output information 77 (i.e., multiple output charts 78-1 to 78-M) as the maximum value. In the threshold processing unit 74, the determined maximum value is compared with a threshold. If the maximum value exceeds the threshold, the flag generation unit 26 generates a flag. If the maximum value does not reach the threshold, no flag is generated. Alternatively, instead of determining the maximum value, processing for determining the top K object scores can be performed. In this case, the top K flags are generated simultaneously based on the top K object scores. K is an integer greater than or equal to 2. However, the maximum value is referenced when generating the chart.

[0065] The chart generation unit 28 uses bars to represent the maximum value of each display frame and generates a chart by repeating these bars. Alternatively, instead of a chart representing multiple maximum values ​​in a time series sequence, it can generate a chart representing multiple average values ​​in a time series sequence. Moving average processing can also be applied to the chart representing the maximum values. The generated chart also includes bars representing the maximum values ​​that do not meet the display criteria.

[0066] exist Figure 3 In the example shown, the output information 77 (i.e., multiple output images 78-1 to 78-M) is sent to the image generation unit 30 unchanged. In this embodiment, the image generation unit 30 generates a two-dimensional image based on the output information 77. Specifically, the object score, i.e., the lesion probability, extracted from each unit is transformed into color. The transformed color is mapped to a two-dimensional space. The multiple output images 78-1 to 78-M have different sizes, but they share a common coordinate system. Each color is mapped to this common coordinate system. A portion of the output information 77 can be used as the mapping object.

[0067] If the output information 77 contains information that satisfies the flag display conditions (first information) and information that does not satisfy the flag display conditions (second information), the flag generation unit 26 generates a flag based only on the information that satisfies the flag display conditions (maximum value condition and threshold condition) (first information). On the other hand, in this case, the graph generation unit 30 generates a two-dimensional graph based on both the information that satisfies the flag display conditions (first information) and the information that does not satisfy the flag display conditions (second information). The chart generation unit 28 generates a chart based only on the information that satisfies the maximum value condition.

[0068] In addition, when generating multiple output maps 78-1 to 78-M, feature maps 92 and 94 extracted from the middle of the detection process are used.

[0069] exist Figure 4The first example of generating a two-dimensional image is shown. In intermediate information 100, in the illustrated example, lesions are detected at two coordinates, 102 and 104. In reality, lesions are detected at dozens or hundreds of coordinates within a single display frame.

[0070] The probability of a lesion at coordinate 102 is (0.8), and the probability of a lesion at coordinate 104 is (0.4). The size of the lesion detected at coordinate 102 is shown by reference numeral 102A, and the size of the lesion detected at coordinate 104 is shown by reference numeral 104A.

[0071] In the first generation example, the lesion probability is transformed into a color code. For example, the lesion probability (0.8) at coordinate 102 is transformed into color code (204), and the lesion probability (0.4) at coordinate 104 is transformed into color code (120). Reference numeral 106 indicates the transformed information. A two-dimensional image 108 is generated by mapping the colors (R value, G value, B value) corresponding to each color code to a two-dimensional space. Reduction and magnification processing can also be performed during the generation of the two-dimensional image 108. Interpolation and smoothing processing can also be applied after color mapping.

[0072] exist Figure 5 The second example of generating a two-dimensional diagram is shown. The intermediate information 100, which has already been described, is processed as follows. A local region 102B with a certain area is defined based on the size 102A determined at coordinate 102, and similarly, a local region 104B with a certain area is defined based on the size 104A determined at coordinate 104. Local regions 102B and 104B can be generated, for example, by reducing the size of 102A and 104A.

[0073] The lesion probability (0.8) at coordinate 102 is transformed into a color code (204), and this color code is assigned to the entire local region 102B. Similarly, the lesion probability (0.4) at coordinate 104 is transformed into a color code (120), and this color code is assigned to the entire local region 104B. By mapping the colors (R value, G value, B value) corresponding to each color code to a two-dimensional space based on the information 110 thus constructed, a two-dimensional image 112 is generated.

[0074] Alternatively, in the second generation example, dimensions 102A and 104A can be used as local regions 102B and 104B without any changes.

[0075] exist Figure 6An example of color transformation is shown. Reference numeral 114 denotes a set of color transformation functions. Specifically, the color transformation function set 114 consists of the B transformation function 118B, the G transformation function 118G, and the R transformation function 118R. The horizontal axis represents the color index, which corresponds to the lesion score. The vertical axis represents relative brightness. Figure 6 The image shows a color band 116 representing the colors defined by each color index. Figure 6 The color transformation function set 114 shown is just one example; a variety of color transformation function sets, i.e., various color bands, can be used. It can also provide grayscale representation for lesion scoring.

[0076] exist Figure 7 A variation of the chart is shown. Chart 120 consists of multiple bars representing the probability of lesions. Index 122 indicates the current time. A difference in value is generated at threshold line 126. This difference signifies a change in the threshold. The portion 126B that is earlier in time than the difference in value represents a relatively high threshold, while the portion 126A that is later in time represents a relatively low threshold. The current threshold can be changed by moving index 124 up and down. The shape of threshold line 126 changes as the threshold changes.

[0077] For example, tomographic images, two-dimensional images, and charts can be generated and displayed based on frame data strings stored in the image memory. In this case, these images may be displayed not as moving images, but as still images corresponding to specific past timings. In this case, the specific past timing can be specified by element 128. Element 128 is a marker that moves along the horizontal axis of chart 120. For example, a specific past time point in time when a lesion was detected can be specified by element 128.

[0078] exist Figure 8 Other examples of displayed images are shown. Display image 40A includes tomographic image 42, two-dimensional image 44A, graphs 46A, 46B, etc. Tomographic image 42 includes lesion 50, with an overlay of a sign 48 indicating it. In this other example, two types of lesions (e.g., tumor and non-tumor) are detected, and the probability of each lesion, i.e., a lesion score, is calculated for each type.

[0079] In the two-dimensional graph 44A, for example, the probability of the first lesion is represented by the brightness of a blue hue, and the probability of the second lesion is represented by the brightness of a red hue. The relationship between the probability of the first lesion and the brightness of the blue hue is represented by the first color band 52A, and the relationship between the probability of the second lesion and the brightness of the red hue is represented by the second color band 52B. Lines 55A and 55B representing thresholds are displayed on each color band 52A and 52B.

[0080] Alternatively, the first two-dimensional plot representing the probability distribution of the first lesion and the second two-dimensional plot representing the probability distribution of the second lesion can be generated and displayed separately. On the tomographic image 42, the first and second lesions are identified by the same label 48 without distinguishing between them, but different labels can also be displayed for each lesion category.

[0081] Chart 46A shows the time-varying probability (maximum value) of the first lesion. Chart 46B shows the time-varying probability (maximum value) of the second lesion. Chart 46A includes line 64A as a display element representing the threshold, and Chart 46B includes line 64B as a display element representing the threshold.

[0082] According to the above-described implementation, in addition to markers, two-dimensional diagrams and graphs are displayed as supplementary examination information. The two-dimensional diagrams and graphs are displayed regardless of whether markers are displayed. Therefore, by observing this displayed information, the examiner can obtain more detailed information about the object being examined. This can assist in the detailed examination of lesions notified via marker display, or verify the appropriateness of the marker display. Furthermore, the object being examined can be evaluated even without markers being displayed.

Claims

1. An ultrasonic diagnostic device, characterized in that, Include: The detection unit (22) performs detection processing to detect lesions in ultrasound images; The marker generation unit (26) generates a marker to notify the lesion based on the output information of the detection unit (22); Reference information generation units (28, 30) generate reference information based on the output information, representing at least one of the spatial distribution of the probability of lesions and the temporal change of the probability of lesions, as auxiliary examination information different from the marker; and The display (32) displays the ultrasonic image, the mark, and the reference information in real time. The reference information generation unit includes: The chart generation unit (28) generates a chart representing the time-varying probability of the lesion as reference information. The chart generation unit (28) determines a representative value of the probability of lesions within a frame for each frame and generates the chart based on multiple representative values ​​determined from multiple frames.

2. The ultrasonic diagnostic device according to claim 1, characterized in that, The mark is displayed on the ultrasound image. The reference information is displayed in the vicinity of the ultrasound image.

3. The ultrasonic diagnostic device according to claim 1, characterized in that, The reference information generation unit includes: The graph generation unit (30) generates a two-dimensional graph representing the spatial distribution of the probability of the lesion. The reference information includes the two-dimensional diagram and the chart.

4. The ultrasonic diagnostic device according to claim 3, characterized in that, If the output information contains first information that satisfies the sign display conditions and second information that does not satisfy the sign display conditions, the sign generation unit (26) generates the sign based on the first information, and the image generation unit (30) generates the two-dimensional image based on the first information and the second information.

5. The ultrasonic diagnostic device according to claim 1, characterized in that, The representative value for each frame is the maximum value.

6. The ultrasonic diagnostic device according to claim 1, characterized in that, The flag generation unit (26) generates the flag when the output information meets the flag display conditions. The flag display conditions include a condition requiring that the probability of a lesion determined by the output information exceeds a threshold. The chart includes display elements representing the threshold.

7. The ultrasonic diagnostic device according to claim 6, characterized in that, The chart generation unit (28) changes the displayed elements according to the change of the threshold.

8. An ultrasonic image processing method, characterized in that, Include: Detection step (22) involves performing detection processing to detect lesions in ultrasound images; The marker generation step (26) generates a marker to notify the lesion based on the output information characterizing the execution result of the detection process; Reference information generation steps (28, 30): Based on the output information, generate reference information that characterizes at least one of the spatial distribution of the probability of the lesion and the temporal change of the probability of the lesion, as examination auxiliary information different from the marker; and In step (32), the ultrasonic image, the marker, and the reference information are displayed in real time. In the reference information generation step, a graph representing the time change of the lesion probability is generated as the reference information, wherein a representative value of the lesion probability within a frame is determined for each frame, and the graph is generated based on multiple representative values ​​determined from multiple frames.

9. A computer program product comprising a computer program, characterized in that the computer program is executed by an information processing device in the following steps: Detection step (22) involves performing detection processing to detect lesions in ultrasound images; The marker generation step (26) generates a marker to notify the lesion based on the output information characterizing the execution result of the detection process; Reference information generation steps (28, 30): Based on the output information, generate reference information that characterizes at least one of the spatial distribution of the probability of the lesion and the temporal change of the probability of the lesion, as examination auxiliary information different from the marker; and In step (32), the ultrasonic image, the marker, and the reference information are displayed in real time. In the reference information generation step, a graph representing the time change of the lesion probability is generated as the reference information, wherein a representative value of the lesion probability within a frame is determined for each frame, and the graph is generated based on multiple representative values ​​determined from multiple frames.

Images