IMAGE PROCESSING APPARATUS, IMAGE PROCESSING METHOD, AND IMAGE PROCESSING PROGRAM

Abstract

To provide an image processing device capable of improving visibility of features expressed by each of two sheets of medical images, in a composite image for display generated by composing two sheets of colored medical images according to an identical portion of an analyte.SOLUTION: The image processing device which composes first and second medical images according to the identical portion of the analyte expressed by a color image comprises: a base composite image generation unit which superimposes the luminance component image of the first medical image onto the second medical image and generates a base composite image; a contribution degree determination unit which on the basis of the second medical image, determines a distribution degree concerning a color which each pixel region of the first medical image should give to each pixel region of the base composite image, for each pixel region; and a color correction unit which adds the color component of the first medical image to the pixel value of each pixel of the base composite image, with a weight corresponding to the contribution degree, and generates the composite image for display.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] The present invention relates to an image processing device, an image processing method, and an image processing program. [Background technology]

[0002] There is known an ultrasound diagnostic device that generates a color Doppler image of the inside of a subject by transmitting an ultrasound beam to the subject and detecting the time and intensity of the transmitted ultrasound beam until it is reflected by tissues, blood flow, etc. inside the body and returns (see, for example, Patent Documents 1, 2, and 3).

[0003] Generally, color Doppler images are generated by overlaying a B image (Brightness mode Imaging), which is a grayscale representation of a tomographic image inside a living body, with a flow image (Color Flow Imaging or Tissue Doppler Imaging) that shows blood flow information (blood flow velocity, blood flow direction, power, velocity variance) and the movement of body tissues such as the myocardial wall in color.

[0004] In the following description, for convenience of explanation, a signal that generates a B image is referred to as a "B signal," and a signal that generates a flow image is referred to as a "flow signal." [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Publication No. 11-169368 [Patent Document 2] JP 2008-284287 A [Patent Document 3] JP 2017-042606 A Summary of the Invention [Problem to be solved by the invention]

[0006] In recent years, in response to demands for clearer representation of internal structures and tissue textures in a form that matches the ease of viewing of images felt by users and the color characteristics of display devices (such as LCDs), B images are often represented as color images (hereinafter also referred to as "colored B images") rather than grayscale images. For example, such B images are images in which the signal (e.g., signal intensity) of the reflected wave of ultrasound from within the subject is converted into color information using a color map. Note that a color image means an image in which the pixel value of each pixel of the image is represented as a combination of values ​​(i.e., density) of two or more color components such as in the RGB color space.

[0007] However, when a flow image is synthesized with such a colored B image to generate a color Doppler image, the color tone of both images changes in the synthesized color Doppler image, and the visibility of the structural characteristics of tissues and the characteristics of the spread and velocity of blood flow expressed by color tends to deteriorate.

[0008] In particular, since color Doppler images combine images showing two different conditions, blood flow and a tomographic image inside a living body, it is necessary to make it easy to visually distinguish the signal strength of the components of those signals. For example, even in areas where the speed and power of the flow signal are low, it is preferable that the blood flow that changes successively is displayed without interruption and the exact position of thin blood vessels is displayed stably.

[0009] 1 to 3 show examples of color Doppler images generated by three types of image synthesis methods according to the conventional techniques.

[0010] Fig. 1 is a diagram showing an example of a color Doppler image generated by combining a grayscale B image with a flow image. As can be seen from Fig. 1, when the B image is a grayscale image, no color mixing occurs, resulting in a color Doppler image in which the characteristics of both the B image and the flow image are relatively easy to see. However, in this case, it is not possible to add color to the B image or adjust the display color of the B image (for example, adjust the hue or saturation).

[0011] Fig. 2 is a diagram showing an example of a color Doppler image generated by image synthesis of a flow image so as to be overlaid on a colored B image. As can be seen from Fig. 2, in the color Doppler image generated by this method, the B image is hidden in the area where the flow image is displayed. Therefore, there is a problem that the visibility of the tissue structure represented by the B image is poor, and the relationship between the behavior of the blood flow shown on the flow image and the behavior of the tissue shown on the B image cannot be visually recognized.

[0012] In addition, there is also a method in which, when overwriting a colored B image with a flow image, whether to display a B image or a flow image for each pixel forming the image is determined according to the echo intensity or color intensity, and based on the determination result, the pixel value of either the B image or the flow image is selected for each pixel to generate a color Doppler image. However, even with this method, the configuration is such that only one of the B image or the flow image is displayed for each pixel region, and the above-mentioned problems cannot be solved.

[0013] Fig. 3 is a diagram showing an example of a color Doppler image generated by superimposing a colored B image and a flow image by transparent synthesis (also called alpha blending). In this example, the colored B image and the flow image are weighted and added at a predetermined transparency ratio (i.e., α value) to generate a color Doppler image. As can be seen from Fig. 3, in such a color Doppler image, the impression of the color tone of the colored B image and / or the flow image changes depending on the α value, and the visibility of the structural features of tissues and the characteristics of the spread and speed of blood flow expressed by chromaticity (representing a tone expressed by hue and saturation; the same applies below) decreases.

[0014] The present disclosure has been made in consideration of the above-mentioned problems, and aims to provide an image processing device, an image processing method, and an image processing program that enable improved visibility of features expressed by each of two medical images in a composite image for display generated by image-combining two colored medical images relating to the same part of a subject. [Means for solving the problem]

[0015] The present disclosure, which solves the above-mentioned problems, is 1. An image processing device that synthesizes first and second medical images of the same part of a subject, the first and second medical images being expressed as color images, the image processing device comprising: a base composite image generating unit that generates a base composite image by performing image synthesis so as to superimpose a luminance component image, in which only a luminance component is extracted from the first medical image, on the second medical image; a contribution determining unit that determines, for each pixel region, a contribution of color that each pixel region of the first medical image should give to each pixel region of the base composite image based on at least a luminance component or a color component of a pixel value of each pixel region of the second medical image; a color correction unit that generates a composite image for display by adding color components of pixel values ​​of each pixel region of the first medical image to pixel values ​​of each pixel region of the base composite image with weights according to the contribution rates; The image processing device includes:

[0016] In other respects, 1. An image processing method for synthesizing first and second medical images of the same part of a subject, the first and second medical images being expressed as color images, the method comprising the steps of: a process of performing image synthesis so as to superimpose a luminance component image, from which only luminance components are extracted, on the second medical image to generate a base synthetic image; determining, for each pixel region, a degree of color contribution that each pixel region of the first medical image should give to each pixel region of the base composite image based on at least a luminance component or a color component of a pixel value of each pixel region of the second medical image; a process of generating a composite image for display by adding color components of pixel values ​​of each pixel region of the first medical image to pixel values ​​of each pixel region of the base composite image with weights according to the contribution rates; The image processing method includes the steps of:

[0017] In other respects, A program for causing a computer to execute image processing for image synthesis of first and second medical images relating to the same part of a subject, the first and second medical images being represented as color images, the program comprising: a process of performing image synthesis so as to superimpose a luminance component image, from which only luminance components are extracted, on the second medical image to generate a base synthetic image; determining, for each pixel region, a degree of color contribution that each pixel region of the first medical image should give to each pixel region of the base composite image based on at least a luminance component or a color component of a pixel value of each pixel region of the second medical image; a process of generating a composite image for display by adding color components of pixel values ​​of each pixel region of the first medical image to pixel values ​​of each pixel region of the base composite image with weights according to the contribution rates; It is an image processing program having the following structure. Effect of the Invention

[0018] According to the image processing device of the present disclosure, in a composite image for display generated by image-compositing two colored medical images relating to the same part of a subject, it is possible to improve the visibility of the features represented by each of the two medical images. [Brief description of the drawings]

[0019] [Figure 1] FIG. 1 shows an example of a color Doppler image generated by combining a flow image with a grayscale B image. [Diagram 2] FIG. 1 shows an example of a color Doppler image generated by image synthesis of a flow image so as to be overlaid on a colored B image. [Diagram 3] FIG. 1 shows an example of a color Doppler image generated by superimposing a colored B image and a flow image using transparency synthesis (also called alpha blending). [Figure 4] FIG. 1 is a diagram showing the configuration of an ultrasonic diagnostic apparatus according to an embodiment of the present invention. [Diagram 5] FIG. 1 is a diagram showing a configuration of an image processing unit according to an embodiment of the present invention. [Figure 6]FIG. 1 is a diagram showing an example of an image generated by processing in each part of an image processing unit; [Figure 7] FIG. 13 is a diagram showing an example of a B image generated by a B image generating unit; [Figure 8] FIG. 1 is a diagram showing an example of a flow image generated by a flow image generating unit; [Figure 9] FIG. 13 is a diagram showing an example of a luminance component image of a B image generated by a division unit; [Figure 10] FIG. 13 is a diagram showing an example of a color component image of a B image generated by a division unit; [Figure 11] FIG. 13 is a diagram showing an example of a contribution degree image generated by a contribution degree determination unit; [Figure 12] FIG. 13 is a diagram showing an example of a base composite image generated by a base composite image generating unit; [Figure 13] FIG. 1 is a diagram showing an example of a color contribution component image generated by a color correction unit; [Figure 14] FIG. 13 is a diagram showing an example of a composite image for display generated by a color correction unit; [Figure 15] An example of a color map for the B image. [Figure 16] FIG. 1 shows an example of a color map for a flow image. [Figure 17] FIG. 13 is a diagram showing an example of a contribution degree determination table; [Figure 18] FIG. 1 is a diagram showing a configuration of a color correction unit according to an embodiment of the present invention. [Figure 19] FIG. 1 is a diagram showing the configuration of an ultrasound diagnostic apparatus (image processing unit) according to Modification 1. [Figure 20] FIG. 13 is a diagram showing a color map for a B image to be referred to by a B image generating unit according to the first modification example. [Figure 21] FIG. 13 is a diagram showing the configuration of an ultrasound diagnostic apparatus (image processing unit) according to Modification 2. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0020] Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In this specification and the drawings, components having substantially the same functions are denoted by the same reference numerals, and redundant description will be omitted.

[0021] In this disclosure, the term "pixel region" refers to a region of one pixel, or a region that forms one section with a plurality of pixels.

[0022] [Overall configuration of ultrasound diagnostic equipment] Hereinafter, the configuration of an ultrasound diagnostic device according to one embodiment of the present invention (hereinafter, referred to as "ultrasound diagnostic device U") will be described with reference to Fig. 4. Note that in this embodiment, as an example of the image processing device of the present disclosure, a mode is shown in which the image processing device is applied to generate a color Doppler image by image-compositing a B image and a flow image generated by the ultrasound diagnostic device U. However, the subject of image composition by the image processing device of the present disclosure may be other medical images, such as an elasticity image generated by the ultrasound diagnostic device U.

[0023] FIG. 4 is a diagram showing the configuration of an ultrasound diagnostic apparatus U according to this embodiment.

[0024] The ultrasound diagnostic device U includes an ultrasound probe 1, a transmitter 2, a receiver 3, a scanner 4, a B signal generator 5, a flow signal generator 6, an image processor 7, and a display 8. The “image processing device” of the present disclosure is applied to the image processor 7 of the ultrasound diagnostic device U.

[0025] The ultrasound diagnostic apparatus U is used to visualize the shape, properties, or dynamics inside a subject as ultrasound images and perform image diagnosis.

[0026] The ultrasonic probe 1 converts a voltage pulse generated by a transmitting unit 2 into an ultrasonic beam and transmits it into the subject, receives ultrasonic echoes reflected within the subject, converts them into electrical signals, and outputs them to a receiving unit 3. The ultrasonic probe 1 includes, for example, a plurality of transducers (piezoelectric elements) arranged in a matrix, and a channel switching unit (selector) for switching and controlling the on / off of the driving state of the plurality of transducers individually or in block units (hereinafter referred to as "channels").

[0027] The transmission unit 2 is a transmission circuit that transmits a voltage pulse as a drive signal to the ultrasonic probe 1. The transmission unit 2 includes, for example, a high-frequency pulse oscillator and a pulse setting unit. The voltage pulse generated by the high-frequency pulse oscillator is adjusted to the voltage amplitude, pulse width, and timing set by the pulse setting unit, and transmitted to each channel of the ultrasonic probe 1.

[0028] The transmitting unit 2 has a pulse setting unit for each of the multiple channels of the ultrasonic probe 1, and is capable of setting the voltage amplitude, pulse width, and timing of the voltage pulse for each of the multiple channels. For example, the transmitting unit 2 can change the target depth by setting an appropriate delay time for each of the multiple channels, and can also generate different pulse waveforms for generating B images and flow images (for example, one voltage pulse is sent when generating B images, and four voltage pulses are sent when generating flow images).

[0029] The receiving unit 3 is a receiving circuit that receives and processes the received signals related to the ultrasonic echoes generated by the ultrasonic probe 1. The receiving unit 3 includes a preamplifier for each channel, an AD converter for each channel, and a receiving beamformer. The preamplifier amplifies the weak received signals. The AD converter converts the amplified received signals (analog signals) into digital signals. The receiving beamformer combines the signals of multiple channels into one by phasing and adding the received signals (digital signals) of each channel, and outputs the combined signal to the B signal generating unit 5 or the flow signal generating unit 6. Hereinafter, this output signal will be referred to as the "received echo signal."

[0030] The scanning unit 4 controls the ultrasonic probe 1, the transmitting unit 2, and the receiving unit 3. The scanning unit 4 controls a channel switching unit provided in the ultrasonic probe 1 to switch among the multiple channels to be driven. The scanning unit 4 then drives (scans) the multiple channels in sequence, causing the ultrasonic probe 1 to transmit ultrasonic beams into the subject in the order along the scanning direction. In this way, the scanning unit 4 causes the B signal generating unit 5 and the flow signal generating unit 6 to generate two-dimensional data along the scanning direction and the depth direction.

[0031] In addition to the above, the signal processing circuit is provided with a transmission / reception switching unit (not shown) for switching a signal system with the ultrasonic probe 1, and a processing system switching unit (not shown) for switching a destination to which a received echo signal is sent from the receiving unit 3. The scanning unit 4 controls the switching of the transmission / reception switching unit to switch the signal system between when a voltage pulse is sent from the transmitting unit 2 to the ultrasonic probe 1 and when a received signal generated by the ultrasonic probe 1 is sent to the receiving unit 3. The scanning unit 4 also controls the switching of the processing system switching unit to send the received echo signal from the receiving unit 3 to the B signal generating unit 5 when generating a B image, and to send the received echo signal from the receiving unit 3 to the flow signal generating unit 6 when generating a flow image.

[0032] However, this configuration is merely an example, and it is needless to say that the transmission / reception switching unit and processing system switching unit of the signal processing circuit are not necessarily provided. For example, the B signal generating unit 5 and the flow signal generating unit 6 may generate B signals and flow signals from the same received echo signal. In that case, it becomes possible to generate B images and flow images, which will be described later, in a synchronized manner.

[0033] The B signal generating unit 5 acquires the received echo signal from the receiving unit 3 and generates a B signal. The B signal is a signal that indicates the temporal change in the signal intensity of the ultrasonic echo in the depth direction (transmission direction of the ultrasonic beam). When the ultrasonic probe 1 scans with the ultrasonic beam, the B signal generating unit 5 sequentially accumulates the B signals obtained at each scanning position in a line memory and generates two-dimensional data that constitutes a frame unit of the B image.

[0034] The B signal generating unit 5 includes, for example, an envelope detection circuit, a dynamic filter, and a logarithmic compression circuit. The envelope detection circuit detects the signal strength by envelope detection of the received echo signal. The logarithmic compression circuit performs logarithmic compression on the signal strength of the received echo signal detected by the envelope detection circuit. The dynamic filter is a bandpass filter whose frequency characteristics change according to the depth, and removes noise components contained in the received echo signal.

[0035] The flow signal generating unit 6 acquires the received echo signal from the receiving unit 3 and generates a flow signal. The flow signal is a signal generated by frequency analysis of ultrasonic echoes from blood flow or moving body tissue, and indicates their velocity, power, and velocity variance (Turbulence). The flow signal is calculated, for example, based on the phase difference of ultrasonic echoes from the same depth position of ultrasonic pulses continuously transmitted by the ultrasonic probe 1. The flow signal generating unit 6, like the B signal generating unit 5, sequentially accumulates flow signals obtained at each scanning position in a line memory when the ultrasonic probe 1 scans with an ultrasonic beam, and generates two-dimensional data that becomes a frame unit of a flow image.

[0036] The flow signal generating unit 6 includes, for example, an MTI filter, a quadrature detection circuit, and an autocorrelation calculation unit. The MTI (Moving Target Indication) filter is a low-pass filter that removes clutter components (ultrasonic echoes from tissues) from stationary tissues from the received echo signal. The quadrature detection circuit mixes a reference signal that is in phase with the transmitted ultrasonic wave and a reference signal that is out of phase with the transmitted ultrasonic wave by π / 2 with respect to the received echo signal to generate a quadrature detection signal (hereinafter referred to as an "IQ signal"). The autocorrelation calculation unit extracts a flow signal (velocity, power, variance) by autocorrelation calculation based on the IQ signals of ultrasonic echoes from the same depth position of continuously transmitted ultrasonic pulses.

[0037] The image processing unit 7 receives frame-by-frame B signals and flow signals for constituting image data from the B signal generating unit 5 and the flow signal generating unit 6, respectively. The image processing unit 7 generates a B image based on the input B signals and generates a flow image based on the input flow signals, synthesizes these images, and outputs them to the display unit 8 as a color Doppler image (details will be described later).

[0038] Here, the B signal and the flow signal in each frame mean a unit of a group of signals required to construct one image. However, the B signal and the flow signal may be signals for a part of an image. For example, the flow image may be superimposed only on a specified region of interest in the B image.

[0039] The B signal and the flow signal input from the B signal generating unit 5 and the flow signal generating unit 6 to the image processing unit 7 may be subjected to coordinate conversion processing or data interpolation processing via a DSC (Digital Scan Converter).

[0040] The display unit 8 is a monitor that displays the composite image for display (in this embodiment, a color Doppler image) output from the image processing unit .

[0041] The B signal generating unit 5, the flow signal generating unit 6, and the image processing unit 7 are realized by a digital arithmetic circuit composed of, for example, a DSP (Digital Signal Processor). Each of these components includes a buffer memory, and intermediate data in the process of each arithmetic process and two-dimensional data in frame units generated in each arithmetic process are stored in the buffer memory.

[0042] [Detailed configuration of the image processing unit] Hereinafter, a detailed configuration of the image processing unit 7 according to the present embodiment will be described with reference to FIGS.

[0043] Fig. 5 is a diagram showing the configuration of the image processing unit 7 according to this embodiment. Note that the arrows shown in Fig. 5 represent the flow of signal processing.

[0044] FIG. 6 is a diagram showing an example of an image generated in the processing of each unit of the image processing unit 7. FIGS. 7 to 14 are diagrams showing the images shown in FIG. 6 individually. FIG. 7 is a diagram showing an example of a B image generated by the B image generating unit 71. FIG. 8 is a diagram showing an example of a flow image generated by the flow image generating unit 72. FIG. 9 is a diagram showing an example of a luminance component image of the B image generated by the dividing unit 73. FIG. 10 is a diagram showing an example of a color component image of the B image generated by the dividing unit 73. FIG. 11 is a diagram showing an example of a contribution degree image generated by the contribution degree determining unit 75. FIG. 12 is a diagram showing an example of a base composite image generated by the base composite image generating unit 74. FIG. 13 is a diagram showing an example of a color contribution component image generated by the color correcting unit 76. FIG. 14 is a diagram showing an example of a composite image for display generated by the color correcting unit 76.

[0045] The image processing unit 7 includes a B image generating unit 71, a flow image generating unit 72, a dividing unit 73, a base composite image generating unit 74, a contribution degree determining unit 75, and a color correcting unit .

[0046] The B image generating unit 71 acquires the B signals generated by the B signal generating unit 5 on a frame-by-frame basis, and generates a B image by converting the B signals corresponding to each pixel region into pixel values. In this case, the B image generating unit 71 generates the B image using, for example, a function (hereinafter referred to as a "color map for B image") that converts the signal intensity of the B signal into pixel values, which is stored in advance in a storage unit (for example, a ROM included in the image processing unit 7). The B image generating unit 71 converts the B signal into pixel values ​​such that the luminance increases as the signal intensity of the B signal increases. That is, an image region with high luminance in the B image is a region in which the tissue structure, etc. within the subject is clearly expressed, and is a region that characterizes the tissue structure within the subject.

[0047] FIG. 15 is a diagram showing an example of a color map for a B image. In this embodiment, the B image is generated as a colored image by the color map for the B image as shown in FIG. 15. That is, the pixel value of each pixel region of the B image is not expressed by only one-bit luminance, but is, for example, a value expressed in 256 gradations for each of the RGB components (i.e., a color space vector related to the RGB components). In the color map for the B image, for example, the signal intensity of the B signal is converted into the value of each of the RGB components by applying three functions f0-R, f0-G, and f0-B that convert the signal intensity of the B signal into a value expressed in 256 gradations for each of the RGB components. Note that the pixel value of a pixel region where the signal intensity of the B signal is large is usually expressed in a state where all of the RGB components are large.

[0048] The flow image generating unit 72 acquires the flow signals generated by the flow signal generating unit 6 in units of frames, and converts the flow signals corresponding to each pixel region into pixel values ​​to generate a flow image. In this case, the flow image generating unit 72 generates the flow image using a function (hereinafter referred to as a "color map for flow images") that converts flow signals into pixel values, which is stored in advance in a storage unit (e.g., a ROM included in the image processing unit 7). Here, the flow image generating unit 72 determines pixel values ​​based on two components of flow signals, velocity and power. Typically, the flow image generating unit 72 converts the flow signals into pixel values ​​such that the luminance increases as the power of the flow signal (e.g., blood flow rate) increases.

[0049] Fig. 16 is a diagram showing an example of a color map for a flow image. Note that, in the velocity component on the horizontal axis shown in Fig. 16, the center position of the graph represents a velocity of 0, the left side of the graph center represents a velocity in the direction away from the graph, and the right side of the graph center represents a velocity in the direction toward the graph.

[0050] The pixel value of the flow image is, for example, a color space vector related to RGB components, and each RGB component is expressed in 256 gradations. In the color map for the flow image, for example, the direction of the speed toward the ultrasonic probe 1 is set to be represented by a large R component and red (area D in FIG. 16), and conversely, the direction away from the ultrasonic probe 1 is set to be represented by a large B component and blue (area B in FIG. 16). Also, in the color map for the flow image, the larger the power, the larger all the RGB color components are set (area B in FIG. 16) (area D in FIG. 16). On the other hand, when the power is small, it is highly likely that the component is a background noise component, so the ratio of the RGB color components is set to be brown, approximately 12:10:5 (area A in FIG. 16) (area E in FIG. 16). Furthermore, when the velocity is small, it is highly likely that this is a motion artifact caused by unnecessary Doppler signals from objects other than blood flow, so similarly, the ratio of RGB color components is set to approximately 12:10:5, resulting in brown (area C in Figure 16).

[0051] The division unit 73 divides the pixel values ​​of each pixel region of the B image into a luminance component and a color component, and generates an image in which only the luminance component is extracted from the colored B image (hereinafter referred to as a "luminance component image"), and an image in which only the color component is extracted from the colored B image (hereinafter referred to as a "color component image"). Note that the division unit 73 generates the luminance component image and the color component image such that an image obtained by adding the luminance component image and the color component image becomes the original colored B image.

[0052] The division unit 73 calculates the luminance component of the pixel value of each pixel region of the B image (i.e., the pixel value of the luminance component image) as the average value of each RGB component of the pixel value of each pixel region of the B image, for example, as shown in the following equation (1). I B1 =avg(I B0R ,I B0G ,I B0B )…(1) (However, I B1 : Pixel value of the luminance component image, avg(I B0R ,I B0G ,I B0B): R component of pixel value of B image I B0R and the G component of the pixel value of the B image, I B0G and the B component of the pixel value of the B image I B0B (average of and

[0053] Then, the division unit 73 calculates the color components of the pixel values ​​of each pixel region of the B image (i.e., the pixel values ​​of the color component image) by subtracting the luminance components of the pixel values ​​of each pixel region of the B image calculated by equation (1) from the RGB components of the pixel values ​​of each pixel region of the B image, for example, as in the following equations (2), (3), and (4). I B2R =I B0R -I B1 …(2) I B2G =I B0G -I B1 …(3) I B2B =I B0B -I B1 …(4) (However, I B2R : R component of the color component image, I B2G : G component of the color component image, I B2B : B component of the color component image, I B0R : R component of pixel value of B image, I B0G : G component of pixel value of B image, I B0B : B component of pixel value of image B, I B1 : luminance component image pixel value)

[0054] The base composite image generating unit 74 generates a base composite image by adding the luminance component of the pixel value of each pixel region of the B image to the pixel value of each pixel region of the flow image. For example, the base composite image generating unit 74 associates each pixel region of the B image (luminance component image) with each pixel region of the flow image, and calculates the pixel value of each pixel region of the base composite image using the following equations (5) to (8). Note that the weighting coefficient w B1 , w F is set to an appropriate value (e.g., w B1 :0.5,w F :0.5).

[0055] Here, the base composite image generation unit 74 calculates the luminance component of the pixel value of the base composite image by adding the luminance component of the pixel value of the B image and the luminance component of the pixel value of the flow image, and then calculates the RGB components of the pixel value of this base composite image by adding the color component of the pixel value of the flow image (the color component from which the luminance component has been subtracted in advance, using processing similar to that of equations (2), (3), and (4) above) to the luminance component of the pixel value of this base composite image.

[0056] I Out1a =I B1 ·w B1 +I Fa ·w F …(5) (However, I Out1a : Luminance component of pixel value of base composite image, I B1 : Pixel value of the luminance component image, I Fa : Luminance component of pixel value of flow image, w B1 : weighting factor for B image, w F : weighting coefficient for flow image)

[0057] I Out1R =I FR +I Out1a …(6) I Out1G =I FG +I Out1a …(7) I Out1B =I FB +I Out1a …(8) (However, I Out1R : R component of pixel value of base composite image, I Out1G : G component of pixel value of base composite image, I Out1B : B component of pixel value of base composite image, I FR : The value obtained by subtracting the brightness component from the R component of the pixel value of the flow image, I FG : The value obtained by subtracting the luminance component from the G component of the pixel value of the flow image, I FB : The value obtained by subtracting the luminance component from the B component of the pixel value of the flow image, w B1 : weighting factor for B image, w F : weighting coefficient for flow image)

[0058] The image synthesis process by the base synthetic image generating unit 74 is based on the same idea as the method of generating a color Doppler image according to the conventional technology described with reference to Fig. 1. That is, when generating a base synthetic image, the B image to be synthesized is expressed only by brightness, so that no color mixing occurs on the base synthetic image, and the features of both the B image and the flow image (for example, the features of the tissue structure inside the subject and the features of the spread of blood flow) can be visually recognized.

[0059] The contribution degree determination unit 75 determines, for each pixel region, the contribution degree (hereinafter simply referred to as "contribution degree") regarding the color that each pixel region of the B image should give to each pixel region of the base composite image based on the luminance component of the pixel value of each pixel region of the B image (i.e., the pixel value of each pixel region of the luminance component image) and the luminance component of the pixel value of each pixel region of the flow image. For example, the contribution degree determination unit 75 compares the luminance of the B image and the luminance of the flow image for each corresponding pixel region, and determines the contribution degree for each pixel region based on the relationship.

[0060] The contribution degree is determined, for example, for each pixel area of ​​the image size of the B image in units of one pixel. The contribution degree acts like a mask image that is applied to the color component image of the B image (hereinafter, also referred to as a "contribution degree image").

[0061] The contribution determination unit 75 determines the contribution of each pixel region from the luminance component of the pixel value of each pixel region of the B image and the luminance component of the pixel value of each pixel region of the flow image, for example, using a contribution determination table previously stored in a memory unit (for example, a ROM possessed by the image processing unit 7).

[0062] 17 is a diagram showing an example of a contribution degree determination table. The contribution degree of each pixel region is, for example, 0≦w BcThe contribution degree is calculated as a scalar value of ≦1. For example, the contribution degree takes a value of 1.0 in a pixel region that displays only the B image, and takes a value of 0.0 in a pixel region that displays only the blood flow of the flow image. The contribution degree is determined to a value close to 1.0 in a pixel region where the impression of the B image is greater than the impression of the flow image, and is determined to a value close to 0.0 in a pixel region where the impression of the color flow image is greater than the impression of the B image. Specifically, as shown in the contribution degree determination table in FIG. 17, the contribution degree is determined based on the relationship between the luminance component of the pixel value of the B image and the luminance component of the pixel value of the flow image, and the larger the luminance component of the pixel value of the B image relative to the luminance component of the pixel value of the flow image, the larger the contribution degree is determined to be.

[0063] However, unlike a general mask image, the contribution image does not have a binary value of 0 or 1, but has a gradation between 0.0 and 1.0. This allows the colors of the B image and the flow image to be reflected in the pixel regions that express the respective characteristics of the image. For example, a large contribution (i.e., weight) is given to pixel regions that should strongly express the color of the B image, and a small contribution (i.e., weight) is given to pixel regions that do not, thereby determining the degree to which the color of each pixel region of the color component image (image B) appears in the final composite image for display.

[0064] B images generally have characteristic image information related to tissue structure over a wide range of the image in order to represent internal structures and the texture of tissues. On the other hand, flow images generally have characteristic image information related to the speed and volume of blood flow only in localized regions in the image where blood flow exists, and do not have characteristic image information in other regions. Therefore, by generating a "contribution degree" that indicates how much the color of the B image affects the composite image for display, which is low in regions where blood flow exists and high in regions where blood flow does not exist, it becomes possible to reflect the color of the B image over a wide range in the final composite image for display, while retaining the color of the flow image in regions where blood flow exists.

[0065] In addition, in areas where the speed and power of the flow signal are low, the contribution of the B image is increased so that the color of the B image becomes stronger. This allows the characteristics of the tissue structure represented by the B image in such areas to be clearly displayed, while the continuously changing blood flow represented by the flow image is displayed without interruption, and the exact positions of thin blood vessels are stably displayed.

[0066] The color correction unit 76 adds the color components of the pixel values ​​of each pixel region of the B image to the pixel values ​​of each pixel region of the base composite image with weights according to the contributions to generate a composite image for display. As a result, in the composite image for display, the colors of the B image are restored to image regions where the impression of the B image is greater than the impression of the color flow image.

[0067] 18 is a diagram showing the configuration of a color correction unit 76 according to this embodiment. For example, the color correction unit 76 first performs a contribution image weighting process (i.e., mask process) on the color component image of the B image to generate a color contribution component image of the B image. Then, the color correction unit 76 adds the color components of the pixel values ​​of each pixel region of the color contribution component image of the B image to the pixel values ​​of each pixel region of the base composite image to generate a composite image for display that is ultimately to be displayed on the display unit 8.

[0068] The color correction unit 76 calculates the pixel value of each pixel region of the composite image for display using, for example, the following equations (9), (10), and (11). I Out2R =I B2R ·w B2 +I Out1R …(9) I Out2G =I B2G ·w B2 +I Out1G …(10) I Out2B =I B2B ·w B2 +I Out1B …(11) (However, I Out2R : R component of pixel value of composite image for display, I Out2G : G component of pixel value of composite image for display, I Out2B: B component of pixel value of composite image for display, I Out1R : R component of pixel value of base composite image, I Out1G : G component of pixel value of base composite image, I Out1B : B component of pixel value of base composite image, I B2R : R component of pixel value of color component image, I B2G : G component of pixel value of color component image, I B2B : B component of the pixel value of the color component image, w B2 : Contribution of each color of the RGB components of the color component image)

[0069] In this manner, the image processing unit 7 according to this embodiment generates a base composite image (in which both are reflected by the magnitude of brightness) that reflects both the spatial information of the entire object represented by the B image and the flow image (for example, tissue structure, spread of blood flow, etc.), and then, for each image region, strongly reflects the color tone of the image between the B image and the flow image that has the greater local feature value of the object on the base composite image, thereby generating a composite image for display.

[0070] This makes it possible to generate a composite image for display that maintains the visibility of the spatial information of the entire object represented by each of the B image and flow image (e.g., tissue structure, blood flow spread, etc.) while improving the visibility of the local features of each of the B image and flow image (e.g., areas with large tissue movement or areas with fast blood flow speeds, etc.) (see Figure 14).

[0071] Moreover, according to the image synthesis method by the image processor 7 of this embodiment, the flow image is not overlaid on the B image, and the B image is not hidden. Therefore, in the color Doppler image generated by the image processor 7 of this embodiment, it is possible to visually confirm the relationship between the behavior of the blood flow shown in the flow image and the behavior of the tissue shown in the B image. In addition, in such a color Doppler image, even in an area where the velocity and power of the flow signal are small, the blood flow that changes successively is displayed without interruption, and the accurate position of a thin blood vessel is stably displayed.

[0072] (Variation 1) Fig. 19 is a diagram showing the configuration of an ultrasound diagnostic apparatus U (image processing unit 7) according to Modification 1. Fig. 20 is a diagram showing a B image color map referred to by a B image generating unit 71 according to Modification 1 (see Figs. 20B and 20C).

[0073] Usually, in the ultrasound diagnostic device U, colored B images are used not only for generating color Doppler images but also for independent display. Therefore, in the ultrasound diagnostic device U according to the above embodiment, the B image generating unit 71 uses a color map for B images (see FIG. 20A) to directly convert B signals (i.e., signal strength of reflected waves from inside the subject) into pixel values ​​of each color component, and generates a colored B image from the B signals.

[0074] In this case, however, when performing image synthesis processing of a B image and a flow image, as in the ultrasound diagnostic apparatus U according to the above embodiment, a process is required in which the B image generated as a color image by the B image generation unit 71 is divided into a color component image and a luminance component image by the division unit 73.

[0075] In the ultrasound diagnostic apparatus U according to this modification, in order to reduce the computational load when generating a color Doppler image, the B image generating unit 71 uses a first function f b (hereinafter, also referred to as the "luminance component image generation map") (see FIG. 20B) is used to generate a luminance component image of the B image, and a second function f c -B, f c -G, f c -R (hereinafter, also referred to as a "color component image generation map") (see FIG. 20C) is used to generate a color component image of the B image. This makes it possible to omit the configuration of the division unit 73.

[0076] In addition, the luminance component image generation map and color component image generation map referenced by the B image generation unit 71 in this modified example are set, for example, as shown in the following equations (12), (13), (14), and (15) so as to correspond to the color map for the B image in the above embodiment.

[0077] y = avg(R, G, B) … (12) (where y is the brightness of the brightness component image generation map, avg(R, G, B) is the average value of the RGB components of the color map for the B image)

[0078] CR=Ry …(13) CG=Gy …(14) CB=By …(15) (However, CR: R component of the color component image generation map, CG: G component of the color component image generation map, CB: B component of the color component image generation map, R: R component of the color map for B image, G: G component of the color map for B image, B: B component of the color map for B image)

[0079] Incidentally, as the image data format of the B image according to this modified example, a data format capable of storing luminance and chromaticity separately is used.

[0080] According to the ultrasound diagnostic device U of this modification, at the stage of generating the B image, it is possible to generate the luminance component image and the color component image of the B image separately. This makes it possible to omit the process of dividing the luminance component image and the color component image of the B image from the color image of the B image, thereby reducing the computation load when generating a color Doppler image.

[0081] (Variation 2) FIG. 21 is a diagram showing the configuration of an ultrasound diagnostic apparatus U (image processing unit 7) according to the second modification.

[0082] The image processing unit 7 of this modified example has multiple image synthesis units for synthesizing two colored medical images relating to the same part of the subject, and by cascading the multiple image synthesis units, it is possible to synthesize three or more medical images relating to the same part of the subject.

[0083] 21 shows a structure in which a first image synthesis unit 70a and a second image synthesis unit 70b are connected in a two-stage cascade. Each of the first image synthesis unit 70a and the second image synthesis unit 70b is composed of a division unit 73, a base synthesis image generation unit 74, a contribution determination unit 75, and a color correction unit 76. Each of the first image synthesis unit 70a and the second image synthesis unit 70b synthesizes two input medical images by the same method as that described in the above embodiment.

[0084] Here, the first image synthesis unit 70a receives the first medical image and the second medical image. The first image synthesis unit 70a synthesizes the first medical image and the second medical image to output a first synthetic image for display. The second image synthesis unit 70b receives the first synthetic image for display output from the first image synthesis unit 70a and a third medical image. The second image synthesis unit 70b synthesizes the third medical image and the first synthetic image for display to output a second synthetic image for display. The ultrasound diagnostic device U displays the second synthetic image for display on the display unit 8, for example.

[0085] In this way, by connecting in multiple stages, it becomes possible to synthesize three or more medical images. For example, if a B image is synthesized with a highlighted image of nerve tissue and then a flow image, it is possible to provide a medical image in which three types of information about the inside of the body, namely the internal structure, the location of the nerve tissue, and blood flow information, can be visually recognized just by looking at one medical image.

[0086] (Variation 3) The ultrasound diagnostic device U according to the above embodiment is configured to determine the degree of contribution of color that each pixel region of the B image should give to each pixel region of the base composite image based on the magnitude relationship between the luminance component of the pixel value of the B image and the luminance component of the pixel value of the flow image. This configuration is useful in that it makes it possible to distinguish, for each pixel region, a pixel region that gives a strong impression of the B image from a pixel region that gives a strong impression of the flow image, and to determine the degree of contribution in terms of gradation.

[0087] From the viewpoint of reducing the computation load, the above comparison process for determining the contribution degree may be omitted. That is, the contribution degree determining unit 75 may determine the contribution degree based only on the image information (luminance component or color component) of the flow image. In this case, the contribution degree determining unit 75 may determine the contribution degree so that, for example, the contribution degree is small for an area where the signal strength of the flow signal is large, and the contribution degree is large for an area where the signal strength of the flow signal is small.

[0088] In this case, however, there is a risk that the visibility of the tissue structure of image B may be partially deteriorated in the finally generated composite image for display. Therefore, the image synthesis method according to this modification is a method to be adopted when aiming at reduction of the computation load even at the expense of deterioration of image quality.

[0089] (Variation 4) In general, when determining whether a feature of a B image or a flow image is present in a given region on a color Doppler image, it is more useful to compare the luminance components than to compare only one color component (e.g., the R component) because the luminance component represents the signal intensity of each of the B signal and the flow signal.

[0090] However, from the perspective of ensuring visibility, when it is desired to avoid combining specific colors or when it is desired to emphasize the presence of a specific color, it may be more useful to compare the color components of the B image and the flow image rather than comparing the luminance components of the B image and the flow image.

[0091] From this viewpoint, the contribution degree determination unit 75 may be configured to determine the contribution degree based on the chromaticity in the first color space vector of the B image and the chromaticity in the second color space vector of the flow image. In this case, the contribution degree determination unit 75 may be configured to determine the contribution degree by comparing the chromaticity in the same color space vector between the B image and the flow image, or may be configured to determine the contribution degree by comparing the chromaticity in different color space vectors.

[0092] (Variation 5) In the above embodiment, the signal intensity of the B signal is converted into RGB components to configure the entire image area of ​​the B image as a color image. However, the B image applied to the image synthesis process according to the present disclosure may be one in which only a part of the entire image area is configured as a color image.

[0093] For example, the B image input to the image processing unit 7 may be obtained by coloring the area of ​​the puncture needle detected in a grayscale B image generated by converting the signal intensity of the B signal into luminance.

[0094] In addition, the B image input to the image processing unit 7 may be obtained by coloring the area of ​​nerve tissue detected in a grayscale B image generated by converting the signal intensity of the B signal into luminance.

[0095] Even when such an image is used as the B image, in the image synthesis method of the prior art, when the image is synthesized with the flow image, color mixing occurs in some image areas, which may degrade the visibility of the B image and / or the flow image. However, in accordance with the image synthesis method of the present disclosure, such deterioration in visibility is suppressed, and a composite image for display can be generated that improves the visibility of the respective features of the B image and the flow image.

[0096] (Variation 6) In the above embodiment, the contribution degree determining section 75 compares the luminance of the B image and the luminance of the flow image for each corresponding pixel region on a pixel-by-pixel basis, and determines the contribution degree for each pixel region.

[0097] However, the contribution degree determination unit 75 may compare the luminance of the B image and the luminance of the flow image for each pixel region that corresponds to each other, each pixel region being a section made up of a plurality of adjacent pixels, and determine the contribution degree for each pixel region to generate a contribution degree image. In this case, the contribution degree determination unit 75 may determine the contribution degree for each pixel region, for example, by using the average value of the luminance in the pixel regions to be compared.

[0098] Alternatively, the contribution degree determining section 75 may determine the contribution degree for each feature pattern appearing in the B image and / or for each feature pattern appearing in the flow image.

[0099] (Variation 7) In the above embodiment, the base composite image generating unit 74 generates the base composite image by adding the luminance component of the pixel value of each pixel region of the B image to the pixel value of each pixel region of the flow image.

[0100] However, the base composite image generating unit 74 may use other functions for image synthesis as long as it generates a base composite image by superimposing a luminance component image of the B image on the flow image. For example, it may use an operation that compares the luminance components of both images and adopts the larger value (such as a function max(a, b)), or an operation that adopts the smaller value (such as a function min(a, b)). However, in order to clearly express the spatial information of the entire object represented by each of the B image and the flow image, it is preferable that both the luminance component of the B image and the luminance component of the flow image are reflected on the base composite image.

[0101] (Variation 8) In the above embodiment, the chromaticity of the B image and the flow image is expressed in the RGB color space. However, the expression format of the chromaticity of the B image and the flow image is not limited to this, and the chromaticity of the B image and the flow image may be expressed in the HSV color space, the HSL color space, the YUV color space, or the like.

[0102] (Variation 9) In the above embodiment, the contribution degree determination unit 75 expresses the contribution degree as a scalar value, and the contribution degree for each color of the RGB components of the color component image is set to the same value. However, the contribution degree may be expressed as a vector in which a different weight can be set for each component of the color space vector. By doing so, it is possible to make the color of the blood flow particularly easy to see.

[0103] (Variation 10) In the above embodiment, image synthesis of a B image and a flow image generated by an ultrasound diagnostic device U is shown as an example of an application target of the image processing device according to the present disclosure. However, the image processing device according to the present disclosure can perform image synthesis on various other colored medical images. For example, such medical images may be medical images captured by an X-ray imaging device, medical images captured by an MRI inspection device, and medical images captured by a thermography imaging device.

[0104] (Variation 11) In the above embodiment, the image processing unit 7 is realized by a hardware configuration. However, a part or all of the configuration may be realized by a software configuration under program control of a microcomputer or the like.

[0105] Although the specific examples of the present invention have been described in detail above, these are merely examples and do not limit the scope of the claims. The technology described in the claims includes various modifications and alterations of the specific examples exemplified above. [Industrial Applicability]

[0106] According to the image processing device of the present disclosure, in a composite image for display generated by image-compositing two colored medical images relating to the same part of a subject, it is possible to improve the visibility of the features represented by each of the two medical images. [Explanation of symbols]

[0107] U Ultrasound diagnostic equipment 1 Ultrasonic probe 2. Transmitter 3. Receiving section 4 Scanning section 5 B signal generation section 6 Flow signal generator 7 Image Processing Section 70a First image synthesis unit 70b Second image synthesis unit 71 B Image Generation Unit 72 Flow image generation unit 73 Split part 74 Base synthetic image generation unit 75 Contribution Determination Unit 76 Color Correction Section 8 Display

Claims

1. 1. An image processing device that synthesizes first and second medical images related to the same part of a subject, the first and second medical images being expressed as color images, the image processing device comprising: a base composite image generating unit that generates a base composite image by performing image synthesis so as to superimpose a luminance component image, in which only a luminance component is extracted from the first medical image, on the second medical image; and a contribution determining unit that determines, for each pixel region, a contribution of color that each pixel region of the first medical image should give to each pixel region of the base composite image based on at least a luminance component or a color component of a pixel value of each pixel region of the second medical image; a color correction unit that generates a composite image for display by adding color components of pixel values ​​of each pixel region of the first medical image to pixel values ​​of each pixel region of the base composite image with weights according to the contribution rates; An image processing device comprising:

2. The contribution degree determination unit compares a luminance component or a color component of a pixel value of each pixel region of the second medical image with a luminance component or a color component of a pixel value of each pixel region of the first medical image for each corresponding pixel region, and determines the contribution degree for each pixel region. The image processing device according to claim 1 .

3. The contribution degree determiner determines the contribution degree in a gradational manner such that the contribution degree increases as the luminance component of the pixel value of each pixel region of the first medical image relative to the luminance component of the pixel value of each pixel region of the second medical image increases. The image processing device according to claim 2 .

4. The base composite image generating unit adds a luminance component of a pixel value of each pixel region of the first medical image to a pixel value of each pixel region of the second medical image to generate the base composite image. The image processing device according to claim 1 .

5. a division unit that divides pixel values ​​of each pixel region of the first medical image into a luminance component and a color component and generates a luminance component image and a color component image of the first medical image; The image processing device according to claim 1 .

6. a first medical image generating unit that generates the first medical image based on first sample values ​​related to in-vivo information at each position in the subject obtained by the first imaging process; a second medical image generating unit that generates the second medical image based on second sample values ​​related to in-vivo information at each position in the subject obtained by the second imaging process; The image processing device according to claim 1 , further comprising:

7. the first medical image generation unit generates a luminance component image of the first medical image using a first function that converts the first sample value into luminance, and generates a color component image of the first medical image using a second function that converts the first sample value into chromaticity; The image processing device according to claim 6.

8. The image processing device is applied to an ultrasound diagnostic device, the first medical image is a B image obtained by a first ultrasonic scan of the inside of the subject by the ultrasonic diagnostic device, The second medical image is a flow image obtained by a second ultrasonic scan of the subject by the ultrasonic diagnostic device. The image processing device according to claim 1 .

9. a plurality of image synthesis units each including the base synthesis image generation unit, the contribution degree determination unit, and the color correction unit; By cascading a plurality of the image synthesis units, it is possible to synthesize three or more medical images relating to the same part of the subject. The image processing device according to claim 1 .

10. 1. An image processing method for synthesizing first and second medical images of a same part of a subject, the first and second medical images being expressed as color images, the method comprising the steps of: a process of performing image synthesis so as to superimpose a luminance component image, in which only a luminance component is extracted from the first medical image, on the second medical image to generate a base synthetic image; determining, for each pixel region, a degree of contribution of color that each pixel region of the first medical image should give to each pixel region of the base composite image based on at least a luminance component or a color component of a pixel value of each pixel region of the second medical image; a process of generating a composite image for display by adding a color component of a pixel value of each pixel region of the first medical image to a pixel value of each pixel region of the base composite image with a weight according to the contribution degree; An image processing method comprising the steps of:

11. A program for causing a computer to execute image processing for image synthesis of first and second medical images relating to the same part of a subject, the first and second medical images being represented as color images, the program comprising: a process of performing image synthesis so as to superimpose a luminance component image, in which only a luminance component is extracted from the first medical image, on the second medical image to generate a base synthetic image; determining, for each pixel region, a degree of contribution of color that each pixel region of the first medical image should give to each pixel region of the base composite image based on at least a luminance component or a color component of a pixel value of each pixel region of the second medical image; a process of generating a composite image for display by adding a color component of a pixel value of each pixel region of the first medical image to a pixel value of each pixel region of the base composite image with a weight according to the contribution degree; An image processing program having the following steps:

Images