Ultrasound diagnostic device, image processing device, and program

Abstract

To generate a natural rendering image while keeping a calculation amount.SOLUTION: An ultrasonic diagnostic device includes a three-dimensional data generation section, an acquisition section, and a rendering section. The three-dimensional data generation section generates three-dimensional data on the basis of echo reflection intensity acquired by a probe. The acquisition section acquires a plurality of parameters and the three-dimensional data. The rendering section generates a rendering image of the three-dimensional data through the use of a color attenuation value considering propagation of light based on the echo reflection intensity and a base color parameter included in the parameters.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The embodiments disclosed in this specification and the drawings relate to an ultrasound diagnostic apparatus, an image processing apparatus, and a program. [Background technology]

[0002] A technology is known in which an ultrasound diagnostic device acquires the echo reflection intensity of ultrasound as three-dimensional data and generates a rendering image. Recently, rendering images are also generated based on global illumination, which is a rendering method that takes light sources and shadows into account in the calculation. When rendering using this method, in CT (Computed Tomography) and MRI (Magnetic Resonance Imaging), the data acquired by each device is associated with clinical tissue sites, so colors such as ambient light color, reflection color, and attenuation color are assigned to the data and rendering is performed. On the other hand, in an ultrasound diagnostic device, the echo reflection intensity is not associated with clinical tissue sites, so various colors cannot be assigned to the echo reflection intensity as in CT and MRI data. For this reason, a coloring method different from that of CT and MR is required.

[0003] In addition, in an ultrasound diagnostic device, since a rendering image is displayed in real time while a scan is being performed, it is desirable to suppress the amount of calculation, and reflection and attenuation in the optical characteristics are typically used for color specification. Increasing the number of optical characteristics to be calculated allows for the generation of a natural rendering image, but this comes with problems such as complicated color settings, heavy calculations, and the need for high-performance hardware.

[0004] Conventionally, when performing global illumination rendering using only reflection and attenuation, the following methods have been used, but each method has its own problems. One is a method in which a single reflection color and attenuation color are assigned to the echo reflection intensity above a certain threshold that constitutes the object being observed. However, with this method, especially when the light source is positioned directly in front, no clear shadow is cast on the image, so the whole is displayed in a single color, and the visibility of the contours can become extremely poor. The other is a method in which the reflection color is linked to the level of the echo reflection intensity to depict the contours. However, with this method, when the light source is moved, the color does not change and the contours continue to be depicted, which can impair the three-dimensional effect and reality. Therefore, there is a demand for generating natural rendering images while maintaining the amount of calculations. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2019-181168 A Summary of the Invention [Problem to be solved by the invention]

[0006] One of the problems to be solved by the embodiments disclosed in this specification and the drawings is to generate a natural rendering image while keeping the amount of calculations to a minimum. However, the problems to be solved by the embodiments disclosed in this specification and the drawings are not limited to the above problem. Problems corresponding to the effects of each configuration shown in the embodiments described later can also be positioned as other problems. [Means for solving the problem]

[0007] An ultrasound diagnostic apparatus according to an embodiment includes a three-dimensional data generating unit, an acquiring unit, and a rendering unit. The three-dimensional data generating unit generates three-dimensional data based on echo reflection intensity acquired by a probe. The acquiring unit acquires a plurality of parameters and the three-dimensional data. The rendering unit generates a rendering image of the three-dimensional data using a color attenuation value that takes into account light propagation based on the echo reflection intensity and a base color parameter included in the plurality of parameters. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a block diagram showing an example of the configuration of an ultrasonic diagnostic apparatus according to the first embodiment. [Diagram 2] FIG. 2 is a block diagram for explaining an overview of the global illumination image generation process in the first embodiment. [Diagram 3] FIG. 3 is a flowchart showing an example of the operation of a processing circuit that executes a global illumination image generation process in the first embodiment. [Figure 4] FIG. 4 is a block diagram illustrating the opacity curve generation process of FIG. [Diagram 5] FIG. 5 is a block diagram illustrating the color attenuation curve generation process of FIG. [Figure 6] FIG. 6 is a block diagram illustrating the light characteristics map generation process of FIG. [Figure 7] FIG. 7 is a block diagram illustrating the global illumination rendering process of FIG. [Figure 8] FIG. 8 is a diagram showing a global illumination image in which a light source is in front in the first embodiment. [Figure 9] FIG. 9 is a diagram showing a global illumination image in which the light source is on the left side in the first embodiment. [Figure 10] FIG. 10 is a block diagram illustrating an example of the configuration of an image processing apparatus according to the second embodiment. [Figure 11]FIG. 11 is a conventional view of a global illumination image with a frontal light source. [Figure 12] FIG. 12 is a conventional view of a global illumination image with the light source on the left. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an embodiment of an ultrasound diagnostic apparatus will be described in detail with reference to the drawings.

[0010] (First embodiment) Fig. 1 is a block diagram showing an example of the configuration of an ultrasonic diagnostic apparatus according to the first embodiment. The ultrasonic diagnostic apparatus 1 in Fig. 1 includes an apparatus main body 100 and an ultrasonic probe 101. The apparatus main body 100 is connected to an input device 102 and an output device 103. The apparatus main body 100 is also connected to an external device 104 via a network NW. The external device 104 is, for example, a server equipped with a PACS (Picture Archiving and Communication Systems).

[0011] The ultrasonic probe 101 performs an ultrasonic scan on a scan region in a living body P, which is a subject, under the control of the device main body 100, for example. The ultrasonic probe 101 has, for example, a plurality of piezoelectric transducers, a matching layer provided between the plurality of piezoelectric transducers and a case, and a backing material that prevents the propagation of ultrasonic waves from the plurality of piezoelectric transducers backward in the radiation direction. The ultrasonic probe 101 is, for example, a two-dimensional array probe in which a plurality of ultrasonic transducers are arranged along a first element array direction (elevation direction) and a second element array direction (azimuth direction). The ultrasonic probe 101 is detachably connected to the device main body 100. The ultrasonic probe 101 may be provided with a button that is pressed during offset processing, an operation for freezing an ultrasonic image (freeze operation), and the like.

[0012] The piezoelectric transducers generate ultrasonic waves based on a drive signal supplied from an ultrasonic transmission circuit 110 (described later) of the device main body 100. This causes ultrasonic waves to be transmitted from the ultrasonic probe 101 to the living body P. When ultrasonic waves are transmitted from the ultrasonic probe 101 to the living body P, the transmitted ultrasonic waves are reflected one after another at discontinuous surfaces of the acoustic impedance in the body tissue of the living body P, and are received as reflected wave signals by the piezoelectric transducers. The amplitude of the received reflected wave signal depends on the difference in acoustic impedance at the discontinuous surface where the ultrasonic waves are reflected. In addition, when the transmitted ultrasonic pulse is reflected by the surface of a moving blood flow or a heart wall, the reflected wave signal undergoes a frequency shift due to the Doppler effect depending on the velocity component in the ultrasonic transmission direction of the moving body. The ultrasonic probe 101 receives the reflected wave signal from the living body P and converts it into an electric signal.

[0013] 1 illustrates an example of the connection relationship between one ultrasonic probe 101 and the device main body 100. However, multiple ultrasonic probes can be connected to the device main body 100. Which of the multiple connected ultrasonic probes is to be used for ultrasonic scanning can be arbitrarily selected by, for example, a software button on a touch panel described later.

[0014] The device main body 100 is a device that generates an ultrasonic image based on a reflected wave signal received by an ultrasonic probe 101. The device main body 100 has an ultrasonic transmission circuit 110, an ultrasonic reception circuit 120, an internal storage circuit 130, an image memory 140, an input interface 150, an output interface 160, a communication interface 170, and a processing circuit 180.

[0015] The ultrasonic transmission circuit 110 is a processor that supplies a drive signal to the ultrasonic probe 101. The ultrasonic transmission circuit 110 is realized by, for example, a trigger generation circuit, a delay circuit, and a pulser circuit. The trigger generation circuit repeatedly generates a rate pulse for forming a transmission ultrasonic wave at a predetermined rate frequency. The delay circuit provides each rate pulse generated by the trigger generation circuit with a delay time for each of the multiple piezoelectric transducers required to focus the ultrasonic waves generated from the ultrasonic probe into a beam and determine the transmission directivity. The pulser circuit applies a drive signal (drive pulse) to the multiple ultrasonic transducers provided in the ultrasonic probe 101 at a timing based on the rate pulse. By changing the delay time provided to each rate pulse by the delay circuit, the transmission direction from the surfaces of the multiple piezoelectric transducers can be arbitrarily adjusted.

[0016] Furthermore, the ultrasonic transmission circuit 110 can arbitrarily change the output intensity of the ultrasonic waves by the drive signal. In the ultrasonic diagnostic device, the output intensity can be increased to reduce the influence of ultrasonic wave attenuation in the living body P. By reducing the influence of ultrasonic wave attenuation, the ultrasonic diagnostic device can obtain a reflected wave signal with a large S / N ratio at the time of reception.

[0017] Generally, when an ultrasonic wave propagates through a living body P, the strength of the ultrasonic vibration (also called acoustic power), which corresponds to the output intensity, attenuates. The attenuation of acoustic power occurs due to absorption, scattering, reflection, and the like. The degree of reduction in acoustic power depends on the frequency of the ultrasonic wave and the distance in the direction of ultrasonic radiation. For example, the degree of attenuation increases by increasing the frequency of the ultrasonic wave. Also, the degree of attenuation increases as the distance in the direction of ultrasonic radiation increases.

[0018] The ultrasonic receiving circuit 120 is a processor that performs various processes on the reflected wave signal received by the ultrasonic probe 101 to generate a received signal. The ultrasonic receiving circuit 120 generates a received signal based on the reflected wave signal of the ultrasonic wave acquired by the ultrasonic probe 101. Specifically, the ultrasonic receiving circuit 120 is realized by, for example, a preamplifier, an A / D converter, a demodulator, and a beamformer. The preamplifier amplifies the reflected wave signal received by the ultrasonic probe 101 for each channel and performs a gain correction process. The A / D converter converts the gain-corrected reflected wave signal into a digital signal. The demodulator demodulates the digital signal. For example, the beamformer gives a delay time required to determine the reception directivity to the demodulated digital signal and adds up multiple digital signals to which the delay time has been given. The addition process of the beamformer generates a received signal in which the reflection component from the direction according to the reception directivity is emphasized. In the following, the "ultrasonic reflected wave signal" and the "received signal" are collectively referred to as "echo signal". Therefore, the "strength of the received signal" may be rephrased as "the reflection strength of the echo signal (echo reflection strength)."

[0019] The internal storage circuit 130 has a processor-readable storage medium, such as a magnetic storage medium, an optical storage medium, or a semiconductor memory. The internal storage circuit 130 stores a program for implementing ultrasonic transmission and reception, a program related to a global illumination (GI) image generation process described later, and various data. The program and various data may be stored in the internal storage circuit 130 in advance, for example. The program and various data may be stored in a non-transient storage medium, distributed, read from the non-transient storage medium, and installed in the internal storage circuit 130. The internal storage circuit 130 stores B-mode image data, contrast image data, image data related to a blood flow image, and three-dimensional data generated by the processing circuit 180 according to an operation input via the input interface 150. The internal storage circuit 130 can also transfer the stored image data and three-dimensional data to an external device 104 or the like via the communication interface 170.

[0020] The internal storage circuit 130 may be a drive device that reads and writes various information between the internal storage circuit 130 and a portable storage medium such as a CD drive, a DVD drive, or a flash memory. The internal storage circuit 130 can also write the stored data to the portable storage medium and store the data in the external device 104 via the portable storage medium.

[0021] The image memory 140 has a processor-readable storage medium, such as a magnetic storage medium, an optical storage medium, or a semiconductor memory. The image memory 140 stores image data corresponding to a plurality of frames immediately before a freeze operation input via the input interface 150. The image data stored in the image memory 140 is displayed continuously (cine display), for example. The image memory 140 is not limited to storing image data, and may store three-dimensional data.

[0022] The internal storage circuit 130 and the image memory 140 do not necessarily have to be realized by independent storage devices. The internal storage circuit 130 and the image memory 140 may be realized by a single storage device. Also, the internal storage circuit 130 and the image memory 140 may each be realized by a plurality of storage devices.

[0023] The input interface 150 accepts various instructions from an operator via the input device 102. The input device 102 is, for example, a mouse, a keyboard, a panel switch, a slider switch, a trackball, a rotary encoder, an operation panel, and a touch panel. The input interface 150 is connected to the processing circuit 180 via, for example, a bus, converts an operation instruction input by the operator into an electric signal, and outputs the electric signal to the processing circuit 180. Note that the input interface 150 is not limited to only those that connect to physical operation components such as a mouse and a keyboard. For example, a circuit that receives an electric signal corresponding to an operation instruction input from an external input device provided separately from the ultrasound diagnostic device 1 and outputs the electric signal to the processing circuit 180 is also included as an example of the input interface.

[0024] The output interface 160 is, for example, an interface for outputting an electric signal from the processing circuit 180 to the output device 103. The output device 103 is any display such as a liquid crystal display, an organic EL display, an LED display, a plasma display, or a CRT display. The output device 103 may be a touch panel display that also serves as the input device 102. The output device 103 may further include a speaker that outputs sound in addition to the display. The output interface 160 is connected to the processing circuit 180 via, for example, a bus, and outputs an electric signal from the processing circuit 180 to the output device 103.

[0025] The communication interface 170 is connected to the external device 104 via a network NW, for example, and performs data communication with the external device 104 .

[0026] The processing circuitry 180 is, for example, a processor that functions as the core of the ultrasound diagnostic device 1. The processing circuitry 180 executes a program stored in the internal storage circuitry 130 to realize a function corresponding to the program. The processing circuitry 180 has, for example, a B-mode processing function 181, a Doppler processing function 182, an image generating function 183, a three-dimensional data generating function 184 (three-dimensional data generating unit), an acquisition function 185A (acquisition unit), an opacity curve generating function 185B, a color attenuation curve generating function 185C (color attenuation curve generating unit), a light characteristic map generating function 185D (light characteristic map generating unit), a rendering function 185E (rendering unit), a display control function 186 (display control unit), and a system control function 187. Note that the Doppler processing function 182 may be omitted from the functions of the processing circuitry 180 since it has little relation to this embodiment.

[0027] The B-mode processing function 181 is a function that generates B-mode data based on the reception signal (echo signal) received from the ultrasonic reception circuit 120. In the B-mode processing function 181, the processing circuit 180 performs, for example, envelope detection processing and logarithmic compression processing on the reception signal received from the ultrasonic reception circuit 120, and generates data (B-mode data) in which the signal strength (echo reflection strength) of the reception signal is expressed as a brightness value (luminance value). The generated B-mode data is stored in a RAW data memory (not shown) as B-mode RAW data on a two-dimensional ultrasonic scan line (raster).

[0028] Furthermore, the processing circuitry 180 can execute harmonic imaging by using the B-mode processing function 181. Harmonic imaging is an imaging method that uses not only fundamental wave components contained in the reflected wave signal of ultrasonic waves but also harmonic components (harmonic components). Harmonic imaging includes, for example, tissue harmonic imaging (THI) that does not use a contrast agent, and contrast harmonic imaging (CHI) that uses a contrast agent.

[0029] In THI, harmonic components can be extracted using an imaging method called the Amplitude Modulation (AM) method, the Phase Modulation (PM) method, or the AMPM method, which is a combination of the AM and PM methods.

[0030] In the AM, PM and AMPM methods, ultrasonic waves with different amplitudes and phases are transmitted multiple times along the same scanning line. This allows the ultrasonic receiving circuit 120 to generate multiple pieces of reflected wave data along each scanning line, and output the generated reflected wave data. The processing circuit 180 extracts harmonic components by performing addition and subtraction processing of the multiple pieces of reflected wave data along each scanning line according to the modulation method using the B-mode processing function 181. The processing circuit 180 then performs envelope detection processing and the like on the reflected wave data of the harmonic components to generate B-mode data.

[0031] In addition, in the CHI, for example, a frequency filter is used to extract harmonic components. The processing circuitry 180 can separate reflected wave data (harmonic components) whose reflection source is the contrast agent from reflected wave data (fundamental wave components) whose reflection source is tissue in the living body P by using a B-mode processing function 181. This allows the processing circuitry 180 to select harmonic components from the contrast agent using a filter and generate B-mode data for generating contrast image data.

[0032] The B-mode data for generating the contrast image data is data that represents the echo reflection intensity with the contrast agent as a reflection source as a brightness value. The processing circuit 180 can also extract a fundamental wave component from the reflected wave data of the living body P to generate B-mode data for generating tissue image data.

[0033] The Doppler processing function 182 is a function that generates data (Doppler information) that extracts motion information based on the Doppler effect of a moving object within a ROI (Region Of Interest) set in a scan region by performing frequency analysis on a reception signal received from the ultrasonic reception circuit 120. The generated Doppler information is stored in a RAW data memory (not shown) as Doppler RAW data (also called Doppler data) on a two-dimensional ultrasonic scan line.

[0034] Specifically, the processing circuitry 180 estimates, for example, an average velocity, an average variance value, an average power value, etc., as motion information of a moving body at each of a plurality of sample points using the Doppler processing function 182, and generates Doppler data indicating the estimated motion information. The moving body is, for example, blood flow, tissue such as a heart wall, or a contrast agent. The processing circuitry 180 according to this embodiment estimates, for each of a plurality of sample points using the Doppler processing function 182, an average velocity of blood flow, a variance value of blood flow velocity, a power value of blood flow signal, etc., as motion information of blood flow (blood flow information), and generates Doppler data indicating the estimated blood flow information.

[0035] The image generating function 183 is a function that generates B-mode image data based on data generated by the B-mode processing function 181. For example, in the image generating function 183, the processing circuitry 180 converts (scan converts) a scan line signal sequence of an ultrasonic scan into a scan line signal sequence of a video format represented by a television or the like, and generates image data for display (display image data). Specifically, the processing circuitry 180 performs RAW-pixel conversion on the B-mode RAW data stored in the RAW data memory, for example, coordinate conversion according to the scanning form of the ultrasonic wave by the ultrasonic probe 101, thereby generating two-dimensional B-mode image data (also referred to as ultrasonic image data) composed of pixels. In other words, the processing circuitry 180 generates a plurality of ultrasonic images (medical images) corresponding to a plurality of consecutive frames by transmitting and receiving ultrasonic waves using the image generating function 183.

[0036] Furthermore, the processing circuitry 180 generates Doppler image data in which blood flow information is visualized, for example, by performing RAW-pixel conversion on the Doppler RAW data stored in the RAW data memory. The Doppler image data is average velocity image data, variance image data, power image data, or image data combining these. The processing circuitry 180 generates, as the Doppler image data, color Doppler image data in which blood flow information is displayed in color, and Doppler image data in which one blood flow information is displayed in a grayscale wave shape.

[0037] The three-dimensional data generating function 184 is a function that generates three-dimensional B-mode data (three-dimensional data) based on the reception signal received from the ultrasound receiving circuit 120. In the three-dimensional data generating function 184, the processing circuit 180 generates three-dimensional data by using the B-mode data generated by the B-mode processing function 181 to assign brightness values ​​to voxels arranged in a three-dimensional space. This three-dimensional data may be called volume data. Since the brightness value corresponds to the echo reflection intensity, it may be interpreted that the echo reflection intensity is assigned to the voxels of the volume data. Therefore, hereinafter, the "brightness value of the volume data" may be used in a substantially similar sense to the "echo reflection intensity".

[0038] The acquisition function 185A is a function for acquiring data related to the GI image generation processing described later. Specifically, the acquisition function 185A causes the processing circuit 180 to acquire a plurality of parameters, such as parameters input by a user and parameters set as defaults in the ultrasound diagnostic device 1, and three-dimensional data. The plurality of parameters include, for example, an opacity parameter, a base color parameter, and a rendering parameter. A detailed description of these parameters will be given later.

[0039] The opacity curve generation function 185B is a function that generates an opacity curve. The opacity curve is a graph that associates echo reflection intensity with an opacity value. The opacity value is a value of opacity, and is expressed, for example, as a value from "zero" to "1." For example, an opacity value of "zero" represents transparency, and an opacity value of "1" represents opacity.

[0040] By the opacity curve generating function 185B, the processing circuit 180 generates an opacity curve based on the opacity parameters. The opacity parameters include, for example, a transfer function that defines the shape of the opacity curve and a threshold value that defines the boundary between transparent and opaque. This transfer function may be linear or nonlinear. A plurality of threshold values ​​may be provided, and for example, two boundaries, transparent and opaque, may be defined. In this embodiment, a graph of the opacity curve is represented by a look-up table (LUT) and used in the processing described later. The processing by the opacity curve generating function 185B may be called an opacity curve generating process.

[0041] The color attenuation curve generating function 185C is a function that generates a color attenuation curve. The color attenuation curve is a graph that associates an opacity value with a color attenuation value. The color attenuation value is a value that indicates how much the base color is attenuated, taking into account the propagation of light, and is provided for each element of the color expression format. That is, a color attenuation curve is provided for each element of the color expression format. In this embodiment, RGB is used as the color expression format. Note that the color expression format is not limited to RGB, and may be a color space (e.g., HSV and HLS) expressed by three components of hue, saturation, and lightness (or luminance), or a color space (e.g., YUV, YCbCr, and YPbPr) expressed using a luminance signal Y and two color difference signals.

[0042] The processing circuit 180 generates a color attenuation curve based on the base color parameters and the opacity curve by the color attenuation curve generating function 185C. The base color parameters include, for example, a combination of RGB (color map) that represents the color of the object (material) to be displayed. The base color may be interpreted as a reflected color that is a color reflected from an object. The shape of the color attenuation curve has at least one different shape among a plurality of elements. Therefore, there are two or more transfer functions that define the shape of the color attenuation curve. In addition, this transfer function may be linear or nonlinear. In other words, the processing circuit 180 generates a color attenuation curve by using at least two different transfer functions for each element of the color expression format. In this embodiment, the graph of the color attenuation curve is represented by an LUT and is used in the processing described later. In addition, the processing by the color attenuation curve generating function 185C may be called a color attenuation curve generating process.

[0043] The light characteristics map generating function 185D is a function for generating a light characteristics map. A light characteristics map is a function in which a physical phenomenon (e.g., attenuation of color) expected when light hits a material is quantified and stored in voxels arranged in a three-dimensional space. In this embodiment, a photon map of a photon mapping method described later is generated as the light characteristics map.

[0044] The above photon is a definition in the photon mapping method, which discretizes light to represent it in a computer and carries light energy per unit time. In this photon mapping method, a number of photons set in advance in the system or a number of photons set by the user are collided in three-dimensional data and placed in a scene. The scene is a three-dimensional space in which a light characteristic map is created.

[0045] The light characteristic map generating function 185D causes the processing circuit 180 to generate a light characteristic map based on the opacity curve, the color attenuation curve, the rendering parameters, and the three-dimensional data. The rendering parameters include, for example, information on the position of the light source directed toward the material. The processing by the light characteristic map generating function 185D may be referred to as a light characteristic map generating process.

[0046] The rendering function 185E is a function for generating a rendering image. The rendering image includes, for example, a volume rendering image and a global illumination image. In this embodiment, a rendering image that does not take a light source into consideration is defined as a volume rendering image, and a rendering image that takes a light source into consideration is defined as a global illumination image.

[0047] A volume rendering image is obtained by performing volume rendering on volume data. In volume rendering, the brightness and color of each voxel on display are set according to the brightness value assigned to the voxel of the volume data (the brightness value of the volume data). Then, volume rendering displays a projection image of the voxel projected from an arbitrary viewpoint.

[0048] On the other hand, a global illumination image is generated by performing a rendering process using the above-mentioned photon map. For the rendering process, for example, a ray tracing method is used. In this embodiment, a global illumination image is generated as the rendering image.

[0049] The rendering function 185E causes the processing circuit 180 to generate a global illumination image based on the opacity curve, the base color parameters, the rendering parameters, the light characteristic map, and the three-dimensional data. The rendering parameters include, for example, information on the position of the viewpoint. The processing by the rendering function 185E may be called a global illumination rendering process.

[0050] The display control function 186 is a function that displays an image based on various ultrasound image data generated by the image generation function 183 on a display as the output device 103. Specifically, for example, the processing circuitry 180 uses the display control function 186 to control the display of an image based on image data including B-mode image data, Doppler image data, or both generated by the image generation function 183 on the display.

[0051] More specifically, the processing circuitry 180 converts (scan converts) a scan line signal sequence of an ultrasonic scan into a scan line signal sequence of a video format represented by a television or the like, using the display control function 186, to generate display image data. The processing circuitry 180 may also execute various processes, such as dynamic range, brightness, contrast, and gamma curve correction, as well as RGB conversion, on the display image data. The processing circuitry 180 may also add supplementary information, such as text information of various parameters, scales, and body marks, to the display image data. The processing circuitry 180 may also generate a user interface (GUI: Graphical User Interface) for an operator to input various instructions via an input device, and display the GUI on a display.

[0052] Furthermore, the processing circuit 180 may display a global illumination image (GI image) generated by the rendering function 185E using the display control function 186. The processing circuit 180 may display a GUI related to the setting of the GI image together with the GI image. This allows the user to change the parameters displayed on the GUI in real time to change the GI image to a desired display. The parameters that the user can change are, for example, an opacity parameter, a base color parameter, and a rendering parameter.

[0053] The system control function 187 is a function that comprehensively controls the operation of the entire ultrasound diagnostic apparatus 1. For example, in the system control function 187, the processing circuitry 180 controls the ultrasound transmission circuitry 110 and the ultrasound reception circuitry 120 based on parameters related to the transmission and reception of ultrasound.

[0054] Above, a description has been given of the configuration of the ultrasound diagnostic apparatus in the first embodiment. Next, an overview and operation of the global illumination image generation processing in the first embodiment will be described.

[0055] 2 is a block diagram for explaining an overview of the global illumination image generation process in the first embodiment. The processing circuit 180 generates a global illumination image by executing a global illumination image generation process 200 based on the acquired parameters and three-dimensional data. The global illumination image generation process 200 includes, for example, an opacity curve generation process 210, a color attenuation curve generation process 220, a light characteristic map generation process 230, and a global illumination rendering process 240. Note that the opacity curve generation process 210 and the color attenuation curve generation process 220 may generate an opacity curve and a color attenuation curve based on default parameters if the user does not set parameters related to these processes.

[0056] Fig. 3 is a flowchart showing an example of the operation of a processing circuit that executes a global illumination image generation process in the first embodiment. The global illumination image generation process in Fig. 3 is started, for example, when a user executes a mode (illumination mode) that displays a global illumination image. Note that the flowchart in Fig. 3 explains the process for one frame.

[0057] (Step ST110) When the global illumination image generation process is started, the processing circuitry 180 executes the acquisition function 185A. When the acquisition function 185A is executed, the processing circuitry 180 acquires a plurality of parameters and three-dimensional data. The plurality of parameters include at least one of parameters input by a user and parameters set as defaults in the ultrasound diagnostic device 1. The three-dimensional data is generated by the three-dimensional data generation function 184 based on the echo reflection intensity acquired in real time by the ultrasound probe 101.

[0058] (Step ST120) After acquiring the parameters and the three-dimensional data, the processing circuit 180 executes the opacity curve generating function 185B. When the opacity curve generating function 185B is executed, the processing circuit 180 generates an opacity curve based on the opacity parameters. The opacity curve generating process by the opacity curve generating function 185B will be described below with reference to FIG. 4.

[0059] Fig. 4 is a block diagram for explaining the opacity curve generation process of Fig. 2. The processing circuitry 180 generates an opacity curve by executing the opacity curve generation process 210 on the opacity parameters. Specifically, the opacity curve generation process 210 uses a transfer function and a threshold value included in the opacity parameters to generate an LUT related to an opacity curve with the echo reflection intensity as a key. The processing circuitry 180 calculates the value of the opacity curve LUT using, for example, the following formula (1).

[0060]

number

[0061] (Step ST130) After generating the opacity curve, the processing circuit 180 executes the color attenuation curve generating function 185C. When the color attenuation curve generating function 185C is executed, the processing circuit 180 generates a color attenuation curve based on the base color parameters and the opacity curve. The base color parameters are set based on a preset color map or a color map specified by the user. The color attenuation curve generating process by the color attenuation curve generating function 185C will be described below with reference to FIG. 5.

[0062] Fig. 5 is a block diagram for explaining the color attenuation curve generation process of Fig. 2. The processing circuit 180 generates a color attenuation curve by executing the color attenuation curve generation process 220 on the base color parameters and the opacity curve. Specifically, the color attenuation curve generation process 220 creates an LUT for a color attenuation curve using the opacity value as a key for each of RGB included in the base color parameters. The processing circuit 180 calculates the value of the color attenuation curve LUT corresponding to each of RGB using, for example, the following formulas (2) to (4).

[0063]

number

[0064]

number

[0065]

number

[0066] In formulas (2) to (4), Extinction[I][R] indicates the color attenuation value for the red (R) component. Similarly, Extinction[I][G] indicates the color attenuation value for the green (G) component, and Extinction[I][B] indicates the color attenuation value for the blue (B) component. Er, Eg, and Eb indicate the numerical values ​​of each RGB element. The numerical values ​​of the elements are expressed in floating-point format. For example, when each RGB element is expressed in 8 bits, the numerical values ​​from "zero" to "255" are treated as corresponding to the numerical values ​​from "zero" to "1.0". The color attenuation value is also expressed in floating-point format.

[0067] When Opacity[I] is greater than zero, the calculation of Extinction[I][R] is different from Extinction[I][G] and Extinction[I][B]. Specifically, Extinction[I][R] has a smaller rate of decrease in value than Extinction[I][G] and Extinction[I][G]. This corresponds to a different shape of the color decay curve.

[0068] (Step ST140) After generating the color attenuation curve, the processing circuit 180 executes the light characteristics map generating function 185D. When the light characteristics map generating function 185D is executed, the processing circuit 180 generates a light characteristics map based on the opacity curve, the color attenuation curve, the rendering parameters, and the three-dimensional data. The light characteristics map generating process by the light characteristics map generating function 185D will be described below with reference to FIG. 6.

[0069] Fig. 6 is a block diagram for explaining the light characteristics map generation process of Fig. 2. The processing circuit 180 generates a light characteristics map by executing a light characteristics map generation process 230 on the opacity curve, the color attenuation curve, the rendering parameters, and the three-dimensional data. Specifically, the light characteristics map generation process 230 applies a photon mapping method using the information on the position of the light source included in the rendering parameters and the three-dimensional data to generate a photon map.

[0070] In addition, various parameters can be set for each photon in the photon map. In this embodiment, a color attenuation value for each of RGB is assigned to each photon. Specifically, the processing circuit 180 reads out an opacity value and a color attenuation value based on an echo reflection intensity value included in a voxel of three-dimensional data corresponding to a position of a scene where the photon is placed, an opacity curve LUT, and a color attenuation curve LUT. Then, the processing circuit 180 assigns a color attenuation value to each photon.

[0071] (Step ST150) After generating the light characteristic map, the processing circuitry 180 executes the rendering function 185E. When the rendering function 185E is executed, the processing circuitry 180 generates a global illumination image based on the opacity curve, the base color parameters, the rendering parameters, the light characteristic map, and the three-dimensional data. The global illumination rendering process by the rendering function 185E will be described below with reference to FIG. 7.

[0072] Fig. 7 is a block diagram for explaining the global illumination rendering process of Fig. 2. The processing circuit 180 generates a global illumination image by executing a global illumination rendering process 240 on the opacity curve, the base color parameters, the rendering parameters, the light characteristic map, and the three-dimensional data. Specifically, the global illumination rendering process 240 generates a projection image (global illumination image) by projecting the voxels of the three-dimensional data to which the photon map is applied, based on the information on the position of the viewpoint included in the rendering parameters. An opacity value based on the echo reflection intensity and a base color are set for the voxels on the projection surface.

[0073] Furthermore, the display color of the global illumination image is calculated by applying the color attenuation value included in the photon map to the numerical values ​​of each of the RGB elements included in the base color parameters. When the color attenuation value is calculated using the above formulas (2) to (4), the display color will have greater attenuation of G and B compared to R, so that a large amount of R component will remain in places where light attenuation occurs, such as shadow areas. In other words, the color tone changes according to the echo reflection intensity.

[0074] (Step ST160) After generating the global illumination image, the processing circuit 180 causes the display control function 186 to display the global illumination image on a display serving as the output device 103. After the processing of step ST160, the global illumination image generation processing ends.

[0075] The process of FIG. 3 may be repeatedly executed until the user ends the illumination mode or changes to another mode.

[0076] Fig. 8 is a diagram showing a global illumination image in the first embodiment in which the light source is in front. In Fig. 8, the viewing direction is also in front, so the light from the light source travels in the same direction as the viewing direction. In the global illumination image 800 in Fig. 8, changes in color tone are represented by gray, and shadow areas are represented by hatching. This is the same for the subsequent global illumination images.

[0077] The right hand of a fetus is depicted in a global illumination image 800 of FIG. 8. Focusing on the palm 810 of the right hand, a color change occurs in a part different from the shaded part. This makes the shape of the palm clear in the palm 810. Focusing on a cross-section 820 of the right hand, similar to the palm 810, there is a color change, so the shape of the structure (e.g., bones) can be clearly seen. On the other hand, such a color change does not occur in conventional methods. Below, the conventional diagram will be explained with reference to FIG. 11.

[0078] FIG. 11 is a conventional diagram of a global illumination image with a light source in front. In the global illumination image 1100 of FIG. 11, the right hand of a fetus is drawn in the same way as in the global illumination image 800. When focusing on the palm 1110 of the right hand, the shadowed part is expressed, but the shape of the palm is unclear because there is no change in color. In addition, when focusing on the cross-section 1120 of the right hand, the shape of the structure cannot be captured because there is no change in color, just like in the palm 1110.

[0079] Fig. 9 is a diagram showing a global illumination image in which the light source is on the left side in the first embodiment. In Fig. 9, since the viewing direction is the front, the light from the light source travels from left to right with respect to the viewing direction.

[0080] In the global illumination image 900 of FIG. 9, the whole body of a fetus is drawn. When focusing on the fetus's head 910, the left side of the face is in shadow, making the outline of the face unclear. When focusing on a structure 920 other than the fetus, the boundary can be identified by the change in color. On the other hand, in the conventional method, the outline is emphasized, which may result in an unnatural appearance. The conventional image will be described below with reference to FIG. 12.

[0081] Fig. 12 is a conventional diagram of a global illumination image with the light source on the left side. In the global illumination image 1200 of Fig. 12, the whole body of a fetus is drawn, similar to the global illumination image 900. When focusing on the head 1210 of the fetus, the contours of the left head, left cheek, and left ear are drawn as clear edges, which makes it look unnatural. In addition, when focusing on a structure 1220 other than the fetus, edges are drawn, similar to the head 1210, even though it is a smooth area.

[0082] In summary, in generating a global illumination image, the ultrasound diagnostic device according to the first embodiment links the degree of color attenuation to the echo reflection intensity, and sets the reflected color as a single color that is not linked to the echo reflection intensity. In addition, this ultrasound diagnostic device links a color attenuation curve that specifies the degree of color attenuation to an opacity curve, so that the higher the echo, the more the color is transmitted, and the lower the echo, the more the color is not transmitted. As a result, in the generated global illumination image, at the boundary where the echo reflection intensity changes greatly, not only a simple change in color shade but also a change in RGB value occurs, so that the boundary part is depicted as a color change and can be captured as a natural contour. On the other hand, the shadow part due to the way the light hits, i.e., the position of the light source, is depicted in the same way as in the past, so the reality of the change in the shadow is maintained. Therefore, this ultrasound diagnostic device can obtain a result that achieves both realism and improved visibility compared to the past with the same amount of calculation as the calculation amount using the same color attenuation and reflected color as in the past.

[0083] As described above, the ultrasound diagnostic apparatus according to the first embodiment generates three-dimensional data based on the echo reflection intensity acquired by the probe, acquires a plurality of parameters and the three-dimensional data, and generates a rendering image of the three-dimensional data using a color attenuation value that takes into account the propagation of light based on the echo reflection intensity and a base color parameter included in the plurality of parameters.

[0084] Therefore, the ultrasound diagnostic device according to the first embodiment can generate a natural rendering image while maintaining the amount of calculation by making the attenuation of color correspond to the echo reflection intensity. Since the amount of calculation is the same as when the attenuation of color is not made to correspond to the echo reflection intensity, natural contours can be depicted at the edge of a structure without impairing real-time performance. In addition, since color variations occur due to the characteristics of speckles in ultrasound images, not only at the edge of a structure, the multiple colors are transparent, providing an effect approximating actual color unevenness, and also providing the effect of making smooth parts of the structure look more natural.

[0085] Second embodiment In the first embodiment, a description has been given of an image processing apparatus having a plurality of functions relating to global illumination image generation processing. In the second embodiment, a description will be given of an image processing apparatus having these plurality of functions.

[0086] 10 is a block diagram showing an example of the configuration of an image processing device according to the second embodiment. The image processing device 300 in FIG. 10 is connected to an input device 301 and an output device 302. The image processing device 300 is also connected to a medical imaging device 303 via a network NW. The medical imaging device 303 corresponds to, for example, an ultrasonic diagnostic device. The input device 301 and the output device 302 are substantially similar to the input device 102 and the output device 103 in FIG. 1.

[0087] The image processing device 300 is a device that executes a global illumination image generation process and generates a global illumination image. The image processing device 300 has a memory circuit 310, an input interface 320, an output interface 330, a communication interface 340, and a processing circuit 350. The storage circuitry 310 includes a processor-readable storage medium, such as a magnetic storage medium, an optical storage medium, or a semiconductor memory. The storage circuitry 310 stores a program related to the global illumination image generation process, various data, and the like. The program and various data may be stored in the storage circuitry 310 in advance, for example. Also, for example, the program and various data may be stored in a non-transient storage medium and distributed, and may be read from the non-transient storage medium and installed in the storage circuitry 310. The storage circuitry 310 also stores B-mode image data, contrast image data, image data related to blood flow images, and three-dimensional data generated by the medical imaging device 303 according to an operation input via the input interface 320.

[0088] The storage circuit 310 may be a drive device that reads and writes various information between the storage circuit 310 and a portable storage medium such as a CD drive, a DVD drive, or a flash memory. The storage circuit 310 can write stored data to the portable storage medium and store the data in an external device via the portable storage medium.

[0089] The input interface 320 accepts various instructions from an operator via the input device 301. The input device 301 is, for example, a mouse, a keyboard, a panel switch, a slider switch, a trackball, a rotary encoder, an operation panel, and a touch panel. The input interface 320 is connected to the processing circuit 350 via, for example, a bus, converts an operation instruction input by the operator into an electric signal, and outputs the electric signal to the processing circuit 350. Note that the input interface 320 is not limited to only those that connect to physical operation components such as a mouse and a keyboard. For example, a circuit that receives an electric signal corresponding to an operation instruction input from an external input device provided separately from the image processing device 300 and outputs the electric signal to the processing circuit 350 is also included as an example of the input interface.

[0090] The output interface 330 is an interface for outputting, for example, an electrical signal from the processing circuit 350 to the output device 302. The output device 302 is any display such as a liquid crystal display, an organic EL display, an LED display, a plasma display, or a CRT display. The output device 302 may be a touch panel display that also serves as the input device 301. The output device 302 may further include a speaker that outputs sound in addition to the display. The output interface 330 is connected to the processing circuit 350 via, for example, a bus, and outputs an electrical signal from the processing circuit 350 to the output device 302.

[0091] The communication interface 340 is connected to the medical imaging apparatus 303 via, for example, a network NW, and performs data communication with the medical imaging apparatus 303 .

[0092] The processing circuitry 350 is, for example, a processor that functions as the core of the image processing device 300. The processing circuitry 350 executes a program stored in the storage circuitry 310 to realize a function corresponding to the program. The processing circuitry 350 has, for example, an acquisition function 351A (acquisition unit), an opacity curve generation function 351B, a color attenuation curve generation function 351C (color attenuation curve generation unit), a light characteristics map generation function 351D (light characteristics map generation unit), a rendering function 351E (rendering unit), and a display control function 352 (display control unit).

[0093] The acquisition function 351A is a function for acquiring data related to the GI image generation process. Specifically, the acquisition function 351A enables the processing circuit 350 to acquire parameters input by a user, parameters set as defaults in the image processing device 300, and three-dimensional data from the medical imaging device 303.

[0094] In addition, each of the opacity curve generation function 351B, the color attenuation curve generation function 351C, the light characteristics map generation function 351D, the rendering function 351E, and the display control function 352 has substantially the same functions as, for example, the opacity curve generation function 185B, the color attenuation curve generation function 185C, the light characteristics map generation function 185D, the rendering function 185E, and the display control function 186 in the first embodiment.

[0095] Therefore, the image processing device according to the second embodiment can be expected to provide the same effects as those of the first embodiment.

[0096] According to at least one of the embodiments described above, a natural rendering image can be generated while keeping the amount of calculations to a minimum.

[0097] Although some embodiments have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, modifications, and combinations of embodiments can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope of the invention and its equivalents as described in the claims, as well as in the scope and spirit of the invention. [Explanation of symbols]

[0098] 1. Ultrasound diagnostic equipment 100 Device body 101 Ultrasound probe 102,301 Input Devices 103,302 Output Device 104 External device 110 Ultrasonic transmission circuit 120 Ultrasonic receiving circuit 130 Internal memory circuit 140 Image Memory 150,320 Input Interface 160,330 Output Interface 170,340 Communication Interface 180,350 Processing circuit 181 B-mode processing function 182 Doppler processing function 183 Image generation function 184 3D data generation function 185A,351A acquisition function 185B, 351B Opacity curve generation function 185C, 351C Color attenuation curve generation function 185D, 351D Light characteristic map generation function 185E,351E Rendering function 186,352 Display control functions 187 System Control Functions 200 Global illumination image generation processing 210 Opacity curve generation process 220 Color attenuation curve generation process 230 Light characteristic map generation processing 240 Global illumination rendering process 300 Image processing device 303 Medical Imaging Equipment 310 Memory circuit 800,900,1100,1200 Global illumination images 810,1110 Palm 820,1120 Cross section 910,1210 head 920,1220 Structure NW Network

Claims

1. a three-dimensional data generating unit that generates three-dimensional data based on the echo reflection intensity acquired by the probe; an acquisition unit that acquires a plurality of parameters and the three-dimensional data; a rendering unit that generates a rendering image of the three-dimensional data by using a color attenuation value that takes into account the propagation of light based on the echo reflection intensity and a base color parameter included in the plurality of parameters; Equipped with a color attenuation curve generating unit that generates a color attenuation curve based on the base color parameters and an opacity curve that associates the echo reflection intensity and the opacity value; Further comprising: the color attenuation value is calculated based on the echo reflection intensity, the opacity curve, and the color attenuation curve; The rendering unit generates the rendering image by performing a global illumination rendering process on the three-dimensional data.

2. The color attenuation curve corresponds to the opacity value and the color attenuation value. The ultrasonic diagnostic apparatus according to claim 1 .

3. The plurality of parameters further includes a rendering parameter including information about a position of a light source directed toward the object; a light characteristic map generating unit that generates a light characteristic map storing the color attenuation values ​​based on the opacity curve, the color attenuation curve, the rendering parameters, and the three-dimensional data; Further comprising: The rendering parameters further include information on a viewpoint position, the rendering unit generates the rendering image based on the opacity curve, the base color parameters, the rendering parameters, the light characteristics map, and the three-dimensional data. The ultrasonic diagnostic apparatus according to claim 2 .

4. The light characteristics map generating unit generates the light characteristics map using a photon mapping method. The ultrasonic diagnostic apparatus according to claim 3.

5. The color attenuation curve is provided for each element of the color expression format. The ultrasonic diagnostic apparatus according to any one of claims 2 to 4.

6. The color representation format is any one of RGB, HSV, HLS, YUV, YCrCb, and YPbPr. The ultrasonic diagnostic apparatus according to claim 5 .

7. the color attenuation curve generating unit generates the color attenuation curve by using at least two different transfer functions for each element of the color expression format.

7. The ultrasonic diagnostic apparatus according to claim 5 or 6.

8. The base color parameters are set based on a preset color map or a color map specified by the user. The ultrasonic diagnostic apparatus according to any one of claims 1 to 7.

9. an acquisition unit that acquires three-dimensional data generated based on a plurality of parameters and echo reflection intensities acquired by a probe; a rendering unit that generates a rendering image of the three-dimensional data by using a color attenuation value that takes into account the propagation of light based on the echo reflection intensity and a base color parameter included in the plurality of parameters; Equipped with a color attenuation curve generating unit that generates a color attenuation curve based on the base color parameters and an opacity curve that associates the echo reflection intensity and the opacity value; Further comprising: The color attenuation value is calculated based on the echo reflection intensity, the opacity curve, and the color attenuation curve. The rendering unit generates the rendering image by executing a global illumination rendering process on the three-dimensional data.

10. Computer, a first means for acquiring three-dimensional data generated based on a plurality of parameters and echo reflection intensities acquired by a probe; a second means for generating a rendering image of the three-dimensional data by using a color attenuation value that takes into account the propagation of light based on the echo reflection intensity and a base color parameter included in the plurality of parameters; Functioning as a a third means for generating a color attenuation curve based on the base color parameters and an opacity curve that associates the echo reflection intensity and the opacity value; Further functioning as The color attenuation value is calculated based on the echo reflection intensity, the opacity curve, and the color attenuation curve. The second means is a program for generating the rendered image by executing a global illumination rendering process on the three-dimensional data.

Images