Ultrasound diagnostic device, image processing device, method and procedure

Abstract

An ultrasound diagnostic apparatus is provided that generates natural-looking plotted images while maintaining computational efficiency. The ultrasound diagnostic apparatus of the described embodiment includes a three-dimensional data generation unit, an acquisition unit, and a plotting unit. The three-dimensional data generation unit generates three-dimensional data based on echo reflection intensity acquired by the probe. The acquisition unit acquires multiple parameters and the three-dimensional data. The plotting unit generates a plotted image of the three-dimensional data using color attenuation values ​​that take into account the propagation of light based on echo reflection intensity and primary color parameters included in the multiple parameters.

Description

[0001] Cross-reference to related applications

[0002] This application claims priority based on Japanese Patent Application No. 2021-067854, filed on April 13, 2021, the entire contents of which are incorporated herein by reference. Technical Field

[0003] The embodiments disclosed in this specification and accompanying drawings relate to ultrasound diagnostic apparatus, image processing apparatus, methods, and procedures. Background Technology

[0004] Techniques exist for generating rendering images by acquiring the echo reflection intensity of ultrasound waves using ultrasound diagnostic devices as three-dimensional data. Furthermore, recently, rendering images based on global illumination have been developed, a method that considers light sources and shadows during computation. When using this method, in CT (Computed Tomography) or MRI (Magnetic Resonance Imaging), since the data acquired by each device is correlated with clinical tissue sites, colors such as ambient light color, reflected color, and attenuation color are assigned to the data before rendering. However, in ultrasound diagnostic devices, since the echo reflection intensity is not correlated with clinical tissue sites, various colors cannot be assigned to the echo reflection intensity as in CT or MRI data. Therefore, a different coloring method than CT or MRI is required.

[0005] Furthermore, in ultrasound diagnostic devices, since images are displayed in real time while scanning is being performed, it is desirable to minimize computational load. Therefore, in color specification, reflection and attenuation from the optical properties are used representatively. While increasing the number of optical properties to be calculated, generating natural-looking images presents challenges: color settings become more complex, computation becomes more demanding, and high-performance hardware becomes required.

[0006] Previously, when rendering global illumination using only reflection and attenuation, the following methods were used, each with its own technical problems. One method assigned a single reflection color and attenuation color to the echo reflection intensity above a certain threshold constituting the observed object. However, this method, especially when the light source is located directly opposite, suffers from poor outline clarity because no clear shadows fall on the image, resulting in the entire image being represented by a single color. Another method depicted outlines by linking the reflection color to the echo reflection intensity. However, this method suffers from the problem of the color not changing as the light source moves, continuously depicting outlines and compromising three-dimensionality and realism. Therefore, there is a requirement to generate a natural-looking rendered image while maintaining computational efficiency.

[0007] Existing technical documents

[0008] Japanese Patent Application Publication No. 2019-181168 Summary of the Invention

[0009] The technical problem to be solved by this invention is to generate natural rendering images while maintaining computational load.

[0010] The ultrasound diagnostic device according to the technical solution includes a three-dimensional data generation unit, an acquisition unit, and a rendering unit. The three-dimensional data generation unit generates three-dimensional data based on the echo reflection intensity acquired by the probe. The acquisition unit acquires multiple parameters, including primary color parameters, and three-dimensional data. The rendering unit uses color attenuation values, taking into account the propagation of light based on the echo reflection intensity, and the primary color parameters to generate a rendered image of the three-dimensional data.

[0011] Invention Effects

[0012] Generate natural-looking rendering images while maintaining computational efficiency. Attached Figure Description

[0013] Figure 1 This is a block diagram showing an example of the configuration of the ultrasound diagnostic apparatus according to the first embodiment.

[0014] Figure 2 This is a block diagram illustrating the general outline of the global illumination image generation process in the first embodiment.

[0015] Figure 3 This is a flowchart illustrating an example of the operation of the processing circuit performing the global illumination image generation process of the first embodiment.

[0016] Figure 4 This is an explanation Figure 2 A flowchart for generating and processing the opacity curve.

[0017] Figure 5 This is an explanation Figure 2 A flowchart for generating and processing color attenuation curves.

[0018] Figure 6 This is an explanation Figure 2 A block diagram of the optical property mapping generation process.

[0019] Figure 7 This is an explanation Figure 2 A block diagram of global illumination rendering.

[0020] Figure 8 This is a diagram showing a global illumination image of the light source in the first embodiment being in front.

[0021] Figure 9 This is a diagram showing a global illumination image with the light source on the left side in the first embodiment.

[0022] Figure 10 This is a block diagram illustrating a configuration example of the image processing apparatus according to the second embodiment.

[0023] Figure 11 This is a previous image illustrating a global illumination image with the light source facing forward.

[0024] Figure 12 This is a previous diagram illustrating a global illumination image with the light source on the left.

[0025] Label Explanation

[0026] 1…Ultrasound diagnostic device; 100…Device body; 101…Ultrasound probe; 102, 301…Input devices; 103, 302…Output devices; 104…External devices; 110…Ultrasound transmitting circuit; 120…Ultrasound receiving circuit; 130…Internal storage circuit; 140…Image memory; 150, 320…Input interface; 160, 330…Output interface; 170, 340…Communication interface; 180, 350…Processing circuit; 181…B-mode processing function; 182…Doppler processing function; 183…Image generation function; 184…3D data generation function; 185A, 351A…Acquisition function; 185B, 351B…Opacity curve generation function; 185C, 351C… Color decay curve generation function; 185D, 351D… Light property mapping generation function; 185E, 351E… Drawing function; 186, 352… Display control function; 187… System control function; 200… Global illumination image generation and processing; 210… Opacity curve generation and processing; 220… Color decay curve generation and processing; 230… Light property mapping generation and processing; 240… Global illumination drawing processing; 300… Image processing device; 303… Medical camera device; 310… Storage circuit; 800, 900, 1100, 1200… Global illumination image; 810, 1110… Palm; 820, 1120… Cross-section; 910, 1210… Head; 920, 1220… Structure; NW… Network. Detailed Implementation

[0027] The ultrasound diagnostic apparatus of the embodiments described herein includes a three-dimensional data generation unit, an acquisition unit, and a rendering unit. The three-dimensional data generation unit generates three-dimensional data based on the echo reflection intensity acquired by the probe. The acquisition unit acquires multiple parameters, including primary color parameters, and three-dimensional data. The rendering unit uses color attenuation values, taking into account the propagation of light based on the echo reflection intensity, and the primary color parameters to generate a rendered image of the three-dimensional data.

[0028] The following is a reference to the appendix. Figure 1 The implementation method of the ultrasound diagnostic device will be described in detail.

[0029] (First Embodiment)

[0030] Figure 1 This is a block diagram showing an example of the configuration of the ultrasound diagnostic apparatus according to the first embodiment. Figure 1 The ultrasound diagnostic device 1 has a main body 100 and an ultrasound probe 101. The main body 100 is connected to an input device 102 and an output device 103. In addition, the main body 100 is connected to an external device 104 via a network NW. The external device 104 is, for example, a server equipped with a PACS (Picture Archiving and Communication Systems).

[0031] The ultrasonic probe 101 performs an ultrasonic scan on a scanning area within a biological body P, which is the subject of the examination, under control, for example, by the device body 100. The ultrasonic probe 101, for example, has multiple piezoelectric vibrators, a matching layer disposed between the multiple piezoelectric vibrators and a housing, and a backing member to prevent ultrasonic waves from propagating rearward from the multiple piezoelectric vibrators relative to the radiation direction. The ultrasonic probe 101 is, for example, a two-dimensional array probe in which multiple ultrasonic vibrators are arranged along a first element arrangement direction (elevation direction) and a second element arrangement direction (azimuth direction). The ultrasonic probe 101 is detachably connected to the device body 100. The ultrasonic probe 101 may also be equipped with buttons that can be pressed during offset processing and during operations such as pausing the ultrasonic image (pause operation).

[0032] Multiple piezoelectric transducers generate ultrasonic waves based on drive signals supplied from the ultrasonic transmitting circuit 110 (described later) provided in the device body 100. Ultrasonic waves are then transmitted from the ultrasonic probe 101 to the biological body P. When ultrasonic waves are transmitted from the ultrasonic probe 101 to the biological body P, the transmitted ultrasonic waves are sequentially reflected by the discontinuities of the acoustic impedance of the body tissue of the biological body P, and are received as reflected wave signals by the multiple piezoelectric transducers. The amplitude of the received reflected wave signal depends on the difference in acoustic impedance on the discontinuities that reflected the ultrasonic waves. Furthermore, in cases where the transmitted ultrasonic pulse is reflected by a moving blood flow or the surface of the heart wall, the reflected wave signal is frequency-shifted due to the Doppler effect, depending on the velocity component of the ultrasonic wave transmission direction of the moving body. The ultrasonic probe 101 receives the reflected wave signal from the biological body P and converts it into an electrical signal.

[0033] exist Figure 1 The diagram illustrates the connection between an ultrasonic probe 101 and the device body 100. However, multiple ultrasonic probes can be connected to the device body 100. Which of the connected ultrasonic probes is used for ultrasonic scanning can be arbitrarily selected, for example, via a software button on a touch panel described later.

[0034] The main body 100 is a device that generates an ultrasonic image based on the reflected wave signal received by the ultrasonic probe 101. The main body 100 includes an ultrasonic transmitting circuit 110, an ultrasonic receiving circuit 120, an internal storage circuit 130, an image memory 140, an input interface 150, an output interface 160, a communication interface 170, and a processing circuit 180.

[0035] The ultrasonic transmitting circuit 110 is a processor that supplies drive signals to the ultrasonic probe 101. The ultrasonic transmitting circuit 110 is implemented, for example, by a trigger generation circuit, a delay circuit, and a pulse generator circuit. The trigger generation circuit repeatedly generates rate pulses at a predetermined rate frequency to form transmitted ultrasonic waves. The delay circuit applies a delay time to each of the multiple piezoelectric vibrators generated by the trigger generation circuit, which is necessary to determine the transmission directionality of the ultrasonic waves generated from the ultrasonic probe into a beam. The pulse generator circuit applies drive signals (drive pulses) to the multiple ultrasonic vibrators disposed on the ultrasonic probe 101 based on the timing of the rate pulses. By varying the delay time applied to each rate pulse by the delay circuit, the transmission direction from the surface of the multiple piezoelectric vibrators can be arbitrarily adjusted.

[0036] Furthermore, the ultrasonic transmitting circuit 110 can arbitrarily change the output intensity of the ultrasonic wave via a drive signal. In the ultrasonic diagnostic device, by increasing the output intensity, the effect of ultrasonic wave attenuation within the biological body P can be reduced. By reducing the effect of ultrasonic wave attenuation, the ultrasonic diagnostic device can obtain a reflected wave signal with a relatively large signal-to-noise ratio (S / N) upon reception.

[0037] Generally, if ultrasound propagates within a living organism P, the intensity of the vibrations (also known as acoustic power) of the output ultrasound waves attenuates. This attenuation occurs through absorption, scattering, and reflection. Furthermore, the degree of reduction in acoustic power depends on the frequency of the ultrasound wave and the distance along its direction of propagation. For example, increasing the frequency of the ultrasound wave increases the attenuation. Moreover, the longer the distance along the direction of propagation, the greater the attenuation.

[0038] The ultrasonic receiving circuit 120 is a processor that performs various processing on the reflected wave signal received by the ultrasonic probe 101 and generates a received signal. The ultrasonic receiving circuit 120 generates a received signal based on the reflected wave signal of the ultrasonic wave obtained by the ultrasonic probe 101. Specifically, the ultrasonic receiving circuit 120 is implemented, for example, by a preamplifier, an A / D converter, a demodulator, and a beamformer. The preamplifier amplifies the reflected wave signal received by the ultrasonic probe 101 according to each channel and performs gain correction processing. The A / D converter converts the gain-corrected reflected wave signal into a digital signal. The demodulator demodulates the digital signal. The beamformer, for example, assigns a delay time to the demodulated digital signal required to determine the receiving directivity, and adds multiple digital signals with the assigned delay time. Through the addition processing of the beamformer, a received signal is generated that emphasizes the reflected component from the direction corresponding to the receiving directivity. Furthermore, the "reflected wave signal of the ultrasonic wave" and the "received signal" are collectively referred to as the "echo signal." Therefore, the "intensity of the received signal" can also be referred to as the "reflection intensity of the echo signal (echo reflection intensity)."

[0039] The internal storage circuit 130 may be a storage medium that can be read by a processor, such as a magnetic storage medium, an optical storage medium, or a semiconductor memory. The internal storage circuit 130 stores programs for implementing ultrasound transceiver, programs for global illumination (GI) image generation processing (described later), and various data. The programs and various data may also be pre-stored in the internal storage circuit 130. Furthermore, the programs and various data may be distributed from a non-transient storage medium, read from the non-transient storage medium, and installed into the internal storage circuit 130. In addition, the internal storage circuit 130 stores B-mode image data, contrast image data, blood flow image data, and three-dimensional data generated by the processing circuit 180, according to operations input via the input interface 150. The internal storage circuit 130 may also transfer the stored image data and three-dimensional data to an external device 104, etc., via the communication interface 170.

[0040] Alternatively, the internal storage circuit 130 can also be a drive device that reads and writes various information between itself and removable storage media such as CD drives, DVD drives, and flash memory. The internal storage circuit 130 can also write stored data to removable storage media, thereby storing the data in the external device 104 via the removable storage media.

[0041] Image memory 140 is a storage medium, such as a magnetic storage medium, optical storage medium, or semiconductor memory, that can be read by a processor. Image memory 140 stores image data corresponding to multiple frames input via input interface 150 before the pause operation. The image data stored in image memory 140 is displayed continuously, for example (video display). In addition, image memory 140 is not limited to storing image data; it can also store three-dimensional data.

[0042] The internal storage circuit 130 and image memory 140 described above do not necessarily have to be implemented by separate storage devices. The internal storage circuit 130 and image memory 140 may also be implemented by a single storage device. Furthermore, the internal storage circuit 130 and image memory 140 may also be implemented by multiple storage devices.

[0043] Input interface 150 receives various instructions from the operator via input device 102. Input device 102 may be, for example, a mouse, keyboard, panel switch, slider switch, trackball, rotary encoder, operation panel, or touch panel. Input interface 150 is connected to processing circuit 180 via a bus, for example, to convert the operation instructions input by the operator into electrical signals and output the electrical signals to processing circuit 180. Furthermore, input interface 150 is not limited to interfaces connected to physical operating components such as mice and keyboards. For example, circuitry that receives electrical signals corresponding to operation instructions input from external input devices that are separately located from the ultrasound diagnostic device 1 and outputs such electrical signals to processing circuit 180 is also included in the example of an input interface.

[0044] Output interface 160 is, for example, an interface used to output electrical signals from processing circuit 180 to output device 103. Output device 103 can be any display such as a liquid crystal display, organic EL display, LED display, plasma display, CRT display, etc. Output device 103 can also be a touch panel type display that also functions as input device 102. Output device 103 may also include a speaker for outputting sound in addition to the display. Output interface 160 is connected to processing circuit 180 via a bus, for example, to output electrical signals from processing circuit 180 to output device 103.

[0045] The communication interface 170 is connected to the external device 104, for example, via a network NW, to perform data communication with the external device 104.

[0046] The processing circuit 180 is, for example, a processor that functions as the central processing unit of the ultrasound diagnostic device 1. The processing circuit 180 executes a program stored in the internal storage circuit 130 to implement the functions corresponding to that program. The processing circuit 180 includes, for example, a B-mode processing function 181, a Doppler processing function 182, an image generation function 183, a three-dimensional data generation function 184 (three-dimensional data generation unit), an acquisition function 185A (acquisition unit), an opacity curve generation function 185B, a color attenuation curve generation function 185C (color attenuation curve generation unit), an optical property mapping generation function 185D (optical property mapping generation unit), a drawing function 185E (drawing unit), a display control function 186 (display control unit), and a system control function 187. Furthermore, the Doppler processing function 182 is less relevant to this embodiment, so it can be omitted from the functions of the processing circuit 180.

[0047] B-mode processing function 181 is a function that generates B-mode data based on the received signal (echo signal) received from the ultrasonic receiving circuit 120. In B-mode processing function 181, processing circuit 180 performs envelope detection processing and logarithmic compression processing on the received signal received from ultrasonic receiving circuit 120, for example, to generate data (B-mode data) that expresses the signal strength (echo reflection intensity) of the received signal in terms of brightness values ​​(brightness values). The generated B-mode data is stored as B-mode RAW data on two-dimensional ultrasonic scan lines (gratings) in a RAW data memory (not shown).

[0048] Furthermore, the processing circuit 180 can perform harmonic imaging via the B-mode processing function 181. Harmonic imaging is an imaging method that utilizes not only the fundamental wave component contained in the reflected wave signal of ultrasound, but also the higher harmonic components (harmonic components). Examples of harmonic imaging include tissue harmonic imaging (THI) without contrast agents and contrast harmonic imaging (CHI) using contrast agents.

[0049] In THI, harmonic components can be extracted using the amplitude modulation (AM) method, the phase modulation (PM) method, and an image method called AMPM, which combines the AM and PM methods.

[0050] In the AM, PM, and AMPM methods, ultrasonic waves with different amplitudes and phases are transmitted multiple times along the same scan line. As a result, the ultrasonic receiving circuit 120 generates multiple reflected wave data along each scan line and outputs the generated reflected wave data. The processing circuit 180 performs addition and subtraction processing on the multiple reflected wave data from each scan line using the B-mode processing function 181, corresponding to the modulation method, to extract harmonic components. Furthermore, the processing circuit 180 performs envelope detection processing on the reflected wave data of the harmonic components, etc., to generate B-mode data.

[0051] Furthermore, in CHI, for example, a frequency filter is used to extract harmonic components. The processing circuit 180, through the B-mode processing function 181, is able to separate the reflected wave data (high harmonic components) from the contrast agent and the reflected wave data (fundamental components) from the tissue within the biological body P. Thus, the processing circuit 180 can use a filter to select the high harmonic components from the contrast agent, generating B-mode data for generating contrast image data.

[0052] The B-mode data used to generate contrast image data is data that represents the intensity of the echo reflection from the contrast agent as a reflection source using brightness values. Furthermore, the processing circuit 180 can also extract the fundamental component from the reflected wave data of the biological organism P to generate B-mode data for generating tissue image data.

[0053] The Doppler processing function 182 generates data (Doppler information) by performing frequency analysis on the received signal received from the ultrasonic receiving circuit 120, extracting motion information based on the Doppler effect. This motion information refers to the motion of a moving body within a region of interest (ROI) defined in the scanning area. The generated Doppler information is stored as Doppler RAW data (also called Doppler data) on a two-dimensional ultrasonic scan line in a RAW data memory (not shown).

[0054] Specifically, the processing circuit 180 uses the Doppler processing function 182 to estimate, for example, the average velocity, average variance, and average power value at multiple sample points as motion information of a moving body, and generates Doppler data representing the estimated motion information. The moving body can be, for example, blood flow, tissue such as the heart wall, or a contrast agent. In this embodiment, the processing circuit 180 uses the Doppler processing function 182 to estimate the average velocity, variance of blood flow velocity, and power value of blood flow signal at multiple sample points as motion information (blood flow information), and generates Doppler data representing the estimated blood flow information.

[0055] Image generation function 183 is a function that generates B-mode image data based on data generated by B-mode processing function 181. For example, in image generation function 183, processing circuit 180 transforms (scan conversion) the scan line signal train of ultrasound scanning into a scan line signal train in a video format, such as television, to generate image data for display. Specifically, processing circuit 180 generates two-dimensional B-mode image data (also called ultrasound image data) composed of pixels by performing RAW-pixel transformation on B-mode RAW data stored in RAW data memory, such as coordinate transformation corresponding to the scanning pattern of ultrasound waves from ultrasound probe 101. In other words, processing circuit 180 generates multiple ultrasound images (medical images) corresponding to multiple consecutive frames by transmitting and receiving ultrasound waves through image generation function 183.

[0056] Furthermore, the processing circuit 180 generates Doppler image data that visualizes blood flow information, for example, by performing a RAW-pixel transformation on the Doppler RAW data stored in the RAW data memory. The Doppler image data is average velocity image data, variance image data, power image data, or a combination of these. The processing circuit 180 generates, as Doppler image data, color Doppler image data that displays blood flow information in color, and Doppler image data that displays blood flow information as a wave shape in grayscale.

[0057] The 3D data generation function 184 is a function that generates 3D B-mode data (3D data) based on the received signal received from the ultrasonic receiving circuit 120. In the 3D data generation function 184, the processing circuit 180 uses the B-mode data generated by the B-mode processing function 181 to assign brightness values ​​to voxels arranged in 3D space to generate 3D data. This 3D data can also be referred to as volume data. In addition, since the brightness value corresponds to the echo reflection intensity, it can also be interpreted as assigning echo reflection intensity to the voxels of the volume data. Therefore, the "brightness value of the volume data" can be used in a meaning that is approximately the same as "echo reflection intensity".

[0058] The acquisition function 185A is a function that acquires data related to the GI image generation process described later. Specifically, through the acquisition function 185A, the processing circuit 180 acquires multiple parameters and three-dimensional data, such as parameters input by the user and parameters set by default in the ultrasound diagnostic device 1. Among the multiple parameters are, for example, opacity parameters, primary color parameters, and drawing parameters. Detailed explanations of these parameters will be provided later.

[0059] The Opacity Curve Generation function 185B generates an opacity curve. An opacity curve is a graph that plots the reflection intensity against the opacity value. The opacity value is a value representing the transparency, for example, from "zero" to "1". For instance, an opacity value of "0" represents transparency, and an opacity value of "1" represents opacity.

[0060] The processing circuit 180 generates an opacity curve based on opacity parameters using the opacity curve generation function 185B. The opacity parameters include, for example, a transfer function that defines the shape of the opacity curve and a threshold that defines the transparent or opaque boundary. The transfer function can be either linear or non-linear. Multiple thresholds can also be set; for example, two boundaries, one for transparency and one for opacity, can be defined. Furthermore, in this embodiment, it is assumed that the graph of the opacity curve is represented using a look-up table (LUT) and used in the processing described later. Additionally, the processing performed by the opacity curve generation function 185B can also be referred to as opacity curve generation processing.

[0061] The color decay curve generation function 185C is a function that generates color decay curves. A color decay curve is a graph that establishes a corresponding relationship between opacity values ​​and color decay values. Color decay values ​​are values ​​that take into account the propagation of light and represent the degree to which a basic color decays; they are set according to each element of the color representation. That is, the color decay curve is set according to each element of the color representation. In this embodiment, it is assumed that RGB is used as the color representation. However, the color representation is not limited to RGB; it can also be a color space represented by three components: hue, chroma, and lightness (or brightness) (e.g., HSV and HLS), or a color space represented using the lightness signal Y and two color difference signals (e.g., YUV, YCbCr, and YPbPr).

[0062] The processing circuit 180 generates a color decay curve based on primary color parameters and an opacity curve using the color decay curve generation function 185C. The primary color parameters include, for example, a combination of RGB values ​​(color map) representing the color of the object (material) to be displayed. The primary color can also be referred to as the color reflected from the object, i.e., the reflected color. The shape of the color decay curve is at least one different shape among multiple elements. Therefore, there are two or more transfer functions defining the shape of the color decay curve. Furthermore, this transfer function can be either linear or non-linear. In other words, the processing circuit 180 generates a color decay curve using at least two different transfer functions for each element of the color representation. In this embodiment, it is assumed that the graph of the color decay curve is represented by a LUT and used in the processing described later. Furthermore, the processing performed by the color decay curve generation function 185C can also be referred to as color decay curve generation processing.

[0063] The optical property map generation function 185D is a function that generates optical property maps. An optical property map is generated by quantifying and storing the imagined physical phenomena (such as color attenuation) when light shines on a material into voxels arranged in three-dimensional space. In this embodiment, a photon map, described later, is generated as an optical property map.

[0064] The aforementioned photon is defined in the photon mapping method, which discretizes light for computer representation and transmits light energy per unit time. In this photon mapping method, a pre-set number of photons in the system or a user-defined number of photons are used for collision calculations within three-dimensional data and then configured into a scene. The scene is a three-dimensional space used to create the light property mapping.

[0065] Using the light property mapping generation function 185D, the processing circuit 180 generates a light property map based on the opacity curve, color attenuation curve, drawing parameters, and three-dimensional data. The drawing parameters include, for example, information about the position of the light source facing the material. Alternatively, the processing performed by the light property mapping generation function 185D can also be referred to as light property mapping generation processing.

[0066] The drawing function 185E is a function that generates drawn images. These drawn images may include, for example, volumetric drawn images and global illumination images. In this embodiment, a drawn image that does not consider the light source is defined as a volumetric drawn image, and a drawn image that takes the light source into account is defined as a global illumination image.

[0067] Volume rendering images are obtained by performing volume rendering on volume data. In volume rendering, the brightness and color of each voxel are set according to the brightness values ​​assigned to the voxels in the volume data (the brightness values ​​of the volume data). Based on this, volume rendering displays a projected image formed by projecting the voxels from any viewpoint.

[0068] On the other hand, the global illumination image is rendered using the photon mapping described above. In this rendering process, ray tracing is used, for example. In this embodiment, the global illumination image is generated as the rendered image.

[0069] Using the drawing function 185E, the processing circuit 180 generates a global illumination image based on the opacity curve, primary color parameters, drawing parameters, light characteristic mapping, and 3D data. Drawing parameters include, for example, information about the viewpoint's position. Alternatively, the processing performed by the drawing function 185E can be referred to as global illumination drawing processing.

[0070] Display control function 186 is a function that causes the display, which is the output device 103, to display an image based on various ultrasonic image data generated by image generation function 183. Specifically, for example, processing circuit 180 controls the display on the screen based on B-mode image data, Doppler image data, or image data including both generated by image generation function 183 through display control function 186.

[0071] More specifically, the processing circuit 180, through the display control function 186, converts, for example, the scan line signal train of an ultrasonic scan into a scan line signal train in a video format, such as television (scan conversion), and generates display image data. Furthermore, the processing circuit 180 can perform various processing on the display image data, such as dynamic range, brightness, contrast, gamma curve correction, and RGB conversion. Additionally, the processing circuit 180 can add various parameter character information, scales, body markings, and other auxiliary information to the display image data. Furthermore, the processing circuit 180 can generate a graphical user interface (GUI) for the operator to input various instructions via an input device, enabling the display to show the GUI.

[0072] Furthermore, the processing circuit 180 can also display the global illumination image (GI image) generated by the drawing function 185E via the display control function 186. Additionally, the processing circuit 180 can also display a GUI (User-defined interface) showing settings related to the GI image along with the GI image. Thus, by changing the parameters displayed on the GUI, the user can make the GI image change in real time to achieve the desired display. Parameters that the user can change include, for example, opacity parameters, primary color parameters, and drawing parameters.

[0073] System control function 187 is a function that comprehensively controls the operation of the entire ultrasonic diagnostic device 1. For example, in system control function 187, processing circuit 180 controls ultrasonic transmitting circuit 110 and ultrasonic receiving circuit 120 based on parameters related to the transmission and reception of ultrasonic waves.

[0074] The configuration of the ultrasound diagnostic apparatus of the first embodiment has been described above. Next, the outline and operation of the global illumination image generation process of the first embodiment will be described.

[0075] Figure 2 This is a block diagram illustrating the general outline of the global illumination image generation process in the first embodiment. The processing circuit 180 generates a global illumination image by performing a global illumination image generation process 200 based on acquired parameters and three-dimensional data. The global illumination image generation process 200 includes, for example, an opacity curve generation process 210, a color attenuation curve generation process 220, a light property mapping generation process 230, and a global illumination rendering process 240. Furthermore, the opacity curve generation process 210 and the color attenuation curve generation process 220 can generate opacity curves and color attenuation curves based on default parameters even when the user has not set any parameters for these processes.

[0076] Figure 3 This is a flowchart illustrating an example of the operation of the processing circuit performing the global illumination image generation process of the first embodiment. Figure 3 The global illumination image generation process begins, for example, by the user executing a mode (illumination mode) to display the global illumination image. Additionally, Figure 3 The flowchart illustrates the processing of one frame.

[0077] (Step ST110)

[0078] If global illumination image generation processing begins, processing circuit 180 executes acquisition function 185A. If acquisition function 185A is executed, processing circuit 180 acquires multiple parameters and three-dimensional data. The multiple parameters include at least one of parameters input by the user and parameters defaulted to the ultrasound diagnostic device 1. The three-dimensional data is generated by three-dimensional data generation function 184 based on the echo reflection intensity acquired in real time by the ultrasound probe 101.

[0079] (Step ST120)

[0080] After acquiring the parameters and 3D data, the processing circuit 180 executes the opacity curve generation function 185B. If the opacity curve generation function 185B is executed, the processing circuit 180 generates an opacity curve based on the opacity parameters. The following uses... Figure 4 The opacity curve generation process performed by the opacity curve generation function 185B is explained.

[0081] Figure 4 This is an explanation Figure 2The diagram shows the block diagram of the opacity curve generation process. Processing circuit 180 generates an opacity curve by performing an opacity curve generation process 210 on the opacity parameters. Specifically, the opacity curve generation process 210 uses the transfer function and threshold contained in the opacity parameters to generate a LUT with echo reflection intensity as a keyword for the opacity curve. Processing circuit 180 calculates the value of the opacity curve LUT, for example, using the following equation (1).

[0082]

[0083] In equation (1), I represents the echo reflection intensity value. Opacity[I] represents the opacity value (the value of opacity). C trans This represents the transfer function. th This indicates the threshold for defining the transparent boundary. The echo reflection intensity value I, for example, when represented in 8 bits, is a value from "zero" to "255".

[0084] (Step ST130)

[0085] After generating the opacity curve, the processing circuit 180 executes the color decay curve generation function 185C. If the color decay curve generation function 185C is executed, the processing circuit 180 generates a color decay curve based on the primary color parameters and the opacity curve. The primary color parameters are set based on a preset color map or a user-specified color map. The following uses... Figure 5 The color attenuation curve generation process performed by the color attenuation curve generation function 185C is explained.

[0086] Figure 5 This is an explanation Figure 2 A block diagram of the color decay curve generation process is provided. The processing circuit 180 generates a color decay curve by performing a color decay curve generation process 220 on the primary color parameters and the opacity curve. Specifically, the color decay curve generation process 220 creates LUTs for the color decay curves with opacity values ​​as keywords for each of the RGB values ​​included in the primary color parameters. The processing circuit 180 calculates the values ​​of the color decay curve LUTs corresponding to RGB values, for example, using equations (2) to (4) below.

[0087]

[0088]

[0089]

[0090] In equations (2) to (4), Extinction[I][R] represents the color attenuation value for the element with respect to red (R). Similarly, Extinction[I][G] represents the color attenuation value for the element with respect to green (G), and Extinction[I][B] represents the color attenuation value for the element with respect to blue (B). Er, Eg, and Eb represent the numerical values ​​of the respective RGB elements. The numerical values ​​of the elements are represented using a floating decimal point. For example, when each RGB element is represented using 8 bits, values ​​from "zero" to "255" are mapped to values ​​from "zero" to "1.0". Furthermore, the color attenuation value is also represented using a floating decimal point.

[0091] When Opacity[I] is greater than zero, the operation of Extinction[I][R] differs from that of Extinction[I][G] and Extinction[I][B]. Specifically, the rate of decrease in value of Extinction[I][R] is smaller compared to Extinction[I][B] and Extinction[I][G]. This corresponds to a different shape of the color decay curve.

[0092] (Step ST140)

[0093] After generating the color decay curve, the processing circuit 180 executes the light property map generation function 185D. If the light property map generation function 185D is executed, the processing circuit 180 generates a light property map based on the opacity curve, color decay curve, drawing parameters, and 3D data. The following uses... Figure 6 The optical property mapping generation process performed by the optical property mapping generation function 185D is explained.

[0094] Figure 6 This is an explanation Figure 2 The diagram illustrates the light property mapping generation process. Processing circuit 180 generates a light property map by performing a light property mapping generation process 230 on the opacity curve, color attenuation curve, drawing parameters, and 3D data. Specifically, the light property mapping generation process 230 uses the light source position information and 3D data contained in the drawing parameters to apply a photon mapping method to generate a photon map.

[0095] Furthermore, various parameters can be set for each photon in the photon mapping. In this embodiment, it is assumed that each photon is assigned a color attenuation value for RGB. Specifically, the processing circuit 180 reads out the opacity value and color attenuation value based on the echo reflection intensity value, opacity curve LUT, and color attenuation curve LUT contained in the voxel of the three-dimensional data corresponding to the position of the scene where the photons are arranged. Then, the processing circuit 180 assigns a color attenuation value to each photon.

[0096] (Step ST150)

[0097] After generating the light property map, the processing circuit 180 executes the drawing function 185E. If the drawing function 185E is executed, the processing circuit 180 generates a global illumination image based on the opacity curve, primary color parameters, drawing parameters, light property map, and 3D data. The following uses... Figure 7 The global illumination rendering process performed by the rendering function 185E is explained.

[0098] Figure 7 This is an explanation Figure 2 The diagram illustrates the global illumination rendering process. Processing circuit 180 generates a global illumination image by performing global illumination rendering processing 240 on opacity curves, primary color parameters, rendering parameters, light property mapping, and 3D data. Specifically, global illumination rendering processing 240 generates a projected image (global illumination image) that projects voxels of 3D data using photon mapping, based on the viewpoint position information contained in the rendering parameters. For voxels on the projection surface, opacity values ​​and primary colors are set based on echo reflection intensity.

[0099] Furthermore, the displayed color of the global illumination image is calculated by applying the color attenuation value contained in photon mapping to the values ​​of the individual RGB elements included in the primary color parameters. When calculating the color attenuation value using the above equations (2) to (4), the displayed color exhibits greater attenuation of G and B compared to R. Therefore, in places where light attenuation occurs, such as in shadows, the R component remains more abundant. In other words, the hue changes according to the intensity of the echo reflection.

[0100] (Step ST160)

[0101] After generating the global illumination image, the processing circuit 180 uses the display control function 186 to cause the display, which is the output device 103, to display the global illumination image. After the processing in step ST160, the global illumination image generation process ends.

[0102] Alternatively, you can also Figure 3 The process is repeated until the user ends the lighting mode or changes to another mode.

[0103] Figure 8 This is a diagram showing a global illumination image of the light source in the first embodiment being in front. Figure 8 In this context, since the viewpoint is also facing forward, the light from the light source travels in the same direction as the viewpoint. Figure 8 In the global illumination image 800, hue variations are represented by gray, and areas that act as shadows are represented by shading. This is also done in subsequent global illumination images.

[0104] exist Figure 8 The global illumination image 800 depicts the right hand of a fetus. If one focuses on the palm 810 of the right hand, a change in tone occurs in the area different from the shadow. Thus, the shape of the hand is clearly discernible in the palm 810. Furthermore, if one focuses on the cross-section 820 of the right hand, similarly to the palm 810, the shape of structures (e.g., bones) can be clearly captured due to the change in tone. On the other hand, such a change in tone does not occur in conventional methods. Hereinafter, using... Figure 11 Explanation of previous diagrams.

[0105] Figure 11 This is a previous image of a global illumination image with the light source facing forward. Figure 11 In the global illumination image 1100, the right hand of the fetus is depicted in the same way as in the global illumination image 800. If one focuses on the palm 1110 of the right hand, although the shadow is shown, the shape of the palm becomes indistinct because no change in tone occurs. Furthermore, if one focuses on the cross-section 1120 of the right hand, similarly to the palm 1110, the shape of the structure cannot be discerned because no change in tone occurs.

[0106] Figure 9 This is a diagram showing a global illumination image with the light source on the left side in the first embodiment. Figure 9 In the image, since the viewpoint is facing forward, the light from the light source travels from left to right relative to the viewpoint.

[0107] exist Figure 9 The global illumination image 900 depicts the entire fetus. If focusing on the fetus's head 910, the facial contours become blurred due to the shadow on the left side of the face. Furthermore, if focusing on structures other than the fetus 920, boundaries can be understood through variations in tone. On the other hand, in conventional methods, there are cases where the contours are emphasized, resulting in an unnatural appearance. Hereinafter, using... Figure 12 Explanation of previous diagrams.

[0108] Figure 12 This is a previous image of global illumination with the light source on the left. Figure 12 In the global illumination image 1200, the entire body of the fetus is depicted, similar to that in the global illumination image 900. However, if focusing on the fetus's head 1210, the outlines of the left side of the head, left cheek, and left ear are clearly depicted as edges, creating an unnatural appearance. Furthermore, if focusing on structures other than the fetus 1220, similarly to the head 1210, even though these are smooth areas, edges are depicted.

[0109] In summary, the ultrasound diagnostic apparatus of the first embodiment, in generating a global illumination image, links the degree of color attenuation with the echo reflection intensity, setting the reflected color to a single color that is not linked to the echo reflection intensity. Furthermore, this ultrasound diagnostic apparatus links a color attenuation curve with an opacity curve at a predetermined degree of color attenuation, so that higher echoes allow more color transmission, and lower echoes allow less color transmission. Through these methods, in the generated global illumination image, at boundaries where the echo reflection intensity changes significantly, not only does the color intensity change, but the RGB values ​​also change, so the boundary portion is depicted as a color change, capturing a natural outline. On the other hand, since the light illumination method, i.e., the shadow portion caused by the position of the light source, is depicted unchanged, the realism of shadow changes is maintained. Therefore, this ultrasound diagnostic apparatus can achieve a result that balances realism and discernibility more effectively than before, with the same amount of computational effort as the conventional apparatus using color attenuation and reflected color.

[0110] As explained above, the ultrasound diagnostic apparatus of the first embodiment generates three-dimensional data based on the echo reflection intensity obtained by the probe, acquires multiple parameters including primary color parameters and three-dimensional data, and generates a plotted image of the three-dimensional data using color attenuation values ​​and primary color parameters that take into account the propagation of light based on the echo reflection intensity.

[0111] Therefore, the ultrasound diagnostic apparatus of the first embodiment can generate natural drawn images while maintaining computational complexity by corresponding the color attenuation to the echo reflection intensity. Since the computational complexity remains unchanged compared to the case where the color attenuation is not corresponding to the echo reflection intensity, natural contours can be drawn at the ends of structures without compromising real-time performance. Furthermore, since the color discrepancies caused by the spectral characteristics of the ultrasound image occur not only at the ends of structures, the transmission of multiple color hues can achieve an effect similar to real-world color unevenness, and the smooth parts of the structure also appear more natural.

[0112] (Second Implementation)

[0113] In the first embodiment, multiple functions related to global illumination image generation processing were described. In the second embodiment, an image processing apparatus having these multiple functions will be described.

[0114] Figure 10 This is a block diagram illustrating a configuration example of the image processing apparatus according to the second embodiment. Figure 10The image processing device 300 is connected to the input device 301 and the output device 302. Furthermore, the image processing device 300 is connected to the medical imaging device 303 via a network NW. The medical imaging device 303 is, for example, an ultrasound diagnostic device. Additionally, the input device 301 and the output device 302 are connected to... Figure 1 The input device 102 and the output device 103 are substantially the same.

[0115] The image processing apparatus 300 is an apparatus that generates a global illumination image by performing global illumination image generation processing. The image processing apparatus 300 includes a storage circuit 310, an input interface 320, an output interface 330, a communication interface 340, and a processing circuit 350.

[0116] The storage circuit 310 includes a storage medium that can be read by a processor, such as a magnetic storage medium, an optical storage medium, or a semiconductor memory. The storage circuit 310 stores programs and various data related to global illumination image generation processing. The programs and various data may also be pre-stored in the storage circuit 310. Furthermore, they may be distributed from a non-transient storage medium, read from a non-transient storage medium, and installed into the storage circuit 310. In addition, the storage circuit 310 stores B-mode image data, contrast image data, blood flow image data, and three-dimensional data generated by the medical imaging device 303, according to operations input via the input interface 320.

[0117] Alternatively, the storage circuit 310 can also be a drive device that reads and writes various information between a CD drive, DVD drive, and a removable storage medium such as flash memory. The storage circuit 310 can also write stored data to a removable storage medium, and then store the data to an external device via the removable storage medium.

[0118] Input interface 320 receives various instructions from the operator via input device 301. Input device 301 may be, for example, a mouse, keyboard, panel switch, slider switch, trackball, rotary encoder, operation panel, or touch panel. Input interface 320 is connected to processing circuit 350 via a bus, for example, to convert the operation instructions input by the operator into electrical signals and output the electrical signals to processing circuit 350. Furthermore, input interface 320 is not limited to interfaces connected to physical operating components such as mice and keyboards. For example, circuitry that receives electrical signals corresponding to operation instructions input from external input devices that are separately located from image processing device 300 and outputs these electrical signals to processing circuit 350 is also included in the example of an input interface.

[0119] Output interface 330 is, for example, an interface used to output electrical signals from processing circuit 350 to output device 302. Output device 302 can be any display such as a liquid crystal display, organic EL display, LED display, plasma display, CRT display, etc. Output device 302 can also be a touch panel type display that also functions as input device 301. Output device 302 may also include a speaker for outputting sound in addition to the display. Output interface 330 is connected to processing circuit 350 via a bus, for example, to output electrical signals from processing circuit 350 to output device 302.

[0120] The communication interface 340 is connected to the medical camera device 303, for example, via a network NW, and performs data communication with the medical camera device 303.

[0121] The processing circuit 350 is, for example, a processor that functions as the central processing unit of the image processing apparatus 300. The processing circuit 350 executes a program stored in the storage circuit 310 to implement the functions corresponding to that program. The processing circuit 350 includes, for example, an acquisition function 351A (acquisition unit), an opacity curve generation function 351B, a color attenuation curve generation function 351C (color attenuation curve generation unit), a light characteristic mapping generation function 351D (light characteristic mapping generation unit), a drawing function 351E (drawing unit), and a display control function 352 (display control unit).

[0122] The acquisition function 351A is a function that acquires data related to the GI image generation and processing. Specifically, through the acquisition function 351A, the processing circuit 350 acquires parameters input by the user, parameters set by default for the image processing device 300, and three-dimensional data from the medical imaging device 303.

[0123] In addition, the opacity curve generation function 351B, the color attenuation curve generation function 351C, the light property mapping generation function 351D, the drawing function 351E, and the display control function 352 each have, for example, substantially the same functions as the opacity curve generation function 185B, the color attenuation curve generation function 185C, the light property mapping generation function 185D, the drawing function 185E, and the display control function 186 of the first embodiment.

[0124] Therefore, the image processing apparatus of the second embodiment can be expected to have the same effect as that of the first embodiment.

[0125] According to at least one of the embodiments described above, it is possible to generate natural-looking rendering images while maintaining computational load.

[0126] Several embodiments have been described, but these embodiments are merely illustrative and not intended to limit the scope of the invention. These embodiments can be implemented in a wide variety of other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their variations are included in the scope or spirit of the invention, and are included in the scope of the invention as described in the claims and its equivalents.

Claims

1. An ultrasonic diagnostic device, characterized in that, have: The three-dimensional data generation unit generates three-dimensional data based on the echo reflection intensity obtained by the probe; The acquisition unit acquires multiple parameters, including primary color parameters, and the aforementioned three-dimensional data; The color attenuation curve generation unit, based on the aforementioned primary color parameters and the aforementioned echo reflection intensity and opacity values, establishes a corresponding opacity curve, and generates a color attenuation curve corresponding to the aforementioned opacity values ​​and color attenuation values; and The rendering unit generates a rendered image of the three-dimensional data using the color attenuation value and the primary color parameter, taking into account the propagation of light based on the echo reflection intensity. The color attenuation value is calculated based on the echo reflection intensity, the opacity curve, and the color attenuation curve. The aforementioned color decay curves are set according to each element of the color's representation. The display color of the image drawn from the above three-dimensional data is calculated by applying the respective color attenuation values ​​to the values ​​of each element of the color representation contained in the primary color parameters.

2. The ultrasonic diagnostic device as described in claim 1, characterized in that, The aforementioned parameters also include drawing parameters, which contain information about the position of the light source being directed toward the object. The aforementioned ultrasound diagnostic device also includes an optical property mapping generation unit, which generates an optical property mapping that stores the aforementioned color attenuation value based on the aforementioned opacity curve, the aforementioned color attenuation curve, the aforementioned drawing parameters, and the aforementioned three-dimensional data. The above drawing parameters also include information about the viewpoint's position. The above-mentioned drawing unit generates the above-mentioned drawing image based on the above-mentioned opacity curve, the above-mentioned primary color parameters, the above-mentioned drawing parameters, the above-mentioned light characteristic mapping and the above-mentioned three-dimensional data.

3. The ultrasonic diagnostic device as described in claim 2, characterized in that, The aforementioned optical property mapping generation unit generates the aforementioned optical property mapping using the photon mapping method.

4. The ultrasonic diagnostic device as described in claim 1, characterized in that, The above colors are represented in one of the following formats: RGB, HSV, HLS, YUV, YCrCb, and YPbPr.

5. The ultrasonic diagnostic device as described in claim 1 or 4, characterized in that, The color decay curve generation unit generates the color decay curve by using at least two different transfer functions for each element of the color representation.

6. The ultrasonic diagnostic device according to any one of claims 1 to 4, characterized in that, The aforementioned drawing unit generates the aforementioned drawing image by performing global illumination drawing processing.

7. The ultrasonic diagnostic device according to any one of claims 1 to 4, characterized in that, The above primary color parameters are set based on a pre-defined color map or a user-specified color map.

8. The ultrasonic diagnostic device according to any one of claims 1 to 4, characterized in that, The above-mentioned drawing unit generates the above-mentioned drawing image in such a way that the higher the echo reflection intensity, the more color is transmitted, and the lower the echo reflection intensity, the less color is transmitted.

9. The ultrasonic diagnostic device as described in claim 8, characterized in that, The color attenuation curves described above show that the higher the echo reflection intensity, the more transparent the color, and the lower the echo reflection intensity, the less transparent the color.

10. An image processing apparatus, characterized in that, have: The acquisition unit acquires multiple parameters, including primary color parameters, and three-dimensional data generated based on the echo reflection intensity acquired by the probe. The color attenuation curve generation unit, based on the aforementioned primary color parameters and the aforementioned echo reflection intensity and opacity values, establishes a corresponding opacity curve, and generates a color attenuation curve corresponding to the aforementioned opacity values ​​and color attenuation values; and The rendering unit generates a rendered image of the three-dimensional data using the color attenuation value and the primary color parameter, taking into account the propagation of light based on the echo reflection intensity. The color attenuation value is calculated based on the echo reflection intensity, the opacity curve, and the color attenuation curve. The aforementioned color decay curves are set according to each element of the color's representation. The display color of the image drawn from the above three-dimensional data is calculated by applying the respective color attenuation values ​​to the values ​​of each element of the color representation contained in the primary color parameters.

11. An image processing method, characterized in that, The following steps are required: It acquires multiple parameters, including primary color parameters, and three-dimensional data generated based on the echo reflection intensity obtained by the probe; Based on the above primary color parameters and the above echo reflection intensity and opacity values, a corresponding opacity curve was established, and a corresponding color attenuation curve was generated based on the above opacity values ​​and color attenuation values. as well as A rendering image of the aforementioned three-dimensional data is generated using the aforementioned color attenuation value, which takes into account the propagation of light based on the aforementioned echo reflection intensity, and the aforementioned primary color parameters. The aforementioned color attenuation value is calculated based on the aforementioned echo reflection intensity, the aforementioned opacity curve, and the aforementioned color attenuation curve. The aforementioned color decay curves are set according to each element of the color's representation. The display color of the image drawn from the above three-dimensional data is calculated by applying the respective color attenuation values ​​to the values ​​of each element of the color representation contained in the primary color parameters.

12. A storage medium storing a program, characterized in that, The above procedure enables the computer to function as a mechanism for: An apparatus that acquires multiple parameters, including primary color parameters, and generates three-dimensional data based on the echo reflection intensity obtained by the probe; Based on the above primary color parameters and the above echo reflection intensity and opacity values, a corresponding opacity curve is established, and a mechanism is generated that establishes the above opacity values ​​and color attenuation values ​​into a corresponding color attenuation curve. as well as The mechanism that generates the three-dimensional image using the color attenuation value and the primary color parameter, taking into account the propagation of light based on the echo reflection intensity, calculates the color attenuation value based on the echo reflection intensity, the opacity curve, and the color attenuation curve. The aforementioned color decay curves are set according to each element of the color's representation. The display color of the image drawn from the above three-dimensional data is calculated by applying the respective color attenuation values ​​to the values ​​of each element of the color representation contained in the primary color parameters.

Images