Scale adaptive ultrasound surface shading

Abstract

An ultrasound system (210) includes an interface, a memory (151), and a processor (152). The interface receives two-dimensional ultrasound images. The memory (151) stores instructions and the two-dimensional ultrasound images. The processor (152) executes the instructions. The instructions, when executed by the processor (152), cause the ultrasound system (210) to perform the following operations: for each pixel of a plurality of pixels of each two-dimensional ultrasound image, identify a best scale for applying surface shading based on a size of at least one blood vessel in the two-dimensional ultrasound image; apply the surface shading to at least one first pixel based on identifying the best scale for the at least one first pixel; and apply the surface shading to at least one second pixel based on identifying the best scale for the at least one second pixel.

Description

Technical Field

[0001] This invention relates to imaging visualization. In particular, it relates to the two-dimensional to three-dimensional visualization conversion in ultrasound image visualization. Background Technology

[0002] A new ultrasound visualization technique called FlowViewer has recently been developed to provide 3D-like renderings of 2D ultrasound flow imaging data. FlowViewer visualization is applicable to all ultrasound flow imaging modalities, including color Doppler, energy Doppler (also known as CPA), directional energy Doppler (also known as dCPA), microfluidic imaging (also known as MFI), and high-resolution MFI (also known as MFI-HD). FlowViewer visualization can be applied during real-time 2D ultrasound imaging and in post-processing. The foundation of FlowViewer visualization is the application of the known Brin-Von reflection model for 3D surfaces to 2D ultrasound images. More specifically, the FlowViewer visualization algorithm requires two input 2D ultrasound images (a luminance image and a red / green / blue (RGB) image). The 2D luminance image is used to create the illumination effect of the 2D RGB image. For example, the 2D RGB image can include a color-coded Doppler velocity map. Two-dimensional luminance images may include a power component of a Doppler signal, which is used to (a) indirectly determine whether a pixel corresponds to a flow during color Doppler flow imaging; and (b) directly determine whether a pixel corresponds to a flow during energy Doppler flow imaging (e.g., when in CPA, dCPA, MFI, MFI-HD).

[0003] In stream viewer visualization techniques, 2D luminance images are processed using low-pass gradient filters to generate a 3D gradient vector field for applying surface shading to the 2D RGB image. The resulting surface is defined by its 3D normal vector, which corresponds to the smoothed 3D gradient vector field. A smoothed 3D surface of the 3D gradient vector field is generated by convolving the 2D luminance image with a Gaussian kernel, providing the 3D gradient vector field for applying surface shading to the 2D RGB image. The scale of the Gaussian kernel is a parameter that controls the smoothness of the 3D surface of the gradient vector field and the size of the details that will be enhanced by lighting effects. The size of the Gaussian kernel is determined by the scale parameter and reflects the amount of smoothing applied. The gradient vector field has three components (Gx, Gy, and 1). Gx is the horizontal derivative of the pre-filtered 2D luminance image, and Gy is the vertical derivative of the pre-filtered 2D luminance image. The Gx and Gy components of the smoothed 3D surface of the gradient vector field are multiplied by a height parameter that controls the “kurtosis” of the output image. The generated gradient vector field is then fed into a 3D surface surface, which generates a diffuse and specular reflection map. This map is combined with the input RGB image to generate the final 3D-like stream viewer output image.

[0004] In the surface shading module, a 3D vector [Gx; Gy; 1] normal to the surface N(x, y) is calculated at each pixel x, y. The diffuse component, which is the dot product of N(x, y) and the light source vector L, is then obtained. Next, H(x, y), a half-angle vector between the light source direction L and the observer direction V, is calculated at each pixel x, y. The specular component S(x, y) is then obtained by raising the dot product of N(x, y) and H(x, y) to a power of the glossiness parameter. Then, according to... At each pixel x, y, the output RGB image Out(x, y) is calculated based on the input RGB image In(x, y), where AmbientC, DiffuseC, and SpecularC are coefficient parameters. The AmbientC coefficient parameter comes from the input RGB image, while the DiffuseC and SpecularC coefficient parameters come from the diffuse and specular reflection maps generated from the 3D surface based on the gradient vector field.

[0005] The two main drawbacks of the aforementioned technique stem from the arbitrary choice of the scale of the single Gaussian kernel used for the gradient filter. The first drawback is that the scale is not set according to the size of the blood vessels present in the RGB image, which hinders optimization of results in visual separation. The second drawback is that the technique does not work well when multiple blood vessel sizes exist in the same image (e.g., large veins near arterioles), forcing users to accept suboptimal compromises. Summary of the Invention

[0006] The object of this invention is, in particular, to provide improved visualization. This is achieved by generating a virtual lighting effect on a two-dimensional image, wherein the pixels of the two-dimensional image are modified to generate a three-dimensional visualization effect.

[0007] This invention is defined by the independent claims. Advantageous embodiments are defined by the dependent claims.

[0008] According to one aspect of this disclosure, an ultrasound system includes an interface, a memory, and a processor. The interface receives two-dimensional ultrasound images. The memory stores instructions and the two-dimensional ultrasound images. The processor executes the instructions. When executed by the processor, the instructions cause the ultrasound system to perform the following operations: for each pixel of a plurality of pixels in each two-dimensional ultrasound image, identifying an optimal scale for applying surface coloring based on the size of at least one blood vessel in the two-dimensional ultrasound image; applying surface coloring to the at least one first pixel based on the identified optimal scale; and applying surface coloring to the at least one second pixel based on the identified optimal scale.

[0009] According to another aspect of this disclosure, a method for applying virtual illumination to an ultrasound image includes: receiving a two-dimensional ultrasound image; identifying an optimal scale for applying surface shading based on the size of at least one blood vessel in the two-dimensional ultrasound image for each of a plurality of pixels in the two-dimensional ultrasound image; applying surface shading to the at least one first pixel based on the identified optimal scale for at least one first pixel; and applying surface shading to the at least one second pixel based on the identified optimal scale for at least one second pixel.

[0010] According to another aspect of this disclosure, an ultrasound system includes an interface, a memory, and a processor. The interface receives two-dimensional ultrasound images. The memory stores instructions and the two-dimensional ultrasound images. The processor executes the instructions. When executed by the processor, the instructions cause the ultrasound system to perform the following operations: for each pixel of a plurality of pixels in each two-dimensional ultrasound image, identifying an optimal scale for applying surface coloring based on the size of at least one blood vessel in the two-dimensional ultrasound image; for each pixel of the plurality of pixels in each two-dimensional ultrasound image, selecting a corresponding filter from a filter bank based on the identified optimal scale for the pixel, and obtaining a luminance pixel corresponding to the pixel from the corresponding filter; and applying surface coloring to each pixel of the plurality of pixels by using the luminance pixels from the filters in the filter bank, based on the identified optimal scale for the pixel.

[0011] These and other aspects of the invention will become apparent and will be illustrated with reference to the embodiments described below. Attached Figure Description

[0012] Exemplary embodiments are best understood in conjunction with the accompanying drawings and the following detailed description. It should be emphasized that the various features are not necessarily drawn to scale. In fact, dimensions may be arbitrarily increased or decreased for clarity of discussion. Where applicable and practicable, the same reference numerals refer to the same elements.

[0013] Figure 1 The illustration depicts a system for scale-adaptive ultrasonic surface coloring according to a representative embodiment.

[0014] Figure 2 Another system for scale-adaptive ultrasonic surface coloring is illustrated according to a representative embodiment.

[0015] Figure 3A The illustration depicts a method for scale-adaptive ultrasonic surface coloring according to a representative embodiment.

[0016] Figure 3BThe illustration shows another method for scale-adaptive ultrasonic surface coloring according to a representative embodiment.

[0017] Figure 4 The illustration shows the effect of different filter scaling in scale-adaptive ultrasonic surface coloring according to a representative embodiment.

[0018] Figure 5 The illustration shows a hybrid process for scale-adaptive ultrasonic surface coloring according to a representative embodiment.

[0019] Figure 6 The illustration shows scale estimation in scale-adaptive ultrasonic surface coloring according to a representative embodiment.

[0020] Figure 7 The illustration compares single-scale and multi-scale methods in scale-adaptive ultrasonic surface coloring according to representative embodiments.

[0021] Figure 8 The illustration shows a comparison of the final coloring results obtained using a single-scale method and a multi-scale method in scale-adaptive ultrasonic surface coloring according to representative embodiments.

[0022] Figure 9 The illustration shows a computer system on which a method for scale-adaptive ultrasonic surface coloring is implemented according to another representative embodiment. Detailed Implementation

[0023] In the following detailed description, representative embodiments with specific details disclosed are set forth for purposes of explanation and not limitation, in order to provide a thorough understanding of embodiments according to this teaching. However, other embodiments consistent with this disclosure but departing from the specific details disclosed herein remain within the scope of the appended claims. Descriptions of known systems, devices, materials, methods of operation, and methods of manufacture may be omitted to avoid obscuring the description of representative embodiments. However, systems, apparatuses, materials, and methods within the scope of this teaching are applicable to those skilled in the art and may be used according to representative embodiments. It should be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting. The definitions and explanations of terms herein supplement the technical and scientific meanings of terms commonly understood and accepted in the art of this teaching.

[0024] It should be understood that although the terms first, second, third, etc., may be used herein to describe various elements or components, these elements or components should not be limited by these terms. These terms are only used to distinguish one element or component from another. Therefore, without departing from the teachings of the inventive concept, the first element or component discussed below may be referred to as the second element or component.

[0025] As used in the specification and appended claims, the singular forms of the terms “a,” “an,” and “the” are intended to include both the singular and plural forms unless the context clearly specifies otherwise. Additionally, when used herein, the terms “comprising” and / or “including” and / or similar terms indicate the presence of the stated features, elements, and / or components, but do not exclude the presence or addition of one or more other features, elements, components, and / or groups thereof. The term “and / or” as used herein includes any and all combinations of one or more of the associated listed items.

[0026] Unless otherwise stated, when an element or component is referred to as "connected to," "coupled to," or "proximity to" another element or component, it should be understood that the element or component may be directly connected or coupled to the other element or component, or that there may be intermediate elements or components. That is, these and similar terms cover the situation in which one or more intermediate elements or components may be used to connect two elements or components. However, when an element or component is referred to as "directly connected" to another element or component, this only covers the situation in which two elements or components are connected to each other without any intermediate or intermediate elements or components.

[0027] Therefore, through various aspects, embodiments, and / or specific features or sub-components of this disclosure, this disclosure is intended to provide one or more advantages as specifically pointed out below.

[0028] As described in this article, the technique of creating a pseudo-3D appearance by adding virtual lighting effects to ultrasound images to improve the visualization of blood vessels can be enhanced by applying surface shading calculations at multiple scales (e.g., to consider multiple blood vessel sizes existing in a single image) and selecting the optimal scale on a pixel-by-pixel basis.

[0029] Figure 1 The illustration shows a system 100 for scale-adaptive ultrasonic surface coloring according to a representative embodiment.

[0030] Figure 1 System 100 is a system for scale-adaptive ultrasonic surface coloration and includes components that can be supplied together or deployed in a distributed manner. The system includes an ultrasonic probe 110, an ultrasonic base 120, and a display 180.

[0031] The ultrasound probe 110 includes at least a transducer array 113 and processing circuitry 115. The transducer array 113 comprises an array of individual transducer elements, including a first transducer element 1131, a second transducer element 1132, up to an Xth transducer element 113X. The transducer array 113 may include three or more (e.g., tens, hundreds, or even thousands) transducer elements. The processing circuitry 115 may include a combination of a memory storing instructions and a processor executing those instructions, for example, processing ultrasound images received via the transducer array 113. The processing circuitry 115 may also, or alternatively, include application-specific integrated circuit (ASIC) circuitry elements.

[0032] The ultrasound base 120 includes a first interface 121, a second interface 122, a third interface 123, and a controller 150. The third interface 123 is a user interface and can be used by a user to interact with the ultrasound base 120 (e.g., input commands and receive outputs). Figure 1 In this configuration, the controller 150 includes a memory 151 for storing instructions and a processor 152 for executing instructions. In some embodiments, Figure 1 The controller 150 may also include a third interface 123, and may also be considered to include a display 180 as a user interface.

[0033] The controller 150 includes at least a memory 151 for storing instructions and a processor 152 for executing instructions. Figure 4 The document describes a computer that can be used to implement controller 150, but controller 150 may include more than Figure 1 or Figure 4 More or fewer elements as depicted. In some embodiments, Figure 1 The various components of system 100 may include a controller (e.g., controller 150).

[0034] Display 180 may be located locally on controller 150 or may be remotely connected to controller 150. Display 180 includes a graphical user interface 181 (GUI) for displaying the various types of images and data described herein. Display 180 may be connected to controller 150 via a local wired interface, such as an Ethernet cable, or via a local wireless interface, such as a Wi-Fi connection. Display 180 may be connected to other user input device interfaces through which a user can input commands, including a mouse, keyboard, thumbwheel, etc. Display 180 may be a monitor, such as a computer monitor, a display on a mobile device, an augmented reality display, a television, an electronic whiteboard, or another screen configured to display electronic images. Display 180 may also include one or more input interfaces (e.g., input interfaces that can be connected to other elements or components as described above), and an interactive touchscreen configured to display prompts to the user and collect touch input from the user.

[0035] Controller 150 may directly perform some of the operations described herein, and may indirectly perform other operations described herein. For example, controller 150 may indirectly control operations, for instance, by generating and transmitting content to be displayed on display 180. Controller 150 may directly control other operations (e.g., logical operations performed by processor 152 executing instructions from memory 151) based on input received via an interface from electronic components and / or a user. Therefore, when processor 152 executes instructions from memory 151, the process implemented by controller 150 may include steps not directly performed by controller 150.

[0036] When executed by processor 152, instructions stored in memory 151 cause system 100 to perform the following operations: for each of a plurality of pixels in a two-dimensional ultrasound image, identify an optimal scale for applying surface coloring based on the size of at least one blood vessel in the two-dimensional ultrasound image. The plurality of pixels may include all pixels in the two-dimensional ultrasound image, or may include a subset of all pixels in the two-dimensional ultrasound image, such as every other pixel in rows and / or columns of pixels in the entire two-dimensional ultrasound image, and / or pixels in portions of the two-dimensional ultrasound image that do not include the entire two-dimensional ultrasound image. The instructions also cause system 100 to perform the following operations: apply surface coloring to at least one first pixel based on the identified optimal scale for at least one first pixel; and apply surface coloring to at least one second pixel based on the identified optimal scale for at least one second pixel. In some embodiments, the optimal scale for at least one first pixel and at least one second pixel are the same and therefore a single universal scale, such as the average optimal scale calculated across the entire image by considering all image pixels of the image. In other embodiments, surface coloring is applied to at least one first pixel at a first scale based on the first scale identified as the optimal scale for at least one first pixel; and surface coloring is applied to at least one second pixel at a second scale based on the second scale identified as the optimal scale for at least one second pixel.

[0037] exist Figure 1 In this design, controller 150 may include two separate modules. These two separate modules may be separable in terms of the program stored in memory 151 and / or in terms of different processors or application-specific circuitry (ASICs) used to implement the modules. The first module can automatically calculate the scale of one or more blood vessels present in the RGB image to achieve optimal illumination. Parameter optimization is automated to make controller 150 more "plug-and-play" and require less manual adjustment, thus accelerating the workflow. The second module can locally process the scale of the luminance image to generate a luminance map that automatically adapts to different regions of anatomical structures of varying sizes that may exist in the two-dimensional ultrasound image. When multiple vessel sizes are present, the second module improves the visual aspects of the vessels while providing better vessel differentiation.

[0038] System 100 combines elements that can be derived from... Figure 5The module is implemented in further detail with respect to three related components. The module can be implemented as a hardware component (e.g., an application-specific integrated circuit) or by a hardware component and / or by a software component executed by one or more processors. The first component may include a scale estimation module that takes one or more images as input and generates a scale map. The scale map is a two-dimensional image that provides a pixel-wise estimate of the size of structures present in the vicinity of a pixel. The second component is a multi-scale filtering method that processes the luminance image to generate a luminance map, which is used to produce a smooth gradient vector field suitable for the local vascular scale. The filtering method may be based on a low-pass first-derivative filter bank that convolves the input image with N>1 scale-increasing filters and computes a set of N finite-difference gradient vector fields, one for each scale. The third component may include a multi-scale coloring module that applies coloring computations according to a surface reflection model (e.g., Brin-Von), calculating surface normals based on the (input) scale map value at the pixel by using different gradient vectors for each pixel.

[0039] Figure 2 Another system for scale-adaptive ultrasonic surface coloring is illustrated according to a representative embodiment.

[0040] Figure 2 System 200 includes ultrasound system 210. Additionally, while smartphone A and smartphone B are not necessarily considered... Figure 2 The system 200 is part of the system, but the ultrasound application 299A on smartphone A and the ultrasound application 299B on smartphone B can be considered as part of the system 200. Figure 2 Part of system 200. Although not shown, Figure 2 The arrangement may also include instruments for measuring arterial pressure.

[0041] The ultrasonic system 210 includes a transducer array 213, a lens 214, a controller 250, a user interface 223, and a wireless communication circuit 290. Figure 2In this system, controller 250 includes a memory 251 for storing instructions and a processor 252 for executing instructions. In some embodiments, controller 250 may also be considered to include a user interface 223. In some implementations, smartphones A and B may be considered part of system 200, for example, when issued by the same entity that owns or otherwise controls ultrasound system 210. Smartphones A and B communicate with ultrasound system 210 via network 201 and wireless communication circuitry 290. Network 201 may include one or more of a wireless local area network (LAN) (e.g., a Wi-Fi network) and / or a wired or wireless wide area network (WAN) (e.g., a cellular network and / or the Internet). Ultrasound system 210 may include an ultrasound probe (e.g., a portable ultrasound probe) that works in conjunction with ultrasound application 299A and / or ultrasound application 299B to perform ultrasound examinations.

[0042] When executed by processor 252, instructions stored in memory 251 cause system 200 to perform the following operations: for each pixel of a plurality of pixels in each two-dimensional ultrasound image, identify an optimal scale for applying surface coloring based on the size of at least one blood vessel in the two-dimensional ultrasound image. The plurality of pixels may include all pixels in the two-dimensional ultrasound image, or may include a subset of all pixels in the two-dimensional ultrasound image, such as every other pixel, and / or pixels in the two-dimensional ultrasound image that do not include a portion of the entire two-dimensional ultrasound image. The instructions also cause system 200 to perform the following operations: apply surface coloring to at least one first pixel based on the identified optimal scale for at least one first pixel; and apply surface coloring to at least one second pixel based on the identified optimal scale for at least one second pixel. In some embodiments, Figure 1 Some or all of the functions attributable to controller 150 are not implemented by controller 250, but by... Figure 2 The process is performed on either smartphone A or smartphone B. Additionally, in some embodiments, the optimal scale for at least one first pixel and at least one second pixel is the same, and is therefore a single universal scale, such as the average optimal scale calculated across the entire image by considering all image pixels. In other embodiments, surface shading is applied to at least one first pixel based on the first scale identified as the optimal scale for at least one first pixel; and surface shading is applied to at least one second pixel based on the second scale identified as the optimal scale for at least one second pixel.

[0043] exist Figure 2In this embodiment, controller 250 may include two separate modules. These two separate modules may be separable in terms of the program stored in memory 251 and / or in terms of different processors or application-specific circuitry (ASICs) used to implement the modules. The first module may automatically calculate the scale of one or more blood vessels present in the RGB image to achieve optimal illumination. Parameter optimization is automated to make controller 150 more "plug-and-play" and require less manual adjustment, thus accelerating the workflow. The second module may locally process the scale of the luminance image to generate a luminance map that automatically adapts to different regions of the two-dimensional ultrasound image where anatomical structures of different sizes may exist. When multiple vessel sizes are present, the second module improves the visual aspects of the vessels while providing better vessel differentiation. In some embodiments, the first and second modules may be implemented in separate components, for example, if the first module is implemented in controller 250 and instances of the second module are implemented in smartphones A and B. In some embodiments, the first and second modules may be implemented in smartphones A and B, rather than in controller 250.

[0044] Figure 2 The arrangement in the diagram combines three related elements, which can be derived from... Figure 5 The modules are implemented in further detail. Modules may be implemented as hardware elements (e.g., application-specific integrated circuits) or by hardware elements and / or as software elements or by software elements executed by one or more processors. A first element may include a scale estimation module that takes one or more images as input and generates a scale map. The scale map is a two-dimensional image that provides a pixel-wise estimate of the size of structures present in the vicinity of a pixel. A second element is a multi-scale filtering method that processes the luminance image to generate a luminance map used to produce a smooth gradient vector field suitable for the local vascular scale. The filtering method may be based on a low-pass first-derivative filter bank that convolves the input image with N>1 scale-increasing filters and computes a set of N finite-difference gradient vector fields, one for each scale. A third element may include a multi-scale coloring module that applies coloring computations according to a surface reflection model (e.g., Brin-Von), calculating surface normals based on the (input) scale map value at the pixel by using different gradient vectors for each pixel.

[0045] Figure 3A The illustration depicts a method for scale-adaptive ultrasonic surface coloring according to a representative embodiment.

[0046] Figure 3A The method can be composed of, including Figure 1 The controller 150 in the system 100 executes, or is controlled by, the system 100. Figure 2 The controller 250 and / or one or more of smartphone A and / or smartphone B are executed.

[0047] Figure 3A The method begins at S310 with receiving a two-dimensional ultrasound image. S310 may include receiving a single two-dimensional ultrasound image or more two-dimensional ultrasound images, and then selecting one of the two-dimensional ultrasound images, for example, based on quality control. The two-dimensional ultrasound image may include a two-dimensional red / green / blue (RGB) ultrasound image (e.g., a color-coded Doppler velocity map).

[0048] At S320, a two-dimensional luminance image is received or generated. The two-dimensional luminance image may include the power component of the Doppler signal, which is the basis of the Doppler velocity map when the two-dimensional ultrasound image includes a color-coded Doppler velocity map.

[0049] At S325, a two-dimensional ultrasound image and a two-dimensional luminance image are input. For example, the two-dimensional ultrasound image and the two-dimensional luminance image can be input for processing by a hardware module in a controller 150, controller 250, or smartphone A or smartphone B as described herein, and / or by a software module executed by the controller 150, controller 250, or smartphone A or smartphone B.

[0050] At S330, at least one blood vessel is identified. More than one blood vessel can be identified, and each of the multiple such blood vessels can have a different size. Blood vessels (one or more) can be identified using an image analysis program trained to recognize blood vessels and the characteristics of such blood vessels (e.g., the size of each of the multiple blood vessels). For example, at least one blood vessel may include a first blood vessel having a first size and a second blood vessel having a second size.

[0051] At S340, the scale of each blood vessel in at least one blood vessel is calculated. An optimal scale can be identified for each pixel among multiple pixels in the 2D ultrasound image. The optimal scale for each pixel is used to apply surface shading based on the size of at least one blood vessel in the 2D ultrasound image. The method in Figure 3 may include automatically calculating the scale of each blood vessel in at least one blood vessel in each 2D ultrasound image to produce optimal illumination for the 2D ultrasound image. For example, the estimated blood vessel size can take values ​​between 1 pixel and 32 pixels in radius, and the scale can be calculated for each blood vessel based on the range of pixel values. As an example, the scale of the first blood vessel can be calculated as 1 pixel, and the scale of the second blood vessel can be calculated as 3 pixels, where the pre-filtering range (sigma) is between .25 pixels and 8 pixels.

[0052] For example, when the blood vessels identified at S330 include a first blood vessel and a second blood vessel, a first pixel can be used for the first blood vessel, and a first scale can be identified based on a first size of the first blood vessel. A second pixel can be used for the second blood vessel, and a second scale can be identified based on a second size of the second blood vessel. One or more scales of one or more pixels can be identified based on the first size of the first blood vessel, the second size of the second blood vessel, and other factors such as proximity to the nearest edge of the first blood vessel and / or the second blood vessel.

[0053] At S360, a luminance map is generated to identify the optimal scale. The method in Figure 3 may include generating a luminance map for each two-dimensional ultrasound image, adapted to the size of each vessel in at least one vessel, to identify the optimal scale for applying surface shading to the two-dimensional ultrasound image. The luminance map may map different optimal scales for each pixel in the two-dimensional ultrasound image or a subset of pixels in the two-dimensional ultrasound image (e.g., every other pixel in rows and / or columns and / or pixels in a portion of the two-dimensional ultrasound image smaller than the entire two-dimensional ultrasound image). As mentioned above, the estimated vessel size, expressed in radius, can take values ​​ranging from 1 pixel to 32 pixels, and the pre-filtered scale (sigma) can range between .25 pixels and 8 pixels. Different optimal scales can be mapped in the luminance map based on these ranges for each pixel or for a subset of pixels.

[0054] At S370, a two-dimensional luminance is applied to the two-dimensional ultrasound image by applying a luminance map from S360, thereby applying surface shading at a first scale and at a second scale. As explained herein, the luminance map is a derivative of the original two-dimensional luminance image and may include a gradient vector field customized for each pixel, allowing surface shading to be applied to at least one first pixel and at least one second pixel. Applying surface shading gives the two-dimensional ultrasound image a three-dimensional appearance.

[0055] Applying two-dimensional luminance at S370 may include: applying surface shading to at least one first pixel at the first scale, based on identifying a first scale as the optimal scale for at least one first pixel. Applying two-dimensional luminance at S370 may also include: applying surface shading to at least one second pixel at the second scale, based on identifying a second scale as the optimal scale for at least one second pixel. At S370, more than two scales can be used because each of the multiple pixels can be applied at different scales.

[0056] At S380, it is determined whether the two-dimensional ultrasound image (one or more) received at S310 is the final two-dimensional ultrasound image to be received. If the two-dimensional ultrasound image (one or more) received at S310 is the final two-dimensional ultrasound image (S380 = Yes), then the method ends.

[0057] If the two-dimensional ultrasound image (one or more) received at S310 is not the final two-dimensional ultrasound image (S380 = No), then the method returns to S310 to receive one or more additional two-dimensional ultrasound images.

[0058] Figure 3B The illustration shows another method for scale-adaptive ultrasonic surface coloring according to a representative embodiment.

[0059] Figure 3B The method can be composed of, including Figure 1 The controller 150 in the system 100 executes, or is controlled by, the system 100. Figure 2 The controller 250 and / or one or more of smartphone A and / or smartphone B are executed.

[0060] Figure 3B The method also begins at S310 with receiving a two-dimensional ultrasound image. S310 may include receiving a single two-dimensional ultrasound image or more two-dimensional ultrasound images, and then selecting one of the two-dimensional ultrasound images, for example, based on quality control.

[0061] At S320, a two-dimensional luminance image is received or generated.

[0062] At S330, at least one blood vessel is identified.

[0063] At S341, a two-dimensional scale map is generated by calculating the scale of each blood vessel in at least one blood vessel. A two-dimensional scale map can be generated for each two-dimensional ultrasound image. Based on the size of the blood vessel present near at least one first pixel and the size of the blood vessel near at least one second pixel in the two-dimensional ultrasound image, the scale map can reflect a first scale of at least one first pixel and a second scale of at least one second pixel.

[0064] At S345, one or more filters are selected from the filter bank. The filters may include: a first filter selected from the filter bank based on a first scale for each of at least one first pixel, and a second filter selected from the filter bank based on a second scale for each of at least one second pixel.

[0065] At S350, one or more gradient vector fields are generated. A gradient vector field should be understood as an image comprising a two-dimensional array of three-dimensional vectors. The gradient vector fields (one or more) can be adapted to the size of blood vessels near at least one first pixel and the size of blood vessels near at least one second pixel in a two-dimensional ultrasound image. The gradient vector fields (one or more) can be generated by using a first filter and a second filter selected from a filter bank at S345.

[0066] At S361, surface shading is applied at a first scale and at a second scale. Surface shading can be applied to each two-dimensional ultrasound image by using different gradient vectors for each of at least one first pixel and at least one second pixel.

[0067] At S380, it is determined whether the two-dimensional ultrasound image (one or more) received at S310 is the final two-dimensional ultrasound image to be received. If the two-dimensional ultrasound image (one or more) received at S310 is the final two-dimensional ultrasound image (S380 = Yes), then the method ends.

[0068] If the two-dimensional ultrasound image (one or more) received at S310 is not the final two-dimensional ultrasound image (S380 = No), then the method returns to S310 to receive one or more additional two-dimensional ultrasound images.

[0069] Figure 4 The illustration shows the effect of different filter scaling in scale-adaptive ultrasonic surface coloring according to a representative embodiment.

[0070] Figure 4 The illustration shows the different levels of detail produced by five different gradient filter scales applied to the same underlying RGB image. As shown, the lowest scale of the input RGB image on the left shows the most detail in terms of depth, as indicated by the arrows, representing the tissue. For each input RGB image on the right, the level of detail gradually decreases, such that the highest scale of the RGB image on the right shows the least detail in terms of depth, representing the tissue indicated by the arrows. (Using...) Figure 1 System 100 or Figure 2 The scaling mechanism can automatically optimize the scale based on the size of blood vessels present in an RGB image to achieve optimal results in visual separation. Automated scaling can be applied to RGB images containing multiple blood vessel sizes.

[0071] Figure 4 It can display standard color ultrasound images, with colors including red, blue, gray, and black. It can be used for... Figure 4 The brightness values ​​of each pixel are adjusted (virtual lighting) to provide a virtualized 3D appearance. The brightness values ​​can be varied to provide a more realistic effect. Figure 4The ultrasound images used in this study estimate different heights / depths of surfaces, where the correspondence is generally that the brighter the pixel, the higher the surface at that pixel. Gray and black are used for the background B-mode in the ultrasound images. Blue and red indicate anatomical structures (e.g., blood vessels and veins) with opposite blood flow directions relative to the transducer, and the shading, reflected by different luminance values, provides a highlight to the virtualized 3D impression. By constructing surfaces, a different channel than that used in the color ultrasound images is used to provide a highlight to the 3D effect in the color ultrasound images. The colors in the color ultrasound images depend on the blood flow velocity estimated from the Doppler signal, while the surface height depends on the power of the Doppler signal used to estimate the blood flow velocity.

[0072] Conventional colorization of ultrasound images has at least two major drawbacks. The first problem is that a single-scale parameter is used to pre-filter the ultrasound image before colorization. For example... Figure 4 As shown, different scales provide completely different effects. Figure 4 The illustration shows that the optimal scale value is directly related to the size of the blood vessel being scanned. Therefore, ideally, pre-selecting a single scale should not be the basis for scaling, as blood vessel size varies widely depending on the anatomical view and the patient being examined.

[0073] The second problem is that ultrasound images typically include multiple blood vessels of different sizes (e.g., the large vein of the portal vein and the small artery of the hepatic artery). Therefore, even if the user can interactively adjust a single scale via UI controls, it is impossible to select the optimal single scale for all blood vessels present in different parts of the same ultrasound image. Furthermore, the optimal scale is actually determined by… Local blood vessel size It is known that the size of local blood vessels can vary greatly between different pixels, so ideally the scale should be adaptively adjusted at different parts of the image to match the size of local blood vessels.

[0074] As described in this paper, the first problem can be solved by automatically estimating the size of blood vessels in an image and then setting the image scale pixel-by-pixel based on the size of the blood vessels. The second problem can be solved by automatically estimating the local blood vessel size pixel-by-pixel. The result can be scaling based on blood vessel size and / or multi-scale scaling.

[0075] The optimal scale can be generated for each pixel on the scale map based on the input ultrasound image. A scale can be generated for the entire image, or a scale value can be generated for each pixel of the image, as this allows for the provision of the optimal scale through automated means. Alternatively, different scales can be generated, for example, by automatically estimating the scale of each vessel on a pixel-by-pixel basis.

[0076] Illumination, which is typically derived from a single scale, is now performed pixel-by-pixel. Instead of being set by the user, a single scale value is now automatically generated for the entire image. Illumination is calculated using multi-scale values ​​generated pixel-by-pixel. The multi-scale map is pixel-by-pixel, but not necessarily for every single pixel in the ultrasound image. Interpolation can be used, and any level of resolution can be used. Furthermore, the pixel-by-pixel scale map can have a different overall size than the ultrasound image (e.g., for only a subset of the ultrasound image). For example, the scale can be estimated for every other pixel.

[0077] Figure 5 The illustration shows a hybrid process for scale-adaptive ultrasonic surface coloring according to a representative embodiment.

[0078] exist Figure 5 In this process, luminance image processing is performed at steps S510, S520, S530, S540, and S550. At step S550, RGB image processing is also performed.

[0079] Specifically, at S510, scale estimation is applied to the luminance image, and at S530, a scale map is generated based on the scale estimation.

[0080] At S520, the filter bank scales #1…N are applied to the luminance image to generate gradient vector fields #1…N at S540. As previously described, the filtering method can be based on a low-pass first-order derivative filter bank that convolves the input image with filters of scales N>1 and computes a set of N finite-difference gradient vector fields, one for each scale. For example, the set of gradient vector fields could include seven gradient vector fields for seven different scales.

[0081] The scale map generated at S530 is applied to the RGB image by using a 1...N gradient vector field generated at S540 to color at S550 in a multi-scale manner.

[0082] At S560, the output image is obtained by applying the scale map to the RGB image using the 1...N gradient vector field at S550.

[0083] exist Figure 5The starting point is a two-dimensional RGB image and a luminance image. The luminance image is input into scale estimation processing at S510 to generate a scale map at S530. A filter bank is applied to the luminance image at S520 to generate a gradient vector field at S540. At S550, the scale map and the gradient vector field generated by the filter bank are combined with the RGB image to apply multi-scale colorization, and the result is output as the output image. The RGB image and luminance image are for a single-layer ultrasound image. Regardless of whether a global single-scale version or a pixel-by-pixel version is intended, the RGB image can be segmented first.

[0084] From this article and Figure 5 The teachings add three related elements: scale estimation applied to the luminance image by the first module at S510, filter bank scales #1…N applied to the luminance image by the second module at S520, and multi-scale colorization performed at S550. The first element may include a scale estimation module that takes one or more images as input and generates a scale map. The scale map is a two-dimensional image that provides a pixel-wise estimate of the size of structures present in the vicinity of a pixel. The second element is a multi-scale filtering method that processes the luminance image to generate a luminance map used to produce a smooth gradient vector field adapted to the local vascular scale. The filtering method may be based on a low-pass first-derivative filter bank that convolves the input image with filters of N>1 increasing scales and computes a set of N finite-difference gradient vector fields, one for each scale. The third element may include a multi-scale shading module that applies shading calculations based on a surface reflection model (e.g., Brin-Von) and calculates the surface normal by using different gradient vectors for each pixel, based on the input scale map value at that pixel.

[0085] Figure 6 The illustration shows scale estimation in scale-adaptive ultrasonic surface coloring according to a representative embodiment.

[0086] Figure 6 The processing flow of the first module used for scale estimation is illustrated. First, the luminance image is masked by applying a threshold dependent on the noise level. Based on the binary mask, an accurate Euclidean distance transform can be generated using, for example, the Maurer algorithm, which encodes the distance from any given pixel to the nearest boundary point. These first steps are common to both single-scale and multi-scale methods.

[0087] In the single-scale method, the optimal scale (S) for automatic adaptation can be estimated by approximating the radius along the centerline, as follows: Where d(x) is the distance transformation value at pixel x. The higher d(x), the closer x is to the center line or central axis of the shape. The weighted average equation described above does not calculate the standard average distance within the shape (which would underestimate vessel size), but rather approximates the average radius along the center line by assigning greater weight to higher distance values ​​(corresponding to points closer to the center line) and less weight to lower distance values ​​(corresponding to points closer to the boundary). The advantage of this technique lies in its versatility and limited complexity, as it does not require explicit calculation of the center line curve.

[0088] The same principle can be used for local scaling, rather than global scaling over the entire shape. For this purpose, the distance transform is locally processed using a Gaussian kernel with standard deviation σ, as follows: exist Figure 6 In the diagram, masking luminance is shown at 601 and applied to generate a distance transform at 602. The distance transform is applied to generate a global scale map at 603 and a local scale map at 604. The global scale map at 603 is based on the ratio of the sum of distance^2(x) to the sum of distance(x). The local scale map at 604 is based on the scale factor. and The ratio.

[0089] exist Figure 6 In China, regarding such Figure 5 The scale estimation at S510 allows the luminance image to be thresholded to produce masked luminance at 601 (as shown for black and white). The masked luminance at 601 reflects whether the signal is above or below the threshold; therefore, if below the threshold, the color signal is a black pixel, and if above the threshold, the color signal is valid and is a white pixel. Figure 6 The binary image of the masking luminance at 601 provides the shape of the different blood vessels already present in the ultrasound image, and this may be key to determining the blood vessel size for each pixel.

[0090] Next Figure 6In the diagram, a distance transform is generated at position 602. The distance transform at position 602 is generated from the masking luminance at position 601 by providing variable values ​​reflecting the distance from the pixel to the nearest edge of the blood vessel. Blue can be used to reflect a relatively close distance to the nearest edge, yellow can be used to reflect a relatively medium distance, and red can be used to reflect a relatively far distance. Red can be used for the centers(one or more) of large blood vessels, as these centers have the farthest distance to the edge of the blood vessel in the distance transform at position 602. The global scale map at position 603 reflects the result of applying the first summation function shown on the right to the distance transform at position 602, while the local scale map at position 604 reflects the result of applying the second summation function shown on the right to the distance transform at position 602. The global scale map at position 603 can have a binary appearance reflecting a dedicated average value using a single scale value, while the local scale map at position 604 can have more than two colors, such as dark blue outside the blood vessel, light blue around the edges of smaller and larger blood vessels, and yellow at the center(one or more) of large blood vessels. The local scale map involves different scales for pixel variations based on the size of the blood vessel in the pixel. The local scale map at 604 can be pixel-by-pixel or for a subset of pixels (e.g., every other pixel).

[0091] The global scale diagram 603 can be simply analogized to a perfect blood vessel, such as a solid tube with a centerline and a uniform radius from the centerline to the edge of the tube. For each point along the centerline, a size estimate of the blood vessel can be generated, such that the scale will reflect the average of the size estimates generated for each point along the centerline. The global scale then provides an estimate of the size of the solid tube as a blood vessel.

[0092] Blood vessels with perfect structures (such as solid tubes) are almost nonexistent. However, the distance transformation at 602 reflects the highest value that would appear at the relative center of the blood vessel, thus providing an approximation of the centerline. For the equation, distance d is used as a weight in the weighted map to average the distances across the global scale map at 603. The average value along the centerline is taken, weighted according to the probability that a pixel lies on the centerline, and the resulting probability is proportional to the distance.

[0093] based on Figure 6In processing, pixels of two anatomical structures of the same size can have similar or identical luminance values, even if the two structures are of different types (e.g., organs and veins). On the other hand, pixels of two anatomical structures of different sizes can have significantly different luminance values, even if the two structures are of the same type (e.g., two veins). Even if two anatomical structures are intertwined, similar or different luminance values ​​may be produced as long as the processing can distinguish between the different anatomical structures. Luminance values ​​are assigned based on a scale reflecting the size of the anatomical structure of the pixel. The luminance value is a function of the size of the anatomical structure.

[0094] Figure 7 The illustration compares single-scale and multi-scale methods in scale-adaptive ultrasonic surface staining according to representative embodiments. That is, Figure 7 Single-scale and multi-scale methods for processing luminance images to generate luminance maps are compared.

[0095] like Figure 6 As shown, in the single-scale method, a low-pass filter with a scale σ given by the global scale map calculated by the first module can be used to process the luminance image.

[0096] In multi-scale methods, luminance images can be processed pixel-by-pixel based on local scale values. For this purpose, a multi-scale implementation based on a Gaussian scale space can be used. A scale space is generated by applying low-pass filters across all scales obtained from the local scale map. The scale map is then used to select the most suitable scale for each pixel. This implementation avoids pixel-by-pixel binary decision-making that promotes temporal consistency of processing. Furthermore, this implementation does not use image intensity information, making it independent of image type. To reduce computational cost, the filter bank can be computed only at a finite number of scales. The processed luminance map values ​​for a pixel at a given scale s_(n+ε) are obtained by linear interpolation between the pixel values ​​at consecutive scales s_n and s_(n+1).

[0097] exist Figure 7 In the diagram, a luminance map is shown at 701 and applied to generate a single-scale map at 703 and a multi-scale map at 704. The luminance map at 701 can be provided along with the RGB image for processing as described herein. A scale space 740 of a filter scale group 750 is shown, comprising seven filter scale maps from a first filter scale map 751 to a seventh filter scale map 757. These seven filter scales comprise the filter scale space of the luminance map and can be selectively applied pixel-by-pixel to the multi-scale map at 704 to generate a processed luminance map in the lower right. The single-scale map at 703 is applied to the luminance map at 701 to generate a processed luminance map in the upper right.

[0098] exist Figure 7 Multiple scale maps can be provided in filter scale group 750. Seven different scale maps from the first filter scale map 751 to the seventh filter scale map 757 are shown. The luminance value of a pixel depends on the size of the local anatomy. Figure 7 In the filter scale group 750, the filter scale map is used to establish luminance values ​​against the processed luminance map in the lower right corner against the background of virtual lighting.

[0099] exist Figure 7 In this context, filter scale group 750 is used for preprocessing of the virtual lighting to pre-filter the luminance image before any subsequent processing. Figure 7 Instead of a low-pass filter, such as the one traditionally used to apply a single-scale Gaussian filter to the entire ultrasound image, a filter bank is used to select a scale map for each pixel being processed, so as to take into account the size of the blood vessels represented by a subset of pixels. Figure 7 In this process, the scale pre-filter parameters depend on the scale map from the filter bank, eliminating the need for complex individual scale values. The scale space 740 can include a color table, for example, from dark blue to light blue to green to yellow, with seven different colors corresponding to the filter scale maps in the filter scale bank 750. Different scale maps correspond to different levels of blur in the image. The scale maps in the filter scale bank 750 are pre-computed. As an example, the filter scale bank 750 can include four, eight, or sixteen scale maps, and the available resolution can vary based on the number of scale maps to be used. The system produces results using the relative positions of pixels in the underlying image and the scale maps. Additionally, the system can interpolate using two scale maps. The resulting processed luminance map depends on the scale maps in the filter bank.

[0100] Because of the use of filter scale group 750, the scale value of pixels that do not correspond to blood vessels can be zero, while the scale value of pixels that correspond to blood vessels will vary according to the size of the corresponding blood vessel. The scale map in filter scale group 750 will be used to calculate the illumination of each pixel in a pixel-by-pixel setting.

[0101] Figure 8 The illustration compares the final coloring results obtained using a single-scale method and a multi-scale method in scale-adaptive ultrasonic surface coloring according to a representative embodiment.

[0102] The second module is used for implementation. Figure 8 Multi-scale shading is applied to the RGB image, and discrete gradients in the x and y directions are calculated based on the processed luminance map. These discrete gradients are then fed into the surface shading model to illuminate the RGB image and provide a 3D effect. Figure 8A comparison of the final coloring is provided when using single-scale and multi-scale methods. Figure 8 In each of the three sets of images (left, middle, and right), the hepatic artery is better distinguished from the portal vein in images obtained using a multi-scale approach.

[0103] exist Figure 8 In the diagram, the original shading is shown on the left side of the user interface 881. The lighting using single-scale shading is shown on the middle user interface 881. The lighting using multi-scale shading is shown on the right side of the user interface 881.

[0104] exist Figure 8 In order to apply virtual lighting to a surface, the surface normal is first required. The lighting calculation depends on the position of one or more light sources and the orientation of the surface normal relative to the light sources. An X vector, a Y vector, and a third vector perpendicular to the surface are needed. The third vector is calculated by computing the gradient of the image. Scale space is used for the gradient, reflecting the differences between adjacent pixels, as this provides the slope of the image as the spatial derivative of the image reflecting the slope of the surface.

[0105] exist Figure 8 In the diagram, arrows point to different anatomical structures, which the system distinguishes by using dimensions to determine scale values. For example, one arrow might point to the larger portal vein, while another might point to the smaller hepatic artery. Virtual lighting applied on a pixel-by-pixel basis provides improved discrimination, even when such anatomical structures are adjacent or close together (as shown by the smaller portion indicated by the left arrow relative to the larger anatomical structure indicated by the right arrow). Different blood vessels can be distinguished using shading in the color version.

[0106] exist Figure 8 In the image, the entire middle image uses only a single scale, while the image on the right uses multi-scale lighting. The user interface clearly demonstrates this to the operator. Figure 8 The details of the anatomical structures, including the different structures that are connected or close to each other.

[0107] Figure 9 The illustration shows a computer system on which a method for scale-adaptive ultrasonic surface coloring is implemented according to another representative embodiment.

[0108] refer to Figure 9The computer system 900 includes a set of software instructions that can be executed to cause the computer system 900 to perform any of the methods or computer-based functions disclosed herein. The computer system 900 can operate as a standalone device or can be connected to other computer systems or peripheral devices, for example, using a network 901. In an embodiment, the computer system 900 performs logic processing based on digital signals received via an analog-to-digital converter.

[0109] In a networked deployment, computer system 900 operates as a server or client user computer in a server-client user network environment, or as a peer-to-peer computer system in a point-to-point (or distributed) network environment. Computer system 900 can also be implemented as or incorporated into various devices, such as workstations, fixed computers, mobile computers, personal computers (PCs), laptops, tablets, or any other machine capable of executing a set of (sequential or other) software instructions specifying the actions to be taken by that machine, including a controller (e.g., controller 150 or controller 250). Computer system 900 can be incorporated as a device or incorporated into a device that is itself part of an integrated system including additional devices. In embodiments, computer system 900 can be implemented using electronic devices that provide voice, video, or data communication. Furthermore, although computer system 900 is illustrated in the singular, the term "system" should also be considered as including any collection of systems or subsystems that individually or jointly execute one or more sets of software instructions to perform one or more computer functions.

[0110] like Figure 9As illustrated, computer system 900 includes processor 910. Processor 910 can be considered a representative example of a processor of a controller and executes instructions to implement some or all aspects of the methods and processes described herein. Processor 910 is tangible and non-transient. As used herein, the term "non-transient" is not interpreted as a permanent state characteristic, but rather as a state characteristic that will last for a period of time. The term "non-transient" explicitly excludes transient characteristics, such as carrier waves or signals, or other forms of characteristics that exist only temporarily at any time and place. Processor 910 is an article of manufacture and / or a machine part. Processor 910 is configured to execute software instructions to perform the functions described in the various embodiments herein. Processor 910 may be a general-purpose processor or may be a portion of an application-specific integrated circuit (ASIC). Processor 910 may also be a microprocessor, microcomputer, processor chip, controller, microcontroller, digital signal processor (DSP), state machine, or programmable logic device. Processor 910 may also be logic circuitry, including programmable gate arrays (PGAs), such as field-programmable gate arrays (FPGAs), or another type of circuitry including discrete gate and / or transistor logic. The processor 910 may be a central processing unit (CPU), a graphics processing unit (GPU), or both. Additionally, any processor described herein may include multiple processors, parallel processors, or both. Multiple processors may be included in a single device or multiple devices, or coupled to a single device or multiple devices.

[0111] As used herein, the term "processor" encompasses any electronic component capable of executing programs or machine-executable instructions. References to computing devices that include "processor" should be interpreted as including more than one processor or processing core, as in a multi-core processor. A processor can also refer to a collection of processors within a single computer system or distributed across multiple computer systems. The term computing device should also be interpreted as a collection or network of computing devices, each comprising one or more processors. A program has software instructions that are executed by one or more processors, which may be located within the same computing device or distributed across multiple computing devices.

[0112] Computer system 900 also includes main memory 920 and static memory 930, wherein the memories in computer system 900 communicate with each other and with processor 910 via bus 908. Either or both of main memory 920 and static memory 930 can be considered representative examples of the memory of a controller and store instructions for implementing some or all aspects of the methods and processes described herein. The memory described herein is a tangible storage medium for storing data and executable software instructions, and is non-transient during the time period in which the software instructions are stored. As used herein, the term "non-transient" should not be construed as a permanent state characteristic, but rather as a state characteristic that will persist for a period of time. The term "non-transient" explicitly excludes transient characteristics, such as carrier waves or signals, or other forms of characteristics that exist only temporarily at any time and place. Main memory 920 and static memory 930 are articles of manufacture and / or machine parts. Main memory 920 and static memory 930 are computer-readable media from which a computer (e.g., processor 910) can read data and executable software instructions. Each of the main memory 920 and static memory 930 may be implemented as one or more of random access memory (RAM), read-only memory (ROM), flash memory, electrically programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, removable disks, magnetic tapes, optical disc read-only memory (CD-ROM), digital versatile disks (DVDs), floppy disks, Blu-ray discs, or any other form of storage medium known in the art. The memory may be volatile or non-volatile, secure and / or encrypted, insecure and / or unencrypted.

[0113] "Memory" is an example of a computer-readable storage medium. Computer memory is any memory that can be directly accessed by a processor. Examples of computer memory include, but are not limited to, RAM, registers, and register files. The reference to "computer memory" or "memory" should be interpreted as potentially referring to multiple memories. Memory can be, for example, multiple memories within the same computer system. Memory can also be multiple memories distributed across multiple computer systems or computing devices.

[0114] As shown in the figure, the computer system 900 also includes a video display unit 950, such as a liquid crystal display (LCD), an organic light-emitting diode (OLED), a flat panel display, a solid-state display, or a cathode ray tube (CRT). Additionally, the computer system 900 includes an input device 960 (e.g., a keyboard / virtual keyboard, a touch-sensitive input screen, or voice input with voice recognition), and a cursor control device 970 (e.g., a mouse, a touch-sensitive input screen, or a touchpad). The computer system 900 may also optionally include a disk drive unit 980, a signal generation device 990 (e.g., a speaker or a remote control), and / or a network interface device 940.

[0115] In one embodiment, such as Figure 9 As shown, the disk drive unit 980 includes a computer-readable medium 982 in which one or more software instruction sets 984 (software) are embedded. The software instruction sets 984 are read from the computer-readable medium 982 for execution by the processor 910. Additionally, when executed by the processor 910, the software instructions 984 perform one or more steps of the methods and processes described herein. In embodiments, the software instructions 984 reside wholly or partially within main memory 920, static memory 930, and / or processor 910 during execution by the computer system 900. Furthermore, the computer-readable medium 982 may include the software instructions 984, or receive and execute the software instructions 984 in response to a propagation signal, causing a device connected to the network 901 to transmit voice, video, or data through the network 901. The software instructions 984 may be sent or received via the network 901 via a network interface device 940.

[0116] In the embodiments, dedicated hardware implementations (e.g., application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), programmable logic arrays, and other hardware components) are configured to implement one or more methods described herein. One or more embodiments described herein may be implemented using two or more specific interconnected hardware modules or devices with associated control and data signals that can communicate between and through the modules. Therefore, this disclosure covers software, firmware, and hardware implementations. Nothing in this application should be construed as being implemented or feasible solely using software and not hardware (e.g., tangible non-transient processors and / or memory).

[0117] According to various embodiments of this disclosure, the methods described herein can be implemented using a hardware computer system that executes software programs. Additionally, in exemplary non-limiting embodiments, implementations may include distributed processing, component / object distributed processing, and parallel processing. Virtual computer system processing can implement one or more of the methods or functions described herein, and the processors described herein can be used to support virtual processing environments.

[0118] Accordingly, scale-adaptive ultrasound surface shading enables the creation of pseudo-3D appearances by applying surface shading calculations at multiple scales and selecting the optimal scale on a pixel-by-pixel basis (e.g., to take into account multiple vessel sizes present in a single image), thereby achieving improved visualization of vessels in techniques that add virtual lighting effects to ultrasound Doppler images.

[0119] The teachings herein can be provided as improvements to flow viewers and / or on systems for obstetric, vascular, and musculoskeletal applications, such as the EPIQ or Affiniti systems. These teachings are applicable to all flow imaging modalities in flow viewers, including color Doppler, energy Doppler (also known as CPA), directional energy Doppler (also known as dCPA), microfluidic imaging (also known as MFI), and high-resolution MFI (also known as MFI-HD) during real-time imaging and post-processing.

[0120] While scale-adaptive ultrasonic surface colorization has been described with reference to several exemplary embodiments, it should be understood that the language used is descriptive and illustrative, not restrictive. Changes may be made within the scope of the appended claims (as described and modified herein) without departing from the scope of scale-adaptive ultrasonic surface colorization in its various aspects. Although scale-adaptive ultrasonic surface colorization has been described with reference to specific means, materials, and embodiments, it is not intended to be limited to the specific content disclosed; rather, it extends to all functionally equivalent structures, methods, and uses (e.g., within the scope of the appended claims).

[0121] The illustrations of the embodiments described herein are intended to provide a general understanding of the structure of various embodiments. The illustrations are not intended to be a complete description of all elements and features of this disclosure described herein. Many other embodiments will be apparent to those skilled in the art upon reading this disclosure, and other embodiments can be utilized and derived from this disclosure, allowing structural and logical substitutions and changes to be made without departing from the scope of this disclosure. Additionally, the illustrations are merely representative and may not be drawn to scale. Some scales in the illustrations may be exaggerated, while others may be minimized. Therefore, this disclosure and the accompanying drawings should be considered illustrative rather than restrictive.

[0122] One or more embodiments of this disclosure may be referred to herein individually and / or collectively by the term "invention," merely for convenience and not intended to actively limit the scope of this application to any particular invention or inventive concept. Furthermore, while specific embodiments have been illustrated and described herein, it should be understood that any subsequent arrangements designed to achieve the same or similar purpose may replace the specific embodiments shown. This disclosure is intended to cover any and all subsequent modifications or variations of the various embodiments. Combinations of the above embodiments and other embodiments not specifically described herein will be apparent to those skilled in the art upon reading this specification.

[0123] Furthermore, in the foregoing detailed embodiments, various features may be combined or described in a single embodiment for the purpose of simplifying this disclosure. This disclosure should not be construed as reflecting an intention that the claimed embodiments require more features than expressly recited in each claim. Rather, as reflected in the foregoing claims, the inventive subject matter may involve fewer features than any of the disclosed embodiments. Therefore, the following claims are incorporated into the detailed embodiments, each of which stands independently and defines the claimed subject matter.

[0124] The foregoing description of the disclosed embodiments is provided to enable those skilled in the art to practice the concepts described in this disclosure. Therefore, the subject matter disclosed above is to be considered illustrative rather than restrictive, and the appended claims are intended to cover all such modifications, enhancements, and other embodiments falling within the scope of the claims. Accordingly, to the fullest extent permitted by law, the scope of this disclosure should be determined by the broadest permissible interpretation of the appended claims and their equivalents, and should not be construed as limited or restricted by the foregoing detailed description.

Claims

1. An ultrasound system (210), comprising: The interface receives two-dimensional ultrasound images; as well as The processor (152) is configured as follows: For each pixel of the two-dimensional ultrasound image, the optimal scale for applying surface coloring is identified based on the size of at least one blood vessel in the two-dimensional ultrasound image (S340 / S341). Based on the identification of the optimal scale for at least one first pixel, surface coloring is applied to the at least one first pixel (S361 / S370). as well as Based on the identification of the optimal scale for at least one second pixel, surface coloring is applied to the at least one second pixel (S361 / S370).

2. The ultrasound system (210) according to claim 1, wherein, The processor (152) is also configured to: Based on identifying a first scale as the optimal scale for the at least one first pixel, surface shading is applied to the at least one first pixel at the first scale; as well as Based on identifying the second scale as the optimal scale for the at least one second pixel, surface coloring is applied to the at least one second pixel at the second scale. The at least one blood vessel includes a first blood vessel having a first size and a second blood vessel having a second size. Wherein, the at least one first pixel includes pixels of the first blood vessel, and the first size is identified based on the first scale of the first blood vessel. Wherein, the at least one second pixel includes the pixel of the second blood vessel, and the second size is identified based on the second scale of the second blood vessel.

3. The ultrasound system (210) according to claim 1, wherein, The processor (152) is also configured to: Receive the two-dimensional luminance image of the two-dimensional ultrasound image; A luminance map is generated based on the luminance image; and The two-dimensional luminance map is applied to the two-dimensional ultrasound image to apply surface shading to the at least one first pixel and the at least one second pixel, wherein the surface shading is applied to give the two-dimensional ultrasound image a three-dimensional appearance.

4. The ultrasound system (210) according to claim 1, wherein, The two-dimensional ultrasound image includes red / green / blue RGB images.

5. The ultrasound system (210) according to claim 1, wherein the processor (152) is further configured to: Automatically calculate the scale of each blood vessel in at least one blood vessel in the two-dimensional ultrasound image to produce optimal illumination for the two-dimensional ultrasound image; and A luminance map suitable for the size of each blood vessel in the at least one blood vessel is generated to identify the optimal scale for applying surface coloring to the two-dimensional ultrasound image.

6. The ultrasound system (210) according to claim 2, wherein, The processor (152) is also configured to: For the two-dimensional ultrasound image, a two-dimensional scale map is generated based on the size of the blood vessels near the at least one first pixel and the size of the blood vessels near the at least one second pixel in the two-dimensional ultrasound image. The two-dimensional scale map reflects the first scale of the at least one first pixel and the second scale of the at least one second pixel. Generate a gradient vector field suitable for the following: the size of blood vessels near at least one first pixel and the size of blood vessels near at least one second pixel in the two-dimensional ultrasound image, and For each of the at least one first pixel and the at least one second pixel, coloring is applied to the two-dimensional ultrasound image by using different gradient vectors from the gradient vector field.

7. The ultrasound system (210) according to claim 6, wherein, The gradient vector field is generated by using a first filter selected from a filter bank based on the first scale for each first pixel in at least one first pixel. And for each of the at least one second pixel, it is generated by using a second filter selected from the filter bank based on the second scale.

8. The ultrasound system (210) according to any one of claims 1 to 7, further comprising: An interface for displaying received two-dimensional ultrasound images and / or for displaying processed ultrasound images according to any one of claims 1 to 7.

9. A method for applying virtual illumination to an ultrasound image, the method comprising: Receive two-dimensional ultrasound images; For each pixel in a plurality of pixels in a two-dimensional ultrasound image, the optimal scale for applying surface coloring is identified based on the size of at least one blood vessel in the two-dimensional ultrasound image (S340 / S341). Based on the identification of the optimal scale for at least one first pixel, surface coloring is applied to the at least one first pixel (S361 / S370). as well as Based on the identification of the optimal scale for at least one second pixel, surface coloring is applied to the at least one second pixel (S361 / S370).

10. The method of claim 9, further comprising: Based on identifying a first scale as the optimal scale for the at least one first pixel, surface shading is applied to the at least one first pixel at the first scale; as well as Based on identifying the second scale as the optimal scale for the at least one second pixel, surface coloring is applied to the at least one second pixel at the second scale. The at least one blood vessel includes a first blood vessel having a first size and a second blood vessel having a second size. Wherein, the first pixel is used for the first blood vessel, and the first scale is identified based on the first size of the first blood vessel. The second pixel is used for the second blood vessel, and the second scale is identified based on the second size of the second blood vessel.

11. The method of claim 10, further comprising: Receive the two-dimensional luminance image of the two-dimensional ultrasound image; Generate a luminance map based on the luminance image; as well as The two-dimensional luminance map is applied to the two-dimensional ultrasound image to apply surface shading to the at least one first pixel and the at least one second pixel, wherein the surface shading is applied to give the two-dimensional ultrasound image a three-dimensional appearance.

12. The method according to claim 9, further comprising: The scale of each blood vessel in at least one blood vessel in the two-dimensional ultrasound image is automatically calculated to produce the optimal illumination effect for the two-dimensional ultrasound image; as well as For the two-dimensional ultrasound image, a luminance map suitable for the size of each blood vessel in the at least one blood vessel is generated to identify the optimal scale for applying surface coloring to the two-dimensional ultrasound image.

13. The method of claim 9, further comprising: For the two-dimensional ultrasound image, a two-dimensional scale map is generated based on the size of the blood vessels near the at least one first pixel and the size of the blood vessels near the at least one second pixel in the two-dimensional ultrasound image. The two-dimensional scale map reflects the first scale of the at least one first pixel and the second scale of the at least one second pixel. Generate a gradient vector field suitable for the following: the size of blood vessels near at least one first pixel and the size of blood vessels near at least one second pixel in the two-dimensional ultrasound image, and For each of the at least one first pixel and the at least one second pixel, coloring is applied to the two-dimensional ultrasound image by using different gradient vectors from the gradient vector field.

14. The method according to claim 13, wherein, The gradient vector field is generated by using a first filter selected from a filter bank based on the first scale for each first pixel in at least one first pixel. And for each of the at least one second pixel, it is generated by using a second filter selected from the filter bank based on the second scale.

15. A computer program product comprising instructions that, when executed by a processor, cause the processor to perform the method according to any one of claims 9 to 14.

Images