Method and system for segmentation and rendering of inverted data

Abstract

To preprocess three dimensional data prior to generating inverted renders.SOLUTION: The preprocessing includes segmenting the 3D data to remove portions of the data associated with noise such that those portions do not appear in the generated render. The segmentation may include applying a mask to the 3D data. The mask may be generated by sorting data points in the 3D data set into a first set or a second set. The 3D data may be filtered prior to generating the mask. The mask may be adjusted based on feature recognition. The preprocessing may allow the visualization of hypoechoic regions of interest in a volume.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] This application relates to inverse rendering of 3D data. More specifically, this application relates to preprocessing of 3D data prior to inverse rendering of the data.

Background Art

[0002] An ultrasonic image is generated by transmitting an ultrasonic signal to a subject (e.g., a test subject) and processing an echo received from the transmitted signal. In a medical image, the echo is typically generated by a tissue structure (e.g., cell wall, binding fiber) that reflects the ultrasonic signal. The ultrasonic signal can be transmitted through a volume to generate (e.g., render) a three-dimensional (3D) image of the volume and receive an echo from the volume. For example, in medical imaging, a 3D image of an organ (e.g., heart, liver) or a part of an organ can be generated. The 3D image may be in color or grayscale where the color and / or brightness of a voxel corresponds to the intensity of the echo signal at that position. That is, a voxel having the highest intensity signal is rendered with the brightest value (e.g., white in grayscale), and a voxel having the lowest intensity signal is rendered with the darkest value (e.g., black in grayscale). Positive rendering tends to emphasize acoustically reflective structures, while low signal areas are not emphasized.

[0003] Certain features within a volume may be of interest even if they provide little or no echogenicity (e.g., hypoechoic). For example, blood vessels, the gallbladder, and some lesions may be rendered as dark areas in a 3D image of the anatomical structure of interest. Ultrasound systems can produce negative or "inverted renderings" of 3D images. In inverted rendering, voxels with the highest intensity signals are rendered at the darkest values ​​(e.g., black in grayscale), and voxels with the lowest intensity signals are rendered at the brightest values ​​(e.g., white in grayscale). Inverted rendering may allow users to more easily observe hypoechoic areas of interest. [Overview of the project] [Problems that the invention aims to solve]

[0004] However, low-signal regions that do not contain features of interest (e.g., poorly coupled tissue, shadowing, or areas obscured by noise) are also low-echo and can be emphasized in inverted rendering. In some cases, low-signal regions may dominate the inverted rendering, obscuring the low-echo regions of interest. This can reduce or eliminate the effectiveness of inverted rendering. [Means for solving the problem]

[0005] Methods and systems for preprocessing 3D data before generating inverted renderings are described. Preprocessing 3D data may include segmenting the 3D data into two or more sets, for example, data to be rendered and data that is not allowed to contribute to the 3D rendered image. Segmentation may be performed by applying a mask to the 3D dataset. In some examples, the mask is a binary mask (for example, data points are divided into two sets). classification(This is done). Preprocessing may include reducing noise before segmentation. Reducing noise may include applying a smoothing kernel and / or an adaptive smoothing (e.g., persistence) algorithm. Reducing noise may improve segmentation performance depending on the application. Preprocessing 3D data before generating inverted renderings can reduce obscuration of low-echo regions of interest due to noise in some applications.

[0006] According to at least one example disclosed herein, the method converts the data points of a 3D dataset into one of a first set or a second set. classification This is a step in which the 3D dataset defines the first volume and the first set classification The mask data points corresponding to the data points of the 3D dataset are set to the first value, and the second set classification The process may include the steps of setting the mask data points corresponding to the data points of the 3D dataset to a second value, where the outer boundary of the mask set's data points to the first value defines a second volume within the first volume; applying the mask to the 3D dataset such that the data points of the 3D dataset corresponding to the data points of the 3D dataset outside the second volume are discarded, and a subset of the 3D dataset is generated; and generating an inverted rendering from the subset of the 3D dataset.

[0007] According to at least one example disclosed herein, the system includes a non-temporal computer-readable medium containing a three-dimensional dataset defining a volume, and the data points of the 3D dataset into one of a first set or a second set. classification And, in the first set classification Generate mask data points corresponding to the data points of the 3D dataset, and then add them to the second set. classification The mask data points corresponding to the data points in the 3D dataset are assigned to the first value and to the second set. classificationThis may include a processor configured to determine, as a second value, the data points of a mask corresponding to the data points of a 3D dataset, as a first value, the outer boundary of the data points of the mask set, define a second volume within a first volume based on the outer boundary, apply the mask to the 3D dataset to discard the data points of the 3D dataset corresponding to the data points outside the second volume, and generate an inverted rendering from a subset of the 3D dataset. [Brief explanation of the drawing]

[0008] [Figure 1] This shows a typical ultrasound scan of the volume within the liver. [Figure 2] This shows other representative ultrasound scans of liver volume. [Figure 3] This is a block diagram of an ultrasound imaging system configured according to some embodiments of the present disclosure. [Figure 4] The images show a typical ultrasound scan of a volume including the gallbladder and a scan processed according to some embodiments of the present disclosure. [Figure 5] The following are representative processed ultrasonic scans, masks, and inverted renderings of processed ultrasonic scans according to several embodiments of the present disclosure. [Figure 6] This disclosure shows typical inverted renderings of representative ultrasonic scans without segmentation and typical inverted renderings of representative ultrasonic scans with segmentation, according to some embodiments of this disclosure. [Figure 7] This is a flowchart of a method according to some embodiments of the present disclosure. [Figure 8] A block diagram shows an exemplary processor according to some embodiments of the present disclosure. [Modes for carrying out the invention]

[0009] The following description of specific exemplary embodiments is illustrative in nature and is not intended in any way to limit the invention or its applications or uses. The following detailed description of embodiments of the System and Method refers to the accompanying drawings, which form part of this specification and illustrate specific embodiments that can carry out the System and Method described. These embodiments are described in sufficient detail to enable those skilled in the art to carry out the disclosed System and Method, and it should be understood that other embodiments may be used, and structural and logical modifications may be made without departing from the spirit and scope of the System. Furthermore, for clarity, detailed descriptions of specific features are omitted where they are obvious to those skilled in the art, so as not to obscure the description of the System. Accordingly, the following detailed description should not be interpreted restrictively, and the scope of the System is defined solely by the appended claims.

[0010] As mentioned above, positive rendering of a 3D dataset visualizes voxels associated with high-intensity echo signals as light values ​​or light values ​​(e.g., white), while voxels associated with low-intensity echo signals are visualized as dark values ​​(e.g., black). Generally, positive rendering, which seeks to extract the brightest structures (e.g., the most acoustically reflective) in a region of interest, generally functions as intended by utilizing opacity maps and optimization controls based on the expected acoustic properties of the objects being imaged, e.g., tissues and / or organs queried via ultrasound. Opacity maps and optimization controls are included in many current ultrasound imaging systems.

[0011] Inverse 3D rendering has the potential to highlight blood vessels and hypoechoic lesions within uniform / bright tissue, but its usefulness can be significantly limited by the inversion of hypoechoic regions present in the 3D rendering image, even though it does not include features of interest (e.g., areas without signal or noise).

[0012] Figure 1 shows a typical ultrasound scan of a volume within the liver. An example of when inverted rendering is desirable is in gastrointestinal (GI) applications, such as liver imaging. A common problem is that the liver may be imaged through ribs or other difficult acoustic windows. The liver may be contained in the center of the imaging volume, but the outer regions of the volume contain noise. Shadowing from ribs, bones, and / or refraction can generate bright artifacts in inverted rendering. These regions do not contain useful signals but may dominate the final inverted rendering, projecting inward from the outer edges.

[0013] As shown in Figure 1, pane 102 shows a coronal slice of the liver passing through the center of the volume being scanned. Pane 104 shows a sagittal slice of the liver passing through the center of the volume being scanned, and pane 106 shows a transverse slice of the liver passing through the center of the volume being scanned. Two hypoechoic lesions 110 can be observed in all three panels. Several areas of low signal can also be observed, for example, within the window panes of pane 104, areas 112 and 114. These low signal areas 112, 114 may be the result of rib shadows, insufficient acoustic coupling of the transducer to the subject's skin, and / or other factors. Regardless of the cause of the low signal, the low signal areas 112, 114 on the periphery of the volume may appear as bright "walls" in inverted rendering, partially or completely obscuring the hypoechoic lesions 110, and these bright walls may be "cropped" by manually selecting a limited region of interest (ROI) within the volume to be rendered. As shown in panes 104 and 106, the user can interact with the ultrasonic system via the user interface to define the ROI of the volume to be rendered by manipulating the size and position of box 116 and / or line 118.

[0014] Pane 108 shows an inverted rendering of a volume that has been manually trimmed by the user. The hypoechoic lesion 110 is visible in the inverted rendering, but artifacts remain in the orthogonal dimension, which can cause problems when rotating the inverted rendering. Further trimming can remove these artifacts, but it can reduce the ability to rotate the 3D data and keep the lesion 110 within the box 116. Several rendering techniques exist for 3D datasets, including surface rendering and maximum intensity rendering.

[0015] Many rendering techniques project parallel rays through a 3D dataset from a viewpoint selected by the user or pre-set by the system to render a 2D representation (e.g., an image) of the volume associated with the 3D dataset. In surface rendering, the "nearest" voxel is rendered to produce a 2D image, i.e., the voxel first "hit" by the parallel ray is displayed. In maximum intensity rendering, the voxel with the highest intensity along the ray is rendered, or the highest intensity within a given depth range is rendered. Both surface projection and maximum projection techniques can be used for inverted rendering.

[0016] Figure 2 shows a representative ultrasound scan of a liver with blood vessels. Pane 202 shows a representative slice of the scanned volume, and Pane 204 shows an inverted rendering of the entire volume using surface rendering techniques. Low signal around the volume almost completely obscures the blood vessels in the inverted rendering in Pane 204. Panel 206 shows a representative slice of the scanned volume and a user-defined ROI box 207. Pane 208 shows a surface rendering of the trimmed volume defined by the ROI box 207. Blood vessels 209 are visible in the inverted rendering of the trimmed volume, but some details remain obscured due to improper trimming.

[0017] Pane 210 shows a representative slice of the scanned volume, and pane 212 shows a maximum intensity rendering of the entire volume using the maximum intensity technique. Similar to surface rendering, the noise at the periphery of the volume almost completely obscures any features of interest within the volume. Panel 214 shows a representative slice of the scanned volume and an ROI box 215 set by the user. Pane 216 shows a maximum intensity rendering of the trimmed volume defined by the ROI box 215. Blood vessel 217 can be seen in the maximum intensity rendering of the trimmed volume. The maximum intensity rendering can provide somewhat more detail than the surface rendering, as seen in FIG. 2, but the details are still obscured by the noise at the periphery of the volume. Both surface and maximum intensity renderings require not only manual trimming of the rendered volume but also manual user adjustment of imaging parameters such as threshold, opacity, and transparency.

[0018] As shown in FIGS. 1 and 2, for example, there is a high user workflow burden to isolate features of interest, such as using a narrow ROI box to exclude cluttered regions and adjusting settings such as threshold and transparency. In post-acquisition analysis, if it is known a priori that low echo features are captured within the volume, this process can be time-consuming but may be acceptable. In live (4D) imaging for diagnostic purposes, interference artifacts and workflow burden can significantly limit the usefulness of the maximum intensity rendering technique.

[0019] Ideally, inverse rendering in 3D or 4D should enable simple visualization of low echo regions with minimal workflow load. In the present disclosure, a "smart inversion" approach that automates the manner of segmenting a 3D data set to project only regions of a volume known to contain signals from regions of interest (e.g., tissue) rather than signals due to noise (also simply referred to as noise). In the final inverse rendering, additional emphasis such as emphasizing features of a particular class (e.g., features meeting criteria of "vascular enhancement" vs. "no vascular enhancement") or feature size (e.g., large vessels vs. small vessels) can optionally be included. Some or all of the additional extensions may be controlled by the user.

[0020] As described herein, a system such as an ultrasonic imaging system can receive or acquire a three-dimensional data set that defines a volume. For example, the 3D data set can include data from echo signals obtained by scanning a volume using an ultrasonic transducer. The 3D data set may be stored in a non-transitory computer-readable medium included in the system. The system may include a processor configured to classification place the data points of the 3D data set into one of two sets. classification Based thereon, the processor may generate a mask corresponding to the 3D data set. That is, the mask may have the same dimensions and number of data points as the 3D data set such that each data point within the mask corresponds to a data point within the 3D data set. The data points of the mask corresponding to the data points of the 3D data set placed into the first set may be set to a first value (e.g., 1), and the data points of the mask corresponding to the data points of the 3D data set placed into the second set classification may be set to a second value (e.g., 0). In some examples, the data points may be placed into more than two sets. classification classification

[0021] ​​Data points are set in various ways classification Yes, it is possible. For example, data points into two sets classification This may be done by comparing each data point to a threshold. Data points that exceed the threshold are in the first set. classification Data points below the threshold are then added to the second set. classification Data points equal to the threshold are placed in the first set in some examples, and in the second set in others. classification This is possible. The thresholds used to generate the mask may be pre-configured within the system in some applications. In other applications, the thresholds may be based on the type of examination being performed (e.g., liver examination, cardiac examination). For general examinations, typical values ​​related to the tissue of interest and / or other structures are well known and may be used to set the thresholds. In some embodiments, the system may include a user interface configured to receive user input. The user can manually set thresholds and / or load pre-configured thresholds by providing input through the user interface and / or indicating the type of examination.

[0022] In another example, to a set of data points classification This may be performed by a fuzzy logic operation. For example, a fuzzy logic operation may include a first threshold and a second threshold. The first threshold may be higher than the second threshold. Data points with values ​​exceeding the first threshold may be assigned to a first set, and data points with values ​​below the second threshold may be assigned to a second set. In some examples, data points in the first set may be rendered, and data points in the second set may not be rendered. Data points with values ​​between the two thresholds may be assigned to a third set in some examples. In some examples, data points in the third set may be further subdivided (based on, for example, a sigmoid fuzzy membership function and / or other fuzzy membership functions). classificationIt is possible. While the data points in the third set may contribute to 3D rendering in some examples, they may be rendered differently than those in the first set. For example, data points in the third set may be rendered with lower opacity values ​​(e.g., high translucency). In another example, data points in the third set may be rendered with reduced intensity values.

[0023] In another example, the deep learning algorithm being trained can characterize individual data points based on one or more learned features (e.g., proximity to other data points with similar or different values, intensity gradients across data points). Based on the characterization, the individual data points can be divided into two or more sets. classification It is possible.

[0024] A mask can be applied to a 3D dataset to segment it into data used for inverted rendering and data to be ignored or discarded during inverted rendering. For example, a processor can determine the outer boundary of data points in a mask set to a first value and define a second volume within the first volume based on the outer boundary. The second volume can define a subset of the 3D dataset containing data points in the 3D dataset related to signals from the region of interest. Data points outside the second volume can define data points in the 3D dataset related to noise. Based on the segmentation by the mask, the processor may generate an inverted rendering from the subset of the 3D dataset. The inverted rendering can be generated by any currently known or future technique, e.g., surface rendering or maximum intensity rendering. The inverted rendering may be provided on display to the user and / or stored in system memory for later viewing and / or analysis.

[0025] In some embodiments, the processor may be configured to filter the 3D dataset before generating a mask. Filtering the 3D dataset can reduce the occurrence of misclassifying signals from the region of interest as noise, and vice versa. For example, filtering can reduce the values ​​of at least some data points associated with noise so that fewer data points associated with noise have values ​​above a threshold. In another example, filtering can increase the difference between the values ​​of data points associated with signals and the values ​​of data points associated with noise. This can reduce the number of data points associated with signals from the region of interest that have values ​​below a threshold, and / or the number of data points associated with noise that have values ​​above a threshold. In some embodiments, the results of filtering can be used to set a threshold. A variety of filtering techniques can be used, including, but not limited to, smoothing kernels and adaptive smoothing (e.g., persistence) algorithms, as will be described in more detail below. The filtering technique used may be pre-configured by the system or selected by the user.

[0026] Figure 3 shows a block diagram of an ultrasound imaging system 300 configured according to the principles of the present disclosure. The ultrasound imaging system 300 according to the present disclosure may include a transducer array 314 which can be included in an ultrasound probe 312, for example, an external probe or an internal probe such as an intravascular ultrasound (IVUS) catheter probe. In other embodiments, the transducer array 314 may be in the form of a flexible array configured to conformally apply to the surface of the object being imaged (e.g., a subject). The transducer array 314 is configured to transmit an ultrasound signal (e.g., a beam, wave) and to receive echoes in response to the ultrasound signal. Various transducer arrays can be used, for example, a linear array, a curved array, or a phased array. The transducer array 314 may include, for example, a two-dimensional array of transducer elements that can scan in both elevation and azimuth dimensions for 2D and / or 3D images (as shown). As is generally known, the axial direction is perpendicular to the array plane (in the case of a curved array, the axial direction fans out), the azimuth direction is generally defined by the longitudinal dimension of the array, and the elevation direction is transverse to the azimuth direction.

[0027] In some embodiments, the transducer array 314 can be located within the ultrasonic probe 312 and coupled to a microbeamformer 316 that can control the transmission and reception of signals by the transducer elements in the array 314. In some embodiments, the microbeamformer 316 can control the transmission and reception of signals by active elements in the array 314 (e.g., an active subset of elements in the array that define an active aperture at a given point in time).

[0028] In some embodiments, the microbeamformer 316 may be coupled, for example, by a probe cable or wirelessly, to a transmit / receive (T / R) switch 318, which switches between transmit and receive and protects the main beamformer 422 from high-energy transmit signals. In some embodiments, for example, in a portable ultrasound system, the T / R switch 318 and other elements in the system may be contained within the ultrasound probe 312 rather than within an ultrasound system base that can house image processing electronics. The ultrasound system base typically includes circuitry for signal processing and image data generation, as well as software and hardware components including executable instructions for providing a user interface.

[0029] The transmission of ultrasonic signals from the transducer array 314 under the control of the microbeamformer 316 is guided by a transmit controller 320, which may be coupled to a T / R switch 318 and a main beamformer 322. The transmit controller 320 can control the direction in which the beam is steered. The beam may travel in a straight line (orthogonal) from the transducer array 314 or be steered at different angles for a wider field of view. The transmit controller 420 may also be coupled to a user interface 324 and may receive input from user-controlled user operations. The user interface 324 may include one or more input devices, such as a control panel 352, which may include one or more mechanical controls (e.g., buttons, encoders, etc.), touch-sensitive controls (e.g., trackpads, touchscreens, etc.), and / or other known input devices.

[0030] In some embodiments, the partially beamformed signals generated by the microbeamformer 316 can be coupled to a main beamformer 322, which can combine the partially beamformed signals from individual patches of transducer elements into a fully beamformed signal. In some embodiments, the microbeamformer 316 is omitted, and the transducer array 314 is under the control of the beamformer 322, which performs all beamforming of the signal. In embodiments with or without the microbeamformer 316, the beamformed signals from the beamformer 322 are coupled to a processing circuit 350, which may include one or more processors (e.g., a signal processor 326, a B-mode processor 328, a Doppler processor 360, and one or more image generation and processing components 368) configured to generate an ultrasonic image from the beamformed signals (i.e., beamformed RF data).

[0031] The signal processor 326 can be configured to process the received beamforming RF data in various ways, such as band-pass filtering, decimation, I and Q component separation, and harmonic signal separation. The signal processor 326 may also perform additional signal enhancements, such as speckle reduction, signal composite, and denoising. The processed signal (also called the I and Q components or IQ signal) may be coupled to additional downstream signal processing circuits for image generation. The IQ signal can be coupled to multiple signal paths within the system, each of which can be associated with a specific arrangement of signal processing components suitable for generating different types of image data (e.g., B-mode image data, Doppler image data). For example, the system may include a B-mode signal path 358 that couples the signal from the signal processor 326 to a B-mode processor 328 to generate B-mode image data.

[0032] The B-mode processor can use amplitude detection for imaging of internal structures. The signals generated by the B-mode processor 328 can be coupled to the scan converter 330 and / or the multiplexer formatter 332. The scan converter 330 can be configured to arrange the echo signals into a desired image format from the spatial relationship in which they are received. For example, the scan converter 330 can arrange the echo signals into a two-dimensional (2D) sector shape format, or a three-dimensional format of a pyramidal shape or other shape. The multiplanar formatter 332 can convert echoes received from a point in a common plane within a volume region of the body into an ultrasound image (e.g., a B-mode image) of that plane, for example, as described in U.S. Patent No. 6,443,896 (Detmer). In some embodiments, the scan converter 330 and the multiplanar formatter 332 may be implemented as one or more processors. The volume renderer 334 can generate an image (also called projection, rendering, or rendering) of a 3D dataset viewed from a given reference point, as described, for example, in U.S. Patent No. 6,530,885 (Entrekin et al.).

[0033] The volume renderer 334 may be implemented as one or more processors in some embodiments. The volume renderer 334 can generate renderings such as positive rendering or negative rendering by any known or future known techniques such as surface rendering and maximum intensity rendering.

[0034] According to the principles of this disclosure, the volume renderer 334 can segment a 3D dataset before generating an inverted rendering. Segmentation may be performed by applying a mask to the 3D dataset. The mask divides the data points in the 3D dataset into two sets (e.g., data points to be rendered and data points to be excluded from rendering). classificationThis may be generated by the volume renderer 334. In some examples, the volume renderer 334 can set data points in the mask corresponding to data points in the 3D dataset that have a value above a threshold (e.g., 1) corresponding to a first set of data to be rendered, and data points in the mask corresponding to data points in the 3D dataset that have a value below a second value (e.g., 0) corresponding to a second set of data that will not be rendered. The outer boundary of the data points in the mask corresponding to the first value can define a subset of the 3D dataset. The volume renderer 334 can then generate an inverted rendering of only the subset of the 3D dataset (e.g., the masked 3D dataset). Data points in the 3D volume outside the outer boundary may be ignored or discarded by the volume renderer 334.

[0035] A subset of the 3D dataset may contain data points within the outer boundary set to a second value in the mask. These data points may correspond to low-echo regions of interest in some applications. However, since the volume renderer 334 uses the outer boundary of the data points equal to the first value in the mask, the low-echo regions within the subset are not ignored or discarded by the volume renderer 334 when generating an inverted rendering.

[0036] In some applications, a hypoechoic region of interest can extend beyond the outer boundary defined by the outer boundary. For example, a blood vessel can traverse the entire scanned volume defined by the 3D dataset. In this example, portions of the blood vessel outside the outer boundary defined by the mask may be ignored or discarded by the volume renderer 334 when generating an inverted rendering. Optionally, morphological behavior and / or feature detection can be applied to the mask to preserve features of hypoechoic regions extending beyond a subset of the 3D dataset. These behaviors can adjust the outer boundary of data points, which is set to a first value. Morphological and / or feature detection can use known or future known image segmentation techniques such as the "open," "fill," and "walkout" algorithms. The mask can also be adjusted using artificial intelligence or deep learning algorithms trained to recognize morphological features.

[0037] As mentioned earlier, in some examples, a threshold is used to divide the data points into two sets. classification This is possible. The threshold used to generate the mask can be set in various ways. In any segmentation problem, it is difficult to avoid the existence of data points associated with signals from ROIs below the threshold and data points associated with noise above the threshold. The threshold should be set as accurately as possible. If the threshold is set aggressively low, the inverted rendering may not include parts of the tissue. However, if the threshold is set too high, low-echo regions of interest may become obscured. In some applications, the 3D dataset can be preprocessed before mask generation so that the volume renderer 334 can reduce the number of signal data points with values ​​below the threshold and the number of noise data points with values ​​above the threshold. Various filtering techniques can be used, as will be described in more detail below.

[0038] In some embodiments, the system may include a Doppler signal path 362 that couples the output from the signal processor 326 to the Doppler processor 360. The Doppler processor 360 may be configured to estimate the Doppler shift and generate Doppler image data. The Doppler image data may include color data that is overlaid with B-mode (i.e., grayscale) image data for display. The Doppler processor 360 may be configured, for example, to filter out unwanted signals (i.e., noise or clutter associated with non-moving tissue) using a wall filter. The Doppler processor 360 may be further configured to estimate velocity and power according to known techniques. For example, the Doppler processor may include a Doppler estimator such as an autocorrelationr, where the velocity (Doppler frequency) estimate is based on the argument of a lag-one autocorrelation function and the Doppler power estimate is based on the magnitude of a lag-zero autocorrelation function. Motion can also be estimated by known phase-domain (e.g., parametric frequency estimators such as MUSIC, ESPRIT, etc.) or time-domain (e.g., cross-correlation) signal processing techniques. Instead of, or in addition to, a velocity estimator, other estimators related to the time or spatial distribution of velocity can be used, such as acceleration or time and / or spatial velocity derivative estimators. In some examples, the velocity and power estimates may undergo further threshold detection to further reduce noise, and may also undergo segmentation and post-processing such as filling and smoothing. The velocity and power estimates can then be mapped to a desired range of display colors according to a color map. The color data, also called Doppler image data, can then be coupled to a scan converter 330, where the Doppler image data can be converted to a desired image format and overlaid on a B-mode image of the tissue structure to form a color Doppler or power Doppler image. For example, the Doppler image data may be overlaid on an inverted rendering of the tissue structure. This can enable visualization of fluid flow in low-echo regions (e.g., blood vessels).

[0039] The outputs from the scan converter 330, the multiplanar reformatter 332, and / or the volume renderer 334 (e.g., B-mode images, Doppler images) may be coupled to the image processing device 336 for further enhancement, buffering, and temporary storage before being displayed on the image display 338.

[0040] According to the principles of this disclosure, the image processing apparatus 336 can perform some or all of the pre-processing and / or post-processing of the 3D dataset before the volume renderer 334 generates an inverted rendering. For example, in one embodiment, the image processing apparatus 336 can filter the 3D dataset and / or perform feature recognition on the mask. The image processing apparatus 336 can provide the volume renderer 334 with the filtered 3D dataset and / or the modified mask to generate an inverted rendering. In some embodiments, the image processing apparatus 336 can perform post-processing of the inverted rendering, such as adjusting opacity / transparency settings, smoothing, contrast enhancement, and / or depth color coding.

[0041] The graphics processor 340 can generate graphic overlays for display along with the image. These graphic overlays may include standard identification information such as patient name, date and time of image, and imaging parameters. For these purposes, the graphics processor may be configured to receive input such as typed patient name or other annotations from the user interface 324. The user interface 344 may also be coupled to the multiplanar formatting device 332 for the selection and control of the display of multiplanar format (MPR) images.

[0042] System 300 may include local memory 342. Local memory 342 may be implemented as any suitable non-temporary computer-readable medium (e.g., a flash drive, a disk drive). Local memory 342 can store data generated by System 300, including B-mode images, masks, executable instructions, inputs provided by the user via the user interface 324, or other information necessary for the operation of System 300.

[0043] As described above, the system 300 includes a user interface 324. The user interface 324 may include a display 338 and a control panel 352. The display 338 may include a display device implemented using various known display technologies, such as LCD, LED, OLED, or plasma display technology. In some embodiments, the display 338 may include multiple displays. The control panel 352 may be configured to receive user input (e.g., threshold, filter type, rendering type). The control panel 352 may include one or more hard controls (e.g., buttons, knobs, dials, encoders, mice, trackballs, etc.). In some embodiments, the control panel 352 may additionally or alternatively include soft controls (e.g., GUI control elements or simply GUI controls) provided on a touch-sensitive display. In some embodiments, the display 338 can be a touch-sensitive display including one or more soft controls of the control panel 352. In some embodiments, various components shown in Figure 3 can be combined. For example, the image processing device 336 and the graphics processor 340 can be implemented as a single processor. In another example, the scan converter 330 and the multiplanar reformatter 332 may be implemented as a single processor.

[0044] In some embodiments, the various components shown in Figure 3 may be implemented as separate components. For example, the signal processor 326 may be implemented as a separate signal processor for each imaging mode (e.g., B-mode, Doppler). In some embodiments, one or more of the various processors shown in Figure 3 may be implemented by a general-purpose processor and / or microprocessor configured to perform a specified task. In some embodiments, one or more of the various processors may be implemented as application-specific circuits. In some embodiments, one or more of the various processors (e.g., image processing device 336) may be implemented by one or more graphical processing units (GPUs).

[0045] Figure 8 is a block diagram showing an exemplary processor 800 according to the principles of the present disclosure. Processor 800 may be used to implement one or more processors described herein, for example, the image processing apparatus 336 shown in Figure 3. Processor 800 may include, but is not limited to, any suitable processor type, including a microprocessor, a microcontroller, a digital signal processor (DSP), a field-programmable gate array (FPGA) programmed to form a processor, a graphics processing unit (GPU), an application-specific circuit (ASIC) designed to form a processor, or a combination thereof.

[0046] The processor 800 may include one or more cores 802. A core 802 may include one or more arithmetic logic units 804. In some embodiments, the core 802 may include, in addition to or instead of, an ALU 804, a floating-point logic unit (FPLU) 806 and / or a digital signal processing unit (DSPU) 808.

[0047] The processor 800 may include one or more registers 812 that are communicatively coupled to the core 802. The registers 812 may be implemented using dedicated logic gate circuits (e.g., flip-flops) and / or any memory technology.

[0048] In some embodiments, register 812 may be implemented using static memory. The register can provide data, instructions, and addresses to the core 802. In some embodiments, the processor 800 may include one or more levels of cache memory 810 which are communicably coupled to the core 802. The cache memory 810 can provide computer-readable instructions to the core 802 for execution. The cache memory 810 can provide data for processing by the core 802. In some embodiments, computer-readable instructions may be provided to the cache memory 810 by local memory, for example, local memory connected to an external bus 816. The cache memory 810 can be implemented using any suitable cache memory type, e.g., static random access memory (SRAM), dynamic random access memory (DRAM), and / or any other suitable memory technology.

[0049] The processor 800 may include a controller 814 that can control inputs to the processor 800 from other processors and / or components included in the system (e.g., the control panel 352 and scan converter 330 shown in Figure 3), and / or outputs from the processor 800 to other processors and / or components included in the system (e.g., the display 338 and volume renderer 334 shown in Figure 3). The controller 814 can control data paths in the ALU 804, FPLU 806, and / or DSPU 808. The controller 814 may be implemented as one or more state machines, data paths, and / or dedicated control logic. The gates of the controller 814 may be implemented as standalone gates, FPGAs, ASICs, or any other suitable technology.

[0050] The register 812 and cache memory 810 can communicate with the controller 814 and core 802 via internal connections 820A, 820B, 820C, and 820D. The internal connections may be implemented as buses, multiplexers, crossbar switches, and / or any other suitable connection technology.

[0051] The inputs and outputs of the processor 800 can be provided via a bus 816 which may include one or more conductive wires.

[0052] Bus 816 may be communicatively coupled to one or more components of the processor 800, such as the controller 814, the cache memory 810, and / or the register 812. Bus 816 may be coupled to one or more components of the system, such as the aforementioned display 338 and control panel 352. Bus 816 may be coupled to one or more external memories. The external memory may include read-only memory 832. ROM 832 can be masked ROM, EPROM (Electronically Programmable Read Only Memory), or other suitable technology. The external memory may include random access memory 833. RAM 833 can be static RAM, battery-backed static RAM, dynamic RAM (DRAM), or any other suitable technology. The external memory may include EEPROM (Electrically Erasable Programmable Read Only Memory) 835. The external memory may include flash memory 834. The external memory may include magnetic storage devices such as disk 836. In some embodiments, the external memory may be included in a system such as the ultrasound imaging system 300 shown in Figure 3, for example, in local memory 342.

[0053] As mentioned above, filtering of the 3D dataset may be performed before generating a mask for segmentation. In some applications, filtering the 3D dataset is necessary for segmentation (for example, classification The accuracy of filtering can be improved. Filtering can be performed by various known techniques, such as smoothing with a kernel that averages adjacent voxels, adaptive thresholding, and histogram equalization and remapping. By filtering 3D datasets, a uniform background signal can be generated, creating more separation between noise (e.g., background) and tissue signals, which may allow for more accurate segmentation between noise and signal-containing regions.

[0054] Another filtering technique that can be used is an adaptive smoothing algorithm such as the persistence algorithm (PA). The PA can negatively bias the intensity of voxels that are likely to be noisy based on a measurement or estimation of the true noise floor, and then smooth the values ​​relative to adjacent voxels. In the PA, voxels classified as "true signal" retain their original intensity and remain unblurred, preserving edges while blacking out low-echo regions. The PA may compare data points (e.g., voxels) in a 3D dataset to a noise model to determine their likelihood of being signal or noise. Data points found to be less likely to be noisy are left unchanged, while data points found to be more likely to be noisy have their values ​​reduced. The probability values ​​defining data points that are more or less likely to be noisy may be pre-set in the system or set by the user. Similarly, the amount of reduction in the value of data points that may be noisy may be pre-set in the system or set by the user. The noise model can be generated in various ways. For example, the noise model may be based on the known noise power versus gain of the receiving amplifier of the imaging system. In another example, the noise model may be based on cross-correlation of volumes continuously received from a stationary imaging object such as a phantom or air. Other methods for generating the noise model may also be used. A more detailed description of PA can be found in U.S. Patent Application No. 62 / 621,187, which is incorporated herein by reference for any purpose.

[0055] Depending on the application, the output of filtering may be used to process data points. classificationOne or more thresholds can be set to generate a mask. For example, estimating the noise floor in the PA may be used to set a threshold above the noise floor. In another example, data from histogram equalization may be used to select a threshold between histogram peaks associated with noise and histogram peaks associated with signal. In some examples, such as the fuzzy logic example, the first threshold may be selected based on histogram peaks associated with noise, and the second threshold may be selected based on histogram peaks associated with signal. In some applications, the image and / or data processed from filtering can be used by the user to generate a mask by setting data points. classification To assist the user in selecting one or more thresholds for this purpose, these may be provided.

[0056] Figure 4 shows a typical ultrasound scan of a volume including the gallbladder 400 and the processed (e.g., filtered) scan according to some embodiments of the present disclosure. Pane 402 shows the A-plane of the volume, and pane 404 shows the B-plane of the volume. The periphery of the images in both panes 402 and 404 has pixels associated with noise, but some values ​​of these pixels are similar to those of pixels associated with the tissue surrounding the gallbladder 400. Therefore, generating a mask for the volume including panes 402 and 404 could erroneously segment these areas of noise as tissue. Before generating the mask, panes 402 and 404 can be filtered. In the example shown in Figure 4, PA is used. Pane 406 shows the A-plane to be filtered, and pane 408 shows the B-plane to be filtered. As seen in panes 406 and 408, the intensity (e.g., values) of the noise-related pixels has been reduced. Next, the data points for generating the mask classification This can be done on a filtered 3D dataset. In some examples, classification This can be based on pre-configured, user-driven, or algorithm-based thresholds (one or more). In other examples, classification is deep learning or other classification Techniques can be employed. Optionally, morphological manipulation can be used to remove additional noise, segmentation, and / or holes within "true signal" regions that may be hypoechoic areas of interest, such as lesions and blood vessels. By applying a mask to the volume containing panes 406 and 408 being processed, the tissue and noise regions of the volume can be segmented more precisely.

[0057] Figure 5 shows a typical processed ultrasound scan of a volume, including the gallbladder 500, a mask, and an inverted rendering of the processed ultrasound scan, according to several embodiments of the present disclosure. Pane 502 shows the processed B-plane from Pane 408 of Figure 4. As previously stated, the B-plane is processed by applying a PA filter. Pane 504 shows the mask generated for the B-plane shown in Pane 502. Although only the mask generated for the B-plane is shown, it is understood that masks are generated for all planes in the volume (e.g., the entire 3D dataset defining the scan volume containing the B-plane shown in Pane 502).

[0058] The mask includes a first region 503 and a second region 505. The first region 503 contains data points of the mask set to a first value. The first region 503 corresponds to data points in the 3D dataset that contain signals related to tissue. The second region 505 contains data points of the mask set to a second value. The second region 505 corresponds to data points in the 3D dataset that contain noise. The mask can be applied to a 3D dataset to determine which data points in the 3D dataset are used to generate the inverted rendering. Only a subset of the data in the 3D dataset related to the first region 503 is included in the inverted rendering.

[0059] Pane 506 shows an inverted rendering of the masked 3D dataset (e.g., a subset of the 3D dataset related to the first region 503 of the mask). In the example shown in Figure 5, the inverted rendering is generated using the maximum intensity rendering technique. The gallbladder 500 is visible within the windowpane 506 and is not obscured by the bright noise "wall" around the volume being scanned.

[0060] Optionally, before generating and / or displaying the inverted rendering, the masked 3D dataset may undergo further processing. For example, aspects such as opacity / transparency mapping, smoothing (e.g., based on feature size), contrast enhancement (e.g., based on histogram), and / or color coding may be adjusted. These adjustments may be adaptive based on the imaging system's data and / or presets (e.g., expected values ​​related to the type of examination). At this same stage, further enhancements may be made to adjust the relative intensity / contrast of features determined to meet predetermined, user-driven, and / or algorithmic criteria, e.g., vascular density, lack of vascular enhancement, and / or feature size.

[0061] The resulting segmented and post-processed 3D dataset may be rendered using conventional techniques (e.g., surface rendering, maximum intensity rendering) to produce a rendering of low-echo features within the volume that is less artifact-independent and / or angle-independent. If necessary, current user control over rendering can be retained, allowing the user to modify the segmentation pattern after rendering and / or before generating the rendering and / or post-processing. For example, to customize inverted rendering, the user can adjust which features (e.g., blood vessels near the edges of a volume) and / or regions are included in and / or excluded from the inverted rendering. In some examples, the rendering can be customized by setting thresholds and / or by selecting regions of the volume in selection boxes via the user interface. In another example, during post-processing, the user can enhance features via additional user interface controls (e.g., contrast, opacity).

[0062] Figure 6 shows representative inverted renderings of a representative ultrasound scan without segmentation and a representative ultrasound scan with segmentation, according to several embodiments of the present disclosure. Pane 602 shows an inverted rendering of the volume of liver tissue including the blood vessels 606 shown in Pane 212 of Figure 2. Panel 604 shows an inverted rendering of the volume of liver tissue including the blood vessels 606 after the volume has been segmented by a mask according to the principles of the present disclosure. The inverted rendering shown in Pane 604 shows results comparable to a manually heavily cropped inverted rendering shown in Pane 216 of Figure 2. Thus, preprocessing of 3D datasets according to the principles of the present disclosure can provide better visualization of low-echo regions of interest in inverted renderings, reducing the workflow load.

[0063] Figure 7 is a flowchart 700 of a method according to some embodiments of the present disclosure. The method shown in Figure 7 may be used, for example, to generate inverted renderings shown in pane 506 of Figure 5 and pane 604 of Figure 6, according to the principles of the present disclosure. In some applications, this method can be performed by a system, such as an imaging system, for example, system 300 shown in Figure 3.

[0064] In block 704, "Transfer the data points of the three 3D datasets into one of the first or second sets." classification You may perform the "do" step. The 3D dataset can define the first volume. In some examples, data points from the 3D dataset that exceed a threshold are in the first set. classification Then, data points from the 3D dataset that fall below the threshold are added to the second set. classification In some cases, the threshold is based on the type of organization being scanned. In some cases, the threshold is set by the user. In some cases, the data points are in two or more sets. classification This may happen.

[0065] Optionally, "filtering the 3D dataset" may be performed in block 702 before comparing the data points of the 3D dataset to a threshold. This may result in a larger difference between signal and noise values. Filtering may include applying a 2D or 3D smoothing kernel, histogram equalization and remapping algorithm, and / or an adaptive smoothing (e.g., persistence) algorithm. In some examples, the threshold is set based on the filtering.

[0066] In block 706, you can perform "Generating a mask corresponding to a 3D dataset". classification The mask data points corresponding to the data points of the 3D dataset may be set to the first value, and to the second set classificationThe mask data points corresponding to the data points in the 3D dataset may be set to the second value. classification In some examples, the mask data points may be set to other values ​​corresponding to the additional set. The outer boundary of the mask data points set to the first value can define the second volume within the first volume. In some examples, the outer boundary of the mask data points set to the first value and / or other values ​​can be determined to define the second volume. Optionally, blocks 708 and 710 can be performed. Block 708 performs the "Apply image segmentation to mask" action, where the image segmentation can recognize morphological features extending from the second volume to the first volume, and block 710 performs the "Adjust second volume" step, which can adjust the second volume to include the morphological features recognized in block 708.

[0067] In block 712, "Apply a mask to the 3D dataset" may be performed. The mask may be applied such that data points in the 3D dataset corresponding to data points outside the second volume are discarded, and a subset of the 3D dataset is generated. In block 714, "Generate an inverted rendering from the subset of the 3D dataset" may be performed. In some examples, certain data points in the 3D dataset of the second volume may be rendered differently from other data points. For example, a data point with a value between two thresholds may be rendered with lower opacity than a data point with a value above the higher of the two thresholds (e.g., in a fuzzy logic operation).

[0068] In some embodiments, steps relating to some or all of the blocks can be performed by a volume renderer, for example, the volume renderer 334 of system 300 in Figure 3. In some embodiments, some of the steps may be performed by an image processing device, for example, the image processing device 336 in Figure 3. For example, blocks 702, 704, 706, 708, 710, and / or 712 may be performed by the image processing device, and block 714 may be performed by the volume renderer. In another example, blocks 704 to 714 are performed by the volume renderer, and block 702 is performed by the image processing device. Other divisions of the blocks between the volume renderer and the image processing device can also be used. The systems and methods described herein for preprocessing 3D data before generating inverted renderings can reduce obscuration of low-echo regions of interest due to noise in some applications.

[0069] The systems and methods described herein can reduce the user's workflow burden by automating at least some of the segmentation steps in several applications.

[0070] In various embodiments in which components, systems, and / or methods are implemented using computer-based systems or programmable devices such as programmable logic, it is understood that the above-described systems and methods can be implemented using any of the various known or later developed programming languages ​​such as "C", "C++", "FORTRAN", "Pascal", and "VHDL". Therefore, various storage media, such as magnetic computer disks, optical disks, and electronic memory, can be prepared, which can contain information that can instruct a device, such as a computer, to implement the above-described systems and / or methods. Once a suitable device can access the information and programs contained in the storage media, the storage media can provide the information and programs to the device, and thus the device can perform the functions of the systems and / or methods described herein. For example, if a computer disk containing suitable material such as source files, object files, and executable files is provided to a computer, the computer can receive the information, appropriately configure itself, and perform the functions of the various systems and methods outlined in the above figures and flowcharts in order to perform various functions. That is, the computer can receive various parts of the information relating to different elements of the above-described systems and / or methods from the disk, implement individual systems and / or methods, and coordinate the functions of the individual systems and / or methods described above. In consideration of this disclosure, it should be noted that the various methods and devices described herein may be implemented in hardware, software, and / or firmware. Furthermore, the various methods and parameters are included only as examples and are not included in any limiting sense.

[0071] In consideration of this disclosure, those skilled in the art can implement these teachings in determining the technologies themselves and the equipment required to influence them, while remaining within the scope of the invention. One or more functions of the processors described herein may be incorporated into fewer numbers or a single processing unit (e.g., a CPU), and may be implemented using application-specific integrated circuits (ASICs) or general-purpose processing circuits programmed in response to executable instructions to perform the functions described herein.

[0072] While this system has been described with particular reference to ultrasound imaging systems, it is also envisioned that the system can be extended to other medical imaging systems in which one or more images are obtained in a systematic manner. Therefore, the system can be used to acquire and / or record image information, but is not limited to the kidneys, testes, breasts, ovaries, uterus, thyroid, liver, lungs, musculoskeletal system, spleen, heart, arterial blood and vascular system, and other imaging applications related to ultrasound-guided interventions. Furthermore, the system may include one or more programs that can be used in conjunction with conventional imaging systems to provide the features and advantages of the system. Certain additional advantages and features of this disclosure may be apparent to those skilled in the art by examining this disclosure, or may be experienced by those skilled in the art adopting the novel systems and methods of this disclosure. Another advantage of the system and methods may be that conventional medical imaging systems can be easily upgraded to incorporate the features and advantages of the system, apparatus, and methods.

[0073] Naturally, it is understood that any one example, embodiment, or process described herein may be combined with one or more other examples, embodiments, and / or processes, or may be separated and / or performed between separate devices or device parts of the System, Device, and Method.

[0074] Finally, the above discussion is intended to be merely illustrative of the System and Method, and the appended claims should not be construed as limiting the System and Method to any particular embodiment or group of embodiments. Thus, while the System has been invented in particular detail with reference to exemplary embodiments, it will be understood that a number of modifications and alternative embodiments can be devised without departing from the broader intended spirit and scope of the System and Method described in the following claims. Accordingly, this specification and the drawings should be considered illustrative and not intended to limit the scope of the appended claims.

Claims

1. A step of classifying multiple data points of a three-dimensional dataset acquired by an ultrasound imaging system into one of a first set or a second set, wherein the three-dimensional dataset defines a first volume. A step of generating a mask corresponding to the three-dimensional dataset, wherein the pixel values ​​of the multiple data points of the mask corresponding to the multiple data points of the three-dimensional dataset classified into the first set are set to a first value, the pixel values ​​of the multiple data points of the mask corresponding to the multiple data points of the three-dimensional dataset classified into the second set are set to a second value, and the boundaries of the multiple data points of the mask corresponding to the multiple data points of the three-dimensional dataset classified into the first set define a second volume within the first volume. The steps of applying the mask to the 3D dataset to generate a subset of the 3D dataset consisting of a plurality of data points of the 3D dataset from which a plurality of data points of the 3D dataset corresponding to a plurality of data points outside the second volume have been removed, The steps include generating an inverted rendering from only a subset of the aforementioned 3D dataset, and It has, Multiple data points of the three-dimensional dataset having pixel values ​​above a threshold are classified into the first set, and multiple data points of the three-dimensional dataset having pixel values ​​below the threshold are classified into the second set. method.

2. A non-temporary computer-readable medium containing a three-dimensional dataset acquired by an ultrasonic imaging system, It is a processor, Multiple data points of the three-dimensional dataset are classified into one of the first set or the second set, and the three-dimensional dataset defines a first volume. A mask is generated corresponding to the three-dimensional dataset, the pixel values ​​of multiple data points in the mask corresponding to multiple data points of the three-dimensional dataset classified into the first set are set to a first value, and the pixel values ​​of multiple data points in the mask corresponding to multiple data points of the three-dimensional dataset classified into the second set are set to a second value. Determine the boundaries of multiple data points in a mask corresponding to multiple data points of a three-dimensional dataset classified into the first set, Based on the aforementioned boundary, a second volume is defined within the first volume. The mask is applied to the 3D dataset to generate a subset of the 3D dataset consisting of multiple data points of the 3D dataset from which multiple data points corresponding to multiple data points outside the second volume have been removed. Generate inverted renderings from only a subset of the aforementioned 3D dataset. A processor and It has, Multiple data points of the three-dimensional dataset having pixel values ​​above a threshold are classified into the first set, and multiple data points of the three-dimensional dataset having pixel values ​​below the threshold are classified into the second set. system.

Images