Spatial synthesis in ultrasound imaging

Abstract

A medical image synthesis method is provided which adaptively varies the synthesis method in different regions of an image frame as a function of the pixel position or surrounding pixel value across a series of received image planes to be synthesized. In some embodiments, for each of at least a subset of pixels in the synthesized image, one of a different set of possible synthesis methods is selected, which have maximum intensity pixel selection for regions with the highest average pixel values ​​and minimum intensity pixel selection for regions with the lowest average pixel values.

Description

Technical Field

[0001] The present invention relates to a method of spatial compounding in ultrasonic imaging.

Background Art

[0002] Spatial compounding is a method established in ultrasonic imaging. This technique aims to reduce speckle noise and improve the contrast of an image. Spatial compounding operates by non-coherent combination of two or more transmission events or reception events in different line-of-sight directions for the same object.

[0003] One type of spatial compounding is elevation-direction spatial compounding. Here, a plurality of image planes extending at different angles in the elevation direction are combined to form a composite image.

[0004] Specifically, in ultrasonic imaging, the elevation direction refers to the "out-of-plane" dimension of the ultrasonic beam. The dimensions of the ultrasonic beam can be described in the following three directions.

[0005] The azimuth (lateral) direction is the left-right width of the beam. This is also called the lateral direction or the x-axis direction.

[0006] The axial (range or depth) direction is the direction in which the beam propagates into the tissue downward from the transducer surface. This is also called the y-axis direction.

[0007] The elevation (slice thickness or out-of-plane) direction is the thickness of the beam from the front to the back. This is perpendicular to the azimuth and axial directions and is a direction that is not usually displayed in 2D imaging (i.e., in a B-mode image, it is the direction that enters and exits the image plane). It is also called the Z-axis direction.

[0008] Figure 1 (left) shows a schematic front view of the ultrasonic transducer 4, with the ultrasonic beam 6 propagating from the transducer array on its underside. This view shows the axial (y) and azimuth (x) directions. Figure 1 (right) shows a schematic side view of the beam 6, showing the axial (y) and elevation (z) dimensions of the beam.

[0009] In 2D ultrasound, images are often represented in azimuth and axial directions; while elevation contributes to slice thickness, it is not displayed as part of the image. In 3D ultrasound, volumetric images are acquired in all three dimensions.

[0010] In spatial synthesis in the elevation direction, multiple imaging planes are captured, and each is extended in a different line of sight direction in the elevation direction. This means that the planes are angled in different directions in the elevation direction. This is schematically shown in Figure 1 (right), where the two planes 8a and 8b, the first plane 8a, extends at -1 degree in the elevation direction, and the second plane 8b extends at +1 degree in the elevation direction.

[0011] In 2D array imaging, variable line-of-sight orientation can be achieved by performing receive beamforming parallel to the elevation direction at different angles. The resulting B-mode images are synthesized after log compression. This approach is beneficial because it requires only a single transmit event at the elevation angle, thus maintaining frame rate and preventing potential structural inconsistencies in the case of motion of the imaged object (such as heart movement).

[0012] One example of a spatial synthesis technique in the elevation direction uses elevation synthesis across two elevation planes, -1 degree and 1 degree. Synthesis is achieved by taking the maximum value across each pixel's plane. Figure 2 (right) shows an example of a synthesized image formed by synthesizing two elevation planes, -1 degree and 1 degree. For comparison, Figure 2 (left) shows an image of the same structure formed without synthesis. This image was acquired in an elevation plane with 0 degrees on the axis.

[0013] Conventional elevation-direction synthesis approaches have proven to be an effective solution for improving speckle appearance and reducing "dropout" in cardiac imaging. [Overview of the project] [Problems that the invention aims to solve]

[0014] However, while using the maximum value for all pixels helps to smooth the appearance of speckles, it also amplifies undesirable signals, noise, and clutter in the image. For example, the contrast of off-axis anechoic structures (such as blood vessels) decreases when the maximum value is taken. In particular, such structures should be displayed as dark (low-intensity reflection) in the image. However, using maximum intensity synthesis, these structures appear faded and with reduced contrast. This is illustrated in Figure 3. Figure 3 (top row) shows single image planes of anechoic blood vessels in a phantom, each acquired at a different orientation angle in the elevation direction. Figure 3 (top left) is the image plane oriented at -1 degree elevation. Figure 3 (top center) shows the image plane oriented at 0 degrees elevation (axis plane). Figure 3 (top right) is the image plane oriented at +1 degree elevation. No synthesis is performed on these image planes. Figure 3 (bottom) shows the composite image formed by combining the three image planes in the top row of Figure 3, each using a different synthesis approach. Figure 3 (bottom left) is a composite image formed from the average of the three single elevation planes in Figure 3 (top). Figure 3 (center) is a composite image formed by capturing pixels of the three single elevation planes with the highest intensity pixel at each pixel position.

[0015] It can be seen that blood vessels are more visible in the planes with elevation angles of -1 degree and 0 degrees (upper left and upper center) compared to the plane with elevation angle of 1 degree (upper right). Therefore, despite enhanced speckle smoothing, the contrast of blood vessels is worsened by the standard maximum elevation angle synthesis method (lower center) compared to the method that does not use elevation angle synthesis.

[0016] Using the mean value instead of the maximum value can improve vascular contrast (see Figure 3 (bottom left)), but the vascular contrast is inferior compared to the -1 degree and 0 degree images without elevation compositing.

[0017] Ideally, in the examples shown in Figures 2 and 3, the minimum number of pixels within the blood vessels is taken to maintain the contrast of the vessels. However, this compromises the reduction of speckle in other areas of the image.

[0018] An improved spatial synthesis method that can overcome the above difficulties is valuable. [Means for solving the problem]

[0019] The present invention is defined by the claims.

[0020] According to an embodiment of one aspect of the present invention, a method for spatial synthesis in ultrasonic imaging, The steps include receiving a set of two or more ultrasound image planes representing planes through an imaged object, each extending in a different line of sight direction, A step of generating a composite image from two or more sets of ultrasound image planes, wherein each pixel of the composite image is generated by combining a corresponding pixel in each of the two or more sets of received ultrasound image planes. A method is provided that has this.

[0021] The step of generating the composite image includes a step of selecting a synthesis method from a predetermined set of synthesis methods for each individual pixel of the composite image, wherein the set includes at least a step of selecting the maximum pixel value for the pixel position from the set of received image planes, and a step of selecting the minimum pixel value for the pixel position from the set of received image planes.

[0022] In some embodiments, for each of at least one subset of pixels of the synthetic image, at least the pixels of the subset having the lowest average pixel value in its defined neighborhood across corresponding pixel positions or the set of received image planes are synthesized using the minimum pixel value synthesis method, and the selection is performed such that the pixel positions or their defined neighborhoods across the corresponding set of received image planes are synthesized using the maximum pixel value synthesis method.

[0023] In some embodiments, the method includes generating an average image from a set of two or more received image planes.

[0024] In some embodiments, for at least a subset of pixels of the synthetic image, the selection may be such that at least the pixels of at least the subset within the lowest intensity pixel region of the average image are synthesized using the minimum pixel value synthesis method, and at least the pixels of at least the subset of the highest intensity pixel region of the average image are synthesized using the maximum pixel value selection method.

[0025] In some embodiments, the selection of the synthesis method for each pixel (or each of at least a subset of pixels) may include comparing the average pixel value at the corresponding pixel position, or the average pixel value in their defined neighborhood, with at least one threshold and the average image. Additionally, or alternatively, the selection can be made using a machine learning algorithm. [[ID=%13]]

[0026] Therefore, embodiments of the present invention are based on the concept of individually varying the synthesis type or method used to synthesize each pixel of a synthetic image. In some embodiments, and for at least a subset of pixels, the synthesis method may be selected based on the value of the average pixel (e.g., the mean pixel) of the received image plane at the corresponding pixel position, or the average value of pixels in a defined neighborhood of the corresponding pixel position. In other words, the synthesis method may vary across the entire synthetic image according to the average ultrasonic reflection intensity at that position in the original image. As described above, the general objective is to avoid loss of contrast or contrast resolution of anechoic structures within the image. In at least some embodiments, it is proposed to do this by using a minimum pixel value synthesis method in regions of the image with low intensity (e.g., regions containing anechoic structures) and a maximum pixel value synthesis method in regions of the image with high intensity. There are various specific methods for implementing this selection approach, as will become apparent from the following description.

[0027] One particularly advantageous application of this method is spatial synthesis in the elevation direction in ultrasonic imaging. In this case, a set of two or more ultrasonic image planes represents planes passing through the imaged object that extend in respective different viewing directions in the elevation direction. However, this method can be used for spatial synthesis of image planes angled in any direction. For example, the same method can be used when synthesizing images in a multi-planar format.

[0028] In some embodiments, at least a subset of the aforementioned pixels can be determined, at least in part, based on the variance or spread of pixel values across the image plane at each respective specific pixel position or its defined neighborhood. For example, in some embodiments, at least a subset of pixels may exclude pixels for which the variance of pixel values across the entire image plane at the pixel position or its defined neighborhood exceeds a threshold. For such pixels, a different synthesis method may be used according to a predefined additional rule.

[0029] For example, a high variance (or another measure of spread) indicates a large difference between the pixels being composited. In such situations, given the significant differences between the composited pixels, using maximum or minimum intensity projections may be undesirable. In such situations, it may be preferable to select a different compositing rule according to supplementary rules. For example, in the use case of spatial compositing in the elevation direction, a 0-degree plane value may be more appropriate. This may be added as an additional rule when assigning a compositing method to each pixel.

[0030] The selection of a method for combining each pixel can be performed, for example, by applying a predetermined selection logic, which may include a prioritized sequence of prioritized selection rules, each of which is applied in accordance with all higher-priority selection rules.

[0031] In some embodiments, at least a subset of pixels includes the majority of pixels in the composite image.

[0032] With respect to the averaged image, this may be an averaged image in some embodiments, where each pixel of the averaged image is formed by taking the average of the pixels at the corresponding pixel position in each set of received image planes, or the average of the pixel values ​​in a defined neighborhood of the corresponding pixel position in each set of received image planes. For example, the defined neighborhood is a 3x3 square region or a 4x4 square region (kernel) around each pixel.

[0033] Instead of the average, other forms of averaging can be used, such as the median, mode, weighted average, or truncated average. In some embodiments, each pixel of the averaged image may represent the variance of the pixel values ​​of the corresponding pixel in each set of received image planes.

[0034] In some embodiments, at least one threshold is an (adaptive) threshold function that represents a threshold that changes as a function of depth in the mean image.

[0035] Since the intensity of ultrasound echoes naturally decreases as a function of depth, the threshold used to determine the synthesis method to use may decrease accordingly as a function of depth.

[0036] Furthermore, or in one embodiment, at least one threshold is an adaptive threshold function that represents a threshold that changes as a function of the lateral / azimuth steering angle from which pixels in the average image are acquired. With respect to the lateral steering angle from which pixels are acquired, this refers to the angle at which the ultrasound beam is steering or directed perpendicular to the axial direction (left and right when displaying a standard ultrasound image). The signal-to-noise ratio may change with the steering angle, mainly due to the change in the incident angle of the ultrasound beam with respect to the tissue interface. A large steering angle can result in inefficient reflection of ultrasound, weakening the signal and degrading image quality. To account for these changes in signal intensity and quality, it may be beneficial to adjust the threshold of the synthesis method based on the lateral steering angle.

[0037] In some embodiments, at least one threshold (e.g., a threshold function) is calculated based on the pixel values ​​of the average image.

[0038] For example, in some embodiments, the calculation of the threshold function may include deriving a probability distribution of pixel intensity in the mean image and determining the threshold function based on the probability distribution. For example, in some embodiments, the threshold function may be defined as a function that decays from a starting maximum value, where the starting maximum value is based on the pixel value at a predetermined percentage of the probability distribution, e.g., 90 percent. This can be a linear or nonlinear function (such as an exponential function).

[0039] In some embodiments, the choice of a synthesis method for each pixel of at least a subset of pixels may include using at least one threshold to form a mask and using an average image.

[0040] For example, in some embodiments, the selection of a synthesis method for each of at least a subset of pixels may include forming a binary mask by applying at least one threshold (e.g., a threshold function) to the pixels of the mean image, where the mask pixels are assigned a value of 1 in the binary mask if the mean pixel intensity exceeds at least one threshold, and a value of 0 if the intensity is at the threshold, and vice versa. The selection of a synthesis method for each of at least a subset of pixels may then be performed based on the values ​​of the mask.

[0041] If at least one threshold is a threshold function, each average pixel value is compared to the threshold function value corresponding to the pixel's position. For example, if the threshold function is a function of depth, each average pixel is compared to the threshold function value corresponding to the depth of the associated average pixel value.

[0042] In some embodiments, when the mask value is 1, the maximum intensity synthesis method is used, and when the mask value is 0, the minimum intensity synthesis method is used (or vice versa if the mask assignment is reversed).

[0043] In some embodiments, this method may further include applying an edge-preserving smoothing filter to the binary mask to convert it to a non-binary mask before selecting a blending method for each pixel. In some embodiments, in this case, the defined set of blending methods further uses the average of the pixel values ​​of the set of received image planes at the pixel location. In this set of embodiments, the following selections for the blending method may be used: If the value of the non-binary mask at the pixel location is 1, the maximum intensity blending method is used. If the value of the non-binary mask at the pixel location is 0, the minimum intensity blending method is used. Also, if the value of the mask at the pixel location is between 0 and 1, a weighted combination of the set of blending methods is used.

[0044] The selected weighting may change in proportion to the mask value.

[0045] In some embodiments, at least one threshold is determined based on the application of a trained machine learning algorithm to a set of two or more received image planes.

[0046] In some embodiments, the calculation of binary and / or non-binary masks may include the application of a trained machine learning algorithm to a set of two or more received image planes.

[0047] As briefly mentioned above, in some embodiments, this method may also be a method for spatial synthesis in the elevation direction in ultrasound imaging, where a set of two or more ultrasound image planes represents planes passing through the imaged object extending in each different line of sight direction in the elevation direction. In spatial synthesis in the elevation direction, the ultrasound beam is transmitted at various angles in the plane of elevation or slice thickness, reducing image artifacts and improving structural representation.

[0048] Spatial synthesis can also be performed using an image plane where the line of sight changes across directional axes other than the elevation axis (such as the horizontal or azimuth direction).

[0049] Another application example of this method is when synthesizing images using the multiplanar reformatting method. In this case, spatial synthesis is used to combine 2D slices of a 3D ultrasound dataset to improve the quality of the reformatted image. This is particularly useful in obstetrics and gynecology, as it can provide more detailed images of the fetus and pelvic structure.

[0050] In some embodiments, the received set of two or more image planes includes one image plane extending at a line-of-sight angle of 0 degrees in the elevation direction. That is, this corresponds to a plane at 0 degrees elevation on the axis.

[0051] In some embodiments, the received set of two or more image planes includes one image plane extending with a line of sight angle of 0 degrees in the elevation direction, one image plane extending with a line of sight angle of 0 degrees in the elevation direction, and one image plane extending with a line of sight angle of +1 degree in the elevation direction.

[0052] In some embodiments, this method receives ultrasonic data of an imaged object generated by an array transducer, receives data spanning a volume region having elevation, axial, and transverse / azimuth directions, performs receive beamforming parallel to the elevation direction to generate data corresponding to two or more sets of planes extending in each different line of sight direction in the elevation direction, processes the generated data for the two or more planes to derive each B-mode image corresponding to the image plane extending in each of the two or more line of sight directions.

[0053] This method is a mechanical implementation method. This method may also be a computer-based implementation method. It is not a mental method.

[0054] Another aspect of the present invention is a computer program product comprising computer program code configured to perform a method in accordance with any embodiment described herein or any claim herein, when executed by a processor.

[0055] The present invention can also be implemented in hardware form.

[0056] Accordingly, another aspect of the present invention is a processing apparatus comprising one or more processors, which receives two or more sets of ultrasound image planes representing planes through imaged objects extending in different respective line-of-sight directions, generates a composite image from the two or more sets of ultrasound image planes, wherein each pixel of the composite image is generated by combining pixels corresponding to each of the two or more received sets of ultrasound image planes. The step of generating the composite image comprises, for each individual pixel of the composite image, the step of selecting a synthesis method from a predetermined set of synthesis methods, the set comprising at least the step of selecting the maximum pixel value for a pixel position from the set of received image planes, and the step of selecting the minimum pixel value for a pixel position from the set of received image planes.

[0057] In some embodiments, for at least a subset of pixels in the composite image, the selection is made by combining at least pixels of the subset having the lowest average pixel value in its defined neighborhood across the corresponding pixel location or set of received image planes using the minimum pixel value combination method, and the corresponding pixel location or its defined neighborhood across the set of received image planes using the maximum pixel value combination method.

[0058] In some embodiments, one or more processors may be further adapted to generate an average image from a set of two or more received image planes.

[0059] In some embodiments, the selection of at least a subset of pixels in the composite image may be such that at least pixels of the subset located within the lowest intensity pixel region of the average image are synthesized using the minimum pixel value synthesis method, and at least pixels of the subset located within the highest intensity pixel region of the average image are synthesized using the maximum pixel value selection method.

[0060] In some embodiments, the selection of a synthesis method for each pixel (or each of at least a subset of pixels) may include the step of comparing the average pixel value at the corresponding pixel location, or the average pixel value in their defined neighborhood, with at least one threshold and the average image. Furthermore, or alternatively, the selection may be made using a machine learning algorithm.

[0061] Another aspect of the present invention includes an ultrasonic imaging device and a processing device according to any embodiment of this disclosure or any claim of this application. The processing device is operationally coupled with the ultrasonic imaging device and adapted to receive ultrasonic data generated by the ultrasonic imaging device.

[0062] The ultrasound imaging device may include an ultrasound probe comprising an ultrasound transducer array, such as a 1.5D, 1.75D, or 2D transducer array.

[0063] These and other aspects of the present invention will become apparent from and be explained with reference to the embodiments described below.

[0064] To better understand the present invention and to more clearly illustrate how it can be put into practice, refer to the accompanying drawings as merely examples. [Brief explanation of the drawing]

[0065] [Figure 1]

[0066] The elevation angle, azimuth angle, and axial dimensions of the ultrasonic beam are roughly shown. [Figure 2]

[0067] This shows the effect of known spatial synthesis methods in the elevation direction on image quality. [Figure 3]

[0068] Compared to two composite images formed using two known synthesis methods, this shows different image quality in a single plane in the elevation direction. [Figure 4]

[0069] Steps of an exemplary method according to one or more embodiments of the present invention are outlined below. [Figure 5]

[0070] This is a block diagram of an example of a processing apparatus and system according to one or more embodiments of the present invention. [Figure 6]

[0071] A flowchart illustrating the process of forming a composite image according to one preferred set of embodiments is shown. [Figure 7]

[0072] Examples of threshold functions according to one or more embodiments are shown. [Figure 8]

[0073] This demonstrates the effect of elevation angle blending on image contrast using various methods for compositing pixels from component images. [Figure 9]

[0074] This shows the difference between a fully synthesized image formed from a single synthesis method (maximum intensity projection) and a synthesized image formed using the adaptive synthesis approach of the embodiment of the present invention. [Figure 10]

[0075] The following shows comparative images generated using a single synthesis method (maximum intensity projection method) with two different adaptive synthesis methods according to embodiments of the present invention. [Modes for carrying out the invention]

[0076] The present invention will be described with reference to the drawings.

[0077] The detailed descriptions and specific examples illustrate exemplary embodiments of the apparatus, systems, and methods, but should be understood to be for illustrative purposes only and not intended to limit the scope of the invention. These and other features, aspects, and advantages of the apparatus, systems, and methods of the invention will be better understood from the following description, the appended claims, and the appended drawings. The drawings are for illustrative purposes only and are not drawn to a specific scale. Also, the same reference numerals are used throughout the drawings to indicate the same or similar parts.

[0078] The present invention provides a medical image synthesis method in which the synthesis method adaptively changes in different regions of the image frame as a function of the reflectance intensity at or around each pixel position across the set of image planes to be synthesized. For each pixel of the synthesized image, one of a set of various synthesis methods is selected. In at least some embodiments, and for at least a subset of pixels, these may include maximum intensity pixel selection for the region with the highest average pixel value and minimum intensity pixel selection for the region with the lowest average pixel value. This results in a synthesized image with improved contrast, particularly in anechoic structures, compared to a synthesized image formed using a single synthesis method for all pixels.

[0079] Figure 4 shows the steps of an exemplary method according to one or more embodiments in block diagram form. The steps are summarized before being further described in the form of the exemplary embodiment.

[0080] Method 10 is for spatial synthesis in ultrasound medical image processing. As described above, synthesis in the context of ultrasound imaging refers to the process of combining multiple ultrasound image planes oriented at different angles (or line of sight directions) to form a single image.

[0081] Method 10 involves receiving a set of two or more ultrasound image planes 12 representing a plane through an imaged object extending in different line-of-sight directions. The term “line-of-sight direction” refers to the direction or orientation in which the image plane is angled. Otherwise, it may be called the plane direction of the ultrasound plane.

[0082] This method may further include generating an average image 14 from a set of two or more received image planes. The average image is derived, for example, by calculating the average pixel value from each corresponding pixel position across all received ultrasound image planes, or by calculating the average pixel value of a defined neighborhood around each corresponding pixel position across all received ultrasound image planes. In some examples, the mean may be the mean. For example, if there are three image planes, the value of a particular pixel in the average image can be calculated by adding the values ​​of the corresponding pixels in each of the three planes and dividing the sum by 3. Alternatively, for each pixel position, it can be calculated by taking the average of the pixel values ​​in each image plane within the neighborhood of the pixel position (for example, a 3x3 or 4x4 square neighborhood) and taking the average of these averages across the three image planes.

[0083] This process can be repeated for all pixels, resulting in an average image representing the average intensity of the image plane. Instead of using the average, a different averaging method may be used. In some examples, a weighted average may be used. The variance (or another measure of similarity or spread) of pixels in the received set of image planes is used in some examples, thereby generating a variance image instead of an average image.

[0084] However, the average is determined for each pixel position, and the pixels in the final averaged image may represent their average or correlate their values. For example, the averaged image may be a function of the average of the set of received image planes, or an image correlated with the average.

[0085] This method further includes generating 16 composite images from two or more sets of ultrasound image planes. Each pixel of the composite image is generated by combining the pixels corresponding to each of the two or more sets of received ultrasound image planes.

[0086] Step 16 for generating a composite image includes, for each individual pixel of the composite image, a step of selecting a synthesis method 22 from a predetermined set of synthesis methods, the set of which includes at least the step of selecting the maximum pixel value for a pixel position from a set of received image planes, and the step of selecting the minimum pixel value for a pixel position from a set of received image planes.

[0087] In some embodiments, for at least a subset of pixels, selection 22 combines at least pixels of the subset having the minimum mean pixel value in its defined neighborhood across the corresponding pixel location or set of received image planes using the minimum pixel value combination method, and at least pixels of the subset having the maximum mean pixel value in its defined neighborhood across the set of received image planes using the maximum pixel value combination method.

[0088] In some embodiments, for at least a subset of pixels, selection 22 may be such that at least pixels of the subset within the lowest intensity pixel region of the average image are combined using the minimum pixel value blending method, and at least pixels of the subset within the highest intensity pixel region of the average image are combined using the maximum pixel value blending method. Thus, low-intensity regions of the imaged object (such as anechoic objects like blood vessels) are rendered in the blended image with the minimum intensity pixel value, and high-intensity objects are rendered with the maximum intensity pixel value. Consequently, the contrast of the image is improved compared to blending methods that use the same blending method for all pixels.

[0089] In some embodiments, the selection of a synthesis method 22 for each pixel (or each of at least a subset of pixels) may include the step of comparing the average pixel value at the corresponding pixel location, or the average pixel value in their defined neighborhood, with respect to at least one threshold against the average image. Furthermore, or alternatively, the selection step may be performed by a machine learning algorithm, for example, applied to the average image as input and / or to a set of image planes received as input.

[0090] In some embodiments, at least a subset of pixels described above may be determined in a separate step. For example, this method may include the step of determining at least a subset of pixels based on a statistical or other measure of the pixel value at each pixel location across the received set of image planes. For example, this method may include determining at least a subset of pixels based in part on the variance or spread of pixel values ​​across the image plane at each particular pixel location or its defined neighborhood. For example, in some embodiments, at least a subset of pixels may exclude pixels where the variance of pixel values ​​across the entire image plane at the pixel location or its defined neighborhood exceeds a threshold. For such pixels, different blending methods may be used according to predefined additional rules.

[0091] For example, a high variance (or another measure of spread) indicates a large difference between the pixels being composited. In such situations, given the significant differences between the composited pixels, using the maximum or minimum intensity projection may be undesirable. In such situations, it may be preferable to select a different compositing rule according to supplementary rules. For example, in the use case of spatial compositing in the elevation direction, a 0-degree plane value may be more appropriate. This may be added as an additional rule when assigning a compositing method to each pixel.

[0092] In some embodiments, at least a subset of pixels includes the majority of pixels in the composite image.

[0093] More generally, the selection of a synthesis method for each pixel may, in some embodiments, involve the application of a predefined selection logic, which may include a set of selection rules, each applied to a subset of pixels in the synthesized image according to a criterion. This may include, for example, a hierarchical or prioritized sequence of selection rules, where each selection rule is applied to a particular pixel in accordance with all higher-priority selection rules.

[0094] As described above, the average pixel value may correspond to, for example, the average value of a particular pixel across all image planes, or, for example, the average value of neighboring pixels around a particular pixel, averaged across all image planes. As mentioned above, the threshold on which this average pixel value is compared can be determined in many different ways, and the comparison of thresholds may, in some preferred embodiments, be performed as part of a more complex selection process. For example, the comparison with a threshold is used to form a mask, which is then used to select a synthesis method. Thresholds can typically be set to distinguish different types of structures or regions in an image based on their intensity. For example, high-intensity regions may represent a particular tissue or structure, while low-intensity regions may represent regions filled with different tissues or fluids.

[0095] The received image 12 is combined 24 based on a synthesis method determined for each pixel.

[0096] As described above, this method can also be embodied in hardware form, for example, according to any example or embodiment described herein, or according to any claim of this application, or in the form of a processing unit configured to perform this method.

[0097] For further understanding, Figure 5 shows a schematic diagram of an example of a processing apparatus 32 configured to perform the method according to one or more embodiments of the present invention. The processing apparatus is shown in the context of a system 30 that includes the processing apparatus. Only the processing apparatus represents one aspect of the present invention. System 30 is another aspect of the present invention. The provided system does not have to consist of all the hardware components shown. They may consist of only a subset of them.

[0098] The processing unit 32 includes one or more processors 36 configured to perform the method in accordance with those outlined above, or in accordance with any embodiment described herein or any claim of this application. In the illustrated example, the processing unit further comprises input / output 34 or a communication interface.

[0099] In the illustrative example of Figure 5, the system 30 further comprises an ultrasound acquisition or imaging device 42 for acquiring 2D or 3D ultrasound image data 44, although this is optional. The ultrasound imaging device may include an ultrasound probe containing an ultrasound transducer array, for example, a 1.5D, 1.75D, or 2D transducer array.

[0100] The system 30 may further include in one or more processors 36 of the processing unit 32 a memory 38 for storing computer program code (i.e., computer executable code) in the manner described above, or in accordance with any embodiment described in this disclosure, or in accordance with any claim.

[0101] As described above, the present invention can also be embodied in software form. Accordingly, another aspect of the present invention is a computer program product comprising computer program code configured to perform a method in accordance with any example or embodiment of the invention described herein, or in accordance with any claim of this patent application, when executed on a processor.

[0102] Examples of methods according to a specific set of embodiments will be illustrated by illustrating the above-summarized concept of the present invention. It will be understood that not all features of this specific set of embodiments are essential to the inventive concept, but are described to aid understanding and to provide examples for illustrating the inventive concept. In this set of embodiments, the synthesis method is a method of spatial synthesis in the elevation direction in ultrasound imaging, where a set of two or more ultrasound image planes represents a plane passing through an imaged object extending in each different line of sight direction in the elevation direction. However, the same method described may be applied to synthesize multiple image planes having any variation in the plane direction.

[0103] Figure 6 shows a flowchart illustrating the main steps of an exemplary method according to the series of embodiments described herein.

[0104] This method begins with receiving a set of two or more ultrasound image planes representing planes through an imaged object extending in different line-of-sight directions. In this example, for illustrative purposes, the set consists of two image planes: a first image plane 62 and a second image plane 64. The first image plane 62 is an image plane extending at a line-of-sight angle of -1 degree in the elevation direction. The second image plane 64 is a plane extending at a line-of-sight angle of +1 degree in the elevation direction. In some embodiments, the set may additionally include a third image plane extending at a line-of-sight angle of 0 degrees in the elevation direction.

[0105] The average image 66 is calculated from the received set of image planes 62, 64, as described above. Preferably, the average image is an average image, where each pixel of the average image represents the average of the pixel values ​​in the corresponding pixel positions in each of the received image planes 62, or in their defined neighborhoods.

[0106] Next, a series of intensity-based thresholds are calculated from the average image. In other words, the at least one threshold previously referred to includes multiple thresholds, which are calculated based on the pixel values ​​of the average image. More specifically, in this series of embodiments, a threshold function is calculated, and the threshold is defined as a function of pixel position. In some embodiments, the value of the threshold function may vary as a function of the depth of the pixel in the average image. Furthermore, or in other ways, the threshold function may vary as a function of the steering angle of the lateral / azimuth angle from which the pixels in the image are acquired.

[0107] To explain, as ultrasound propagates through tissue, the wave attenuates as a function of depth, and weaker signals are reflected back to the transducer from deeper structures. As a result, the brightness of the image decreases with increasing depth. When generating an image plane, it is common to increase the gain as a function of depth to compensate for the attenuation effect (using time gain correction, TGC). As a result, both the actual signal and the noise are amplified, so the signal-to-noise ratio tends to decrease as a function of depth. Therefore, to account for these changes, the threshold used in the synthesis method can be adapted as a function of depth. For example, to ensure proper distinction of structures at different depths, a higher threshold can be applied in deeper regions compared to surface regions.

[0108] Regarding the lateral steering angle at which pixels are acquired, this refers to the angle at which the ultrasound beam is steering or directed perpendicular to the axial direction (left and right when displaying a standard ultrasound image). The signal-to-noise ratio can vary with the steering angle, primarily due to changes in the incident angle of the ultrasound beam with respect to the tissue interface. A large steering angle can result in inefficient reflection of ultrasound, weakening the signal and degrading image quality. To account for these changes in signal strength and quality, it may be beneficial to adjust the threshold of the synthesis method based on the lateral steering angle.

[0109] For simplicity, the example described here assumes that a threshold function 68 is calculated that changes only as a function of image depth. Because the threshold changes as a function of pixel position, it is sometimes called an adaptive threshold or adaptive threshold function. Its application is known as "adaptive thresholding."

[0110] To generate a threshold function from an average image, the probability distribution of pixel intensity in the average image is derived, and then the threshold function is defined as a function that decays from a starting maximum value, where the starting maximum value is based on the pixel value at a predetermined percentage of the probability distribution, e.g., 90 percent. In this case, the adjustable threshold function has two main parameters: the starting maximum percentage value and the slope of the threshold curve. An example of a threshold function is shown in Figure 7. In this example, the threshold function is a linear function, but nonlinear functions such as exponential functions can also be considered. In the illustrated example, 90 percent of the distribution of pixel values ​​in the average image 66 is used as the maximum intensity threshold at the shallowest depth. The slope and intensity thresholds at deeper depths were calculated using a decay coefficient of 0.55 dB / cm / MHz. Of course, these values ​​can be adjusted on a case-by-case basis for clinical applications, prioritizing the adoption of the maximum or minimum value as the formulation method.

[0111] The threshold function has a predefined shape or mathematical form, and only the starting maximum value is configured during the execution of the method itself. The shape or curve can be determined based on an estimate of signal loss as a function of depth. For example, this can be tailored to a particular ultrasound imaging device by estimating the signal-to-noise ratio as a function of depth for a batch of acquired images and setting the threshold function to decrease with a shape and velocity that matches the decrease in the average signal-to-noise ratio within the images.

[0112] Returning to the method in Figure 6, once the threshold function is determined, it is used to calculate the binary mask 70 of the average image, where intensities above the threshold are set to 1 and intensities below it are set to 0.

[0113] More specifically, in this method, a threshold function 68 is applied to the pixels of the average image 66 to form a binary mask. If the average pixel intensity at a pixel location exceeds the value of the threshold function corresponding to that pixel location, the mask pixel is assigned a value of 1 in the binary mask; if the intensity falls below the threshold, the mask pixel is assigned a value of 0 (or vice versa).

[0114] In some embodiments, the binary mask 70 can be used alone to determine the blending method for each pixel. For example, if the mask value is 1, the maximum intensity blending method is used, and if the mask value is 0, the minimum intensity blending method is used.

[0115] Alternatively, as shown in Figure 6, this method may further include applying an edge-preserving smoothing filter 72 (e.g., a Kuwahara filter) to the binary mask 70 to convert it to a non-binary mask before selecting a method for combining each pixel. The values ​​of the non-binary mask may vary continuously between 0 and 1.

[0116] Next, using the smoothed mask, the elevation plane can be adaptively combined such that the maximum value is obtained when the mask is equal to 1 for each pixel, the average value is obtained when the mask is between 0 and 1, and the minimum value is obtained when the mask is equal to 0.

[0117] Instead of taking an average when the mask is between 0 and 1, you can use a weighted combination of a series of synthesis methods whose weights change proportionally to the mask pixel values.

[0118] The final composite image 74 may be output to, for example, a display and / or a data store.

[0119] Figures 8 and 9 show the advantages of the proposed adaptive synthesis approach compared to the default maximum pixel synthesis approach.

[0120] Figure 8 shows a series of image planes, each corresponding to a different line of sight with a different elevation angle. Figure 8 (top left) is the image plane oriented at -1 degree elevation. Figure 8 (center) shows the image plane oriented at 0 degrees elevation (axis plane). Figure 8 (top right) shows the image plane oriented at +1 degree elevation. No image merging is performed on these image planes.

[0121] Figure 8 (bottom) shows a composite image formed by combining the three image planes in the top row of Figure 8 using different synthesis approaches. In Figure 8 (bottom left), the average intensity pixel synthesis method was used for all pixels in the image. In Figure 8 (center), the maximum intensity pixel synthesis method was used for all pixels in the image. In Figure 8 (right), an adaptive synthesis method according to an embodiment of the present invention was used.

[0122] Figure 8 shows how the adaptive approach (bottom right) better preserves anechoic vessels seen in a single elevation-direction planar image (Figure 8 (top)) while maintaining the benefits of synthetic speckle smoothing seen in the default global maximum image (Figure 8 (center)).

[0123] Figure 9 illustrates the feasibility of the proposed adaptive synthesis approach. Figure 9 (left) shows a synthesized image formed using a standard maximum intensity pixel synthesis approach for all pixels. Figure 9 (right) shows a synthesized image formed using an adaptive synthesis method according to one or more embodiments of the present invention. The adaptive synthesized image (Figure 9 (right)) demonstrates that the adaptive approach yields a speckle appearance similar to the default approach while suppressing unwanted signal enhancement in anechoic regions within the image. In particular, the white arrows in Figure 9 (left) indicate areas of unwanted signal in anechoic regions within the image. These are better suppressed in the adaptive synthesized image of Figure 9 (right).

[0124] As described above, one application of the adaptive synthesis method according to the embodiment of the present invention is for spatial synthesis in the elevation direction. In this context, one of the relevant considerations is the number of elevation planes used and the elevation angles used.

[0125] According to one embodiment, the set of received ultrasound image planes may include one image plane extending at a line of sight angle of -1 degree in the elevation direction and one image plane extending at a line of sight angle of +1 degree in the elevation direction.

[0126] In addition, it has been found that including a plane at 0 degrees elevation on the axis often improves the adaptive approach. This is illustrated in Figure 10, which shows a set of composite images. Figure 10 (left) shows a composite image formed by compositing planes at -1 degree and +1 degree elevation using a standard maximum intensity compositing approach. Figure 10 (center) shows a composite image formed by compositing planes at -1 degree and +1 degree elevation using an adaptive compositing approach according to an embodiment of the present invention. Figure 10 (right) shows a composite image formed by compositing planes at -1 degree, 0 degree, and +1 degree elevation using an adaptive compositing method according to an embodiment of the present invention.

[0127] In this example, including a plane with an elevation of 0 degrees on the axis of the adaptive approach (Figure 10 (right)) improves vascular contrast compared to an adaptive approach using only planes with elevations of -1 degree and 1 degree (Figure 10 (center)).

[0128] As variations of the proposed method, in some advantageous embodiments, the creation, smoothing, and application of the mask may be improved and optimized for different applications. In the embodiments described above, the binary mask 70 was generated based on a threshold applied to the intensity of pixels in the average image 66. As an alternative, in some embodiments, data-driven machine learning techniques may potentially be used to generate the mask.

[0129] Therefore, to state this more precisely, in some embodiments, at least one threshold (e.g., a threshold function) may be determined based on the application of a trained machine learning algorithm to a set of two or more received image planes. Furthermore, or alternatively, the calculation of binary and / or non-binary masks may include the application of a trained machine learning algorithm to a set of two or more received image planes. More generally, the selection of the synthesis method used for each pixel in the composite image may be performed using a machine learning algorithm. In practice, this can be achieved by using a machine learning algorithm to generate a mask that classifies each pixel position in the composite image in terms of the synthesis method used to form that pixel.

[0130] For example, in some embodiments, an artificial neural network may be trained to take an input set of image planes (or an input mean image) and produce as an output a mask containing a classification of each mask pixel that indicates one of a selected set of possible synthesis methods. This type of task is commonly known in this field as sematic segmentation, classifying each pixel in an image into a specific class. One type of machine learning model suitable for this task is a convolutional neural network (CNN), such as a CNN utilizing the U-Net architecture.

[0131] The training data used to train a machine learning model may include multiple training image sets, each containing a set of images of the same object taken at different viewing angles, and each training image set is further accompanied by a ground truth mask that indicates the classification of each pixel. In particular, the mask may indicate how each pixel is combined. U-Net models are typically trained using pixel-level loss functions such as cross-entropy or dice loss. During training, the model learns to associate specific image features with specific combination methods. Using this learned knowledge, it predicts how each pixel of a new, unidentified image is combined.

[0132] Each ground truth mask can be prepared manually, for example, by a human identifying pixels within an anechoic structure and determining the classification of each pixel based on that. This is just one example of an approach. Instead of a human preparing the ground truth mask, it could be prepared using an automated segmentation algorithm where the structure within the image is segmented, and the algorithm assigns a synthesis method to each pixel based on its relationship to the segmented structure.

[0133] It may be worthwhile to use a machine learning approach that analyzes the input image based not only on the values ​​of individual pixels but also on the surrounding neighboring pixels. This can potentially generate a more consistent mask. One way to achieve this is to use a fully convolutional network (FCN). Because FCNs retain spatial information throughout the network, they can consider the context of each pixel, not just the pixels themselves. Alternatively, U-Net can achieve similar results by appropriately setting the receptive field of the convolutional filter and the network depth. The receptive field refers to the region in the input space that a particular CNN feature looks at. A larger receptive field means that the network can incorporate more context into its predictions.

[0134] In addition to, or instead of, thresholding the intensity of pixels in the received image, the similarity of pixels at the same pixel location across the set of received image planes can be taken into account instead of determining how to blend each pixel location. Examples include the variance of pixels at the same pixel location or a correlation measure between pixels at the same pixel location. Spatial correlation is a statistical measure of how similar pixels at corresponding pixel locations in two or more images are to each other. Regarding variance, a region with high variance indicates a large difference between the pixels being blended. In such situations, using maximum or minimum intensity projections may be undesirable, given the large differences between the blended pixels. In such situations, choosing a 0-degree plane value might be a better choice. This may be added as an additional rule when assigning a blending method to each pixel. For example, an intensity-based mask might be used by default unless the similarity measurement between pixels across the entire set of received image planes at a particular pixel location falls below a threshold; in this case, the pixel value from the 0-degree plane (or another pre-determined one of the received planes, or a specific calculated image based on the received plane) is used.

[0135] The specific embodiment described above applies smoothing to a binary mask to create a non-binary mask. In this regard, according to the set of variations of the embodiment, in addition to applying smoothing to a binary mask, or instead, smoothing can be applied to the individual received image planes 62, 64 before mask creation.

[0136] In embodiments where elevation-direction synthesis is performed, the method may include a step of extracting a set of image planes from a received 3D ultrasound dataset. For example, this could include receiving ultrasound data of an imaged object generated by an array transducer, and receiving data spanning a volume region having elevation, axial, and transverse / azimuth directions. The method may also include performing receive beamforming parallel to the elevation direction to generate data corresponding to a set of two or more planes extending in each of the different line-of-sight directions in the elevation direction. The method may further include processing the generated data for two or more planes to derive separate B-mode images corresponding to the image planes extending in each of the two or more line-of-sight directions.

[0137] The embodiments of the invention described above utilize a processing apparatus. The processing apparatus may generally comprise a single processor or multiple processors. It may be located within a single enclosing device, structure, or unit, or it may be distributed across multiple different devices, structures, or units. Therefore, a reference to a processing apparatus adapted or configured to perform a particular step or task may, alone or in combination, correspond to that step or task performed by one or more of multiple processing components. A skilled person will understand how such a distributed processing device can be implemented. The processing apparatus includes communication modules or inputs / outputs for receiving data and further outputting data to components.

[0138] One or more processors in a processing unit can be implemented in numerous ways using software and / or hardware to perform various required functions. The unit typically uses one or more microprocessors programmed using software (e.g., microcode) to perform the required functions. The unit may also be implemented as a combination of dedicated hardware for performing some functions and one or more programmed microprocessors and associated circuits for performing other functions.

[0139] Examples of circuits used in various embodiments of this disclosure include, but are not limited to, conventional microprocessors, application-specific integrated circuits (ASICs), and field-programmable gate arrays (FPGAs).

[0140] In various implementations, the processor may be associated with one or more storage media, which are volatile and non-volatile computer memories such as RAM, PROM, EPROM, and EEPROM. These storage media may be encoded with one or more programs that perform the required functions when executed on one or more processors and / or controllers. The various storage media may be mounted within the processor or controller, or they may be transportable so that one or more programs stored in the storage media can be loaded into the processor.

[0141] Modifications of the disclosed embodiments can be understood and implemented by those skilled in the art in carrying out the claimed invention, based on a review of the drawings, disclosures, and appended claims. In the claims, the words “comprising” do not exclude other components or steps, and the indefinite articles “a” or “an” do not exclude plurality.

[0142] A single processor or other unit can perform the functions of several of the items listed in the claims.

[0143] The mere fact that certain means are described in mutually different dependent claims does not indicate that combinations of these means cannot be used advantageously.

[0144] Computer programs can be stored / distributed on suitable media such as optical or solid-state media supplied together with or as part of other hardware, but they can also be distributed in other forms, such as via the Internet or other wired or wireless telecommunication systems.

[0145] When the term "adapt" is used in a claim or specification, it means that the term "adapt" is equivalent to the term "constituted".

[0146] No reference numeral in a claim should be construed as limiting the scope.

Claims

1. A method for spatial synthesis in ultrasound imaging, The steps include receiving a set of two or more ultrasound image planes representing planes through an imaged object, each extending in a different line of sight direction, The steps include generating an average image from the set of two or more received image planes, A step of generating a composite image from two or more sets of ultrasound image planes, wherein each pixel of the composite image is generated by combining a corresponding pixel in each of the two or more sets of received ultrasound image planes. It has, The step of generating the composite image includes, for each individual pixel of the composite image, the step of selecting a synthesis method from a predetermined set of synthesis methods, wherein the set includes at least the step of selecting the maximum pixel value for the pixel position from the set of received image planes, and the step of selecting the minimum pixel value for the pixel position from the set of received image planes. For each of at least one subset of pixels in the composite image, the step of selecting a synthesis method for each pixel comprises comparing, with respect to at least one threshold, the average pixel value at the corresponding pixel location in the average image, or the average pixel value in a defined neighborhood thereof, the selection is performed such that, for at least the subset of pixels, at least pixels of the subset located in the lowest intensity pixel region of the average image are synthesized using the minimum pixel value synthesis method, and at least pixels of the subset located in the highest intensity pixel region of the average image are synthesized using the maximum pixel value selection method. method.

2. The method according to claim 1, wherein the at least one threshold is an adaptive threshold function that represents a threshold that changes as a function of depth in the average image and / or as a function of the steering angle of the lateral / azimuth angle from which the pixels in the average image were acquired.

3. The method according to claim 1 or 2, wherein the at least one threshold is calculated based on the pixel values ​​in the average image.

4. The method for calculating the threshold function is: The steps include: deriving the probability distribution of pixel intensity in the average image; A step of defining the threshold function as a function that decays from a starting maximum value, wherein the starting maximum value is based on a predetermined percentage, for example 90 percent, of the pixel values ​​of the probability distribution. The method according to claims 2 and 3.

5. The selection of a synthesis method for each of the aforementioned pixels, at least a subset of pixels, A step of forming a binary mask by applying the at least one threshold to the pixels of the average image, wherein if the average pixel intensity exceeds the at least one threshold, the mask pixel in the binary mask is assigned a value of 1, and if the intensity is below the threshold, the mask pixel is assigned a value of 0, or vice versa. It has, The selection of a synthesis method for each of the at least subsets of the aforementioned pixels is performed based on the value of the mask. The method according to any one of claims 1 to 4.

6. The method according to claim 5, wherein the maximum intensity synthesis method is used when the mask has a value of 1, and the minimum intensity synthesis method is used when the mask has a value of 0.

7. The method further comprises the step of applying an edge-preserving smoothing filter to the binary mask to convert the binary mask into a non-binary mask before selecting the synthesis method for each pixel, The set of synthesis methods defined above further comprises the step of using the average of the pixel values ​​of the set of received image planes at the pixel position, If the value of the non-binary mask at the pixel position is 1, the maximum intensity blending method is used. If the value of the non-binary mask for the pixel position is 0, the minimum intensity synthesis method is used. If the value of the mask for the pixel position is between 0 and 1, a weighted combination of the set of synthesis methods is used. The method according to claim 5.

8. The method according to any one of claims 1 to 7, wherein the at least one threshold is determined based on the application of a trained machine learning algorithm to the set of two or more received image planes.

9. The method according to any one of claims 5 to 7, wherein the calculation of the binary and / or non-binary mask comprises applying a trained machine learning algorithm to the received set of two or more image planes.

10. The method according to any one of claims 1 to 9, wherein the set of two or more ultrasonic image planes represents planes passing through an imaged object extending in different line-of-sight directions in the elevation direction, and preferably the set of two or more received image planes includes one image plane extending at a line-of-sight angle of 0 degrees in the elevation direction.

11. The method according to any one of claims 1 to 10, wherein at least a subset of the pixels is determined at least in part on the dispersion or spread of pixel values ​​across the image plane at each particular pixel location or its defined neighborhood, preferably, the at least subset of pixels excludes pixels where the dispersion or spread of pixel values ​​across the image plane at the pixel location or its defined neighborhood exceeds a threshold.

12. The aforementioned method, A step of receiving ultrasonic data of the imaged object generated by an array transducer, the step of receiving data over a volume region having elevation direction, axial direction, and lateral / azimuth direction, The steps include: performing receive beamforming parallel to the elevation direction to generate data corresponding to two or more sets of planes extending in different line-of-sight directions in the elevation direction; The steps include processing the generated data for the two or more planes to derive B-mode images corresponding to each of the two or more image planes extending in each of the line-of-sight directions, The method according to any one of claims 1 to 11, comprising

13. A computer program product having computer program code configured to cause a processor to perform the method described in any one of claims 1 to 12 when executed by the processor.

14. A processing apparatus, wherein the processing apparatus is The steps include receiving a set of two or more ultrasound image planes representing planes through an imaged object, each extending in a different line of sight direction, The steps include generating an average image from the set of two or more received image planes, A step of generating a composite image from two or more sets of ultrasound image planes, wherein each pixel of the composite image is generated by combining a corresponding pixel in each of the two or more sets of received ultrasound image planes. Having one or more processors configured to perform the following: The step of generating the composite image includes, for each individual pixel of the composite image, the step of selecting a synthesis method from a predetermined set of synthesis methods, wherein the set includes at least the step of selecting the maximum pixel value for the pixel position from the set of received image planes, and the step of selecting the minimum pixel value for the pixel position from the set of received image planes. For each of at least one subset of pixels in the composite image, the step of selecting a synthesis method for each pixel includes comparing, with respect to at least one threshold, the average pixel value at the corresponding pixel location in the average image, or the average pixel value in a defined neighborhood thereof, the selection is performed such that, for at least the subset of pixels, at least pixels of the subset located in the lowest intensity pixel region of the average image are synthesized using the minimum pixel value synthesis method, and at least pixels of the subset located in the highest intensity pixel region of the average image are synthesized using the maximum pixel value selection method. Processing device.

15. It is a system, Ultrasound imaging device, The processing apparatus according to claim 14, which is operably coupled to the ultrasonic imaging device and configured to receive ultrasonic data generated by the ultrasonic imaging device, A system that has

Images