Enhancement method for ultrasonic image and ultrasonic device

Abstract

The present application discloses a method for enhancing ultrasonic images and an ultrasonic device, which are used to overcome the defects such as cumulative errors existing in the zero-phase algorithm and the invisibility of puncture needles in non-linear array images in the related art. By obtaining the lateral displacement of a pixel point at any position in the echo ultrasonic image of the past frame to the echo ultrasonic image of the current frame, and during the calculation of the longitudinal displacement using the zero-phase algorithm, the lateral displacement is used to perform lateral compensation on the longitudinal displacement, the problem of cumulative errors existing in the zero-phase algorithm can be effectively solved. By determining the motion binary image and the energy difference binary image corresponding to the past frame and the current frame and determining the enhancement region, the cumulative errors generated by non-motion factors such as acoustic field changes or non-needle body movements of tissues can be removed, and the enhancement region is enhanced through enhancement parameters, so that the puncture needle in the ultrasonic image is clearly displayed, overcoming the problem of the invisibility of the puncture needle in non-linear array images and further improving the accuracy of the puncture needle enhancement algorithm.

Description

Technical Field

[0001] The present application relates to the field of ultrasonic imaging technology, and particularly relates to a method for enhancing ultrasonic images and an ultrasonic device. Background Art

[0002] The ultrasonic device-guided puncture technology is commonly used in puncture surgeries such as intramuscular injection and tissue aspiration. This technology is used to assist doctors in visually judging the position of the puncture needle, thereby preventing damage to key parts such as nerves. However, due to the smooth surface of the puncture needle, which is prone to specular reflection, it is difficult for the ultrasonic probe to receive the echo signal from the puncture needle, and the visibility of the puncture needle in the ultrasonic image is low. There is still a risk of unnecessary injury caused by misjudgment.

[0003] The defects in the related ultrasonic image puncture needle enhancement methods are as follows: Using the zero-phase algorithm to enhance the puncture needle will have cumulative errors, the puncture needle is invisible in non-linear array images, and there are cumulative errors caused by non-motion factors such as sound field changes or non-needle body movements of tissues.

[0004] Therefore, there are defects in the related technologies that require a method that can overcome the defects of the related technologies and enhance the puncture needle in ultrasonic images. Summary of the Invention

[0005] The purpose of the present application is to provide a method for enhancing ultrasonic images, an ultrasonic device, and a storage medium to overcome the defects of the related technologies.

[0006] In a first aspect, the present application provides a method for enhancing ultrasonic images, the method comprising:

[0007] Obtaining the echo ultrasonic images of the current frame and the past frame;

[0008] Adopting a block matching method based on the correlation matching degree to calculate the lateral displacement from any pixel point in the echo ultrasonic image of the past frame to the echo ultrasonic image of the current frame;

[0009] Based on the lateral displacement, using the zero-phase algorithm to calculate the longitudinal displacement from any pixel point in the echo ultrasonic image of the past frame to the echo ultrasonic image of the current frame;

[0010] Based on the first energy image of the echo ultrasonic image of the past frame and the second energy image of the echo ultrasonic image of the current frame, determining the motion binary image and the energy difference binary image corresponding to the past frame and the current frame;

[0011] Based on the motion binary image and the energy difference binary image, determining the enhancement region;

[0012] Determine an enhancement parameter based on the longitudinal displacement and the motion binary image, and enhance the enhancement region using the enhancement parameter, where the enhancement parameter is used to enhance the enhancement region.

[0013] In a possible implementation manner, the method of calculating the lateral displacement from any pixel point in the echo ultrasound image of the past frame to the echo ultrasound image of the current frame by using the block matching method based on the correlation matching degree specifically includes:

[0014] Take the pixel points at the same position in the echo ultrasound image of the past frame and the echo ultrasound image of the current frame as the central pixel points, take the search neighborhood of the central pixel point in the echo ultrasound image of the past frame as the first search window, and take the search neighborhood of the central pixel point in the echo ultrasound image of the current frame as the second search window;

[0015] Take the matching neighborhood of any pixel point in the first search window as the first matching window, take the matching neighborhood of any pixel point in the second search window as the second matching window, and calculate the correlation between each pixel point in the first matching window and each pixel point in the second matching window; the matching area is smaller than the search area;

[0016] Take the pixel point at the corresponding position in the echo ultrasound image of the current frame with the largest correlation as the matching pixel point of any pixel point in the echo ultrasound image of the past frame in the echo ultrasound image of the current frame, and calculate the lateral displacement Δx(i, j) from any pixel point in the echo ultrasound image of the past frame to the echo ultrasound image of the current frame by using the following lateral displacement formula:

[0017] Δx(i, j) = j m -j

[0018] where j m represents the abscissa of the matching pixel point in the echo ultrasound image of the current frame, and j represents the abscissa of any pixel point in the echo ultrasound image of the past frame.

[0019] In a possible implementation manner, the method of taking the matching neighborhood of any pixel point in the first search window as the first matching window, taking the matching neighborhood of any pixel point in the second search window as the second matching window, and calculating the correlation between each pixel point in the first matching window and each pixel point in the second matching window specifically includes:

[0020] Determine the correlation between each pixel point in the first matching window and each pixel point in the second matching window by using the following correlation calculation formula:

[0021]

[0022] Wherein, R represents correlation, A represents the echo ultrasound image of the past frame, B represents the echo ultrasound image of the current frame, ∑ represents summation within the ranges of Δi ∈ [-h, h] and Δj ∈ [-w, w], i represents the ordinate of any pixel point at a position in the echo ultrasound image of the past frame, j represents the abscissa of any pixel point in the echo ultrasound image of the past frame, i' represents the ordinate of any pixel point at a position in the echo ultrasound image of the current frame, j' represents the abscissa of any pixel point at a position in the echo ultrasound image of the current frame, Δi represents the range of ordinate change, Δj represents the range of abscissa change, and "||" represents the modulus of a complex number.

[0023] In a possible implementation manner, based on the lateral displacement, the zero-phase algorithm is used to calculate the longitudinal displacement from any pixel point position in the echo ultrasound image of the past frame to the echo ultrasound image of the current frame, specifically including:

[0024] Based on any pixel point at a position in the echo ultrasound image of the past frame and the lateral displacement from any pixel point at a position in the echo ultrasound image of the past frame to the echo ultrasound image of the current frame, find the corresponding pixel point of any pixel point at a position in the echo ultrasound image of the past frame in the echo ultrasound image of the current frame;

[0025] Take a correlation neighborhood with the same size for the any pixel point and the corresponding pixel point, and calculate the correlation function of the two images within the correlation neighborhood;

[0026] Based on the correlation function and the longitudinal displacement at the position of the pixel point above the any pixel point, obtain the cross-correlation function of the past frame and the current frame within the correlation neighborhood;

[0027] Based on the cross-correlation function and the longitudinal displacement at the position of the pixel point above the any pixel point, obtain the longitudinal displacement at the position of the any pixel point.

[0028] In a possible implementation manner, the step of finding the corresponding pixel point of any pixel point at a position in the echo ultrasound image of the past frame in the echo ultrasound image of the current frame based on any pixel point at a position in the echo ultrasound image of the past frame and the lateral displacement from any pixel point at a position in the echo ultrasound image of the past frame to the echo ultrasound image of the current frame specifically includes:

[0029] Use the following position determination formula to determine the corresponding pixel point of any pixel point at a position in the echo ultrasound image of the past frame in the echo ultrasound image of the current frame:

[0030] j B = j A+Δx(i A ,j A )

[0031] i B = i A -Δy(i A - 1,j B )

[0032] wherein, i A represents the ordinate of any pixel point at a position in the echo ultrasound image of the past frame, j A represents the abscissa of any pixel point at a position in the echo ultrasound image of the past frame, i B represents the ordinate of the corresponding pixel point, j B represents the abscissa of the corresponding pixel point, Δx(i A ,j A ) represents the horizontal displacement from the position (i A ,j A ) in the echo ultrasound image of the past frame to the echo ultrasound image of the current frame, and Δy(i A - 1,j B ) represents the vertical displacement at the position of the pixel point above any pixel point.

[0033] In a possible implementation manner, taking relevant neighborhoods of the same size for the any pixel point and the corresponding pixel point, and calculating the correlation function between the past frame and the current frame in the relevant neighborhood specifically includes:

[0034] Determining the correlation function between the past frame and the current frame in the relevant neighborhood by using the following correlation function formula:

[0035]

[0036] wherein, C represents the correlation function, A represents the echo ultrasound image of the past frame, B represents the echo ultrasound image of the current frame, ∑ represents summation within the ranges of Δi'∈[-h', h'] and Δj'∈[-w', w'], i A represents the ordinate of any pixel point in the echo ultrasound image of the past frame, j A represents the abscissa of any pixel point in the echo ultrasound image of the past frame, i B represents the ordinate of any pixel point in the echo ultrasound image of the current frame, j B represents the abscissa of any pixel point in the echo ultrasound image of the current frame, Δi' represents the range of vertical coordinate change, and Δj' represents the range of vertical coordinate change.

[0037] In a possible implementation, obtaining the cross-correlation function of the past frame and the current frame in the relevant field based on the correlation function and the longitudinal displacement at the position of the pixel above the pixel at any position specifically includes:

[0038] Determining the cross-correlation function by using the following cross-correlation function formula:

[0039]

[0040] where C′ represents the cross-correlation function, C represents the correlation function, ω c represents the center frequency of the probe, and Δy′ represents the longitudinal displacement at the position of the pixel above the pixel at any position.

[0041] In a possible implementation, obtaining the longitudinal displacement at the pixel at any position based on the cross-correlation function and the longitudinal displacement at the position of the pixel above the pixel at any position specifically includes:

[0042] Calculating the longitudinal displacement at the pixel at any position by using the following time-shift formula:

[0043]

[0044] where Δy(i A ,j A ) represents the longitudinal displacement at the pixel at any position, Δy′ represents the longitudinal displacement at the position of the pixel above the pixel at any position, arg() represents the argument function of a complex number, C′ represents the cross-correlation function, and ω C represents the center frequency of the probe.

[0045] In a possible implementation, determining the first energy image of the echo ultrasound image of the past frame and the second energy image of the echo ultrasound image of the current frame includes:

[0046] Determining the first energy data corresponding to the echo ultrasound image of the past frame and the second energy data corresponding to the echo ultrasound image of the current frame by using the following energy data formula:

[0047] Power = |RF|

[0048] where Power represents the energy data of the echo ultrasound image, RF represents the echo ultrasound data corresponding to the echo ultrasound image of the past frame and the echo ultrasound image of the current frame, and || represents the modulus of a complex number;

[0049] Perform data conversion on the first energy data corresponding to the echo ultrasound image of the past frame and the second energy data corresponding to the echo ultrasound image of the current frame to obtain the first energy image of the echo ultrasound image of the past frame and the second energy image of the echo ultrasound image of the current frame.

[0050] In a possible implementation manner, determining the motion binary image and the energy difference binary image corresponding to the past frame and the current frame based on the first energy image of the echo ultrasound image of the past frame and the second energy image of the echo ultrasound image of the current frame specifically includes:

[0051] Mark the pixel points in the first energy image and the second energy image that are higher than the first threshold dividing line as the first mark, and mark the pixel points that are lower than or equal to the first threshold dividing line as the second mark to obtain the binary images of the past frame and the current frame;

[0052] Compare the binary image of the past frame and the binary image of the current frame, and mark the pixel points with different values in the binary image as the third mark, and mark the pixel points with the same values in the binary image as the fourth mark to obtain the motion binary image corresponding to the past frame image and the current frame image;

[0053] Compare the first energy image of the echo ultrasound image of the past frame and the second energy image of the echo ultrasound image of the current frame to obtain the energy difference image between the past frame and the current frame;

[0054] Mark the pixel points in the energy difference image that are higher than the second threshold dividing line as the fifth mark, and mark the pixel points that are lower than or equal to the second threshold dividing line as the sixth mark to obtain the energy difference binary image corresponding to the past frame and the current frame.

[0055] In a possible implementation manner, determining the enhancement region based on the motion binary image and the energy difference binary image specifically includes:

[0056] Compare the motion binary image and the energy difference binary image, and use the region composed of the pixel points that are simultaneously marked as the third mark and the fifth mark as the enhancement region.

[0057] In a possible implementation manner, determining the enhancement parameter based on the longitudinal displacement and the motion binary image specifically includes:

[0058] Use the following enhancement parameter formula to determine the enhancement parameter:

[0059]

[0060] Among them, (i, j) represents the coordinates of a pixel point, S(i, j) represents the enhancement parameter, Δy(i, j) represents the longitudinal displacement, M(i, j) represents the motion binary image, and Thresh Δy represents a preset threshold value.

[0061] In a second aspect, the present application provides an ultrasonic image enhancement device, and the device includes:

[0062] An image acquisition module, configured to acquire echo ultrasonic images of the current frame and past frames;

[0063] A lateral displacement determination module, configured to calculate the lateral displacement from a pixel point at any position in the echo ultrasonic image of the past frame to the echo ultrasonic image of the current frame by using a block matching method based on the correlation matching degree;

[0064] A longitudinal displacement determination module, configured to calculate the longitudinal displacement from a pixel point at any position in the echo ultrasonic image of the past frame to the echo ultrasonic image of the current frame by using a zero-phase algorithm based on the lateral displacement;

[0065] A binary image determination module, configured to determine a motion binary image and an energy difference binary image corresponding to the past frame and the current frame based on a first energy image of the echo ultrasonic image of the past frame and a second energy image of the echo ultrasonic image of the current frame;

[0066] An enhancement region determination module, configured to determine an enhancement region based on the motion binary image and the energy difference binary image;

[0067] A region enhancement module, configured to determine an enhancement parameter based on the longitudinal displacement and the motion binary image, and enhance the enhancement region by using the enhancement parameter, where the enhancement parameter is used to enhance the enhancement region.

[0068] In a third aspect, the present application provides an ultrasonic device, including: a processor, a memory, a display unit, and a probe;

[0069] The probe is used to transmit ultrasonic signals;

[0070] The display unit is used to display ultrasonic images;

[0071] The processor is respectively connected to the probe and the display unit, and is configured to implement the ultrasonic image enhancement method described in any one of the first aspects above.

[0072] In a fourth aspect, the present application provides a computer-readable storage medium, and when instructions in the computer-readable storage medium are executed by a terminal device, the terminal device can execute the ultrasonic image enhancement method described in any one of the first aspects above.

[0073] In a fifth aspect, the present application provides a computer program product, including a computer program:

[0074] When the computer program is executed by a processor, it implements the method for enhancing an ultrasonic image as described in any one of the above first aspects.

[0075] The technical solutions provided by the embodiments of the present application at least bring the following beneficial effects:

[0076] In the embodiments of the present application, by adopting a block matching method based on correlation matching, the lateral displacement from any pixel point in the echo ultrasonic image of the past frame to the echo ultrasonic image of the current frame is obtained, and during the process of calculating the longitudinal displacement from any pixel point in the echo ultrasonic image of the past frame to the echo ultrasonic image of the current frame by using the zero-phase algorithm, the lateral displacement is used to perform lateral compensation on the longitudinal displacement, which can effectively solve the problem of cumulative error existing in the zero-phase algorithm. By determining the motion binary image and the energy difference binary image corresponding to the past frame and the current frame and determining the enhancement region, the combination of the energy images, the energy difference images and the morphological operations of the front and back frames is realized, which can remove non-motion factors such as sound field changes or cumulative errors generated by non-needle body movements of tissues, etc., and the enhancement region is enhanced by the enhancement parameters, so that the puncture needle is clearly shown in the ultrasonic image, overcoming the problem that the puncture needle is invisible in non-linear array images, and further improving the accuracy of the puncture needle enhancement algorithm. Description of the Drawings

[0077] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required to be used in the embodiments of the present application will be briefly introduced below.

[0078] Figure 1 It is a framework schematic diagram of the ultrasonic device provided by the embodiment of the present application;

[0079] Figure 2 It is a schematic diagram of the principle of the ultrasonic device provided by the embodiment of the present application to implement an ultrasonic image;

[0080] Figure 3 It is an overall flow schematic diagram of the method for enhancing an ultrasonic image provided by the embodiment of the present application;

[0081] Figure 4 It is a flow schematic diagram of step 302 provided by the embodiment of the present application;

[0082] Figure 5 It is a schematic diagram of the echo ultrasonic image of the past frame provided by the embodiment of the present application;

[0083] Figure 6 It is a schematic diagram of the echo ultrasonic image of the current frame provided by the embodiment of the present application;

[0084] Figure 7 Schematic diagram of the first matching window provided by the embodiment of the present application;

[0085] Figure 8 Schematic diagram of the second matching window provided by the embodiment of the present application;

[0086] Figure 9 Schematic diagram centered on the pixel point (i, j) in image A provided by the embodiment of the present application;

[0087] Figure 10 Schematic diagram centered on the pixel point (i′, j′) in image B provided by the embodiment of the present application;

[0088] Figure 11 Schematic diagram of obtaining 9 pixel points in the first step of the three-step search method provided by the embodiment of the present application;

[0089] Figure 12 Schematic diagram of obtaining 9 pixel points in the second step of the three-step search method provided by the embodiment of the present application;

[0090] Figure 13 Schematic diagram of obtaining 9 pixel points in the third step of the three-step search method provided by the embodiment of the present application;

[0091] Figure 14 Schematic flowchart of step 303 provided by the embodiment of the present application;

[0092] Figure 15 Schematic flowchart of determining the first energy image of the echo ultrasound image of the past frame and the second energy image of the echo ultrasound image of the current frame provided by the embodiment of the present application;

[0093] Figure 16 Schematic flowchart of step 304 provided by the embodiment of the present application;

[0094] Figure 17 Schematic diagram of the first energy image of the echo ultrasound image of the past frame and the second energy image of the echo ultrasound image of the current frame provided by the embodiment of the present application;

[0095] Figure 18 Schematic diagram of the longitudinal displacement image and the motion energy difference image provided by the embodiment of the present application;

[0096] Figure 19 Schematic diagram of the enhanced effect image provided by the embodiment of the present application;

[0097] Figure 20 Schematic diagram of the structure of the motion estimation device for the video image provided by the embodiment of the present application. Detailed implementation manners

[0098] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the following will clearly and completely describe the technical solutions in the embodiments of this application with reference to the accompanying drawings in the embodiments of this application. Among them, the described embodiments are some but not all of the embodiments of this application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in this application without creative efforts belong to the scope of protection of this application.

[0099] In related technologies, the enhancement methods for the puncture needle in ultrasonic images are mainly divided into two categories: (1) enhancement methods that require changing the structure of the puncture needle; (2) puncture needle enhancement methods based on image processing. The first category of methods mainly includes: changing the inner cavity of the puncture needle into a curved structure, rotating the puncture needle during the puncture process, transmitting the rotation of the puncture needle to the tissue near the needle tip, and using the color Doppler mode of the ultrasonic device to detect the movement of the tissue at this position, so as to identify the position of the needle tip, etc.; the main limitation of the enhancement of this type of puncture needle structure is that it requires a puncture needle dedicated to ultrasonic-guided puncture surgery, and rotating the puncture needle during the puncture has certain requirements for the doctor's skills. The second category of methods is only based on the image processing algorithm built into the ultrasonic device and has no restrictions on other external factors, so it has a wider applicability.

[0100] Among them, the puncture needle enhancement methods based on image processing are mainly divided into methods based on puncture needle feature recognition and methods based on motion detection.

[0101] 1. The method based on puncture needle feature recognition identifies the structure belonging to the puncture needle through feature detection, such as straight line detection algorithms like Hough transform and Gabor transform, and highlights it, making the puncture needle body with lower brightness or / and incoherence in the original image more clearly and obviously displayed on the screen. This method is mainly applicable to linear array probes because it needs to cooperate with the probe to change the ultrasonic signal emission angle to be basically perpendicular to the needle body, increasing the possibility of the probe receiving the echo signal from the puncture needle, thereby improving the brightness of the needle body; in non-linear array images, to make the emission directions of all array elements perpendicular to the needle body requires more complex control logic, otherwise the puncture needle is invisible in the image and the morphological features of the puncture needle cannot be extracted; on the other hand, in ultrasonic images (especially non-linear array images), there may be organizational structures (such as fascia, etc.) with more obvious straight line features than the puncture needle, which will also cause misjudgment of the puncture needle feature detection algorithm.

[0102] 2. The method based on motion detection identifies motion features according to the changes in the image signals between two or more frames, thereby detecting the tip or the body of the needle during motion and performing enhanced display. This method is applicable to different types of probes, so it is more commonly used in enhancing the puncture needle in ultrasonic images. The detection of motion features also includes various methods such as the method based on the difference between multiple frames of images and the method based on phase.

[0103] In summary, in the related art, the defects in the method for enhancing the puncture needle in ultrasonic images are as follows: there will be cumulative errors when using the zero-phase algorithm to enhance the puncture needle, the puncture needle is invisible in non-linear array images, and there are cumulative errors caused by non-motion factors such as acoustic field changes or non-needle body movements of tissues.

[0104] In view of this, the present application provides a method for enhancing ultrasonic images, an ultrasonic device, and a storage medium to overcome the defects of the related art.

[0105] The inventive concept of the present application can be summarized as follows: The present application obtains the echo ultrasonic images of the current frame and the past frame, and by using the block matching method based on correlation matching, obtains the lateral displacement of any pixel point in the echo ultrasonic image of the past frame to the echo ultrasonic image of the current frame. During the process of calculating the longitudinal displacement of any pixel point in the echo ultrasonic image of the past frame to the echo ultrasonic image of the current frame by using the zero-phase algorithm, the lateral displacement is used to perform lateral compensation on the longitudinal displacement, which can effectively solve the problem of cumulative errors existing in the zero-phase algorithm. By determining the motion binary image and the energy difference binary image corresponding to the past frame and the current frame and determining the enhancement region, the combination of the energy images, the energy difference images, and the morphological operations between the front and rear frames is realized, which can remove the cumulative errors caused by non-motion factors such as acoustic field changes or non-needle body movements of tissues, and enhance the enhancement region through the enhancement parameters, making the puncture needle clearly displayed in the ultrasonic image, overcoming the problem that the puncture needle is invisible in non-linear array images, and further improving the accuracy of the puncture needle enhancement algorithm.

[0106] After introducing the main inventive idea of the embodiments of the present application, the following briefly introduces the application scenarios applicable to the technical solutions of the embodiments of the present application. It should be noted that the following introduced application scenarios are only for illustrating the embodiments of the present application rather than limiting. In specific implementation, the technical solutions provided by the embodiments of the present application can be flexibly applied according to actual needs.

[0107] See Figure 1 As shown, it is the structural block diagram of the ultrasonic device provided by the embodiment of the present application.

[0108] It should be understood that Figure 1 The ultrasonic device 100 shown is only an example, and the ultrasonic device 100 may have more Figure 1More or fewer components as shown may combine two or more components or may have different component configurations. The various components shown in the figures may be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application specific integrated circuits.

[0109] Figure 1 Exemplarily shown is a block diagram of the hardware configuration of the ultrasound device 100 according to an exemplary embodiment.

[0110] As Figure 1 shown, the ultrasound device 100 may include, for example: a processor 110, a memory 120, a display unit 130, and a probe 140; wherein,

[0111] The probe 140 is configured to transmit ultrasonic signals;

[0112] The display unit 130 is configured to display ultrasonic images;

[0113] The memory 120 is configured to store data required for ultrasonic imaging, and may include software programs, application interface data, etc.;

[0114] The processor 110 is respectively connected to the probe 140, the display unit 130, and the memory 120, and is configured to execute the ultrasonic image enhancement method provided in this application.

[0115] Figure 2 It is a schematic diagram of the application principle according to an embodiment of this application. Among them, this part may be implemented by Figure 1 partial modules or functional components of the shown ultrasound device. Only the main components will be described below, and other components such as memories, controllers, control circuits, etc. will not be elaborated here.

[0116] As Figure 2 shown, the application environment may include a user interface 210, a display unit 220 for displaying the user interface, and a processor 230.

[0117] The display unit 220 may include a display panel 221 and a backlight assembly 222. Among them, the display panel 321 is configured to display ultrasonic images, and the backlight assembly 222 is located on the back of the display panel 221. The backlight assembly 222 may include multiple backlight zones (not shown in the figure), and each backlight zone may emit light to light up the display panel 221 in pixels.

[0118] The processor 230 may be configured to control the brightness of the backlight sources of each backlight zone in the backlight assembly 222, and to control the probe to transmit ultrasonic signals and receive ultrasonic echo signals.

[0119] Among them, the processor 230 can process the ultrasonic echo signal to determine an ultrasonic image. To facilitate understanding of the ultrasonic image enhancement method provided in the embodiments of the present application, the following further explains this with reference to the accompanying drawings.

[0120] In a possible implementation manner, the present application provides an ultrasonic image enhancement method, and its overall flowchart is as Figure 3 shown, including the following:

[0121] In step 301, the echo ultrasonic images of the current frame and the past frame are acquired.

[0122] In step 302, a block matching method based on the correlation matching degree is used to calculate the horizontal displacement from any pixel point position in the echo ultrasonic image of the past frame to the echo ultrasonic image of the current frame.

[0123] In step 303, based on the horizontal displacement, a zero-phase algorithm is used to calculate the vertical displacement from any pixel point position in the echo ultrasonic image of the past frame to the echo ultrasonic image of the current frame.

[0124] In step 304, based on the first energy image of the echo ultrasonic image of the past frame and the second energy image of the echo ultrasonic image of the current frame, the motion binary image and the energy difference binary image corresponding to the past frame and the current frame are determined.

[0125] In step 305, based on the motion binary image and the energy difference binary image, the enhancement region is determined.

[0126] In step 306, based on the vertical displacement and the motion binary image, enhancement parameters are determined, and the enhancement region is enhanced using the enhancement parameters, and the enhancement parameters are used to enhance the enhancement region.

[0127] In a possible implementation manner, in step 302, a block matching method based on the correlation matching degree is used to calculate the horizontal displacement from any pixel point position in the echo ultrasonic image of the past frame to the echo ultrasonic image of the current frame. The flowchart of step 302 is as Figure 4 shown, and specifically includes the following steps:

[0128] In step 401, pixel points at any same position in the echo ultrasonic image of the past frame and the echo ultrasonic image of the current frame are taken as the central pixel points. The search neighborhood of the central pixel point of the echo ultrasonic image of the past frame is used as the first search window, and the search neighborhood of the central pixel point of the echo ultrasonic image of the current frame is used as the second search window.

[0129] In step 402, the matching neighborhood of any pixel point in the first search window is used as the first matching window, and the matching neighborhood of any pixel point in the second search window is used as the second matching window, and the correlation between each pixel point in the first matching window and each pixel point in the second matching window is calculated; the matching neighborhood is smaller than the search neighborhood.

[0130] In step 403, the pixel point corresponding to the position in the echo ultrasound image of the current frame with the maximum correlation is used as the matching pixel point of any pixel point in the echo ultrasound image of the past frame in the echo ultrasound image of the current frame, and the following horizontal displacement formula (1) is used to calculate the horizontal displacement Δx(i,j) from any pixel point in the echo ultrasound image of the past frame to the echo ultrasound image of the current frame:

[0131] Δx(i,j) = j m -j (1)

[0132] where j m represents the abscissa of the matching pixel point in the echo ultrasound image of the current frame, and j represents the abscissa of any pixel point in the echo ultrasound image of the past frame.

[0133] For example, as Figure 5 shown, the echo ultrasound image of the past frame includes several pixel points. The pixel point located at the center position in the echo ultrasound image of the past frame is taken as the central pixel point, as Figure 5 shown by the blank circle in. The search neighborhood of this central pixel point is used as the first search window, and the first search window is as Figure 5 shown by the rectangular dashed box in; similarly, as Figure 6 shown, the echo ultrasound image of the current frame includes several pixel points. The pixel point located at the center position in the echo ultrasound image of the current frame is taken as the central pixel point, as Figure 6 shown by the blank circle in. The search neighborhood of this central pixel point is used as the second search window, and the second search window is as Figure 6 shown by the rectangular dashed box in, obtaining two search windows with the same size. Currently, to maximize the operation efficiency, the search window size is set to 11x11 and can be adjusted as needed.

[0134] After determining the first search window and the second search window, the matching neighborhood of any pixel point in the first search window is used as the first matching window, and the first matching window is as Figure 7 shown by the rectangular solid box in; and the matching neighborhood of any pixel point in the second search window is used as the second matching window, and the second matching window is as Figure 8 shown by the rectangular solid box in, obtaining two matching windows with the same size.

[0135] It should be noted that the matching area is smaller than the search area. Currently, to maximize the operation efficiency, the size of the matching window is set to 3x3, which can also be adjusted as needed.

[0136] In a possible implementation manner, calculating the correlation between each pixel point in the first matching window and each pixel point in the second matching window in step 402 specifically includes the following steps:

[0137] Use the following correlation calculation formula (2) to determine the correlation between each pixel point in the first matching window and each pixel point in the second matching window:

[0138]

[0139] Where R represents the correlation, A represents the echo ultrasound image of the past frame, B represents the echo ultrasound image of the current frame, ∑ represents summation within the ranges of Δi∈[-h,h] and Δj∈[-w,w], i represents the ordinate of any pixel point in the echo ultrasound image of the past frame, j represents the abscissa of any pixel point in the echo ultrasound image of the past frame, i′ represents the ordinate of any pixel point in the echo ultrasound image of the current frame, j′ represents the abscissa of any pixel point in the echo ultrasound image of the current frame, Δi represents the range of ordinate change, Δj represents the range of ordinate change, and "||" represents the modulus of a complex number.

[0140] As Figure 9 and Figure 10 shown, taking the pixel point (i,j) in image A and the pixel point (i′,j′) in image B as the centers respectively, that is, Figure 9 and Figure 10 the blank circles within the solid rectangle frames in, select matching neighborhoods with the same size as the matching window. Let the vertical width of the matching window be 2*h + 1 and the horizontal width be 2*w + 1, with the unit being pixel points. Currently, to maximize the operation efficiency, the size of the matching window is set to 3x3, which can also be adjusted as needed. After determining two matching windows with the same size (as shown by the solid rectangle frames in Figure 9 and Figure 10 ), use the above correlation calculation formula (2) to determine the correlation between each pixel point in the first matching window and each pixel point in the second matching window. If it is determined that the pixel point with the maximum correlation R within the above search window is B(i′,j′), then this pixel point will be denoted as (i m ,j m ), this pixel point is the matching pixel point of A(i,j) in image B, and use the above horizontal displacement formula (1) to calculate the horizontal displacement from any pixel point in the echo ultrasound image of the past frame to the echo ultrasound image of the current frame.

[0141] By analogy, there is a corresponding matching pixel point in the echo ultrasound image of the current frame for any pixel point at any position in the echo ultrasound image of the past frame, so as to obtain the lateral displacement from any pixel point in the echo ultrasound image of the past frame to the echo ultrasound image of the current frame.

[0142] Preferably, in order to improve the operation efficiency, the embodiments of the present application may not calculate the correlation of all pixel points in the above search window in sequence, but adopt a simplified search method, including methods such as TSS (Three-Step Search), NTSS (New Three-Step Search), SES (Simple and Fast Search), etc. The embodiments of the present application preferably use the TSS method, and its operation method is as follows:

[0143] Assume that the vertical width of the search window is 2*h + 1, and the horizontal width is 2*w + 1. In the first step, respectively take the central pixel point in the search windows of the past frame and the current frame, and the pixel points with a horizontal distance and a vertical distance of half of the window width from the central pixel point. Taking the central pixel point (i, j) of the search window of the past frame as the central pixel point of the past frame image as an example, determine that the position coordinates are (i + H / 2, j), (i - H / 2, j), (i, j + W / 2), (i, j - W / 2), (i + H / 2, j + W / 2), (i + H / 2, j - W / 2), (i - H / 2, j + W / 2), (i - H / 2, j - W / 2), and the central pixel point (i, j), a total of 9 pixel points. These 9 pixel points are as Figure 11 shown by the pixel points in the circles in the figure. Calculate the correlation R between each pixel point in the matching window centered on the above 9 pixel points in the past frame and each pixel point in the matching window centered on the above 9 pixel points in the current frame. For example, calculate the correlation between each pixel point in the matching window centered on the pixel point (i + H / 2, j) in the past frame and each pixel point in the matching window centered on the pixel point (i + H / 2, j) in the current frame, and so on. For each two pixel points in the past frame and the current frame respectively, calculate the correlation between each pixel point in the matching windows centered on these two pixel points. If the pixel point with the highest correlation is the central pixel point (i, j), that is, the pixel point with the highest correlation is at the same position in the past frame and the current frame, then stop the search.

[0144] In the second step, if the pixel point with the highest correlation is not at the same position in the past frame and the current frame, then use the pixel point with the highest correlation (assuming the coordinate of this pixel point is (i1, j1)) as the central pixel point in the second step, as Figure 12 shown by the pixel point in the upper right corner of the search window in the figure. This pixel point (i1, j1) is any pixel point other than the central pixel point among the above 9 pixel points. For the 8 pixel points with a distance of one-fourth of the window width around it, as Figure 12For the pixel points within the dotted circle, calculate the correlation R between each pixel point within the matching window centered on the above-mentioned 8 pixel points in the past frame and each pixel point within the matching window centered on the above-mentioned 8 pixel points in the current frame. If the pixel point with the highest correlation is the central pixel point (i1, j1), then stop the search.

[0145] In the third step, if the pixel point with the highest correlation is not the central pixel point (i1, j1), then perform the same operation to obtain the central pixel point (i2, j2) in the third step, as Figure 13 For the pixel points within the solid circle in the upper right corner in, for the 8 pixel points at a distance of 1 / 8 of the window width around it, such as Figure 13 For the pixel points within the solid rectangle frame in, perform a similar operation of calculating the correlation again, and select the pixel point (i3, j3) with the highest correlation as the matching pixel point of the pixel point (i, j) in the past frame. Finally, based on this matching pixel point (i3, j3), use the above horizontal displacement formula (1) to obtain the horizontal displacement Δx(i, j) from the pixel point (i, j) at any position in the echo ultrasound image of the past frame to the echo ultrasound image of the current frame.

[0146] Among them, when H and W are less than 8, the number of pixel points to be searched in the above method can be further reduced, thereby reducing the computational amount and improving the computational efficiency.

[0147] In a possible implementation manner, in step 303, based on the horizontal displacement, use the zero-phase algorithm to calculate the longitudinal displacement from the pixel point at any position in the echo ultrasound image of the past frame to the echo ultrasound image of the current frame. The flowchart of step 303 is as Figure 14 shown, and specifically includes the following steps:

[0148] In step 1401, based on the pixel point at any position in the echo ultrasound image of the past frame and the horizontal displacement from the pixel point at any position in the echo ultrasound image of the past frame to the echo ultrasound image of the current frame, find the corresponding pixel point of the pixel point at any position in the echo ultrasound image of the past frame in the echo ultrasound image of the current frame.

[0149] In step 1402, take a correlation neighborhood with the same size for the pixel point at any position and the corresponding pixel point, and calculate the correlation function of the two images within the correlation neighborhood.

[0150] In step 1403, based on the correlation function and the longitudinal displacement at the position of the pixel point above the pixel point at any position, obtain the cross-correlation function between the past frame and the current frame within the correlation neighborhood.

[0151] In step 1404, based on the cross-correlation function and the longitudinal displacement at the position of the pixel point above the pixel point at any position, obtain the longitudinal displacement at the pixel point at any position.

[0152] In a possible implementation, in step 1401, based on any position pixel point in the echo ultrasound image of the past frame and the lateral displacement from any pixel point position in the echo ultrasound image of the past frame to the echo ultrasound image of the current frame, the corresponding pixel point of any position pixel point in the echo ultrasound image of the past frame in the echo ultrasound image of the current frame is found, which specifically includes the following steps:

[0153] Use the following position determination formula (3) to determine the corresponding pixel point of any position pixel point in the echo ultrasound image of the past frame in the echo ultrasound image of the current frame:

[0154] j B = j A + Δx(i A , j A )

[0155] i B = i A - Δy(i A - 1, j B ) (3)

[0156] Wherein, i A represents the ordinate of any position pixel point in the echo ultrasound image of the past frame, j A represents the abscissa of any position pixel point in the echo ultrasound image of the past frame, i B represents the ordinate of the corresponding pixel point, j B represents the abscissa of the corresponding pixel point, Δx(i A , j A ) represents the lateral displacement from the position (i A , j A ) in the echo ultrasound image of the past frame to the echo ultrasound image of the current frame, and Δy(i A - 1, j B ) represents the longitudinal displacement at the position of the pixel point above any position pixel point.

[0157] In the related zero-phase algorithm, jB = jA, ignoring the lateral displacement, resulting in a cumulative error in the finally obtained longitudinal displacement image. The embodiment of the present application compensates step 1401 through the lateral displacement obtained in step 403, that is, the above position determination formula (3) is obtained, which can remove the cumulative error of the related zero-phase algorithm and make the acquired data more accurate.

[0158] In a possible implementation, in step 1402, for any position pixel point and the corresponding pixel point, a correlation neighborhood with the same size is taken, and the correlation function between the past frame and the current frame in the correlation field is calculated, which specifically includes the following content:

[0159] The following correlation function formula (4) is used to determine the correlation function between the past frame and the current frame in the relevant field:

[0160]

[0161] where C represents the correlation function, A represents the echo ultrasound image of the past frame, B represents the echo ultrasound image of the current frame, ∑ represents summation within the ranges of Δi′∈[-h’,h’] and Δj′∈[-w’,w’], i A represents the ordinate of any pixel point at a position in the echo ultrasound image of the past frame, j A represents the abscissa of any pixel point at a position in the echo ultrasound image of the past frame, i B represents the ordinate of any pixel point at a position in the echo ultrasound image of the current frame, j B represents the abscissa of any pixel point at a position in the echo ultrasound image of the current frame, Δi′ represents the range of ordinate change, and Δj′ represents the range of ordinate change.

[0162] It should be added that in the above correlation function formula (4), assuming the longitudinal dimension of the relevant field is 2*h’+1 and the transverse dimension is 2*w’+1, therefore, Δi′∈[-h’,h’], Δj′∈[-w’,w’].

[0163] In a possible implementation manner, in step 1403, based on the correlation function and the longitudinal displacement at the position of the pixel point above any position pixel point, the cross-correlation function between the past frame and the current frame in the relevant field is obtained, specifically including the following content:

[0164] The following cross-correlation function formula (5) is used to determine the cross-correlation function:

[0165]

[0166] where C′ represents the cross-correlation function, C represents the correlation function, ω c represents the center frequency of the probe, and Δy′ represents the longitudinal displacement at the position of the pixel point above any position pixel point.

[0167] In a possible implementation manner, in step 1404, based on the cross-correlation function and the longitudinal displacement at the position of the pixel point above any position pixel point, the longitudinal displacement at any position pixel point is obtained, specifically including the following content:

[0168] The following time shift formula (6) is used to calculate the longitudinal displacement at any position pixel point:

[0169]

[0170] where Δy(i A ,j A) represents the longitudinal displacement at any pixel position, Δy′ represents the longitudinal displacement at the pixel position above any pixel position, arg() represents the argument function of a complex number, C′ represents the cross-correlation function, ω C represents the center frequency of the probe.

[0171] In a possible implementation, the first energy image of the echo ultrasound image of the past frame and the second energy image of the echo ultrasound image of the current frame are determined. The flowchart is as Figure 15 shown and includes the following:

[0172] In step 1501, the first energy data corresponding to the echo ultrasound image of the past frame and the second energy data corresponding to the echo ultrasound image of the current frame are determined using the following energy data formula (7):

[0173] Power = |RF| (7)

[0174] where Power represents the energy data of the echo ultrasound image, RF represents the echo ultrasound data corresponding to the echo ultrasound image of the past frame and the echo ultrasound image of the current frame, and || represents the modulus of a complex number;

[0175] In step 1502, the first energy data corresponding to the echo ultrasound image of the past frame and the second energy data corresponding to the echo ultrasound image of the current frame are subjected to data conversion to obtain the first energy image of the echo ultrasound image of the past frame and the second energy image of the echo ultrasound image of the current frame.

[0176] In a possible implementation, in step 304, based on the first energy image of the echo ultrasound image of the past frame and the second energy image of the echo ultrasound image of the current frame, the motion binary image and the energy difference binary image corresponding to the past frame and the current frame are determined. The flowchart is as Figure 16 shown and includes the following:

[0177] In step 1601, the pixel points in the first energy image and the second energy image with values higher than the first threshold boundary are marked as the first mark, and the pixel points lower than or equal to the first threshold boundary are marked as the second mark to obtain the binary images of the past frame and the current frame.

[0178] In step 1602, the binary image of the past frame and the binary image of the current frame are compared. The pixel points with different values in the binary image are marked as the third mark, and the pixel points with the same values in the binary image are marked as the fourth mark to obtain the motion binary image corresponding to the past frame image and the current frame image.

[0179] In step 1603, the first energy image of the echo ultrasound image of the past frame is compared with the second energy image of the echo ultrasound image of the current frame to obtain the energy difference image between the past frame and the current frame.

[0180] In step 1604, the pixel points in the energy difference image with values higher than the second threshold boundary line are marked as the fifth mark, and the pixel points lower than or equal to the second threshold boundary line are marked as the sixth mark, obtaining the energy difference binary image corresponding to the past frame and the current frame.

[0181] For example, A1 is the first energy image and B1 is the first energy image. Then, the two images are respectively segmented into high-energy and low-energy parts by the threshold segmentation method. The above threshold segmentation method can use an adaptive threshold. In the embodiment of the present application, the histogram threshold is preferably used, that is, taking a certain ratio as the boundary line. Corresponding to the first threshold boundary line, the part of the image with values higher than the first threshold boundary line is the high-energy value (marked as 1, that is, the first mark), and the others are the low-energy values (marked as 0, that is, the second mark), obtaining the binary images of frames A1 and B1. Suppose there are 10,000 pixel points in the energy images of frames A1 and B1, and the preset ratio (the first threshold boundary line) is 1%. Then, the pixel points with values accounting for the first 1% in the image are high-energy values. That is, arranging the values in the image from high to low, the first 100 pixel points are marked as the first mark, and the remaining 9,900 pixel points are marked as the second mark, thereby obtaining the binary images of the past frame and the current frame.

[0182] After obtaining the binary images of frames A1 and B1, the pixel points with different values (0 in one frame and 1 in the other frame) in the above two binary images are considered as the positions where motion exists, that is:

[0183] M = A1 xor B1 (8)

[0184] Where M is the motion binary image, and xor represents exclusive OR. That is, the pixel points that are 0 in A1 and 1 in B1, or 1 in A1 and 0 in B1 are 1 in the motion binary image, corresponding to the third mark, and the other pixel points are 0 in the motion binary image, corresponding to the fourth mark.

[0185] After obtaining the motion binary images of frames A1 and B1, the energy difference image between the two energy images is obtained, that is, the two energy images of A1 and B1 are subtracted and the absolute value is taken, thereby obtaining the energy difference image.

[0186] The energy difference image is segmented into an energy difference binary image by using the above histogram threshold segmentation method. Taking a certain ratio as the dividing line, corresponding to the second threshold dividing line, the part of the image with values higher than this second threshold dividing line is the high energy difference (marked as 1, which is the fifth mark), and the others are the low energy difference (marked as 0, which is the sixth mark), thus obtaining the energy difference binary image. Suppose the energy difference image has 10,000 pixel points and the preset ratio (the second threshold dividing line) is 2%. Then the pixel points with values in the top 2% of the image are the high energy difference. That is, when the values in the image are arranged from high to low, the first 200 pixel points are marked as the fifth mark, and the remaining 9,800 pixel points are marked as the sixth mark, thereby obtaining the energy difference binary image.

[0187] It should be noted that the above first threshold dividing line and second threshold dividing line are determined manually according to experience based on the image display effect, or can also be set as adjustable parameters.

[0188] In a possible implementation manner, after obtaining the motion binary image and the energy difference binary image, based on the motion binary image and the energy difference binary image, an enhanced region is determined, which specifically includes the following steps:

[0189] By comparing the motion binary image and the energy difference binary image, the region composed of the pixel points that are simultaneously marked as the third mark and the fifth mark is used as the enhanced region, that is, the pixel points with high energy difference and motion are taken. It is considered that there is an energy difference caused by motion at these pixel points, and the region composed of these pixel points is called the enhanced region.

[0190] In a possible implementation manner, since the above-obtained enhanced region generally consists of discontinuous pixel points, preferably, the embodiments of the present application can perform morphological processing on the enhanced region, such as performing closing operation first and then opening operation to connect the enhanced region, ensuring the acquisition of a complete enhanced region.

[0191] In addition, since there may still be tissues in motion in the image and the area of the tissues in motion is larger than that of the puncture needle, the embodiments of the present application can choose to perform one of the following several processes:

[0192] (1) When the area of the enhanced region accounts for a proportion of the entire image greater than the specified threshold, the embodiments of the present application consider that there is large-area tissue and motion in this region, and then do not enhance this frame of image. For example, when the area of the enhanced region accounts for more than 20% of the entire image, it is considered that there is large-area tissue and motion, and then this frame of image is not enhanced.

[0193] (2) Calculate the area of each connected component in the enhanced region after morphological operations. If the proportion of the area of each connected component in the entire image exceeds a certain threshold, the embodiments of the present application consider that each connected component belongs to the tissue in motion, and then remove each connected component from the enhanced region. For example, if the proportion of the area of each connected component in the entire image exceeds 30%, the embodiments of the present application consider that each connected component belongs to the tissue in motion, and then remove each connected component from the enhanced region.

[0194] It should be noted that the above threshold is determined manually according to experience based on the image display effect, or can be set as an adjustable parameter.

[0195] In a possible implementation manner, based on the longitudinal displacement and the motion binary image, determine the enhancement parameter, specifically including:

[0196] Use the following enhancement parameter formula (9) to determine the enhancement parameter:

[0197]

[0198] Among them, (i, j) represents the coordinates of the pixel point, S(i, j) represents the enhancement parameter, Δy(i, j) represents the longitudinal displacement, M(i, j) represents the motion binary image, and Thresh Δy represents the preset threshold.

[0199] Among them, the range of the enhancement coefficient S(i, j) is [0, 1]. When Δy(i, j) is greater than or equal to Thresh Δy , the value of S(i, j) is 1.

[0200] Since the human eye is sensitive to light of different wavelengths and is most sensitive to green, the embodiments of the present application prefer to use green to mark the enhanced region, and other colors can also be used. The greater the enhancement parameter, the more obvious the color.

[0201] In a possible implementation manner, for example, collect two consecutive frames of RF data, and its energy images are as Figure 17 shown. The left side is the first energy image of the echo ultrasound image of the past frame, and the right side is the second energy image of the echo ultrasound image of the current frame. After the above steps, the longitudinal displacement image ( Figure 18 left figure) and the motion energy difference image (i.e., the image of the enhanced region, Figure 18 right figure) are obtained respectively, as Figure 18 shown. The left white area in the right figure is the enhanced region. After the above step of enhancing the enhanced region using the enhancement parameter, the enhanced effect image is as Figure 19 shown, Figure 19 The partial region image covered by the ellipse pointed by the arrow in

[0202] In summary, the present application obtains the echo ultrasound images of the current frame and the past frame. By adopting the block matching method based on correlation matching, the lateral displacement from any pixel point in the echo ultrasound image of the past frame to the echo ultrasound image of the current frame is obtained. During the process of calculating the longitudinal displacement from any pixel point in the echo ultrasound image of the past frame to the echo ultrasound image of the current frame by using the zero-phase algorithm, the lateral displacement is used to perform lateral compensation on the longitudinal displacement, which can effectively solve the problem of cumulative error existing in the zero-phase algorithm. By determining the motion binary image and the energy difference binary image corresponding to the past frame and the current frame and determining the enhancement region, the combination of the energy images, the energy difference images and the morphological operations of the front and back frames is realized, which can remove non-motion factors such as sound field changes or cumulative errors generated by non-needle body movements of tissues, etc. And the enhancement region is enhanced by the enhancement parameter, so that the puncture needle is clearly shown in the ultrasound image, overcoming the problem that the puncture needle is invisible in the non-linear array image, and further improving the accuracy of the puncture needle enhancement algorithm.

[0203] Based on the same inventive concept, an embodiment of the present application further provides an enhancement device 2000 for ultrasound images, as Figure 20 shown, the device includes:

[0204] An image acquisition module 2001, configured to acquire the echo ultrasound images of the current frame and the past frame;

[0205] A lateral displacement determination module 2002, configured to calculate the lateral displacement from any pixel point in the echo ultrasound image of the past frame to the echo ultrasound image of the current frame by adopting a block matching method based on correlation matching degree;

[0206] A longitudinal displacement determination module 2003, configured to calculate the longitudinal displacement from any pixel point in the echo ultrasound image of the past frame to the echo ultrasound image of the current frame by using the zero-phase algorithm based on the lateral displacement;

[0207] A binary image determination module 2004, configured to determine the motion binary image and the energy difference binary image corresponding to the past frame and the current frame based on the first energy image of the echo ultrasound image of the past frame and the second energy image of the echo ultrasound image of the current frame;

[0208] An enhancement region determination module 2005, configured to determine the enhancement region based on the motion binary image and the energy difference binary image;

[0209] A region enhancement module 2006, configured to determine an enhancement parameter based on the longitudinal displacement and the motion binary image, and enhance the enhancement region by using the enhancement parameter, where the enhancement parameter is used to enhance the enhancement region.

[0210] In a possible implementation, the block matching method based on the correlation matching degree is adopted to calculate the lateral displacement from a pixel point at any position in the echo ultrasound image of the past frame to the echo ultrasound image of the current frame. The lateral displacement determination module is configured as follows:

[0211] Take the pixel points at the same position in the echo ultrasound image of the past frame and the echo ultrasound image of the current frame as the central pixel points. Take the search neighborhood of the central pixel point in the echo ultrasound image of the past frame as the first search window, and take the search neighborhood of the central pixel point in the echo ultrasound image of the current frame as the second search window;

[0212] Take the matching neighborhood of any pixel point in the first search window as the first matching window, and take the matching neighborhood of any pixel point in the second search window as the second matching window, and calculate the correlation between each pixel point in the first matching window and each pixel point in the second matching window; the matching area is smaller than the search area;

[0213] Take the pixel point at the corresponding position in the echo ultrasound image of the current frame with the maximum correlation as the matching pixel point of the pixel point at any position in the echo ultrasound image of the past frame in the echo ultrasound image of the current frame, and use the following lateral displacement formula to calculate the lateral displacement Δx(i,j) from a pixel point at any position in the echo ultrasound image of the past frame to the echo ultrasound image of the current frame:

[0214] Δx(i,j) = j m -j

[0215] where j m represents the abscissa of the matching pixel point in the echo ultrasound image of the current frame, and j represents the abscissa of the pixel point at any position in the echo ultrasound image of the past frame.

[0216] In a possible implementation, for taking the matching neighborhood of any pixel point in the first search window as the first matching window, taking the matching neighborhood of any pixel point in the second search window as the second matching window, and calculating the correlation between each pixel point in the first matching window and each pixel point in the second matching window, the lateral displacement determination module is configured as follows:

[0217] Adopt the following correlation calculation formula to determine the correlation between each pixel point in the first matching window and each pixel point in the second matching window:

[0218]

[0219] Wherein, R represents the correlation, A represents the echo ultrasound image of the past frame, B represents the echo ultrasound image of the current frame, ∑ represents summation within the range of Δi ∈ [-h, h] and Δj ∈ [-w, w], i represents the ordinate of any pixel point at a position in the echo ultrasound image of the past frame, j represents the abscissa of any pixel point in the echo ultrasound image of the past frame, i' represents the ordinate of any pixel point at a position in the echo ultrasound image of the current frame, j' represents the abscissa of any pixel point at a position in the echo ultrasound image of the current frame, Δi represents the range of ordinate change, Δj represents the range of ordinate change, and "||" represents taking the modulus.

[0220] In a possible implementation manner, based on the lateral displacement, the zero-phase algorithm is used to calculate the longitudinal displacement from any pixel point position in the echo ultrasound image of the past frame to the echo ultrasound image of the current frame, and the longitudinal displacement determination module is configured as follows:

[0221] Based on any pixel point at a position in the echo ultrasound image of the past frame and the lateral displacement from any pixel point at a position in the echo ultrasound image of the past frame to the echo ultrasound image of the current frame, find the corresponding pixel point of any pixel point at a position in the echo ultrasound image of the past frame in the echo ultrasound image of the current frame;

[0222] Take a correlation neighborhood with the same size for the any pixel point and the corresponding pixel point, and calculate the correlation function of the two images within the correlation neighborhood;

[0223] Based on the correlation function and the longitudinal displacement at the position of the pixel point above the any pixel point, obtain the cross-correlation function of the past frame and the current frame within the correlation neighborhood;

[0224] Based on the cross-correlation function and the longitudinal displacement at the position of the pixel point above the any pixel point, obtain the longitudinal displacement at the position of the any pixel point.

[0225] In a possible implementation manner, for finding the corresponding pixel point of any pixel point at a position in the echo ultrasound image of the past frame in the echo ultrasound image of the current frame based on any pixel point at a position in the echo ultrasound image of the past frame and the lateral displacement from any pixel point at a position in the echo ultrasound image of the past frame to the echo ultrasound image of the current frame, the longitudinal displacement determination module is configured as follows:

[0226] Use the following position determination formula to determine the corresponding pixel point of any pixel point at a position in the echo ultrasound image of the past frame in the echo ultrasound image of the current frame:

[0227] j B = j A + Δx(iA , j A )

[0228] i B = i A -Δy(i A -1, j B )

[0229] wherein, i A represents the ordinate of any pixel point at a position in the echo ultrasound image of the past frame, and j A represents the abscissa of any pixel point at a position in the echo ultrasound image of the past frame, and i B represents the ordinate of the corresponding pixel point, and j B represents the abscissa of the corresponding pixel point, and Δx(i A , j A ) represents the horizontal displacement from the position (i A , j A ) in the echo ultrasound image of the past frame to the echo ultrasound image of the current frame, and Δy(i A -1, j B ) represents the vertical displacement at the position of the pixel point above the any position pixel point.

[0230] In a possible implementation manner, for the any position pixel point and the corresponding pixel point, relevant neighborhoods with the same size are taken, and the correlation function between the past frame and the current frame in the relevant neighborhood is calculated. The vertical displacement determination module is configured to:

[0231] Determine the correlation function between the past frame and the current frame in the relevant neighborhood by using the following correlation function formula:

[0232]

[0233] wherein, C represents the correlation function, A represents the echo ultrasound image of the past frame, B represents the echo ultrasound image of the current frame, ∑ represents summation within the ranges of Δi'∈[-h', h'] and Δj'∈[-w', w'], and i A represents the ordinate of any pixel point at a position in the echo ultrasound image of the past frame, and j A represents the abscissa of any pixel point at a position in the echo ultrasound image of the past frame, and i B represents the ordinate of any pixel point at a position in the echo ultrasound image of the current frame, and j B represents the abscissa of any pixel point at a position in the echo ultrasound image of the current frame, Δi' represents the vertical change range, and Δj' represents the vertical change range.

[0234] In a possible implementation, based on the correlation function and the longitudinal displacement at the position of the pixel above the pixel at any position, the cross-correlation function between the past frame and the current frame in the relevant field is obtained, and the longitudinal displacement determination module is configured as follows:

[0235] The cross-correlation function is determined by using the following cross-correlation function formula:

[0236]

[0237] where C′ represents the cross-correlation function, C represents the correlation function, ω c represents the probe center frequency, and Δy′ represents the longitudinal displacement at the position of the pixel above the pixel at any position.

[0238] In a possible implementation, based on the cross-correlation function and the longitudinal displacement at the position of the pixel above the pixel at any position, the longitudinal displacement at the pixel at any position is obtained, and the longitudinal displacement determination module is configured as follows:

[0239] The longitudinal displacement at the pixel at any position is calculated by using the following time-shift formula:

[0240]

[0241] where Δy(i A ,j A ) represents the longitudinal displacement at the pixel at any position, Δy′ represents the longitudinal displacement at the position of the pixel above the pixel at any position, arg() represents the argument function of a complex number, C′ represents the cross-correlation function, and ω C represents the probe center frequency.

[0242] In a possible implementation, the first energy image of the echo ultrasound image of the past frame and the second energy image of the echo ultrasound image of the current frame are determined, and the binary image determination module is configured as follows:

[0243] The first energy data corresponding to the echo ultrasound image of the past frame and the second energy data corresponding to the echo ultrasound image of the current frame are determined by using the following energy data formula:

[0244] Power = |RF|

[0245] where Power represents the energy data of the echo ultrasound image, RF represents the echo ultrasound data corresponding to the echo ultrasound image of the past frame and the echo ultrasound image of the current frame, and || represents the modulus of a complex number;

[0246] Perform data conversion on the first energy data corresponding to the echo ultrasound image of the past frame and the second energy data corresponding to the echo ultrasound image of the current frame to obtain the first energy image of the echo ultrasound image of the past frame and the second energy image of the echo ultrasound image of the current frame.

[0247] In a possible implementation manner, based on the first energy image of the echo ultrasound image of the past frame and the second energy image of the echo ultrasound image of the current frame, determine the motion binary image and the energy difference binary image corresponding to the past frame and the current frame. The binary image determination module is configured to:

[0248] Mark the pixel points in the first energy image and the second energy image that are higher than the first threshold boundary line as the first mark, and mark the pixel points that are lower than or equal to the first threshold boundary line as the second mark to obtain the binary images of the past frame and the current frame;

[0249] Compare the binary image of the past frame and the binary image of the current frame, mark the pixel points with different values in the binary image as the third mark, and mark the pixel points with the same values in the binary image as the fourth mark to obtain the motion binary image corresponding to the past frame image and the current frame image;

[0250] Compare the first energy image of the echo ultrasound image of the past frame and the second energy image of the echo ultrasound image of the current frame to obtain the energy difference image between the past frame and the current frame;

[0251] Mark the pixel points in the energy difference image that are higher than the second threshold boundary line as the fifth mark, and mark the pixel points that are lower than or equal to the second threshold boundary line as the sixth mark to obtain the energy difference binary image corresponding to the past frame and the current frame.

[0252] In a possible implementation manner, based on the motion binary image and the energy difference binary image, determine the enhancement region. The enhancement region determination module is configured to:

[0253] Compare the motion binary image and the energy difference binary image, and use the region composed of the pixel points that are simultaneously marked as the third mark and the fifth mark as the enhancement region.

[0254] In a possible implementation manner, based on the longitudinal displacement and the motion binary image, determine the enhancement parameter, specifically including:

[0255] Use the following enhancement parameter formula to determine the enhancement parameter:

[0256]

[0257] Among them, (i, j) represents the coordinates of a pixel point, S(i, j) represents the enhancement parameter, Δy(i, j) represents the longitudinal displacement, M(i, j) represents the motion binary image, and Thresh Δy represents a preset threshold.

[0258] In an exemplary embodiment, the present application also provides a computer-readable storage medium including instructions, such as a memory 120 including instructions. The above instructions can be executed by a processor 110 of a terminal device 100 to complete the above video image motion estimation method. Optionally, the computer-readable storage medium may be a non-transitory computer-readable storage medium. For example, the non-transitory computer-readable storage medium may be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device, etc.

[0259] In an exemplary embodiment, a computer program product is also provided, including a computer program, which implements the video image motion estimation method provided by the present application when executed by a processor 110.

[0260] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memories, CD-ROMs, optical memories, etc.) containing computer-usable program codes.

[0261] The present application is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, and the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate means for implementing the functions specified in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0262] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured product including instruction means, and the instruction means implements the functions specified in Figure 1 one process or multiple processes and / or blocksFigure 1 The functions specified in one or more boxes.

[0263] These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process. Thus, the instructions executed on the computer or other programmable device provide for implementing the steps of the functions specified in one or more processes and / or boxes Figure 1 One process or more processes and / or boxes Figure 1 The steps of the functions specified in one or more boxes.

[0264] Obviously, those skilled in the art can make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalent technologies, this application is also intended to include these changes and modifications.

Claims

1. An enhancement method for ultrasonic images, characterized in that, the method includes: Obtaining the echo ultrasonic images of the current frame and past frames; Using a block matching method based on correlation matching degree to calculate the horizontal displacement from any pixel point in the echo ultrasonic image of the past frame to the echo ultrasonic image of the current frame; Based on the horizontal displacement, using a zero-phase algorithm to calculate the vertical displacement from any pixel point in the echo ultrasonic image of the past frame to the echo ultrasonic image of the current frame; Based on the first energy image of the echo ultrasonic image of the past frame and the second energy image of the echo ultrasonic image of the current frame, determining the motion binary image and the energy difference binary image corresponding to the past frame and the current frame; Based on the motion binary image and the energy difference binary image, determining the enhancement region; Determining enhancement parameters based on the vertical displacement and the motion binary image, and using the enhancement parameters to enhance the enhancement region, where the enhancement parameters are used to enhance the enhancement region.

2. The method according to claim 1, characterized in that, the step of using a block matching method based on correlation matching degree to calculate the horizontal displacement from any pixel point in the echo ultrasonic image of the past frame to the echo ultrasonic image of the current frame specifically includes: Taking the pixel points at the same position in the echo ultrasonic image of the past frame and the echo ultrasonic image of the current frame as the central pixel points, taking the search neighborhood of the central pixel point in the echo ultrasonic image of the past frame as the first search window, and taking the search neighborhood of the central pixel point in the echo ultrasonic image of the current frame as the second search window; Taking the matching neighborhood of any pixel point in the first search window as the first matching window, taking the matching neighborhood of any pixel point in the second search window as the second matching window, and calculating the correlation between each pixel point in the first matching window and each pixel point in the second matching window; the matching neighborhood is smaller than the search neighborhood; Taking the pixel point at the corresponding position in the echo ultrasonic image of the current frame with the maximum correlation as the matching pixel point of any pixel point in the echo ultrasonic image of the past frame in the echo ultrasonic image of the current frame, and using the following horizontal displacement formula to calculate the horizontal displacement Δx(i,j) from any pixel point in the echo ultrasonic image of the past frame to the echo ultrasonic image of the current frame: Δx(i,j) = j m -j where j m represents the abscissa of a matching pixel point of the echo ultrasound image of the current frame, and j represents the abscissa of a pixel point at any position in the echo ultrasound image of the past frame.

3. The method according to claim 2, characterized in that, the step of taking the matching neighborhood of any pixel point in the first search window as the first matching window, taking the matching neighborhood of any pixel point in the second search window as the second matching window, and calculating the correlation between each pixel point in the first matching window and each pixel point in the second matching window specifically includes: Using the following correlation calculation formula to determine the correlation between each pixel point in the first matching window and each pixel point in the second matching window: Wherein, R represents the correlation, A represents the echo ultrasound image of the past frame, B represents the echo ultrasound image of the current frame, ∑ represents the summation within the ranges of Δi ∈ [-h, h] and Δj ∈ [-w, w], i represents the ordinate of any pixel point at a position in the echo ultrasound image of the past frame, j represents the abscissa of any pixel point in the echo ultrasound image of the past frame, i' represents the ordinate of any pixel point at a position in the echo ultrasound image of the current frame, j' represents the abscissa of any pixel point at a position in the echo ultrasound image of the current frame, Δi represents the range of ordinate change, Δj represents the range of ordinate change, and "||" represents the modulus of a complex number.

4. The method according to claim 2, wherein, based on the lateral displacement, the zero-phase algorithm is used to calculate the longitudinal displacement from any pixel point position in the echo ultrasound image of the past frame to the echo ultrasound image of the current frame, specifically including: Based on any pixel point at a position in the echo ultrasound image of the past frame and the lateral displacement from any pixel point at a position in the echo ultrasound image of the past frame to the echo ultrasound image of the current frame, find the corresponding pixel point of any pixel point at a position in the echo ultrasound image of the past frame in the echo ultrasound image of the current frame; Take relevant neighborhoods of the same size for the any pixel point and the corresponding pixel point, and calculate the correlation function between the past frame and the current frame in the relevant neighborhood; Based on the correlation function and the longitudinal displacement at the position of the pixel point above the any pixel point, obtain the cross-correlation function between the past frame and the current frame in the relevant neighborhood; Based on the cross-correlation function and the longitudinal displacement at the position of the pixel point above the any pixel point, obtain the longitudinal displacement at the position of the any pixel point.

5. The method according to claim 4, wherein, the finding the corresponding pixel point of any pixel point at a position in the echo ultrasound image of the past frame in the echo ultrasound image of the current frame based on any pixel point at a position in the echo ultrasound image of the past frame and the lateral displacement from any pixel point at a position in the echo ultrasound image of the past frame to the echo ultrasound image of the current frame specifically includes: Use the following position determination formula to determine the corresponding pixel point of any pixel point at a position in the echo ultrasound image of the past frame in the echo ultrasound image of the current frame: j B = j A + Δx(i A , j A ) i B = i A - Δy(i A - 1, j B ) where, i A represents the ordinate of any pixel point at any position in the echo ultrasound image of the past frame, j A represents the abscissa of any pixel point at any position in the echo ultrasound image of the past frame, i B represents the ordinate of the corresponding pixel point, j B represents the abscissa of the corresponding pixel point, Δx(i A , j A ) represents the lateral displacement from the position (i A , j A ) in the echo ultrasound image of the past frame to the echo ultrasound image of the current frame, and Δy(i A - 1, j B ) represents the longitudinal displacement at the position of the pixel point above the any-position pixel point.

6. The method according to claim 4, wherein, the taking relevant neighborhoods of the same size for the any pixel point and the corresponding pixel point and calculating the correlation function between the past frame and the current frame in the relevant neighborhood specifically includes: Use the following correlation function formula to determine the correlation function between the past frame and the current frame in the relevant neighborhood: Among them, C represents the correlation function, A represents the echo ultrasound image of the past frame, B represents the echo ultrasound image of the current frame, ∑ represents summation within the ranges of Δi′∈[-h’,h’] and Δj′∈[-w’,w’], i A represents the ordinate of any pixel point at any position in the echo ultrasound image of the past frame, j A represents the abscissa of any pixel point at any position in the echo ultrasound image of the past frame, i B represents the ordinate of any pixel point at any position in the echo ultrasound image of the current frame, j B represents the abscissa of any pixel point at any position in the echo ultrasound image of the current frame, Δi′ represents the ordinate change range, and Δj′ represents the ordinate change range.

7. The method according to claim 4, wherein, the obtaining the longitudinal displacement at the position of the any pixel point based on the cross-correlation function and the longitudinal displacement at the position of the pixel point above the any pixel point specifically includes: Use the following time shift formula to calculate the longitudinal displacement at the position of the any pixel point: where, Δy(i A ,j A ) represents the longitudinal displacement at any of the pixel points, Δy′ represents the longitudinal displacement at the pixel point position above any of the pixel points, arg() represents the argument function of a complex number, C′ represents the cross-correlation function, ω C represents the center frequency of the probe.

8. The method according to claim 1, wherein, determining a first energy image of an echo ultrasound image of a past frame and a second energy image of an echo ultrasound image of a current frame includes: using the following energy data formula to determine a first energy data corresponding to the echo ultrasound image of the past frame and a second energy data corresponding to the echo ultrasound image of the current frame: Power = |RF| wherein, Power represents the energy data of the echo ultrasound image, RF represents the echo ultrasound data corresponding to the echo ultrasound image of the past frame and the echo ultrasound image of the current frame, and || represents complex modulus; performing data conversion on the first energy data corresponding to the echo ultrasound image of the past frame and the second energy data corresponding to the echo ultrasound image of the current frame to obtain a first energy image of the echo ultrasound image of the past frame and a second energy image of the echo ultrasound image of the current frame.

9. The method according to claim 1, wherein, determining a motion binary image and an energy difference binary image corresponding to the past frame and the current frame based on the first energy image of the echo ultrasound image of the past frame and the second energy image of the echo ultrasound image of the current frame specifically includes: marking the pixel points in the first energy image and the second energy image that are higher than the first threshold dividing line as the first mark, and the pixel points that are lower than or equal to the first threshold dividing line as the second mark to obtain the binary images of the past frame and the current frame; comparing the binary image of the past frame and the binary image of the current frame, marking the pixel points with different values in the binary image as the third mark, and the pixel points with the same values in the binary image as the fourth mark to obtain the motion binary image corresponding to the past frame image and the current frame image; comparing the first energy image of the echo ultrasound image of the past frame and the second energy image of the echo ultrasound image of the current frame to obtain the energy difference image between the past frame and the current frame; marking the pixel points in the energy difference image that are higher than the second threshold dividing line as the fifth mark, and the pixel points that are lower than or equal to the second threshold dividing line as the sixth mark to obtain the energy difference binary image corresponding to the past frame and the current frame.

10. An ultrasound device, wherein, comprising: a processor, a memory, a display unit, and a probe; the probe is used for transmitting an ultrasound signal; the display unit is used for displaying an ultrasound image; the processor is respectively connected to the probe and the display unit, and is configured to execute the ultrasound image enhancement method according to any one of claims 1-9.

Images