A method for ultrafast ultrasound puncture enhanced imaging

Abstract

The present invention provides an ultrafast ultrasound puncture enhanced imaging method, wherein the method comprises: S1: in a unit time, in the initial round, a plurality of wide beam signals with long steps and different deflection angles are emitted with ultrafast plane waves, and a specific deflection area is selected according to the direction in which the puncture needle enters according to the user; S2: multiple frames of plane wave deflection images under different emission angles are obtained, and a pre-trained puncture needle angle discrimination neural network is input; S3: the optimal puncture needle rough estimated angle output by the puncture needle angle discrimination neural network is obtained; S4: in the next round of emission, a plurality of wide beam signals with short steps and different deflection angles are emitted to expand specific angles on both sides with the optimal rough estimated angle output in the previous round as the center. The ultrafast ultrasound puncture enhanced imaging method of the present invention solves the problem of correction of puncture needle angle change, puncture path error and needle position judgment in current puncture enhancement.

Description

Technical Field

[0001] The invention relates to the technical field of ultrasonic puncture enhanced imaging, and in particular to an ultrafast ultrasonic puncture enhanced imaging method. Background Art

[0002] At present, ultrasonic puncture enhancement technology is a very common function in clinical practice, and its function is mainly used to guide clinical physicians to perform puncture surgery. The current implementation of puncture enhancement technology is mainly through the matching puncture stand (or other types of navigation hardware) to evaluate the insertion position of the puncture needle, and the software algorithm is used to perform regional enhancement on the needle body. In addition, there is also a puncture enhancement method that does not use the puncture stand to fix the needle body, and evaluates the angle of the needle body by emitting a series of detection signals. With the support of existing technologies, it is generally necessary to add a sequence of ultrasonic images generated by transmitting signals with a fixed deflection angle that is tilted along the insertion direction of the puncture needle. This transmission signal needs to be as perpendicular to the insertion angle of the needle body as possible to obtain the maximum reflection signal. At this time, the needle body image is the most obvious. After obtaining the maximum reflection signal, the display effect of the needle body in the image can be enhanced through image enhancement and mixing, thereby improving the safety of the puncture surgery.

[0003] In the clinical use scenario of puncture enhancement, the situation often faced is that the puncture angle of the needle body changes slightly. After the deflection angle is determined, due to the change in the puncture needle angle, it is impossible to achieve the ideal condition of perpendicular to the needle body insertion angle, which further leads to the inability of the transmission signal with a fixed deflection angle to fully image the puncture needle, further affecting the judgment of the needle insertion angle. In addition, the thinner needle body may undergo partial bending and other deformations during the puncture process (trying to pass through harder tissues), which reduces the enhancement effect of a single deflection angle on the puncture needle and increases the error in depicting the puncture path, affecting the clinical user's judgment of the puncture needle position and the effectiveness of the operation. The above two phenomena can be alleviated by using the puncture stand method mentioned above, but it will greatly reduce the flexibility and freedom of clinical surgery. At the same time, the constant manual correction of the puncture needle angle during the operation will delay the overall progress of the operation and bring many inconveniences.

[0004] During ultrasonic puncture surgery, accurate positioning of the needle and clear images are the most important foundations. At present, ultrasonic puncture enhancement mostly relies on hardware positioning combined with software to achieve the effect of clinical surgical assistance, which provides a certain guarantee for the effectiveness and safety of the surgery. In addition to the assistance of hardware or external equipment, puncture enhancement generally collects image frame signals for specific deflection angles in the transmission sequence to enhance the visualization of the puncture needle. An ultrasonic beam is emitted that forms a large angle with the surface of the puncture needle, so that the echo signal received by the probe increases and the ultrasonic imaging is enhanced. A section of the surface of the puncture needle body will be designed to be spiral, so that the ultrasound forms a high-echo reflection surface on the surface, so that the brightness reflected by the needle body in the image is higher than the surrounding tissue signals, which can better guide the user to perform surgery and improve the accuracy of the puncture process.

[0005] In order to make the puncture needle appear highlighted in the image frame, part of the needle body of the puncture needle is generally made into a spiral design (or other special shapes). This design can improve the visual presentation of the puncture needle, but it also brings certain risks. Generally, the puncture needle is only designed with a thread on a section of the needle body, which is not conducive to the full tracking and positioning of the puncture instrument. In addition, this design will have a certain impact on the force structure of the puncture needle body, resulting in partial uneven force distribution and specific weak points, affecting the puncture needle's anti-bending ability and hardness, so that the puncture needle may bend or even break during the operation. Summary of the invention

[0006] One of the purposes of the present invention is to provide an ultrafast ultrasound puncture enhancement imaging method, which aims to solve the current puncture enhancement correction problem for puncture needle angle changes, puncture path errors and needle position judgment. The patent of this invention combines deep learning and image processing technology to provide more accurate puncture needle positioning and needle bending correction.

[0007] An ultrafast ultrasound puncture enhanced imaging method provided by an embodiment of the present invention comprises:

[0008] S1: The initial round transmits several wide beam signals with long steps and different deflection angles with ultrafast plane waves in a unit time, and selects a specific deflection area according to the user's direction of entry of the puncture needle;

[0009] S2: Acquire multiple frames of plane wave deflection images at different emission angles and input them into a pre-trained puncture needle angle discrimination neural network;

[0010] S3: Obtaining the optimal rough estimated angle of the puncture needle output by the puncture needle angle discrimination neural network;

[0011] S4: In the next round of transmission, the optimal rough estimated angle output in the previous round is used as the center to amplify the specific angle on both sides and transmit a wide beam signal with different deflection angles in short steps;

[0012] S5: The plane wave deflection images obtained under different emission angles are input into a pre-trained puncture needle angle discrimination neural network to obtain the optimal puncture needle accurate angle;

[0013] S6: Determine whether the needle body probability included in the optimal puncture needle precise angle is a single angle. If the needle body is bent or part of the needle body is offset so that a single deflection angle cannot fully image the needle body, repeat step S5 until the two better precise emission angles in the optimal puncture needle precise angle are selected, and compare the angle deviation of the puncture needle offset in the two precise emission angles. If the angle deviation is greater than a specific threshold, it is marked as the puncture needle body may be bent.

[0014] S7. In the subsequent unit time, the puncture needle is transmitted at a specific deflection angle at an optimal precise angle, and the image frame data formed by transmitting the deflection angle only once is used to detect the puncture needle body.

[0015] Preferably, the ultrafast ultrasound puncture enhanced imaging method further comprises:

[0016] S8, if the puncture needle body is marked as possibly bent, then in step S7, the two better accurate emission angles recorded in step S6 are transmitted, and the image frame data received at all deflection angles are used for puncture needle detection;

[0017] S9, the processing of puncture needle detection includes performing eigenvalue decomposition filtering on the image frame, performing Hough transform processing after retaining the information in the image, for locating the connecting line coordinate point where the needle body is located, and fitting the multiple connecting line coordinate points to obtain the puncture path of the puncture needle along the current moving direction;

[0018] S10, performing directional filtering and morphological filtering on the image frame after the puncture needle detection process along the direction of the optimal puncture needle precise angle calculated in step S7, and connecting the cross sections of the puncture needle in the image frame;

[0019] S11, if the puncture needle body is marked as possibly bent, the puncture needle body is spliced ​​in step 9, a specific threshold is given to determine the starting and ending points of the two frames of puncture needle images, and the puncture needle image at the far end of the direction of entry of the puncture needle is mixed with a fixed weight according to the user, and the coordinate point set of the two frames of puncture needle images is fitted to mark the puncture path of the puncture needle along the current direction of travel;

[0020] S12, output image, binarize the detected needle image, judge the puncture needle area with a specific threshold and mark it as the needle coordinates, enhance the needle coordinate points, mix and superimpose the background B image with a fixed weight, and mark the puncture needle path.

[0021] Preferably, in step S2, puncture enhancement requires real-time imaging in clinical practice, and has high requirements for the stability of calculation and judgment. The present invention selects yolov5-lite as the pre-trained neural network, and adopts a first-order recognition network to improve the calculation speed and minimize GPU consumption, so as to avoid the phenomenon of excessive consumption of plane wave ultrasound calculation leading to system freeze and reduced frame rate. The pre-trained input image is a sequence of clinical puncture needle images in different scenes and the coordinate position of the minimum rectangular frame where the puncture needle is located [X1, Y1, W, H], where X1 and Y1 are the coordinate positions of the upper left corner of the rectangular frame, and W and H are the width and height of the marked frame, as follows Figure 1 As shown in . The needle body in the puncture needle image needs to be as complete as possible and the needle tip should be as obvious as possible. Before inputting the network training, the puncture needle image is subjected to multi-scale anisotropic convolution filtering to enhance the image and improve the recognition probability of the puncture needle.

[0022] Preferably, in step S3, 13 frames of plane wave deflection images at different emission angles are input into the training network, and the network outputs the position of the puncture needle in the 13 image frames and the probability of being a puncture needle. The output probabilities of the puncture needles in the sequence are sorted, and the angle of the plane wave emitted by the detection result of the puncture needle image frame with the maximum probability is used as the current priority rough estimate angle N0.

[0023] Preferably, in step S4, the amplification angle is selected as the minimum step size of 5° in step 1, and the plane wave emission deflection range [(N0-5)°, (N0+5)°] required for this round is calculated with a rough estimate of angle N°, and plane wave emission with 11 deflection angles is performed with a step size of 1°.

[0024] Preferably, in the eigenvalue decomposition filtering stage, an image frame with a dimension of m×n is first selected, the column mean is calculated to obtain the result of the difference between the image frame and the column mean, and the covariance matrix is ​​calculated, and the calculation method is as follows:

[0025]

[0026] Among them, Img needle is the input m×n dimensional image frame, N is the larger dimension of the image frame;

[0027] Next, the eigenvalue decomposition result of the covariance matrix is ​​calculated and the eigenvalues ​​are sorted in descending order to [S0, S1, S2…Sn]. The signals below S2 in the descending order eigenvalue list are selected to filter out the noise signal in the image frame.

[0028] Preferably, considering that the puncture needle is generally a straight line at a single angle, the filtered image frame is subjected to Hough transform to separate the straight line segment signal in the image, and the various parameters in the rectangular coordinate system are mapped to polar coordinate parameters through coordinate transformation. The straight line segment in the image can be determined by the distance ρ from the origin to the straight line in the rectangular coordinate system and the polar angle θ from the x-axis to the perpendicular line. After the transformation, the parameters (a, b) of the straight line in the rectangular coordinate system are converted to (ρ, θ). The straight line segment in the image frame can be detected by statistical maximum values. The puncture needle signal in the filtered image frame is transformed from the straight line segment detection to the method of performing maximum value statistics on specific parameters by this method.

[0029] Preferably, in step S10, a two-dimensional Gaussian kernel function filter G with a dimension of k*k is constructed, and the coefficient of distribution of each axis can be determined by the position parameter x∈[-k / 2:1:k / 2] and the scale parameter σ, and its probability density function is:

[0030]

[0031] Among them, exp is the exponential function, x' is the kernel function, σ is the scale parameter, k is selected as 7, σ is 0.5, and the constructed two-dimensional Gaussian kernel filter is used to deflect the precise angle obtained in step 7, and the obtained image frame is directional filtered to enhance the morphological structure of the puncture needle.

[0032] Preferably, in step S11, if the puncture needle is marked as bent (bend=1), two puncture needle image frames are processed in step 10, and the overlapping parts of the two puncture needle images are mixed with a weight of P=0.5. Given a puncture needle amplitude threshold Pth, and taking each column of coordinate points greater than Pth as the puncture needle surface, all coordinate sets {[x0,x1,…,xn],[y0,y1,…,yn]} are counted and fitted to obtain a fitted puncture path.

[0033] Preferably, it is characterized in that in step S12, the mixed puncture needle image is binarized to form a binary mask with the puncture needle body, the coordinate point with a binary mask of 1 in the background B image is extended to a coordinate set of 3 pixels and a weight of P=0.8 is given, and the mixed puncture needle image frame is multiplied by the binary mask and superimposed on the background B image.

[0034] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description, claims, and drawings.

[0035] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0037] Figure 1 A schematic diagram of a network training input image and needle body annotation in an embodiment of the present invention;

[0038] Figure 2 is a schematic diagram of an ultrasonic scanning sequence with different deflection angles in an embodiment of the present invention;

[0039] Figure 3 It is a schematic diagram of the emission at a specific deflection angle with a precise angle of the puncture needle in a subsequent unit time in an embodiment of the present invention;

[0040] Figure 4 is a schematic diagram of a filtered image in an embodiment of the present invention;

[0041] Figure 5 is a schematic diagram of performing Hough transform on a filtered image frame in an embodiment of the present invention;

[0042] Figure 6 It is a schematic diagram of performing directional filtering on the obtained image frame to enhance the morphological structure of the puncture needle in an embodiment of the present invention;

[0043] Figure 7 Schematic diagram of a binary mask with a puncture needle body in an embodiment of the present invention. DETAILED DESCRIPTION

[0044] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0045] An embodiment of the present invention provides an ultrafast ultrasound puncture enhanced imaging method, comprising:

[0046] S1: The initial round transmits several wide beam signals with long steps and different deflection angles with ultrafast plane waves in a unit time, and selects a specific deflection area according to the user's direction of entry of the puncture needle;

[0047] S2: Acquire multiple frames of plane wave deflection images at different emission angles and input them into a pre-trained puncture needle angle discrimination neural network;

[0048] S3: Obtaining the optimal rough estimated angle of the puncture needle output by the puncture needle angle discrimination neural network;

[0049] S4: In the next round of transmission, the optimal rough estimated angle output in the previous round is used as the center to amplify the specific angle on both sides and transmit a wide beam signal with different deflection angles in short steps;

[0050] S5: The plane wave deflection images obtained under different emission angles are input into a pre-trained puncture needle angle discrimination neural network to obtain the optimal puncture needle accurate angle;

[0051] S6: Determine whether the needle body probability included in the optimal puncture needle precise angle is a single angle. If the needle body is bent or part of the needle body is offset so that a single deflection angle cannot fully image the needle body, repeat step S5 until the two better precise emission angles in the optimal puncture needle precise angle are selected, and compare the angle deviation of the puncture needle offset in the two precise emission angles. If the angle deviation is greater than a specific threshold, it is marked as the puncture needle body may be bent.

[0052] S7. In the subsequent unit time, the puncture needle is transmitted at a specific deflection angle at an optimal precise angle, and the image frame data formed by transmitting the deflection angle only once is used to detect the puncture needle body.

[0053] S8, if the puncture needle body is marked as possibly bent, then in step S7, the two better accurate emission angles recorded in step S6 are transmitted, and the image frame data received at all deflection angles are used for puncture needle detection;

[0054] S9, the processing of puncture needle detection includes performing eigenvalue decomposition filtering on the image frame, performing Hough transform processing after retaining the information in the image, for locating the connecting line coordinate point where the needle body is located, and fitting the multiple connecting line coordinate points to obtain the puncture path of the puncture needle along the current moving direction;

[0055] S10, performing directional filtering and morphological filtering on the image frame after the puncture needle detection process along the direction of the optimal puncture needle precise angle calculated in step S7, and connecting the cross sections of the puncture needle in the image frame;

[0056] S11, if the puncture needle body is marked as possibly bent, the puncture needle body is spliced ​​in step 9, a specific threshold is given to determine the starting and ending points of the two frames of puncture needle images, and the puncture needle image at the far end of the direction of entry of the puncture needle is mixed with a fixed weight according to the user, and the coordinate point set of the two frames of puncture needle images is fitted to mark the puncture path of the puncture needle along the current direction of travel;

[0057] S12, output image, binarize the detected needle image, judge the puncture needle area with a specific threshold and mark it as the needle coordinates, enhance the needle coordinate points, mix and superimpose the background B image with a fixed weight, and mark the puncture needle path.

[0058] In step S2, puncture enhancement requires real-time imaging in clinical practice, and has high requirements for the stability of calculation and judgment. The present invention selects yolov5-lite as the pre-trained neural network, and adopts a first-order recognition network to improve the calculation speed and minimize GPU consumption, so as to avoid the phenomenon of excessive consumption of plane wave ultrasound calculation leading to system freeze and reduced frame rate. The pre-trained input image is a sequence of clinical puncture needle images in different scenes and the coordinate position of the minimum rectangular box where the puncture needle is located [X1, Y1, W, H], where X1 and Y1 are the coordinate positions of the upper left corner of the rectangular box, and W and H are the width and height of the marked box, as follows Figure 1 As shown in . The needle body in the puncture needle image needs to be as complete as possible and the needle tip should be as obvious as possible. Before inputting the network training, the puncture needle image is subjected to multi-scale anisotropic convolution filtering to enhance the image and improve the recognition probability of the puncture needle.

[0059] In step S3, 13 frames of plane wave deflection images under different emission angles are input into the training network, and the network outputs the position of the puncture needle and the probability of being a puncture needle in the 13 image frames. The output probability of the puncture needle in the sequence is sorted, and the angle of the plane wave emitted by the detection result of the puncture needle image frame with the maximum probability is used as the current priority rough estimate angle N0.

[0060] In step S4, the amplification angle is selected as the minimum step size of 5° in step 1, and the plane wave transmission deflection range [(N0-5)°, (N0+5)°] required for this round is calculated with a rough estimate of angle N°, and plane wave transmission with 11 deflection angles is performed with a step size of 1°.

[0061] In the eigenvalue decomposition filtering stage, first select an image frame with a dimension of m×n, calculate the column mean to obtain the difference between the image frame and the column mean, and calculate the covariance matrix. The calculation method is as follows:

[0062]

[0063] Among them, Img needle is the input m×n dimensional image frame, N is the larger dimension of the image frame;

[0064] Next, the eigenvalue decomposition result of the covariance matrix is ​​calculated and the eigenvalues ​​are sorted in descending order to [S0, S1, S2…Sn]. The signals below S2 in the descending order eigenvalue list are selected to filter out the noise signal in the image frame.

[0065] Considering that the puncture needle is generally a straight line at a single angle, the Hough transform is performed on the filtered image frame to separate the straight line segment signal in the image. Through coordinate transformation, various parameters in the rectangular coordinate system are mapped to polar coordinate parameters. The straight line segment in the image can be determined by the distance ρ from the origin to the straight line in the rectangular coordinate system and the polar angle θ from the x-axis to the vertical line. After the transformation, the parameters (a, b) of the straight line in the rectangular coordinate system are converted to (ρ, θ). The straight line segment in the image frame can be detected by statistical maximum values. The puncture needle signal in the filtered image frame uses this method to transform the straight line segment detection from the method of performing maximum value statistics on specific parameters.

[0066] In step S10, a two-dimensional Gaussian kernel function filter G with a dimension of k*k is constructed. The coefficient of each axial distribution can be determined by the position parameter x∈[-k / 2:1:k / 2] and the scale parameter σ, and its probability density function is:

[0067]

[0068] Among them, exp is the exponential function, x' is the kernel function, σ is the scale parameter, k is selected as 7, σ is 0.5, and the constructed two-dimensional Gaussian kernel filter is used to deflect the precise angle obtained in step 7, and the obtained image frame is directional filtered to enhance the morphological structure of the puncture needle.

[0069] In step S11, if the puncture needle is marked as bent (bend = 1), two puncture needle image frames will be processed in step 10, and the overlapping parts of the two puncture needle images will be mixed with a weight of P = 0.5. Given a puncture needle amplitude threshold Pth, and taking each column of coordinate points greater than Pth as the puncture needle surface, all coordinate sets {[x0, x1, ..., xn], [y0, y1, ..., yn]} are counted and fitted to obtain the fitted puncture path.

[0070] It is characterized in that in step S12, the mixed puncture needle image is binarized to form a binary mask with the puncture needle body, the coordinate point with a binary mask of 1 in the background B image is extended to a coordinate set of 3 pixels and a weight of P=0.8 is given, and the mixed puncture needle image frame is multiplied by the binary mask and superimposed on the background B image.

[0071] The working principle and beneficial effects of the above technical solution are:

[0072] Step 1: In a unit time, the initial round transmits a number of wide beam signals with long steps and different deflection angles using ultrafast plane waves. A specific deflection area can be selected according to the user's direction of entry of the puncture needle.

[0073] The user can select the left / right side as the puncture needle insertion position on the control panel of the ultrasound device. If the left side is selected, the transmission angle range is [0°, -60°], and the step size is 13 plane wave signals of 5°. If the right side is selected, the transmission angle range is changed to [0°, 60°], and the step size is also 5°.

[0074] Step 2: The obtained plane wave deflection images at multiple frames with different emission angles are input into a pre-trained puncture needle angle discrimination neural network.

[0075] In clinical practice, puncture enhancement requires real-time imaging and high stability of calculation and judgment. The present invention selects yolov5-lite as the pre-trained neural network, and adopts a first-order recognition network to improve the calculation speed and minimize GPU consumption, so as to avoid the phenomenon of excessive consumption of plane wave ultrasound calculation leading to system freeze and reduced frame rate. The pre-trained input image is a sequence of clinical puncture needle images in different scenes and the coordinate position of the minimum rectangular box where the puncture needle is located [X1, Y1, W, H], where X1 and Y1 are the coordinate positions of the upper left corner of the rectangular box, and W and H are the width and height of the marked box, as follows Figure 1 As shown in . The needle body in the puncture needle image needs to be as complete as possible and the needle tip should be as obvious as possible. Before inputting the network training, the puncture needle image is subjected to multi-scale anisotropic convolution filtering to enhance the image and improve the recognition probability of the puncture needle.

[0076] Step 3: Determine the insertion angle of the puncture needle based on the pre-trained neural network, output the probability that the puncture needle may exist at different angles, and determine the optimal puncture needle rough angle contained in the above emission angles.

[0077] 13 frames of plane wave deflection images under different emission angles are input into the training network. The network outputs the location of the puncture needle and the probability of it being a puncture needle in the 13 image frames. The output probability of the puncture needle in the sequence is sorted, and the angle of the plane wave emitted by the detection result of the puncture needle image frame with the maximum probability is used as the current priority rough estimate angle N0.

[0078] Step 4: In the next round of transmission, the optimal rough estimated angle output in the previous round is used as the center to amplify the specific angle on both sides and transmit a wide beam signal with different deflection angles with short steps, as follows Figure 2 shown.

[0079] In the present invention, the amplification angle is selected as the minimum step size of 5° in step 1, and the plane wave emission deflection range [(N0-5)°, (N0+5)°] required for this round is calculated with a rough estimate of angle N°, and plane wave emission with 11 deflection angles is performed with a step size of 1°.

[0080] Step 5: The plane wave deflection images obtained under different emission angles are input into a pre-trained puncture needle angle discrimination neural network, and the optimal puncture needle precise angle contained in the above emission angles is output.

[0081] Similar to step 2, the output probability of the puncture needles in the sequence is sorted, and the puncture angle list under sorting is recorded to form [n0,n1,…,n10],[p0,p1,…,p10] and

[0082] The list of [[x0,y0,w0,h0],[x1,y1,w1,h1],…,[x10,y10,w10,h10]] is the sorted precise deflection angle, probability list and position coordinates of puncture needle recognition.

[0083] Step 6: The probability of the needle body included in the precise angle of the puncture needle is determined to be a single angle. If the needle body is bent or part of the needle body is offset, so that a single deflection angle cannot fully image the needle body, repeat step 5. Until the best first two precise emission angles are selected, and the angle deviation of the puncture needle offset is compared between the two angles. If the angle deviation is greater than a specific threshold, it is marked as a possible bending of the puncture needle body.

[0084] Compare the top 2 puncture needle probabilities (p0, p1) with their position coordinates ([x0, y0, w0, h0], [x1, y1, w1, h1]). Calculate the intersection over union (IOU) of the two position coordinates to determine whether they are highly overlapped (IOU>0.6), and the absolute probability value difference must not be less than 0.25 (|p0–p1|<0.25). Otherwise, it is determined that the puncture needle cannot be fully imaged, and repeat step 5.

[0085] After the above conditions are met, if the absolute difference between the two fine angles (n0, n1) is greater than 5° (|n0–n1|>5), the puncture needle is considered to be bent and marked as bent (bend=1). Otherwise, the puncture needle is considered to be unbent (bend=0).

[0086] Step 7: In the subsequent unit time, the puncture needle is transmitted at a specific deflection angle with a precise angle, and the image frame data formed by transmitting only once at this deflection angle is used to detect the puncture needle body, as follows Figure 3 Displayed in.

[0087] Step 8: Preferably, if the puncture needle body is marked as possibly bent, the first two precise emission angles recorded in step 6 are transmitted in step 7, and the image frame data received at all deflection angles are used for puncture needle detection.

[0088] If the puncture needle is bent (bend=1), two angles (n0, n1) are transmitted and two image frames are retained when a specific deflection angle is performed according to the precise angle of the puncture needle. Otherwise, the image is transmitted according to the maximum probability precise angle (n0) and one image frame is retained.

[0089] Step 9: The processing of puncture needle detection includes performing eigenvalue decomposition filtering on the image frame, retaining the information in the image, and performing Hough transform processing to locate the connecting line coordinate point where the needle body is located. Multiple connecting line coordinate points are fitted to obtain the puncture path of the puncture needle along the current direction of travel.

[0090] In the present invention, in the eigenvalue decomposition filtering stage, firstly, an image frame with a dimension of mxn is selected, its column mean is calculated to obtain the result of the difference between the image frame and the column mean, and its covariance matrix is ​​calculated, and the calculation method is as follows:

[0091]

[0092] Among them, ρmg needle is the input mxn dimensional image frame, N is the larger dimension of the image frame.

[0093] Next, the eigenvalue decomposition result of the covariance matrix is ​​calculated and the eigenvalues ​​are sorted in descending order as [S0, S1, S2...Sn]. The present invention selectively retains the signals below S2 in the descending order eigenvalue list and filters out the noise signal in the image frame. The filtered image is as follows: Figure 4 Displayed in.

[0094] Considering that the puncture needle is generally a straight line at a single angle, the filtered image frame is subjected to Hough transform to separate the straight line segment signal in the image. Through coordinate transformation, the various parameters in the rectangular coordinate system are mapped to polar coordinate parameters. The straight line segment in the image can be determined by the distance ρ from the origin to the straight line in the rectangular coordinate system and the polar angle θ from the x-axis to the vertical line, as follows Figure 5 As shown in . After the conversion, the parameters (a, b) of the straight line in the rectangular coordinate system are converted to (ρ, θ), and the straight line segment in the image frame can be detected by statistical maximum value. The puncture needle signal in the filtered image frame can be transformed from the method of performing maximum value statistics on specific parameters by this method to detect the straight line segment.

[0095] Step 10: Perform directional filtering and morphological filtering on the image frame after the puncture needle detection process along the precise angle calculated in step 7, and connect the possible cross sections of the puncture needle in the image frame.

[0096] Next, a two-dimensional Gaussian kernel function filter G with a dimension of k*k is constructed. The coefficient of the distribution of each axis can be determined by the position parameter x∈[-k / 2:1:k / 2] and the scale parameter σ, and its probability density function is:

[0097]

[0098] Wherein, exp is an exponential function, x' is a kernel function, and σ is a scale parameter. In the present invention, k is selected as 7, σ is selected as 0.5, and the constructed two-dimensional Gaussian kernel filter is used to deflect the precise angle obtained in step 7, and the obtained image frame is subjected to directional filtering to enhance the morphological structure of the puncture needle, as follows Figure 6 Displayed in.

[0099] Step 11: Preferably, if the puncture needle body is marked as possibly bent, the puncture needle body is spliced ​​in step 9, and a specific threshold is given to determine the starting and ending points of the two frames of puncture needle images. According to the direction of entry of the puncture needle set by the user, the puncture needle image at the far end of the direction is mixed with a fixed weight, and the coordinate point set of the two frames of puncture needle images is fitted to mark the puncture path of the puncture needle along the current direction of travel.

[0100] If the puncture needle is marked as bent (bend = 1), two puncture needle image frames will be processed in step 10, and the overlapping parts of the two puncture needle images will be mixed with a weight of P = 0.5. Given the puncture needle amplitude threshold Pth, and taking each column of coordinate points greater than Pth as the puncture needle surface, all coordinate sets {[x0, x1, ..., xn], [y0, y1, ..., yn]} are counted and fitted to obtain the fitted puncture path.

[0101] Step 12: Output the image, binarize the detected needle image, determine the puncture needle area with a specific threshold and mark it as the needle coordinates, enhance the needle coordinates and superimpose them with the background B image with a fixed weight, and mark the puncture needle path.

[0102] Next, the mixed puncture needle image is binary processed to form a binary mask with the puncture needle body, as follows Figure 7 In the figure, the needle body position is 1 and the background is 0. The coordinate set of the coordinate point with a binary mask of 1 in the background B image is extended to 3 pixels and given a weight of P = 0.8. The mixed puncture needle image frame is multiplied by the binary mask and superimposed on the background B image. This method can highlight the visual of the puncture needle body in the image, enhance the display effect of the overall puncture needle in the image, and improve the safety of the puncture surgery. Finally, the puncture path is marked in the image to provide real-time assistance and more accurate guidance for the surgical process.

[0103] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. An ultrafast ultrasound puncture enhanced imaging method, characterized in that: include: S1: The initial round transmits several wide beam signals with long steps and different deflection angles with ultrafast plane waves in a unit time, and selects a specific deflection area according to the user's direction of entry of the puncture needle; S2: Acquire multiple frames of plane wave deflection images at different emission angles and input them into a pre-trained puncture needle angle discrimination neural network; S3: Obtaining the optimal rough estimated angle of the puncture needle output by the puncture needle angle discrimination neural network; S4: In the next round of transmission, the optimal rough estimated angle output in the previous round is used as the center to amplify the specific angle on both sides and transmit a number of wide beam signals with short step lengths and different deflection angles; S5: The plane wave deflection images obtained at different emission angles are input into a pre-trained puncture needle angle discrimination neural network to obtain the optimal puncture needle precise angle for each different emission angle; S6: judging whether the needle body is bent or a single deflection angle is caused by partial needle body deviation according to the needle body probability contained in the optimal puncture needle precise angle of each emission angle; if so, it is impossible to fully image the needle body, then repeating step S5 until the two better precise emission angles among the optimal puncture needle precise angles are selected, and comparing the angle deviation of the puncture needle deviation between the two precise emission angles; if the angle deviation is greater than a specific threshold, then marking the puncture needle body as possibly bent; S7. In the subsequent unit time, the puncture needle is transmitted at a specific deflection angle at an optimal precise angle, and the image frame data formed by transmitting the deflection angle only once is used to detect the puncture needle body.

2. The ultrafast ultrasound puncture enhanced imaging method according to claim 1, characterized in that: Also includes: S8, if the puncture needle body is marked as possibly bent, then in step S7, the two better accurate emission angles recorded in step S6 are transmitted, and the image frame data received at all deflection angles are used for puncture needle detection; S9, processing of puncture needle detection and eigenvalue decomposition filtering of the image frame, performing Hough transform processing after retaining the information in the image, for locating the connecting line coordinate point where the needle body is located, and fitting multiple connecting line coordinate points to obtain the puncture path of the puncture needle along the current moving direction; S10, performing directional filtering and morphological filtering on the image frame after the puncture needle detection process along the direction of the optimal puncture needle precise angle calculated in step S7, and connecting the cross sections of the puncture needle in the image frame; S11, if the puncture needle body is marked as possibly bent, the puncture needle body is spliced ​​in step 9, a specific threshold is given to determine the starting and ending points of the two frames of puncture needle images, and the puncture needle image at the far end of the direction of entry of the puncture needle is mixed with a fixed weight according to the user, and the coordinate point set of the two frames of puncture needle images is fitted to mark the puncture path of the puncture needle along the current direction of travel; S12, output image, binarize the detected needle image, judge the puncture needle area with a specific threshold and mark it as the needle coordinates, enhance the needle coordinate points, mix and superimpose the background B image with a fixed weight, and mark the puncture needle path.

3. The ultrafast ultrasound puncture enhanced imaging method according to claim 1, characterized in that: In step S2, puncture enhancement requires real-time imaging in clinical practice, and has high requirements for the stability of calculation and judgment. The present invention selects yolov5-lite as the pre-trained neural network, and adopts a first-order recognition network to improve the calculation speed and minimize GPU consumption, so as to avoid the phenomenon of system freeze and frame rate reduction caused by excessive consumption of plane wave ultrasound calculation. The pre-training input image is a sequence of clinical puncture needle images in different scenes and the coordinate position of the minimum rectangular frame where the puncture needle is located [X1, Y1, W, H], where X1 and Y1 are the coordinate positions of the upper left corner of the rectangular frame, respectively, and W and H are the width and height of the marking frame, respectively. The needle body in the puncture needle image needs to be as complete as possible and the needle tip is obvious. Before inputting the network training, the puncture needle image is subjected to multi-scale anisotropic convolution filtering to enhance the image and improve the recognition probability of the puncture needle.

4. The ultrafast ultrasound puncture enhanced imaging method according to claim 1, characterized in that: In step S3, 13 frames of plane wave deflection images under different emission angles are input into the training network. The network outputs the position of the puncture needle and the probability of the puncture needle in the 13 image frames, and sorts the output probabilities of the puncture needles in the sequence. The angle of the plane wave emitted by the detection result of the puncture needle image frame with the maximum probability is used as the current priority rough estimate angle N0.

5. The ultrafast ultrasound puncture enhanced imaging method according to claim 1, characterized in that: In step S4, the amplification angle is selected as the minimum step size of 5° in step 1, and the plane wave transmission deflection range [(N0-5)°, (N0+5)°] required for this round is calculated with a rough estimate of angle N°, and plane wave transmission with 11 deflection angles is performed with a step size of 1°.

6. The ultrafast ultrasound puncture enhanced imaging method according to claim 2, characterized in that: In the eigenvalue decomposition filtering stage, we first select a dimension The image frame is taken, the column mean is calculated to obtain the difference between the image frame and the column mean, and the covariance matrix is ​​calculated as follows: in, For input dimensional image frame, is the transpose of the input mxn dimensional image frame, is the larger dimension of the image frame; Next, the eigenvalue decomposition result of the covariance matrix is ​​calculated and the eigenvalues ​​are sorted in descending order to [S0, S1, S2…Sn]. The signals below S2 in the descending order eigenvalue list are selected to filter out the noise signal in the image frame.

7. The ultrafast ultrasound puncture enhanced imaging method according to claim 2, characterized in that: Considering that the puncture needle is generally a straight line at a single angle, the filtered image frame is subjected to Hough transform to separate the straight line segment signal in the image. Through coordinate transformation, the various parameters in the rectangular coordinate system are mapped to polar coordinate parameters. The straight line segment in the image can be obtained by the distance from the origin to the straight line in the rectangular coordinate system. Polar angle from the x-axis to the vertical line OK, after the conversion, the parameters (a, b) of the straight line in the rectangular coordinate system are converted to ( , ), the straight line segments in the image frame can be detected by statistical maxima, and the puncture needle signal in the filtered image frame uses this method to transform the straight line segment detection into a method of performing maximum statistics on specific parameters.

8. The ultrafast ultrasound puncture enhanced imaging method according to claim 2, characterized in that: In step S10, a two-dimensional Gaussian kernel function filter G with a dimension of k*k is constructed, and the coefficient of each axial distribution can be determined by the position parameter ∈[-k / 2:1:k / 2] and the scale parameter Determine, its probability density function is: Among them, exp is the exponential function, is the kernel function, , choose k as 7, The constructed two-dimensional Gaussian kernel filter is used to deflect the precise angle obtained in step 7, and direction filtering is performed on the obtained image frame to enhance the morphological structure of the puncture needle.

9. The ultrafast ultrasound puncture enhanced imaging method according to claim 2, characterized in that: In step S11, if the puncture needle is marked as bent (bend=1), two puncture needle image frames will be processed in step 10, and the overlapping parts of the two puncture needle images will be mixed with a weight of P=0.

5. The puncture needle amplitude threshold Pth is given, and each column of coordinate points greater than Pth is taken as the puncture needle surface. All coordinate sets {[x0,x1,…,xn],[y0,y1,…,yn]} are counted and fitted to obtain the fitted puncture path.

10. The ultrafast ultrasound puncture enhanced imaging method according to claim 2, characterized in that: In step S12, the mixed puncture needle image is binarized to form a binary mask with the puncture needle body, the coordinate point with a binary mask of 1 in the background B image is extended to a coordinate set of 3 pixels and a weight of P=0.8 is given, and the mixed puncture needle image frame is multiplied by the binary mask and superimposed on the background B image.

Images