Flexible ultrasonic probe real-time high-quality imaging method

Abstract

A real-time high-quality imaging method for a flexible ultrasonic probe comprises the following steps: demodulating RF data collected by the flexible probe into I / Q data through Hilbert transform and envelope detection, compressing dense I / Q data from three dimensions of a transmitting channel, a receiving channel and sampling time by applying a sparse processing mechanism, and transmitting the compressed I / Q data to the flexible ultrasonic probe; a space perception shape estimation network is constructed and trained, space and time information of complex channel signals is extracted, shape parameters are obtained according to sparse I / Q data, an ultrasonic image is reconstructed through a delay summation (DAS) beam former according to the estimated shape of the flexible probe, and therefore real-time high-quality imaging of the flexible probe is achieved. According to the method, the guidance data which is more lightweight and contains abundant shape information can be generated, so that the model complexity is reduced, the real-time performance is enhanced, the shape estimation accuracy is remarkably improved, and a real-time, high-quality and accurate shape estimation effect can be realized.

Description

Technical Field

[0001] The present invention relates to a technology in the field of ultrasonic imaging, specifically a flexible ultrasonic probe imaging method based on real-time, high-precision shape estimation, in which the mean square error of the array element is less than 1 / 8 wavelength, the structural similarity exceeds 0.80, and the shape estimation time is only 0.12s. Background Art

[0002] Existing flexible ultrasound probes are prone to misfocusing and distortion during imaging because their shapes change with the surface of the human body. Therefore, accurately obtaining the relative positions of the flexible probe array elements is crucial to achieving high-quality ultrasound imaging. Although some shape estimation methods based on external sensors can directly measure the positions of the array elements, the additional sensing equipment will increase the volume of the probe, increase the cost, and make the imaging system more complex. The iterative shape estimation method estimates the shape of the flexible probe by optimizing the image clarity through multiple iterations. Although no external equipment is required, the shape estimation time is long and the robustness is poor due to the need for multiple iterations. Although the deep learning-based method shortens the shape estimation time, its low shape prediction accuracy limits the practical application of flexible probes. Summary of the invention

[0003] In view of the defects of the prior art that additional displacement monitoring equipment is required, resulting in complex overall structure, long calculation time, low accuracy and poor robustness in the flexible probe shape estimation method, the present invention proposes a real-time high-quality imaging method for a flexible ultrasonic probe. By performing analytical signal extraction and sparse processing on the original RF signal, lighter guidance data containing rich shape information is generated, thereby reducing the model complexity and enhancing the real-time performance. By introducing channel attention and spatial attention mechanisms, the neural network can more accurately focus on the area of ​​interest affected by the probe shape, improve the accuracy of shape estimation, and thus achieve real-time, high-quality and accurate shape estimation effect.

[0004] The present invention is achieved through the following technical solutions:

[0005] The present invention relates to a real-time high-quality imaging method using a flexible ultrasonic probe, comprising:

[0006] Step 1: After demodulating the RF data collected by the flexible probe into I / Q data through Hilbert transform and envelope detection, the sparse processing mechanism is applied to compress the dense I / Q data from three dimensions: the transmitting channel, the receiving channel, and the sampling time, and remove unimportant channel signals and sampling points, thereby reducing the number of model parameters and calculation time.

[0007] Step 2: Construct and train a spatially aware shape estimation network to perform end-to-end mapping between sparse I / Q data and flexible array shapes, fully extract the spatial and temporal information of complex channel signals, and obtain shape parameters based on the sparse I / Q data.

[0008] Step 3: According to the estimated flexible probe shape, the ultrasound image is reconstructed by a delay-sum (DAS) beamformer, thereby achieving real-time high-quality imaging of the flexible probe. Technical Effects

[0009] The present invention converts the original RF signal into a sparse I / Q signal through analytical signal extraction and the designed sparsification processing module to provide guidance data for the shape estimation network; and uses a spatial perception shape network containing channel attention and spatial attention to achieve fast and accurate shape estimation from sparse I / Q signals to the shape of the flexible probe. Compared with the prior art, the present invention does not require additional measurement equipment and repeated iterative estimation. The shape estimation time on the CPU is only 0.12 seconds, and the mean square error of the array element position is less than 1 / 8 wavelength, achieving real-time high-quality imaging of the flexible ultrasound probe. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 It is a flow chart of the present invention;

[0011] Figure 2 This is a schematic diagram of a flexible ultrasound probe model;

[0012] Figure 3 Schematic diagram for analyzing signal extraction;

[0013] Figure 4 Schematic diagram of sparse processing;

[0014] Figure 5 The structure diagram of the spatially aware shape estimation network;

[0015] Figure 6 This is the structural diagram of the spatial perception module;

[0016] Figure 7 It is a schematic diagram of the effect of the embodiment;

[0017] In the figure: (a)-(f) are the predicted probe shapes and reconstructed ultrasound images of anechoic cysts, hyperechoic cysts, hypoechoic cysts, blood vessels, and muscle tissues, respectively. DETAILED DESCRIPTION

[0018] like Figure 1 As shown, this embodiment relates to a method for real-time high-quality imaging using a flexible ultrasonic probe, comprising:

[0019] Step 1: construct a flexible ultrasound probe model, use synthetic aperture imaging technology to collect RF signals and generate RF data.

[0020] The synthetic aperture imaging means that only one array element is excited each time, and all corresponding array elements receive echo signals, and the synthesis of the complete aperture is achieved by sequentially exciting all transmitting array elements.

[0021] like Figure 2 As shown in the figure, the flexible ultrasound probe model for this embodiment is calculated under a given specific curvature, specifically: the coordinates of the nth array element (x n ,z n )for: x n =R·sinθ n , z n =R·(1-cosθ n ), where: θ n is the direction angle between the array element and the coordinate axis, x n and z n are the horizontal and vertical coordinates of the array element respectively.

[0022] The obtained RF data includes: transmitting array element, receiving array element and sampling time.

[0023] In this embodiment, the size of the original RF data is 128×128×2048.

[0024] Step 2: Figure 3 As shown, the Hilbert transform and envelope detection are used to extract the analytical signal from the RF data, and the demodulated I / Q signal is used as input to estimate the shape of the flexible array, including:

[0025] 2.1 Perform Hilbert transform on the RF signal to obtain the analytical signal y i，j (k) = x i，j (k)+jH{x i，j (k)}, i, j = 1, 2, ..., N, where: y i，j (k) is the analytical signal, is the Hilbert transform operation, x i，j (k) is the RF signal transmitted by the i-th array element and received by the j-th array element, and k is the sampling time series.

[0026] 2.2 Extract the envelope of the analytical signal and obtain the demodulated I / Q signal , i, j = 1, 2, ..., N, where: z i，j(k) is the I / Q signal obtained by demodulation. The I / Q signal contains the shallow features of the image and reduces the influence of the carrier signal. It can more directly reflect the changes in the imaging target caused by the probe shape and provide more accurate guidance for the shape estimation task.

[0027] Step 3: Figure 4 As shown, based on the sparse processing mechanism, the dense I / Q signal obtained in step 2 is compressed into sparse I / Q data from three dimensions: the transmitting channel, the receiving channel, and the sampling time to extract key channel features, including:

[0028] Dimension 1: select one of every 8 channels as a transmit channel to eliminate redundant information of adjacent transmit elements and reduce the number of transmit channels from 128 to 32.

[0029] Dimension 2: Among the 32 consecutive receiving channels centered on the transmitting channel, the receiving signals with weak signals and low signal-to-noise ratio are removed.

[0030] Dimension 3: retain sampling points consistent with the depth of the imaging area: for simulated targets, select signals between 401 and 1424 in the sampling sequence, which is equivalent to a depth of approximately 10-30 mm; for in-body targets, select signals between 1 and 1024 in the sampling sequence, which is equivalent to a depth of approximately 0-20 mm.

[0031] In this embodiment, the size of the demodulated I / Q signal is reduced from 128×128×2048 to 32×32×1024 sparse I / Q data.

[0032] Step 4: Figure 5 As shown, a spatially aware shape estimation network is constructed and trained to learn the mapping from sparse I / Q data to the shape of the flexible probe, and then the corresponding flexible probe shape parameters, i.e., the convex function curvature of the flexible probe, are generated according to the sparse I / Q data obtained in step 3.

[0033] The spatial perception shape estimation network includes: multiple dense blocks, spatial perception modules and transition layers, wherein: the dense blocks perform feature extraction and fusion processing based on the output features of the previous layer to obtain richer feature representations, the spatial perception modules adjust the channel weights of the feature maps based on the features output by the dense blocks, adaptively select important channel features, and integrate the morphological channel information generated by the dense blocks, and the transition layers perform dimensionality reduction operations based on the features output by the spatial perception modules to ensure the information flow between dense blocks and improve computational efficiency.

[0034] The dense blocks include multiple 1×1 and 3×3 convolutional layers with the same padding to improve information flow and parameter efficiency.

[0035] The transition layer includes a 1×1 convolution layer and an average pooling layer with a stride of 2, wherein: the convolution and pooling operations are used for the receiving channel and sampling time dimension of sparse I / Q data to ensure smooth information flow and efficient calculation.

[0036] To prevent overfitting and build a lightweight network, this method sets the growth rate to 16 and the block configuration to (6, 12, 24, 16).

[0037] like Figure 5 and Figure 6 As shown, the spatial perception module includes: a channel attention module and a spatial attention module, wherein: the channel attention module utilizes the characteristic relationship between the I / Q signals of different transmission channels, and adaptively selects key channel features by adjusting the weights of different feature maps in the channel domain; the spatial attention module utilizes the spatial characteristic relationship between the I / Q signals of each transmission channel, and adaptively learns important spatial features by emphasizing or suppressing the features of different spatial positions.

[0038] like Figure 5 and Figure 6 As shown in the figure, the channel attention module assigns adaptive weights to different features by compressing and expanding feature channels to generate a channel attention map U'∈R H×W×C After that, the output feature map U'=U*M is obtained by multiplying channel by channel C (U), this module specifically includes: adaptive average pooling layer, linear layer and sigmoid layer, which uses compression and excitation operations to aggregate the morphological channel information in the feature map generated by the dense block.

[0039] The compression mentioned above means: for the input feature U∈R H×W×C , a global average pooling layer is used to compress each spatial domain H×W. The compressed output of the cth element

[0040] The excitation is to capture the dependency between channels using a gating mechanism with two nonlinear fully connected layers with dimensions C / 16 and C. Channel attention map M c =F ex (z, W) = σ(g(z, W)) = σ(W2σ(W1z)), where: z is the compressed output of all elements, σ ​​is the sigmoid function, δ is the ReLU function, W1 and W2 are the parameters of the C / 16 and C layers respectively.

[0041] like Figure 5 and Figure 6As shown in FIG. 1 , the spatial attention module applies average pooling and maximum pooling operations along the channel axis respectively, and then connects to generate a spatial feature descriptor. The module specifically includes: an average pooling layer and a maximum pooling layer connected in parallel, a convolutional layer and a Sigmoid layer, wherein: the average pooling layer and the maximum pooling layer respectively aggregate the morphological spatial information in the feature map generated by the channel attention module and generate a map respectively. and Finally, these graphs are connected into a spatial attention graph Where: σ is the sigmoid function, f 7×7 It is a convolution operation with a convolution kernel of 7×7, and the output feature map U”∈R H×W×C The output feature U"=U'*M is obtained by multiplying point by point in space S (U').

[0042] The spatial perception shape estimation network is trained using the mean square error of the probe shape parameters as the loss function, specifically: Where: M is the number of shape parameters of the flexible probe, p i is the i-th true shape parameter, is the i-th predicted shape parameter.

[0043] Step 5: Reconstruct the ultrasound image from the RF data based on the convex function curvature of the flexible probe through the DAS beamformer to achieve real-time high-quality imaging of the flexible probe. Specifically: Where: x and z represent the horizontal and vertical coordinates of the focus, T is the number of transmitting array elements, R is the number of receiving array elements, t i,j is the propagation time between the transmission of the i-th array element and the reception of the j-th array element (x,z),(x i ,z i ) and (x j ,z j ) represent the coordinates of the focus, transmitting element, and receiving element respectively.

[0044] After specific practical experiments, shape estimation tests were conducted on simulation data sets and on-body data sets: In FieldⅡ, the simulation data set was obtained through a flexible ultrasound probe with a center frequency of 5MHz, 128 array elements, and an array element spacing of 0.3mm, collecting RF signals and shape parameters. The characteristics of the flexible probe were simulated by setting a specific delay for the transmitting array element of the linear probe. A data set of 1000 anechoic cysts, hypoechoic cysts, hyperechoic cysts, and blood vessels was obtained.

[0045] In the Verasonics system (Vantage128, Verasonics, Redmond, WA, USA), the in-body data set was obtained by using a L11-5v ultrasound probe with a center frequency of 5 MHz, 128 array elements, and an array element spacing of 0.3 mm. The experimental configuration was consistent with the simulation experiment. After obtaining 600 in-body data sets containing RF data and shape parameters, the training set, validation set, and test set were divided into 80% / 10% / 10%.

[0046] The original RF data collected by the flexible ultrasonic probe in this embodiment has a size of 128×128×2048. After analytical signal extraction and sparse processing, its size is reduced to 32×32×1024. The sparse I / Q data is normalized to the range of [0,1].

[0047] Through this method, we can obtain Figure 7 The results shown further evaluate the performance of the shape estimation method from three aspects: the accuracy of the flexible probe shape estimation, the quality of the reconstructed image, and the complexity of the shape estimation. Specifically,

[0048] a) The element position mean square error (EPMAE) is used to measure the accuracy of the flexible probe shape between the true shape and the estimated shape, specifically: Where: (x i ,z i )and are the real and estimated lateral and depth coordinates of the i-th array element of the flexible probe, respectively, and N is the number of array elements.

[0049] b) Structural similarity (SSIM) and peak signal-to-noise ratio (PSNR) are used to evaluate the quality of reconstructed images. SSIM evaluates the perceptual similarity between images reconstructed using the true and estimated shapes; PSNR reflects the pixel-level similarity between the two.

[0050] c) Shape estimation time is used to measure the computational complexity of the shape estimation method.

[0051] This embodiment further compares the existing iteration-based method and deep learning-based method. Comparative experiments were conducted on simulated data sets and in vivo data sets, and the quantitative results are summarized in Tables 1 and 2.

[0052] Table 1

[0053] Table 2

[0054] By comparing with the method based on deep learning proposed by minimizing image entropy, maximizing spatial coherence coefficient, and Noda et al., it can be found that the shape estimation performance of this method is better than the other three methods in both simulated and in vivo datasets. Specifically, on the simulated dataset, the values ​​of EPMAE, SSIM, and PSNR are 0.09±0.08, 0.88±0.08, and 28.81±3.61, respectively; on the in vivo dataset, the values ​​of EPMAE, SSIM, and PSNR are 0.11±0.07, 0.80±0.09, and 25.50±3.58, respectively.

[0055] Experimental results show that this method not only performs well in shape estimation accuracy, but also has very high quality of ultrasound images reconstructed by predicting the probe shape, thanks to the rich shape information in the demodulated I / Q signal and the effective extraction of shape features by the spatially aware shape estimation network. In addition, compared with the shape estimation time of more than 30 seconds based on the iterative method, the shape estimation time of this method on the CPU is only 0.12 seconds, ensuring that the flexible probe can be imaged in real time with high quality.

[0056] like Figure 7 As shown in the figure, the flexible probe shape samples obtained by different shape estimation methods and their corresponding ultrasound image reconstruction results are shown in turn. From top to bottom, anechoic cysts, hyperechoic cysts, hypoechoic cysts, blood vessels and muscles are shown in turn. The areas with obvious curvature changes are highlighted and enlarged in the figure to more clearly show the differences between shape estimates. It can be intuitively seen that the proposed method is closest to the true shape in probe shape estimation. The accuracy of the method based on spatial coherence is slightly worse, while the estimation results of the method based on image entropy and Noda et al. are significantly different from the true shape. The correct flexible probe shape can generate high-quality ultrasound images, while the wrong shape will cause the image to be unfocused and distorted. As shown in the figure, the image reconstructed by the predicted shape is visually compared with the image reconstructed by the real and estimated shape through the structural dissimilarity map (DisMap). The darker the blue, the lower the dissimilarity between the predicted image and the reference image. The geometric reconstructed images estimated by other methods have different degrees of deviation in contrast and resolution, resulting in distortion and blur, which is not conducive to clinical diagnosis. In contrast, the image generated by the probe shape estimated by this method is closest to the true image.

[0057] In summary, through the analysis of quantitative and qualitative results of multiple simulations and carrier experiments, it can be seen that the flexible probe imaging method based on real-time and high-accuracy shape estimation in the present invention has achieved excellent performance in various indicators. This shows that the method can achieve fast and accurate shape estimation, enabling the flexible probe to achieve real-time and high-quality ultrasound imaging, showing its potential in clinical applications.

[0058] The above-mentioned specific implementation can be partially adjusted in different ways by those skilled in the art without departing from the principle and purpose of the present invention. The protection scope of the present invention shall be based on the claims and shall not be limited by the above-mentioned specific implementation. Each implementation scheme within its scope shall be subject to the constraints of the present invention.

Claims

1. A method for real-time high-quality imaging using a flexible ultrasonic probe, characterized in that: After the RF data collected by the flexible probe is demodulated into I / Q data through Hilbert transform and envelope detection, the dense I / Q data is compressed from three dimensions of transmit channel, receive channel and sampling time by applying the sparse processing mechanism. Then, the spatial and temporal information of the complex channel signal is extracted by constructing and training a space-aware shape estimation network, and the shape parameters are obtained according to the sparse I / Q data. Then, according to the estimated flexible probe shape, the ultrasound image is reconstructed through a delayed-sum-and-sum (DAS) beamformer, thereby achieving real-time high-quality imaging of the flexible probe.

2. The method for real-time high-quality imaging with a flexible ultrasonic probe according to claim 1, wherein: include: Step 1: construct a flexible ultrasound probe model, use synthetic aperture imaging technology to collect RF signals and generate RF data; Step 2: Use Hilbert transform and envelope detection to extract the analytical signal from the RF data and use the demodulated I / Q signal as input to estimate the shape of the flexible array, including: 2.1 Perform Hilbert transform on the RF signal to obtain the analytical signal y i，j (k) = x i，j (k)+jH{x i，j (k)), i, j = 1, 2, ..., N, where: y i，j (k) is the analytical signal, is the Hilbert transform operation, x i，j (k) is the RF signal transmitted by the i-th array element and received by the j-th array element, and k is the sampling time series; 2.2 Extract the envelope of the analytical signal and obtain the demodulated I / Q signal Where: z i，j (k) is the I / Q signal obtained by demodulation; Step 3: Based on the sparse processing mechanism, the dense I / Q signal obtained in step 2 is compressed into sparse I / Q data from three dimensions: the transmitting channel, the receiving channel, and the sampling time to extract key channel features; Step 4: construct and train a spatially aware shape estimation network to learn the mapping from sparse I / Q data to the shape of the flexible probe, and then generate corresponding flexible probe shape parameters, i.e., the convex function curvature of the flexible probe, according to the sparse I / Q data obtained in step 3; Step 5: Through the DAS beamformer, the ultrasound image is reconstructed from the RF data based on the convex function curvature of the flexible probe, so as to achieve real-time high-quality imaging of the flexible probe.

3. The method for real-time high-quality imaging using a flexible ultrasonic probe according to claim 2, characterized in that: In the step 1, synthetic aperture imaging means: only one array element is excited each time, and all corresponding array elements receive echo signals, and the synthesis of the complete aperture is achieved by sequentially exciting all transmitting array elements; The flexible ultrasound probe model calculates the position of the flexible array element under a given specific curvature, specifically: the coordinates (x n ,z n )for: x n =R·sinθ n , z n =R·(1-cosθ n ), where: θ n is the direction angle between the array element and the coordinate axis, x n and z n are the horizontal and vertical coordinates of the array element respectively.

4. The method for real-time high-quality imaging using a flexible ultrasonic probe according to claim 2, characterized in that: The three dimensions include: Dimension 1: select one of every eight channels as a transmit channel to eliminate the redundant information of adjacent transmit elements and reduce the number of transmit channels from 128 to 32; Dimension 2: Remove the received signals with weak signals and low signal-to-noise ratio from the 32 consecutive receiving channels centered on the transmitting channel; Dimension 3: retain sampling points consistent with the depth of the imaging area: for simulated targets, select signals between 401 and 1424 in the sampling sequence, which is equivalent to a depth of 10-30 mm; for in-body targets, select signals between 1 and 1024 in the sampling sequence, which is equivalent to a depth of 0-20 mm.

5. The method for real-time high-quality imaging using a flexible ultrasonic probe according to claim 2, characterized in that: The spatial perception shape estimation network comprises: a plurality of dense blocks, a spatial perception module and a transition layer, wherein: the dense block performs feature extraction and fusion processing according to the output features of the previous layer, thereby obtaining a richer feature representation; the spatial perception module adjusts the channel weight of the feature map according to the features output by the dense block, adaptively selects important channel features, and integrates the morphological channel information generated by the dense block; the transition layer performs a dimensionality reduction operation according to the features output by the spatial perception module, thereby ensuring the information flow between the dense blocks and improving the computational efficiency; The spatial perception shape estimation network is trained using the mean square error of the probe shape parameters as the loss function, specifically: Where: M is the number of shape parameters of the flexible probe, p i is the i-th true shape parameter, is the i-th predicted shape parameter.

6. The method for real-time high-quality imaging using a flexible ultrasonic probe according to claim 5, characterized in that: The spatial perception module includes: a channel attention module and a spatial attention module, wherein: the channel attention module utilizes the characteristic relationship between the I / Q signals of different transmission channels, and adaptively selects key channel features by adjusting the weights of different feature maps in the channel domain; the spatial attention module utilizes the spatial feature relationship between the I / Q signals of each transmission channel, and adaptively learns important spatial features by emphasizing or suppressing the features of different spatial positions.

7. The method for real-time high-quality imaging using a flexible ultrasonic probe according to claim 6, characterized in that: The channel attention module generates a channel attention map U'∈R by compressing and expanding feature channels to assign adaptive weights to different features. H ×W×C After that, the output feature map U′=U*M is obtained by multiplying channel by channel C (U), this module specifically includes: adaptive average pooling layer, linear layer and sigmoid layer, which uses compression and excitation operations to aggregate the morphological channel information in the feature map generated by the dense block.

8. The method for real-time high-quality imaging using a flexible ultrasonic probe according to claim 7, characterized in that: The compression mentioned above means: for the input feature U∈R H×W×C , a global average pooling layer is used to compress each spatial domain H×W, and the compressed output of the cth element is The excitation is to capture the dependencies between channels using a gating mechanism with two nonlinear fully connected layers with dimensions C / 16 and C respectively. The channel attention map M c =F ex (z, W) = σ(g(z, W)) = σ(W2δ(W1z)), where: z is the compressed output of all elements, σ ​​is the sigmoid function, δ is the ReLU function, W1 and W2 are the parameters of the C / 16 and C layers respectively.

9. The method for real-time high-quality imaging using a flexible ultrasonic probe according to claim 6, characterized in that: The spatial attention module, after applying average pooling and maximum pooling operations respectively along the channel axis, connects to generate spatial feature descriptors. The module specifically includes: parallel average pooling layer and maximum pooling layer, convolution layer and Sigmoid, wherein: the average pooling layer and the maximum pooling layer respectively aggregate the morphological spatial information in the feature map generated by the channel attention module and generate a map respectively. and Finally, these graphs are connected into a spatial attention graph Where: σ is the sigmoid function, f 7 ×7 It is a convolution operation with a convolution kernel of 7×7, and the output feature map U″∈R H×W×C The output feature U″=U′*M is obtained by multiplying point by point in space S (U′).

10. The method for real-time high-quality imaging using a flexible ultrasonic probe according to claim 6, characterized in that: The reconstructed ultrasound image is specifically: each focus Where: x and z represent the horizontal and vertical coordinates of the focus, T is the number of transmitting array elements, R is the number of receiving array elements, t i，j is the propagation time between the transmission of the i-th array element and the reception of the j-th array element (x,z), (x i , z i ) and (x j , z j ) represent the coordinates of the focus, transmitting element, and receiving element respectively.

Images