Ultrasonic sound velocity image reconstruction method and system based on deep learning

Abstract

The present invention discloses an ultrasonic sound velocity image reconstruction method and system based on deep learning, the method comprising: obtaining ultrasonic original channel data; inputting the ultrasonic original channel data into a trained sound velocity image reconstruction model to generate a sound velocity map, the model comprising an encoder and a decoder, the encoder comprising a preliminary feature extraction module, a GCSA module and a subsequent feature processing module, the preliminary feature extraction module is used to extract the original feature map; the GCSA module integrates channel attention, channel shuffling and spatial attention mechanisms, the GCSA module processes the received original feature map to generate an optimized feature map; the subsequent feature processing module is used to further process the optimized feature map to generate an output feature map of the encoder; the decoder receives the output feature map of the encoder, converts the output feature map of the encoder into a sound velocity matrix, thereby generating a sound velocity map. The method of the present invention improves the expression ability of features, so that the generated sound velocity map has higher accuracy.

Description

Technical Field

[0001] The present invention belongs to the field of image reconstruction, and in particular relates to an ultrasonic sound velocity image reconstruction method and system based on deep learning. Background Art

[0002] Ultrasound imaging is an important technology in medical diagnosis and is widely used in the detection and diagnosis of various diseases. It generates image information by emitting high-frequency sound waves and receiving reflected waves. In clinical practice, traditional ultrasound images (such as B-mode images) can only provide limited tissue structure information, and it is difficult to accurately evaluate the elasticity and sound velocity distribution of tissues, which affects the early diagnosis and treatment of diseases. To overcome this limitation, sound velocity imaging has become an important research direction in the field of ultrasound imaging. Sound velocity imaging can provide more information about tissue characteristics, and ultrasound propagation helps to reveal the mechanical properties of tissues and lesion areas. As a quantitative imaging biomarker, speed of sound (SoS) is very sensitive to structural changes caused by pathology, so it has important diagnostic value and carries important information about tissue characteristics. Studies have shown that longitudinal wave imaging has similar diagnostic capabilities to shear wave imaging, and it can provide important information for evaluating elasticity and hardness. The compression wave (longitudinal wave) velocity in tissue is usually determined by measuring the sound wave transmission time between known relative positions. For example, in tumor detection, tumor tissue usually appears to be harder than normal tissue, and longitudinal waves propagate faster, making longitudinal waves more potential in tumor detection. Therefore, combining sound velocity imaging with longitudinal wave propagation characteristics can not only more comprehensively evaluate the mechanical properties of tissues, but also provide more accurate information for early diagnosis and treatment of diseases.

[0003] The acoustic properties of tissues, including sound velocity information, are already reflected in raw ultrasound radio frequency (RF) data, but analysis methods that accurately extract this information are usually accompanied by high computational costs. In addition, sound velocity imaging is highly dependent on the precise measurement of sound wave propagation time, while the resolution of tiny structures is limited by the physical boundaries of existing imaging technology. Different tissues also perform significantly differently in sound velocity imaging. For example, for soft tissues, accurate measurements become particularly challenging due to the small differences in sound velocity.

[0004] At present, the reconstruction technology of ultrasonic sound velocity distribution map of traditional methods is usually based on the sound velocity inversion model, which generates an accurate sound velocity distribution map by extracting sound velocity information from reflected ultrasound data. In 1982, DE Robinson et al. proposed a method to determine the propagation speed of ultrasound in tissue by processing pulse echo data obtained by composite scanning. In 1998, ME Anderson et al. proposed a method for directly estimating the sound velocity in the medium. At the end of this method, the estimator obtains the sound velocity by analyzing the pulse echo data received by a single transducer array after a single transmission. In 2011, Treeby et al. proposed a focusing method for automatic selection of sound velocity after studying different focus functions. In 2020, P Stähli et al. proposed a method for inverse calculation using a Bayesian framework. The SoS image under this method is more stable and has better resistance to phase noise.

[0005] In recent years, deep learning has made significant progress in many fields and has been widely used in traditional fields such as computer vision, such as using deep learning for MRI, CT and ultrasound image reconstruction. For example, in the field of ultrasound imaging, deep learning has been widely used in ultrasound beamforming technology. With the development of deep learning technology, neural network-based sound velocity reconstruction methods have made significant progress. These methods can not only handle complex nonlinear relationships, but also automatically learn deep features extracted from input data, thereby improving the accuracy and detail of the reconstruction results. Micha et al. proposed an alternative method for single-sided pressure wave sound velocity measurement based on channel data, and used a fully convolutional deep neural network to achieve single-sided sound velocity inversion. Khun Jush F et al. studied a DNN-based sound velocity reconstruction method and applied it to channel data collected by a single plane wave for automatic breast ultrasound. Shi M et al. introduced a sound velocity estimation and aberration correction method in a dual-modality PA / US imaging system based on a deep learning framework, which was further enhanced by using physical model data and related sound velocity maps obtained from measurements for transfer learning.

[0006] However, in existing sound velocity reconstruction, the correlation information between channels is often ignored, which may cause the loss of global information and thus affect the model's effective use of information in the feature map. Summary of the invention

[0007] In view of this, the purpose of the present invention is to provide an improved model, which integrates channel attention, channel shuffling and spatial attention mechanisms, aiming to enhance the expressive power of the input feature map by capturing the global dependencies in the feature map.

[0008] The objective of the present invention is achieved through the following technical solutions:

[0009] A deep learning-based ultrasonic sound velocity image reconstruction method, comprising:

[0010] Obtaining ultrasound raw channel data;

[0011] The original ultrasound channel data is input into the trained sound velocity image reconstruction model to generate a sound velocity map, wherein the sound velocity image reconstruction model includes an encoder and a decoder.

[0012] The encoder includes a preliminary feature extraction module, a GCSA module, and a subsequent feature processing module in the direction of the signal flow.

[0013] The preliminary feature extraction module is used to extract the original feature map;

[0014] The GCSA module integrates channel attention, channel shuffling and spatial attention mechanisms. The GCSA module processes the received original feature map and generates an optimized feature map;

[0015] A subsequent feature processing module is used to further process the optimized feature map to generate an output feature map of the encoder;

[0016] The decoder receives the output feature map of the encoder, converts the output feature map of the encoder into a sound speed matrix, and thus generates a sound speed map.

[0017] The present invention also provides an ultrasonic sound velocity image reconstruction system based on deep learning, comprising:

[0018] A data acquisition engine, used to acquire ultrasound raw channel data;

[0019] Model building and training engine, used to build and train the speed of sound image reconstruction model for training;

[0020] The image reconstruction engine is used to generate a sound speed map based on the original ultrasound channel data and the sound speed image reconstruction model, wherein the sound speed image reconstruction model includes an encoder and a decoder.

[0021] The encoder includes a preliminary feature extraction module, a GCSA module, and a subsequent feature processing module in the direction of the signal flow.

[0022] The preliminary feature extraction module is used to extract the original feature map;

[0023] The GCSA module integrates channel attention, channel shuffling and spatial attention mechanisms. The GCSA module processes the received original feature map and generates an optimized feature map;

[0024] A subsequent feature processing module is used to further process the optimized feature map to generate an output feature map of the encoder;

[0025] The decoder receives the output feature map of the encoder, converts the output feature map of the encoder into a sound speed matrix, and thus generates a sound speed map.

[0026] Furthermore, the preliminary feature extraction module includes a first encoding unit, a second encoding unit, a third encoding unit, a fourth encoding unit and a fifth encoding unit in sequence according to the direction of the signal flow.

[0027] in,

[0028] The first encoding unit includes a convolutional layer and a BN layer in sequence according to the direction of the signal flow;

[0029] The second coding unit, the third coding unit and the fourth coding unit have the same structure as the first coding unit, and the convolution layers in the first coding unit to the fourth coding unit adopt strided convolution;

[0030] The fifth encoding unit includes a convolution layer, a maximum pooling layer, a Dropout layer, and a BN layer in sequence according to the direction of the signal flow;

[0031] The subsequent feature processing module includes a sixth encoding unit and a seventh encoding unit, and the sixth encoding unit includes a convolution layer, a maximum pooling layer and a BN layer;

[0032] The seventh coding unit includes a convolution layer, a maximum pooling layer, a Dropout layer and a BN layer.

[0033] Furthermore, the decoder includes a first decoding unit, a second decoding unit, a third decoding unit, a fourth decoding unit, a fifth decoding unit, a sixth decoding unit and a seventh decoding unit in sequence according to the direction of the signal flow, wherein:

[0034] The first decoding unit includes a convolution layer, an upsampling layer and a BN layer in sequence according to the direction of the signal flow;

[0035] The second decoding unit, the third decoding unit and the fourth decoding unit have the same structure as the first decoding unit.

[0036] The output of the convolution layer of the fourth encoding unit is connected with the result after being processed by the convolution layer, the upsampling layer and the BN layer in the third decoding unit;

[0037] The output of the maximum pooling layer of the fifth encoding unit is connected with the result after being processed by the convolution layer, the upsampling layer and the BN layer in the second decoding unit;

[0038] The output of the maximum pooling layer of the sixth encoding unit is connected with the result after being processed by the convolution layer, the upsampling layer and the BN layer in the first decoding unit;

[0039] The fifth decoding unit includes a convolution layer, a resizing layer and a BN layer in sequence according to the direction of the signal flow;

[0040] The sixth decoding unit includes a convolutional layer and a BN layer in sequence according to the direction of the signal flow;

[0041] The seventh decoding unit includes a convolutional layer.

[0042] Furthermore, the GCSA module includes a channel attention submodule and a spatial attention submodule.

[0043] The channel attention submodule receives the preliminary feature map to generate a channel attention map; the channel attention submodule also multiplies the channel attention map with the preliminary feature to obtain an enhanced feature map, and performs channel shuffling on the enhanced feature map to generate a shuffled feature map;

[0044] The spatial attention submodule receives the shuffled feature map and generates a spatial attention map;

[0045] The GCSA module also multiplies the shuffled feature map with the spatial attention map to generate an optimized feature map.

[0046] Furthermore, the channel attention submodule receives the preliminary feature map to generate a channel attention map, which specifically includes:

[0047] S11: receiving a preliminary feature map;

[0048] S12: perform dimension permutation on the preliminary feature map, moving the channel dimension to the last dimension;

[0049] S13: The feature map after dimension permutation is input into a multilayer perceptron. The first layer of the multilayer perceptron reduces the number of channels to one-fourth of the initial number of channels, and then performs a nonlinear transformation through a ReLU activation function. The second layer of the multilayer perceptron restores the number of channels to the initial channel dimension.

[0050] S14: perform dimension permutation again to restore the shape of the feature map generated by S13 to the same shape as the preliminary feature map;

[0051] S15: Pass the result of S14 through the Sigmoid function to generate a channel attention map.

[0052] Furthermore, the spatial attention submodule receives the shuffled feature map and generates a spatial attention map, which specifically includes:

[0053] S21: receiving the shuffled feature map;

[0054] S22: performing a convolution operation on the shuffled feature map so that the number of channels of the feature map generated after the convolution becomes one quarter of the shuffled feature map;

[0055] S23: The feature map generated by S22 is subjected to nonlinear transformation through batch normalization and ReLU activation function;

[0056] S24: Perform a convolution operation on the result of S23 again, so that the number of channels of the feature map generated by the convolution is restored to the initial channel dimension;

[0057] S25: The result of S24 is batch normalized and the Sigmoid activation function is used to generate a spatial attention map.

[0058] The beneficial effects of the present invention are:

[0059] The present invention adds a GCSA module to the model, which integrates channel attention, channel shuffling and spatial attention mechanisms. By introducing the channel attention mechanism and channel shuffling operation, the dependency between channels is fully considered, so that the information in the feature map is fully utilized; then the spatial attention mechanism is introduced to fully capture the spatial information in the local and global features in the image, that is, considering more details in the feature map, and improving the accuracy of sound speed estimation.

[0060] Other advantages, objectives and features of the present invention will be described in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the following examination and study, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be further described in detail below with reference to the accompanying drawings, in which:

[0062] Figure 1 is a schematic diagram of the sound velocity image reconstruction model;

[0063] Figure 2 It is a schematic diagram of the channel attention submodule and the channel shuffle operation;

[0064] Figure 3 is a schematic diagram of the spatial attention submodule;

[0065] Figure 4 These are the test results of different models on simulated data sets;

[0066] Figure 5 These are the experimental results on the CIRS 040GSE phantom;

[0067] Figure 6 These are experimental results on isolated pig tissue with tubular structures. DETAILED DESCRIPTION

[0068] The preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the preferred embodiments are only for illustrating the present invention, rather than for limiting the protection scope of the present invention.

[0069] The present invention proposes a sound velocity image reconstruction method based on deep learning, which is used for sound velocity image reconstruction of ultrasonic radio frequency (RF) data. The method uses a large amount of simulated data as the training input of a deep neural network, and generates a high-precision sound velocity distribution map by fully learning the sound velocity distribution characteristics, thereby obtaining a sound velocity image reconstruction model through training. Through this model, the received ultrasound channel RF data can be converted into a sound velocity matrix to generate a sound velocity distribution map (i.e., a sound velocity map).

[0070] Specifically, a deep learning-based ultrasonic sound velocity image reconstruction method may include:

[0071] Obtaining ultrasound raw channel data;

[0072] The original ultrasound channel data is input into the trained sound velocity image reconstruction model to generate a sound velocity map. The sound velocity image reconstruction model is constructed based on a convolutional neural network. Figure 1 is a schematic diagram of the sound velocity image reconstruction model, such as Figure 1 As shown in the figure, the sound velocity image reconstruction model includes an encoder and a decoder for feature extraction and reconstruction. That is, the raw data of the ultrasound channel is converted into sound velocity values ​​through the encoder and the decoder. The input of the network is the original ultrasound channel data, and the output is a 384 × 384 sound velocity matrix.

[0073] The encoder includes a preliminary feature extraction module, a GCSA module, and a subsequent feature processing module in the direction of the signal flow.

[0074] The preliminary feature extraction module is used to extract the original feature map;

[0075] The GCSA module integrates channel attention, channel shuffling and spatial attention mechanisms. The GCSA module processes the received original feature map and generates an optimized feature map;

[0076] A subsequent feature processing module is used to further process the optimized feature map to generate an output feature map of the encoder;

[0077] The decoder receives the output feature map of the encoder, converts the output feature map of the encoder into a sound speed matrix, and thus generates a sound speed map.

[0078] The present invention also provides an ultrasonic sound velocity image reconstruction system based on deep learning, comprising:

[0079] A data acquisition engine, used to acquire ultrasound raw channel data;

[0080] Model building and training engine, used to build and train the speed of sound image reconstruction model for training;

[0081] The image reconstruction engine is used to generate a sound speed map based on the original ultrasound channel data and the sound speed image reconstruction model, wherein the sound speed image reconstruction model includes an encoder and a decoder.

[0082] The encoder includes a preliminary feature extraction module, a GCSA module, and a subsequent feature processing module in the direction of the signal flow.

[0083] The preliminary feature extraction module is used to extract the original feature map;

[0084] The GCSA module, namely the Global Channel-Spatial Attention Module (GCSA), integrates channel attention, channel shuffling and spatial attention mechanisms. The GCSA module processes the received original feature map to generate an optimized feature map.

[0085] A subsequent feature processing module is used to further process the optimized feature map to generate an output feature map of the encoder;

[0086] The decoder receives the output feature map of the encoder, converts the output feature map of the encoder into a sound speed matrix, and thus generates a sound speed map.

[0087] like Figure 1 As shown, the preliminary feature extraction module includes a first encoding unit, a second encoding unit, a third encoding unit, a fourth encoding unit and a fifth encoding unit in sequence according to the direction of the signal flow.

[0088] in,

[0089] The first encoding unit includes a convolution layer and a batch normalization (BN) layer in sequence according to the direction of the signal flow;

[0090] The second coding unit, the third coding unit, and the fourth coding unit have the same structure as the first coding unit; wherein the convolution layers in the first to fourth coding units all use strided convolution. Specifically, the stride of the strided convolution can be (1,2), wherein the stride "1" in the first dimension indicates that the stride of the convolution kernel in the vertical direction is 1; the stride "2" in the second dimension indicates that the stride of the convolution kernel in the horizontal direction is 2. In the convolution layers of these four coding units, the stride in the horizontal direction is greater than 1, which means that the convolution kernel will skip one pixel each time for convolution. In addition, in these four coding units, the number of data channels has not changed and remains at the initial value (for example, it can be 32). The size of the convolution kernel decreases from the first coding unit to the fourth coding unit.

[0091] The fifth encoding unit includes a convolution layer, a maximum pooling layer, a Dropout layer (i.e., a random discarding layer), and a BN layer (i.e., a batch normalization layer) in the direction of the signal flow.

[0092] The subsequent feature processing module is located after the GCSA module, which includes the sixth coding unit and the seventh coding unit in the direction of the signal flow. The sixth coding unit includes a convolution layer, a maximum pooling layer and a BN layer; the seventh coding unit includes a convolution layer, a maximum pooling layer, a Dropout layer and a BN layer.

[0093] The decoder includes a first decoding unit, a second decoding unit, a third decoding unit, a fourth decoding unit, a fifth decoding unit, a sixth decoding unit and a seventh decoding unit in sequence according to the direction of the signal flow, wherein:

[0094] The first decoding unit includes a convolution layer, an upsampling layer and a BN layer in sequence according to the direction of the signal flow;

[0095] The second decoding unit, the third decoding unit and the fourth decoding unit have the same structure as the first decoding unit.

[0096] The fifth decoding unit includes a convolution layer, a resizing layer and a BN layer in sequence according to the direction of the signal flow;

[0097] The sixth decoding unit includes a convolutional layer and a BN layer in sequence according to the direction of the signal flow.

[0098] The seventh decoding unit is a 1×1 convolutional layer.

[0099] In order to realize the connection between the encoding and decoding paths, a connection (i.e. Concatenate) operation is added here. The specific connection operations can include the following:

[0100] The output of the convolution layer of the fourth encoding unit is connected with the result after being processed by the convolution layer, the upsampling layer and the BN layer in the third decoding unit;

[0101] The output of the maximum pooling layer of the fifth encoding unit is connected with the result after being processed by the convolution layer, the upsampling layer and the BN layer in the second decoding unit;

[0102] The output of the maximum pooling layer of the sixth encoding unit is connected to the result after being processed by the convolution layer, the upsampling layer and the BN layer in the first decoding unit.

[0103] Through the above connection operations, high-level and low-level spatial information can be combined to reduce information loss and improve the training efficiency of the model.

[0104] In the image reconstruction process, if the correlation information between channels is ignored, it may lead to the loss of global information, which in turn affects the effective use of all information in the feature map by the model, and ultimately leads to insufficient capture of global features. In order to solve this problem, the present invention designs a new GCSA module. The module integrates channel attention, channel shuffling and spatial attention mechanisms, aiming to enhance the expressiveness of the input feature map by capturing the global dependencies in the feature map. The GCSA module includes a channel attention submodule and a spatial attention submodule.

[0105] The channel attention submodule can receive the preliminary feature map to generate a channel attention map.

[0106] Figure 2 This is a schematic diagram of the channel attention submodule and channel shuffling operation, as shown in Figure 2 As shown, the channel attention submodule receives the preliminary feature map to generate a channel attention map, which specifically includes:

[0107] S11: Receive preliminary feature map (i.e., as Figure 2 ); the input feature map in );

[0108] S12: Perform dimension permutation on the preliminary feature map, from the original Transformed to , moving the channel dimension to the last dimension, where C represents the original number of channels of the input feature map; H represents the image height of the input feature map; W represents the image width of the input feature map;

[0109] S13: The feature map after dimension permutation is input into a multi-layer perceptron (MLP) to capture the relationship between channels. Specifically, the first layer of the multi-layer perceptron reduces the number of channels to one-fourth of the initial number of channels (i.e., C becomes C / 4), and then performs nonlinear transformation through the ReLU activation function. The second layer of the multi-layer perceptron restores the number of channels to the initial channel dimension (i.e., C / 4 is restored to C). Through MLP, the redundant information between channels can be effectively reduced and important features can be highlighted. In this way, the global dependency between channels can be better captured;

[0110] S14: Dimension permutation is performed again (i.e., inverse permutation, becomes ), restore the shape of the feature map generated by S13 to the same shape as the preliminary feature map;

[0111] S15: Pass the result of S14 through the Sigmoid function to generate a channel attention map.

[0112] Afterwards, the generated channel attention map is multiplied with the feature map of the initial input to obtain the enhanced feature map.

[0113] The above process can be expressed by formula (1): , (1)

[0114] in, is the enhanced feature map, is the input feature map, represents the dimension permutation operation, is the Sigmoid activation function, MLP is the multi-layer perceptron, Represents matrix multiplication.

[0115] If the information between channels is not fully mixed, the expressiveness of the features may be limited, and the effect of channel attention cannot be fully exerted. To further mix and share information, the present invention performs a channel shuffle operation on the enhanced feature map. Specifically, the enhanced feature map is divided into four groups, each containing Then, the grouped feature maps are transposed to disrupt the channel information. Finally, the disrupted feature maps are restored to their original shape. In this way, the information can be better mixed, thus enhancing the feature expression ability.

[0116] The channel shuffle operation can be expressed by formula (2):

[0117] , (2)

[0118] in, is the shuffled feature map, is the enhanced feature map, Represents a channel shuffle operation.

[0119] Spatial information is also crucial for capturing local and global features in images. However, spatial information may not be fully utilized by only channel attention and channel shuffling operations. To this end, the present invention also introduces a spatial attention submodule to fully consider the details in the feature map. The specific process is as follows Figure 3 exhibit.

[0120] like Figure 3 As shown, the spatial attention submodule receives the shuffled feature map and generates a spatial attention map, which may specifically include:

[0121] S21: receiving the shuffled feature map;

[0122] S22: Perform a convolution operation on the shuffled feature map (for example, input a 7×7 convolution layer) so that the number of channels of the feature map generated after the convolution becomes one quarter of the shuffled feature map;

[0123] S23: The feature map generated by S22 is subjected to nonlinear transformation through batch normalization and ReLU activation function;

[0124] S24: Perform another convolution operation on the result of S23 (another 7×7 convolution layer) so that the number of channels of the feature map generated by the convolution is restored to the initial channel dimension C;

[0125] S25: The result of S24 is batch normalized and the Sigmoid activation function is used to generate a spatial attention map.

[0126] After the channel attention submodule and channel shuffling, the present invention inputs the obtained shuffled feature map into a 7×7 convolutional layer, which reduces the number of channels to one-fourth of the original. Then, it is nonlinearly transformed through batch normalization and ReLU activation function. Then, another 7×7 convolutional layer restores the number of channels to the original dimension. , and then undergo a layer of batch normalization, and generate a spatial attention map through the Sigmoid activation function. The above process can be expressed by formula (3):

[0127] , (3)

[0128] in, It is the graph after spatial attention, that is, the spatial attention graph.

[0129] Finally, the shuffled feature map is multiplied with the spatial attention map after the spatial attention submodule to obtain the final output feature map. The above process can be expressed by formula (4):

[0130] , (4)

[0131] in, It is the feature map output by the final encoder.

[0132] The present invention uses three common evaluation indicators to quantitatively evaluate the reconstruction results: Root Mean Square Error (RMSE), Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index (SSIM). Among them, RMSE is used as the main evaluation indicator to measure the error between the reconstruction result and the true value; PSNR and SSIM are used as auxiliary indicators to further evaluate the reconstruction quality from the two dimensions of signal-to-noise ratio and structural similarity.

[0133] The present invention mainly uses the root mean square error (RMSE) as the main indicator for evaluating the accuracy of the sound velocity distribution map generated in deep learning. RMSE measures the degree of deviation between the predicted value and the true value, and is obtained by calculating the square root of the sum of the squares of the differences between the predicted value and the true value. Its value range is from 0 to positive infinity. The smaller the RMSE, the smaller the deviation between the model predicted value and the true value, and the higher the prediction accuracy. The definition of RMSE can be expressed by formula (5):

[0134] , (5)

[0135] in, is the number of experimental samples, is the true value, is the predicted value.

[0136] Peak Signal-to-Noise Ratio (PSNR) is one of the indicators commonly used for image quality assessment. It is used to measure the signal-to-noise ratio between the reconstructed image and the original image, reflecting the quality of the image. In this study, PSNR is used as an auxiliary evaluation indicator to supplement the evaluation of image reconstruction quality. The higher the PSNR value, the smaller the difference between the reconstructed image and the real image, that is, the better the reconstruction quality. The definition of PSNR can be expressed by formula (6):

[0137] , (6)

[0138] in, is the maximum value of the image pixels, is the mean square error, that is The square of .

[0139] Structural Similarity Index (SSIM) is the second auxiliary evaluation index, which is mainly used to evaluate the structural similarity of two images. Unlike PSNR, SSIM comprehensively considers the contrast, brightness and structural information of the image, emphasizes the degree of preservation of image details, and can more accurately reflect the similarity of images. The value range of SSIM is between The closer the value is to 1, the more similar the structures of the two images are. In the process of ultrasonic sound velocity distribution map reconstruction, SSIM can better capture local details and structural information. Therefore, as an auxiliary indicator, it can supplement the structural information that RMSE and SSIM fail to reflect. The definition of SSIM can be expressed by formula (7):

[0140] , (7)

[0141] in, and represents the mean of the original image x and the reconstructed image y, and represents the variance of the original image and the reconstructed image, represents their covariance, and is a constant to avoid the denominator being 0.

[0142] The model in the present invention is trained on 5000 simulated samples, with a training / test split ratio of 0.8, using a stochastic gradient descent optimizer (SGD) with a minimum batch size of 10, setting the momentum to 0.9, the learning rate to 0.00001, and using the mean squared error (MSE) as the loss function. The model is trained for 100 epochs, and the average root mean square error (RMSE) on the training set eventually converges to 20.7377 m / s, and the RMSE on the validation set eventually converges to 25.5701 m / s. In contrast, the RMSE on the training set of the unimproved model (i.e., the model without the GCSA module, only the baseline) converges to 22.5991 m / s on the training set and 27.2553 m / s on the validation set under the same training configuration.

[0143] This study tested the trained model on three different datasets, including simulated data, phantom data, and a 3D scan sequence consisting of 2D imaging slices of a human wrist. The test results are shown in Table 1. The results show that the improved model shows higher performance in most test tasks, further verifying its generalization ability and robustness under different data distributions. The specific results are shown in Table 1.

[0144] Table 1 Model RMSE, PSNR, SSIM performance

[0145]

[0146] The experimental results on the simulated dataset are as follows Figure 4 shown.

[0147] like Figure 4As shown in the figure, ten (corresponding to the ten image columns (a)-(j)) different randomly generated foreign body simulation data are used. Each simulation data has a corresponding ultrasonic delay and sum (DAS) imaging, a real foreign body sound velocity distribution map, a sound velocity distribution map generated by the model before improvement (i.e., the baseline model), and a sound velocity distribution map of the improved model (i.e., the baseline model + GCSA). Figure 4 It can be observed that the sound velocity distribution diagram generated by the basic model in some cases has obvious errors. Figure 4 The images corresponding to column (b) and Figure 4 Taking the image corresponding to column (d) in the figure as an example, the improved model is more accurate in distinguishing different foreign objects. In the case of overlapping foreign objects, the sound velocity distribution map generated by the improved model can effectively distinguish each foreign object instead of blurring them as a whole. The improved model has also been improved in removing artifacts. For example, in Figure 4 In the image corresponding to the (j) column, the basic model generates an artifact that does not exist in the real sound velocity map, but this problem does not exist in the improved model. In contrast, the improved model can more accurately capture the details of the image reconstruction process by enhancing the relationship between channels.

[0148] In the experiment of the present invention, the Vevo F2 LAZR-X device was used for imaging and data acquisition of real data. The Vevo F2 imaging system is a high-resolution, real-time ultrasound device that is mainly used for diagnostic imaging and fluid flow analysis of small animals. It is suitable for laboratory environments and was developed by VisualSonics. The device supports exporting experimental data into multiple formats for subsequent analysis. The phantom and ex vivo porcine tissue experimental data in this experiment were collected using the B-Mode mode of the Vevo LAZR-X device to obtain ultrasound images and ultrasound channel radio frequency data. The probe used in the experiment is UHF29x, with an operating frequency range of 15.0 MHz to 29.0 MHz and a center frequency of 17 MHz. In order to evaluate the accuracy of the sound velocity distribution, this experiment was verified by comparing the structural similarity between the B-Mode image and the sound velocity distribution map, and observing the numerical changes in the sound velocity distribution map. The phantom experiment and the ex vivo animal tissue experiment were carried out respectively.

[0149] (1) Phantom experiment: In this experiment, the multifunctional multi-tissue ultrasonic performance detection phantom provided by CIRS, model 040GSE, was selected. This phantom is the only quality control phantom on the market that provides both elastic targets and all standard B-type imaging test objects. The phantom is made of CIRS's proprietary Zerdine® hydrogel polymer, which can mimic the acoustic properties of human soft tissue. Its dimensions are 17.8cm×12.7cm×20.3cm (length×width×height). The experimental data set comes from an area with a depth of 1.5cm in the phantom, where the background sound velocity value is 1540m / s. The experimental results are as follows: Figure 5 As shown, Figure 5 The first layer of images in the middle represents the situation where there is only one detection target; Figure 5 The second image in the middle shows the situation with two elastic targets. Figure 5 The image corresponding to column (a) shows the B-Mode image generated using the Vevo F2 device. Figure 5 The image in column (b) is the sound speed distribution prediction map generated by the neural network before improvement. Figure 5 The image corresponding to column (c) is the sound velocity distribution prediction map generated by the improved model. It can be seen from the B-Mode image that the target is detected at a depth of 1.5 cm, and the model can accurately detect and distinguish the overall shape of the target from the background.

[0150] The B-Mode image of the second layer shows the imaging in the elastic area at a depth of 1.5 cm, including two elastic targets with different elastic values, 60 kPa and 100 kPa respectively. The target with an elastic value of 60 kPa cannot be detected in the unmodified model. In this experiment, the improved model successfully identified the difference between the two targets and the background sound velocity. The experimentally measured background sound velocity is 1530 m / s, which is close to the theoretical value; the target sound velocity with an elastic value of 60 kPa is 1405 m / s, and the target sound velocity with an elastic value of 100 kPa is 1449 m / s.

[0151] (2) In vitro animal tissue experiment: This experiment selected in vitro pig tissue with tubular structure. In the experiment, a piece of pork with a thickness of 2 cm was selected as the experimental medium, and a tubular structure with a diameter of about 1 cm was embedded inside it as a simulated detection target. Figure 6 As shown, Figure 6 Figure (a) is the B-Mode image of the structure. Figure 6 Figure (b) is the sound velocity distribution prediction diagram generated by the unimproved model. Figure 6 Figure (c) in FIG. 1 is a sound velocity distribution prediction diagram generated by the model proposed in the present invention. It can be observed from the experimental results that the proposed model can accurately detect the overall shape of the target tubular structure (such as Figure 6The model can effectively distinguish the difference between the tubular structure and pork tissue. Further analysis shows that the sound velocity of pork predicted by the model ranges from 1500 m / s to 1580 m / s, which is basically consistent with the theoretical sound velocity of 1540 m / s, verifying the reliability of the model.

[0152] In summary, the present invention adjusts the simulated data set used for training, and designs a new "global channel-spatial attention" module based on the existing neural network to strengthen the information relationship between channels, thereby effectively mining more feature map information. The network architecture shows encouraging performance on simulated data, phantom data, and ex vivo pig tissue data, demonstrating the great potential of the network in ultrasonic sound velocity reconstruction.

[0153] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution, which should be included in the scope of the claims of the present invention.

Claims

1. A method for reconstructing ultrasonic sound velocity images based on deep learning, characterized in that: include: Obtaining ultrasound raw channel data; Inputting the original ultrasound channel data into the trained sound velocity image reconstruction model to generate a sound velocity map, wherein the sound velocity image reconstruction model includes an encoder and a decoder, and the encoder includes a preliminary feature extraction module, a GCSA module and a subsequent feature processing module in sequence according to the direction of the signal flow, and the preliminary feature extraction module is used to extract the original feature map; The GCSA module integrates channel attention, channel shuffling and spatial attention mechanisms. The GCSA module processes the received original feature map and generates an optimized feature map; A subsequent feature processing module is used to further process the optimized feature map to generate an output feature map of the encoder; The decoder receives the output feature map of the encoder, converts the output feature map of the encoder into a sound velocity matrix, and thus generates a sound velocity map; The GCSA module includes a channel attention submodule and a spatial attention submodule. The channel attention submodule receives the preliminary feature map to generate a channel attention map. The channel attention submodule also multiplies the channel attention map with the preliminary feature to obtain an enhanced feature map, and performs channel shuffling on the enhanced feature map to generate a shuffled feature map. The spatial attention submodule receives the shuffled feature map and generates a spatial attention map; The GCSA module also multiplies the shuffled feature map with the spatial attention map to generate an optimized feature map; The channel attention submodule receives the preliminary feature map to generate a channel attention map, which includes: S11: receiving a preliminary feature map; S12: perform dimension permutation on the preliminary feature map, moving the channel dimension to the last dimension; S13: The feature map after dimension permutation is input into a multilayer perceptron. The first layer of the multilayer perceptron reduces the number of channels to one-fourth of the initial number of channels, and then performs a nonlinear transformation through a ReLU activation function. The second layer of the multilayer perceptron restores the number of channels to the initial channel dimension. S14: perform dimension permutation again to restore the shape of the feature map generated by S13 to the same shape as the preliminary feature map; S15: Pass the result of S14 through the Sigmoid function to generate a channel attention map; The spatial attention submodule receives the shuffled feature map and generates a spatial attention map, which includes: S21: receiving the shuffled feature map; S22: performing a convolution operation on the shuffled feature map so that the number of channels of the feature map generated after the convolution becomes one quarter of the shuffled feature map; S23: The feature map generated by S22 is subjected to nonlinear transformation through batch normalization and ReLU activation function; S24: Perform a convolution operation on the result of S23 again, so that the number of channels of the feature map generated by the convolution is restored to the initial channel dimension; S25: The result of S24 is batch normalized and the Sigmoid activation function is used to generate a spatial attention map.

2. The method for reconstructing ultrasonic sound velocity images based on deep learning according to claim 1, characterized in that: The preliminary feature extraction module includes a first coding unit, a second coding unit, a third coding unit, a fourth coding unit and a fifth coding unit in sequence according to the direction of the signal flow, wherein the first coding unit includes a convolutional layer and a BN layer in sequence according to the direction of the signal flow; The second coding unit, the third coding unit and the fourth coding unit have the same structure as the first coding unit, and the convolution layers in the first coding unit to the fourth coding unit adopt strided convolution; The fifth encoding unit includes a convolution layer, a maximum pooling layer, a Dropout layer, and a BN layer in sequence according to the direction of the signal flow; The subsequent feature processing module includes a sixth encoding unit and a seventh encoding unit, and the sixth encoding unit includes a convolution layer, a maximum pooling layer and a BN layer; The seventh coding unit includes a convolution layer, a maximum pooling layer, a Dropout layer and a BN layer.

3. The method for reconstructing ultrasonic sound velocity images based on deep learning according to claim 2, characterized in that: The decoder includes a first decoding unit, a second decoding unit, a third decoding unit, a fourth decoding unit, a fifth decoding unit, a sixth decoding unit and a seventh decoding unit in sequence according to the direction of the signal flow, wherein the first decoding unit includes a convolution layer, an upsampling layer and a BN layer in sequence according to the direction of the signal flow; The second decoding unit, the third decoding unit and the fourth decoding unit have the same structure as the first decoding unit, and the output of the convolution layer of the fourth decoding unit is connected with the result after being processed by the convolution layer, the upsampling layer and the BN layer in the third decoding unit; The output of the maximum pooling layer of the fifth encoding unit is connected with the result after being processed by the convolution layer, the upsampling layer and the BN layer in the second decoding unit; The output of the maximum pooling layer of the sixth encoding unit is connected with the result after being processed by the convolution layer, the upsampling layer and the BN layer in the first decoding unit; The fifth decoding unit includes a convolution layer, a resizing layer and a BN layer in sequence according to the direction of the signal flow; The sixth decoding unit includes a convolutional layer and a BN layer in sequence according to the direction of the signal flow; The seventh decoding unit includes a convolutional layer.

4. An ultrasonic sound velocity image reconstruction system based on deep learning, characterized in that: include: A data acquisition engine, used to acquire ultrasound raw channel data; Model building and training engine, used to build and train the speed of sound image reconstruction model for training; An image reconstruction engine is used to generate a sound velocity map according to the original ultrasound channel data and a sound velocity image reconstruction model, wherein the sound velocity image reconstruction model includes an encoder and a decoder, and the encoder includes a preliminary feature extraction module, a GCSA module and a subsequent feature processing module in sequence according to the direction of the signal flow, and the preliminary feature extraction module is used to extract the original feature map; The GCSA module integrates channel attention, channel shuffling and spatial attention mechanisms. The GCSA module processes the received original feature map and generates an optimized feature map; A subsequent feature processing module is used to further process the optimized feature map to generate an output feature map of the encoder; The decoder receives the output feature map of the encoder, converts the output feature map of the encoder into a sound velocity matrix, and thus generates a sound velocity map; The GCSA module includes a channel attention submodule and a spatial attention submodule. The channel attention submodule receives the preliminary feature map to generate a channel attention map. The channel attention submodule also multiplies the channel attention map with the preliminary feature to obtain an enhanced feature map, and performs channel shuffling on the enhanced feature map to generate a shuffled feature map. The spatial attention submodule receives the shuffled feature map and generates a spatial attention map; The channel attention submodule receives the preliminary feature map to generate a channel attention map, which includes: S11: receiving a preliminary feature map; S12: perform dimension permutation on the preliminary feature map, moving the channel dimension to the last dimension; S13: The feature map after dimension permutation is input into a multilayer perceptron. The first layer of the multilayer perceptron reduces the number of channels to one-fourth of the initial number of channels, and then performs a nonlinear transformation through a ReLU activation function. The second layer of the multilayer perceptron restores the number of channels to the initial channel dimension. S14: perform dimension permutation again to restore the shape of the feature map generated by S13 to the same shape as the preliminary feature map; S15: Pass the result of S14 through the Sigmoid function to generate a channel attention map; The spatial attention submodule receives the shuffled feature map and generates a spatial attention map, which includes: S21: receiving the shuffled feature map; S22: performing a convolution operation on the shuffled feature map so that the number of channels of the feature map generated after the convolution becomes one quarter of the shuffled feature map; S23: The feature map generated by S22 is subjected to nonlinear transformation through batch normalization and ReLU activation function; S24: Perform a convolution operation on the result of S23 again, so that the number of channels of the feature map generated by the convolution is restored to the initial channel dimension; S25: The result of S24 is batch normalized and the Sigmoid activation function is used to generate a spatial attention map.

Images