Photoacoustic image reconstruction model training method and device, equipment and medium

Abstract

The invention provides a photoacoustic image reconstruction model training method and device, equipment and a medium. Comprises: obtaining a model training sample; inputting the model training sample into a to-be-trained photoacoustic image reconstruction model; calling a discrete transformation layer to carry out discrete two-dimensional wavelet transformation on the sparse photoacoustic image to obtain a low-frequency feature map and a high-frequency feature map; calling a high-frequency channel attention layer to perform feature capture on the high-frequency feature map in the vertical direction and the horizontal direction to obtain a high-frequency attention feature map; calling a low-frequency channel attention layer to process the low-frequency feature map to obtain a low-frequency attention feature map; performing image reconstruction on the high-frequency attention feature map, the low-frequency attention feature map and the sparse photoacoustic image to obtain a reconstructed image; calculating a loss value of a to-be-trained photoacoustic image reconstruction model based on the reconstructed image and the dense sampling photoacoustic image serving as a true value; and obtaining a final photoacoustic image reconstruction model under the condition that the loss value is within a preset range.

Description

Technical Field

[0001] The present application relates to the technical field of photoacoustic image reconstruction, and in particular to a photoacoustic image reconstruction model training method, device, equipment and medium. Background Art

[0002] Photoacoustic imaging (PAI) is a non-invasive biomedical imaging technique that combines the high contrast of optical imaging with the deep penetration capability of ultrasound imaging. In PAI, biological tissue generates thermoelastic acoustic waves under the irradiation of laser pulses, which are received by the ultrasound probe and converted into images. Although PAI provides high contrast and resolution, the quality of photoacoustic tomography (PAT) images greatly depends on the sampling density. In order to obtain the best image quality, high sampling rates and acquisition of ultrasound signal scans from multiple angles are required. However, due to equipment and time limitations, sparse sampling is often the only option, which introduces artifacts and blurring in the reconstructed image. Although these problems can be alleviated by increasing the number of sensors or expanding the sampling angle, such solutions will increase system complexity and cost. Therefore, reducing artifacts and improving image quality under sparse sampling conditions remains a key issue.

[0003] In recent years, deep learning has been widely used in medical image processing and reconstruction, such as automatic feature extraction and enhancing the reconstruction quality of sparsely sampled data. These models can learn complex image features and patterns from large amounts of data, effectively improving the shortcomings of traditional algorithms in image reconstruction.

[0004] However, deep learning models, especially recent advanced models, require a lot of computing resources and significant processing time when processing large-scale data, which is not practical in resource-constrained situations (such as portable devices or real-time applications). At the same time, with the continuous development of deep learning, generative models, especially generative adversarial networks (GAN), have become very popular in many fields. They have been used to simulate the reconstruction process of photoacoustic imaging data and effectively improve the visual quality of images. However, in the field of medical imaging, the ability to accurately restore biological structures is crucial, and generative models sometimes sacrifice this accuracy to improve visual effects, which may affect the accuracy and reliability of diagnosis. Summary of the invention

[0005] The technical problem to be solved by the embodiments of the present application is to provide a photoacoustic image reconstruction model training method, device, equipment and medium to effectively improve the accuracy of photoacoustic image reconstruction and increase the feasibility of model deployment in resource-constrained scenarios.

[0006] In a first aspect, an embodiment of the present application provides a photoacoustic image reconstruction model training method, the method comprising:

[0007] Acquire a model training sample, wherein the model training sample includes: a sparse photoacoustic image and a densely sampled photoacoustic image corresponding to the sparse photoacoustic image;

[0008] Inputting the model training samples into a photoacoustic image reconstruction model to be trained, wherein the photoacoustic image reconstruction model to be trained comprises: a discrete transformation layer, a high-frequency channel attention layer, and a low-frequency channel attention layer;

[0009] Calling the discrete transformation layer to perform discrete two-dimensional wavelet transformation on the sparse photoacoustic image to obtain a low-frequency feature map and a high-frequency feature map of the sparse photoacoustic image;

[0010] Calling the high-frequency channel attention layer to capture features of the high-frequency feature map in vertical and horizontal directions to obtain a high-frequency attention feature map;

[0011] Calling the low-frequency channel attention layer to process the low-frequency feature map to obtain a low-frequency attention feature map;

[0012] Performing image reconstruction on the high-frequency attention feature map, the low-frequency attention feature map, and the sparse photoacoustic image to obtain a reconstructed image;

[0013] Based on the reconstructed image and the densely sampled photoacoustic image, calculating a loss value of the photoacoustic image reconstruction model to be trained;

[0014] When the loss value is within a preset range, a final photoacoustic image reconstruction model is obtained.

[0015] In a second aspect, an embodiment of the present application provides a photoacoustic image reconstruction model training device, the device comprising:

[0016] A model sample acquisition module, used to acquire model training samples, wherein the model training samples include: a sparse photoacoustic image and a densely sampled photoacoustic image corresponding to the sparse photoacoustic image;

[0017] A model sample input module, used for inputting the model training sample into a photoacoustic image reconstruction model to be trained, wherein the photoacoustic image reconstruction model to be trained comprises: a discrete transformation layer, a high-frequency channel attention layer and a low-frequency channel attention layer;

[0018] A feature map acquisition module, used for calling the discrete transformation layer to perform discrete two-dimensional wavelet transformation on the sparse photoacoustic image to obtain a low-frequency feature map and a high-frequency feature map of the sparse photoacoustic image;

[0019] A high-frequency feature map acquisition module is used to call the high-frequency channel attention layer to capture features of the high-frequency feature map in the vertical and horizontal directions to obtain a high-frequency attention feature map;

[0020] A low-frequency feature map acquisition module, used to call the low-frequency channel attention layer to process the low-frequency feature map to obtain a low-frequency attention feature map;

[0021] A reconstructed image acquisition module, used to reconstruct the high-frequency attention feature map, the low-frequency attention feature map and the sparse photoacoustic image to obtain a reconstructed image;

[0022] A loss value calculation module, used for calculating the loss value of the photoacoustic image reconstruction model to be trained based on the reconstructed image and the densely sampled photoacoustic image;

[0023] The photoacoustic image reconstruction model acquisition module is used to obtain the final photoacoustic image reconstruction model when the loss value is within a preset range.

[0024] In a third aspect, an embodiment of the present application provides an electronic device, including:

[0025] A processor, a memory, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the photoacoustic image reconstruction model training method described in any one of the above items is implemented.

[0026] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium. When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute any of the above-mentioned photoacoustic image reconstruction model training methods.

[0027] Compared with the prior art, the embodiments of the present application have the following advantages:

[0028] In an embodiment of the present application, by obtaining a model training sample, the model training sample includes: a sparse photoacoustic image and a densely sampled photoacoustic image corresponding to the sparse photoacoustic image. The model training sample is input into the photoacoustic image reconstruction model to be trained, and the photoacoustic image reconstruction model to be trained includes: a discrete transformation layer, a high-frequency channel attention layer, and a low-frequency channel attention layer. The discrete transformation layer is called to perform a discrete two-dimensional wavelet transform on the sparse photoacoustic image to obtain a low-frequency feature map and a high-frequency feature map of the sparse photoacoustic image. The high-frequency channel attention layer is called to capture features in the vertical and horizontal directions of the high-frequency feature map to obtain a high-frequency attention feature map. The low-frequency channel attention layer is called to process the low-frequency feature map to obtain a low-frequency attention feature map. The high-frequency attention feature map, the low-frequency attention feature map, and the sparse photoacoustic image are image reconstructed to obtain a reconstructed image. Based on the reconstructed image and the sparsely sampled photoacoustic image, the loss value of the photoacoustic image reconstruction model to be trained is calculated. When the loss value is within a preset range, the trained photoacoustic image reconstruction model to be trained is used as the final photoacoustic image reconstruction model. The embodiment of the present application is specifically designed to enhance the processing of different frequency subbands in the wavelet domain by deeply integrating wavelet decomposition with a customized attention mechanism. This synergy enables more efficient handling of sparse sampling challenges in PAT while capturing both low-frequency and high-frequency information with lower FLOPs and parameters, making it very suitable for deployment in resource-constrained scenarios. At the same time, the combination of discrete two-dimensional wavelet transform and attention mechanism can effectively improve the accuracy of sparsely sampled photoacoustic image reconstruction. By adopting multi-scale decomposition of the input image, the model effectively captures the global structure and local details, thereby better restoring fine features while maintaining overall structural consistency.

[0029] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 A flowchart of the steps of a photoacoustic image reconstruction model training method provided in an embodiment of the present application;

[0031] Figure 2 A schematic diagram of a high-frequency channel attention layer provided in an embodiment of the present application;

[0032] Figure 3 A schematic diagram of a low-frequency attention module provided in an embodiment of the present application;

[0033] Figure 4 A schematic diagram of the structure of a photoacoustic image reconstruction model training device provided in an embodiment of the present application;

[0034] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0035] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0036] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit the present application. The singular forms "a", "said" and "the" used in the embodiments of the present application and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings.

[0037] Reference Figure 1 , shows a flowchart of the steps of a photoacoustic image reconstruction model training method provided by an embodiment of the present application, such as Figure 1 As shown, the photoacoustic image reconstruction model training method may include: step 101, step 102, step 103, step 104, step 105, step 106, step 107 and step 108.

[0038] Step 101: Obtain model training samples, where the model training samples include: a sparse photoacoustic image and a densely sampled photoacoustic image corresponding to the sparse photoacoustic image.

[0039] In this embodiment, when training the photoacoustic image reconstruction model, a sparse photoacoustic image and a densely sampled photoacoustic image corresponding to the sparse photoacoustic image can be obtained to construct a model training sample, wherein the densely sampled photoacoustic image can be used as the true value of the sparse photoacoustic image for subsequent loss calculation.

[0040] In some examples, sparse photoacoustic images and densely sampled photoacoustic images can be collected through professional datasets and platforms. For example, some research institutions or platforms may release public datasets related to sparse photoacoustic images, which are usually preprocessed and annotated and can be directly used for model training, etc. Or in some scientific research institutions or academic databases, sparse photoacoustic image samples and corresponding densely sampled photoacoustic images contained in relevant research papers and experimental data can be queried.

[0041] In some examples, sparse photoacoustic images and densely sampled photoacoustic images can also be acquired experimentally. For example, by building a photoacoustic imaging system, photoacoustic image data under sparse viewing angles can be acquired. This usually needs to be done in a laboratory environment, requiring professional equipment and experimental skills. During the acquisition process, the parameters of the imaging system, such as the detector scanning angle, the number of detectors, etc., can be adjusted to obtain photoacoustic images of different sparseness.

[0042] It can be understood that the above examples are merely examples listed for a better understanding of the technical solutions of the embodiments of the present application, and are not intended to be the sole limitation to the embodiments.

[0043] After the model training samples are obtained, step 102 is executed.

[0044] Step 102: input the model training samples into the photoacoustic image reconstruction model to be trained, wherein the photoacoustic image reconstruction model to be trained includes: a discrete transformation layer, a high-frequency channel attention layer, and a low-frequency channel attention layer.

[0045] After obtaining the model training samples, the model training samples can be input into the photoacoustic image reconstruction model to be trained. The photoacoustic image reconstruction model to be trained may include: a discrete transformation layer, a high-frequency channel attention layer, and a low-frequency channel attention layer. The processing process of each network layer will be described in detail in the following steps, and this embodiment will not be repeated here.

[0046] Step 103: calling the discrete transformation layer to perform discrete two-dimensional wavelet transformation on the sparse photoacoustic image to obtain a low-frequency feature map and a high-frequency feature map of the sparse photoacoustic image.

[0047] After the model training samples are input into the photoacoustic image reconstruction model to be trained, the discrete transformation layer can be called to perform a discrete two-dimensional wavelet transform on the sparse photoacoustic image to obtain a low-frequency feature map and a high-frequency feature map of the sparse photoacoustic image. Specifically, the discrete transformation layer can be called to perform a discrete two-dimensional wavelet transform on the sparse photoacoustic image to obtain a low-frequency component feature map, a horizontal high-frequency component feature map, a vertical high-frequency component feature map, and a diagonal high-frequency component feature map of the sparse photoacoustic image. The low-frequency component feature map is used as the low-frequency feature map, and the horizontal high-frequency component feature map, the vertical high-frequency component feature map, and the diagonal high-frequency component feature map are used as the high-frequency feature map.

[0048] In practical applications, when processing sparse photoacoustic images, the use of discrete two-dimensional wavelet transform (2D Discrete Wavelet Transform, 2D DWT) can decompose the image into different frequency components, which is very useful for feature extraction and analysis. In this embodiment, in the discrete wavelet transform (DWT), the sparse photoacoustic image is decomposed into four sub-bands (each sub-band is equivalent to a feature map), and the four sub-bands are: LL (low frequency), LH (horizontal high frequency), HL (vertical high frequency) and HH (diagonal high frequency). The LL (low frequency) sub-band contains most of the energy and main information of the image, and usually represents the smooth and low-frequency part of the image. The LH (horizontal high frequency) sub-band captures the high-frequency details of the image in the horizontal direction, such as edges and textures, which helps to detect horizontal edges and details in the image, and is very important for edge detection and feature extraction of the image. The HL (vertical high frequency) sub-band captures the high-frequency details of the image in the vertical direction, which helps to detect edges and details in the image, but is mainly concentrated in the vertical direction. The HH (diagonal high frequency) subband captures the high-frequency details of the image in the diagonal direction, such as diagonal edges and complex textures. The HH subband is equally important for image detail analysis and feature extraction, especially in situations where diagonal edges or complex textures need to be detected.

[0049] In the embodiment of the present application, the input sparse photoacoustic image is first decomposed into four sub-bands through DWT, so that the image shows different features in different frequency bands, which facilitates subsequent feature extraction and processing.

[0050] After the discrete transformation layer is called to perform discrete two-dimensional wavelet transformation on the sparse photoacoustic image to obtain the low-frequency feature map and the high-frequency feature map of the sparse photoacoustic image, step 104 and step 105 are executed.

[0051] Step 104: Call the high-frequency channel attention layer to capture features of the high-frequency feature map in the vertical and horizontal directions to obtain a high-frequency attention feature map.

[0052] After obtaining the high-frequency feature map, the high-frequency channel attention layer can be called to capture the features of the high-frequency feature map in the vertical direction and the horizontal direction to obtain the high-frequency attention feature map. The implementation process can be described in detail in combination with the following specific implementation method.

[0053] In a specific implementation of the present application, the high-frequency channel attention layer may include: a first convolution normalization activation layer, an enhanced axial attention mechanism layer, and a second convolution normalization activation layer. The above step 104 may include:

[0054] Sub-step A1: calling the first convolution normalization activation layer to perform convolution, normalization and activation processing on the high-frequency feature map to obtain a feature map of the high-frequency feature map.

[0055] In this embodiment, the high-frequency channel attention layer may include: a first convolution normalization activation layer, an enhanced axial attention mechanism layer, and a second convolution normalization activation layer. Figure 2 As shown, the high-frequency channel attention layer is Figure 2 The HF-SA shown in Figure 1 has the first convolutional normalized activation layer as Figure 2 The Conv-BN-Tanh on the upper side shown, the second convolutional normalization activation layer is Figure 2 The Conv-BN-Tanh on the lower side shown in the figure has an enhanced axial attention mechanism layer, which is Enhanced CCA.

[0056] After obtaining the high-frequency feature map, the first convolution normalization activation layer can be called to perform convolution, normalization and activation processing on the high-frequency feature map to obtain the feature map of the high-frequency feature map. Figure 2 As shown in the figure, a 3×3 convolutional layer can be used to extract local and global features in the image. Then, BatchNorm (batch normalization) is used to normalize the output of the convolutional layer to accelerate network training and improve model performance. Batch normalization helps to accelerate network training, improve model performance, and reduce the risk of overfitting. Finally, Tanh (hyperbolic tangent activation function) can be used to process the output of batch normalization to obtain a feature map of the high-frequency feature map. Specifically, the hyperbolic tangent activation function is used to introduce nonlinear factors so that the network can handle complex nonlinear relationships.

[0057] After calling the first convolution normalization activation layer to perform convolution, normalization and activation processing on the high-frequency feature map to obtain the feature map of the high-frequency feature map, sub-step A2 is executed.

[0058] Sub-step A2: Call the enhanced axial attention mechanism layer to perform cross-attention processing on the feature map to obtain an enhanced axial attention feature map.

[0059] After obtaining the feature map of the high-frequency feature map, the enhanced axial attention mechanism layer can be called to perform enhanced axial attention processing on the feature map to obtain an enhanced axial attention feature map. The implementation process can be described in detail in conjunction with the following specific implementation method.

[0060] In another specific implementation of the present application, the above sub-step A2 may include:

[0061] Sub-step B1: calling the enhanced axial attention mechanism layer to perform convolution processing on the feature map to obtain a convolution feature map corresponding to the feature map.

[0062] In this embodiment, after obtaining the feature map, the enhanced axial attention mechanism layer can be called to perform convolution processing on the feature map to obtain a convolution feature map corresponding to the feature map. Figure 2 As shown in the right half, the feature vector x of the input feature map can be processed by a 1×1 convolution layer to obtain the convolution feature, and the output is a convolution feature map.

[0063] Sub-step B2: Perform linear transformation on the convolution feature map to obtain a linear transformation feature map.

[0064] Then, the convolution feature map can be linearly transformed to obtain a linearly transformed feature map. Figure 2 As shown in the figure, after obtaining the convolution feature map, the convolution feature map can be linearly transformed by using Norm (normalization), that is, the data in the convolution feature map is adjusted to a specific range or distribution, and the output is the linear transformation feature map.

[0065] Sub-step B3: performing convolution processing on the linear transformation feature map to obtain a Query feature map, a Key feature map and a Value feature map corresponding to the linear transformation feature map.

[0066] After obtaining the linear transformation feature map, the linear transformation feature map can be convolved to obtain the query feature map, key feature map and value feature map corresponding to the linear transformation feature map. Figure 2 As shown in the figure, the linear transformation feature map can be processed by three 1×1Conv (i.e., ordinary convolution layers) and three 3×3Dconv (i.e., depth-wise separable convolution layers) respectively, so that the three features of Query, Key, and Value can be obtained, and the output is the Query feature map, the Key feature map, and the Value feature map.

[0067] In this embodiment, for the high-frequency sub-band, it contains rich edge and texture details, which are determined by local high-frequency changes in different directions. Therefore, the spatial attention mechanism that focuses on extracting the details of the internal sub-band is the best choice. In the above wavelet transform process, the sub-bands generated by the wavelet transform have directional characteristics. Using this feature, the spatial attention mechanism can be adjusted to capture regional correlations on rows and columns, thereby collecting contextual information in the vertical and horizontal directions. This embodiment modifies the basic enhanced axial attention mechanism to better adapt to the processing of high-frequency sub-band inputs in photoacoustic images. Assume that the input is X∈R C×H×W, where C, H, and W represent the number of channels, height, and width, respectively. After performing the basic 1x1 convolution for channel fusion, a 3x3 depth-wise separable convolution is added to ensure that each high-frequency subband can extract local spatial features independently without mixing with the information of other subbands. This design not only preserves the independence of each subband, but also allows fine-grained processing of detailed features within each high-frequency subband. The formulas for the three outputs Q, K, and V are as follows:

[0068]

[0069] After performing convolution processing on the linear transformation feature map to obtain the Query feature map, the Key feature map and the Value feature map corresponding to the linear transformation feature map, sub-step B4 is executed.

[0070] Sub-step B4: Use two convolution kernels of opposite dimensions to perform convolution and feature fusion processing on the Key feature map to obtain a fused Key feature map.

[0071] Sub-step B5: Use two convolution kernels of opposite dimensions to perform convolution and feature fusion processing on the Value feature map to obtain a fused Value feature map.

[0072] After obtaining the Key feature map, two convolution kernels of opposite dimensions can be used to perform convolution and feature fusion processing on the Key feature map to obtain a fused Key feature map. Figure 2 As shown, ASC-K can be used to process the features of the Key feature map to obtain a fused Key feature map.

[0073] After obtaining the Value feature map, two convolution kernels of opposite dimensions can also be used to process the Value feature map to obtain a fused Value feature map. Figure 2 As shown, ASC-V can be used to process the features of the Value feature map to obtain a fused Value feature map.

[0074] ASC-K and ASC-V can use 3×7 and 7×3 convolution kernels to process the Key feature and Value feature respectively to obtain feature extraction. Indicates rearranging the dimensions and then adjusting the dimensions to a new shape. In this embodiment, an axial split convolution block is introduced to expand the receptive field and further enhance the feature extraction in the vertical and horizontal directions. The processed fused Key features and fused Value features can be shown in the following formula:

[0075] K H =Conv 3×7 (K),K W =Conv 7×3 (K)

[0076] V H =Conv 3×7 (V),V W =Conv 7×3 (V)

[0077] Sub-step B6: Process the Query feature map, the fused Key feature map, the fused Value feature map and the convolution feature map to obtain the enhanced axial attention feature map.

[0078] After obtaining the fused Key feature map and the fused Value feature map, the Query feature map, the fused Key feature map, the fused Value feature map and the convolution feature map can be processed to obtain the enhanced axial attention feature map. The improved attention formula can be shown as follows:

[0079]

[0080] In the above formula, It is the final output after applying the attention mechanism to the initial input feature map X. The Softmax function is used to normalize the attention score into a probability distribution. Concat is used to merge features or scores from different paths. γ is used as a parameter to adjust the influence of attention. INF is used to exclude specific elements by assigning low weights during the calculation process.

[0081] Sub-step A3: Call the second convolution normalization activation layer to perform convolution, normalization and activation processing on the enhanced axial attention feature map to obtain the high-frequency attention feature map.

[0082] After obtaining the enhanced axial attention feature map, the second convolution normalization activation layer can be called to perform convolution, normalization and activation processing on the enhanced axial attention feature map to obtain a high-frequency attention feature map. The processing flow of the second convolution normalization activation layer is similar to that of the first convolution normalization activation layer, and this embodiment will not be repeated here.

[0083] Step 105: Call the low-frequency channel attention layer to process the low-frequency feature map to obtain a low-frequency attention feature map.

[0084] After obtaining the low-frequency feature map, the low-frequency channel attention layer can be called to process the low-frequency feature map to obtain a low-frequency attention feature map. Among them, the low-frequency channel attention layer may include: a first low-frequency attention module, a discrete transformation module, a second low-frequency attention module, a high-frequency attention module, an inverse discrete transformation module and a third low-frequency attention module. The processing process of the low-frequency feature map can be described in detail in combination with the following specific implementation method.

[0085] In a specific implementation of the present application, the above step 105 may include:

[0086] Sub-step C1: calling the first low-frequency attention module to process the low-frequency feature map using the MDTA attention mechanism to obtain a first attention feature map.

[0087] In this embodiment, the network structures of the first low-frequency attention module, the second low-frequency attention module and the third low-frequency attention module are the same. The structure of the low-frequency channel attention module can be as follows Figure 3 shown.

[0088] After obtaining the low-frequency feature map, the first low-frequency attention module can be called to process the low-frequency feature map using the MDTA attention mechanism to obtain the first attention feature map. The MDTA (multi-Dconv head transposed attention) mechanism is a multi-head attention mechanism for processing low-frequency feature maps. The processing flow of the first low-frequency attention module can be as follows: Figure 3 As shown, first, the low-frequency feature map can be convolved with a 1×1 convolution kernel to obtain a convolution feature. The 1×1 convolution kernel is used to change the number of channels of the feature map while keeping the spatial resolution unchanged. LayerNorm can perform feature normalization on the convolution features to accelerate training and improve model stability. Then, the MDTA mechanism can be used to capture important information in the normalized feature map. The feature map output by MDTA is then processed again through LayerNorm. Finally, GNFD can be used to further process the feature map, such as denoising and feature enhancement, and the number of channels of the feature map can be changed again through 1×1 convolution to prepare the feature map for use by subsequent layers (such as classifiers).

[0089] The processing of MDTA can be as follows Figure 3 As shown in the lower part of the figure: the input feature representation is processed by 1×1 convolution, then by Norm processing, and then by three 1×1 and 3×3 DConv convolutions to obtain Q, K, and V. Finally, Q, K, and V are processed subsequently to obtain the output of MDTA.

[0090] In the specific implementation, for the sparse image reconstruction task, it is expressed as follows:

[0091] I recon =R(I sparse )-I artifact

[0092] Among them, I recon is the reconstructed clear image, I sparseis the input sparse image, which contains valid structural information and artifacts. Function R() represents the reconstruction algorithm, which processes the sparse data and tries to recover a version close to the original image. artifact Represents structural information and artifacts separated from Isparse.

[0093] The low-frequency subband obtained from the discrete two-dimensional wavelet transform retains the main structure and global information of the image, making it an ideal choice for separating global artifacts from the target structure. To this end, this embodiment introduces a channel attention mechanism MDTA, which analyzes the channel information in Isparse. By enhancing channels containing valid structural information and suppressing channels that may contain artifacts, the feature representation is optimized and the impact of artifacts on the final reconstructed image Irecon is minimized.

[0094] Sub-step C2: calling the discrete transformation module to perform a discrete two-dimensional wavelet transform on the first attention feature map to obtain a sub-low-frequency feature map and a sub-high-frequency feature map.

[0095] After obtaining the first attention feature map, the discrete transformation module can be called to perform a discrete two-dimensional wavelet transform on the first attention feature map to obtain a sub-low-frequency feature map and a sub-high-frequency feature map. That is, a discrete two-dimensional wavelet transform is performed on the output first attention feature map.

[0096] Sub-step C3: calling the second low-frequency attention module to process the sub-low-frequency feature map using the MDTA attention mechanism to obtain a second attention feature map.

[0097] After obtaining the sub-low-frequency feature map, the second low-frequency attention module can be called to process the sub-low-frequency feature map using the MDTA attention mechanism to obtain the second attention feature map. The implementation process is similar to the processing process of the first low-frequency attention module, and this embodiment will not be described in detail here.

[0098] Sub-step C4: Call the high-frequency attention module to process the sub-high-frequency feature map to obtain a third attention feature map.

[0099] After obtaining the sub-high-frequency feature map, the high-frequency attention module can be called to process the sub-high-frequency feature map to obtain a third attention feature map. In this example, the network structure of the high-frequency attention module is the same as the network structure of the high-frequency channel attention layer. The processing process of the sub-high-frequency feature map can refer to the processing process of the high-frequency feature map in the above steps, and this embodiment will not be repeated here.

[0100] Sub-step C5: Call the inverse discrete transformation module to perform an inverse discrete two-dimensional wavelet transform on the second attention feature map and the third attention feature map to obtain a discrete transformation feature map.

[0101] After obtaining the second attention feature map and the third attention feature map, the inverse discrete transformation module can be called to perform inverse discrete two-dimensional wavelet transformation on the second attention feature map and the third attention feature map to obtain a discrete transformation feature map. The inverse discrete two-dimensional wavelet transformation refers to the inverse transformation of the discrete two-dimensional wavelet transformation.

[0102] Sub-step C6: connecting the discrete transformation feature map and the low-frequency feature map by using a jump connection to obtain an updated low-frequency feature map.

[0103] Afterwards, the discrete transformation feature map and the low-frequency feature map can be connected by a skip connection to generate an updated low-frequency feature map.

[0104] Sub-step C7: calling the third low-frequency attention module to process the updated low-frequency feature map using the MDTA attention mechanism to obtain the low-frequency attention feature map.

[0105] Finally, the third low-frequency attention module can be called to process the updated low-frequency feature map using the MDTA attention mechanism to obtain a low-frequency attention feature map.

[0106] It can be understood that the processing flow of the third low-frequency attention module is similar to the processing flow of the above-mentioned first low-frequency attention module, and this embodiment will not be repeated here.

[0107] The embodiment of the present application uses an attention mechanism to focus on extracting internal details of each sub-band, thereby enhancing the local contrast and feature visibility of the image, especially processing edge and texture details in high-frequency sub-bands.

[0108] Step 106: reconstruct the high-frequency attention feature map, the low-frequency attention feature map and the sparse photoacoustic image to obtain a reconstructed image.

[0109] After obtaining the high-frequency attention feature map and the low-frequency attention feature map, the high-frequency attention feature map, the low-frequency attention feature map and the original input sparse photoacoustic image can be reconstructed to obtain a reconstructed image. Specifically, the high-frequency attention feature map and the low-frequency attention feature map can be inversely discretely transformed by two-dimensional wavelet transform to obtain an inverse discrete transformation feature map. The inverse discrete transformation feature map and the sparse photoacoustic image are reconstructed by using residual connection to obtain the reconstructed image.

[0110] Step 107: Based on the reconstructed image and the densely sampled photoacoustic image, a loss value of the photoacoustic image reconstruction model to be trained is calculated.

[0111] After obtaining the reconstructed image, the loss value of the photoacoustic image reconstruction model to be trained can be calculated based on the reconstructed image and the densely sampled photoacoustic image.

[0112] In this embodiment, the loss calculation may be performed in a multi-objective constraint manner, which may include: reconstruction constraint, edge preservation constraint, and frequency domain consistency constraint.

[0113] Among them, the reconstruction constraint measures the pixel-level error between the reconstructed image and the target image, ensuring that the model visually reproduces the overall structure and texture of the target image as closely as possible. Minimizing the reconstruction constraint ensures that the generated image is highly consistent with the target image at the pixel level, preventing the loss of local details. However, relying solely on the reconstruction constraint may result in excessive smoothing of the image. Therefore, this embodiment introduces an edge preservation constraint and a frequency domain consistency constraint to supplement the retention of local details and spectral features.

[0114] As another key objective in multi-objective optimization, the edge preservation constraint ensures that the model focuses on the local structural features of the image and prevents the loss of edges in the target structure of the reconstructed image. By minimizing the edge preservation constraint, the model can effectively capture and retain key edge features in the image and prevent artifacts from causing interruption of the target structure or loss of details. In addition, this constraint helps the model quickly grasp the overall structural features of the image, thereby accelerating convergence.

[0115] In order to compensate for the limitations of spatial domain loss in capturing high-frequency information, this embodiment introduces a frequency domain consistency constraint. This constraint uses Fourier transform to convert image information from the spatial domain to the frequency domain, optimizing the consistency of the model in spectral features. In multi-objective optimization, this goal ensures the integrity of the reconstructed image in the frequency domain, paying special attention to high-frequency details to reduce noise interference. By minimizing this constraint, the model is able to recover important high-frequency information in the frequency domain, ensuring that the spectral features of the reconstructed image are consistent with the target image.

[0116] The formula for the reconstruction constraint can be as follows:

[0117]

[0118] In the above formula, L recon represents the reconstruction constraint, I recon represents the reconstructed image, I target Represents the true target image, N represents the total number of pixels, and i represents the i-th pixel.

[0119] The formula for the edge preserving constraint can be shown as follows:

[0120]

[0121] In the above formula, L edge Indicates edge preservation constraint, Canny() indicates application to reconstruct image Irecon and the target image I target The Canny edge detection operator is used to ensure that edge information is preserved during the reconstruction process.

[0122] The formula for the frequency domain consistency constraint can be shown as follows:

[0123]

[0124] In the above formula, L freq represents the frequency domain consistency constraint, and F() represents the image I recon and the target image I target Fourier transform is performed to ensure consistency in the frequency domain, especially for high-frequency components.

[0125] In this embodiment, in order to adaptively balance the optimization of various objectives during the training process, this embodiment also designs a dynamic weight adjustment mechanism. This mechanism automatically adjusts the weights of different constraints according to the change amplitude of the current iteration. This ensures a balanced approach to multi-objective optimization. Constraints with larger changes will receive higher weights to speed up their optimization process.

[0126] Through the dynamic weight adjustment mechanism, it is possible to ensure that each constraint can adaptively play its role during the training process, thereby achieving effective reconstruction under multi-objective optimization. The embodiment of the present application combines reconstruction constraints (including mean square error (MSE)), edge preservation and frequency domain consistency with dynamic weighting to improve the reconstruction quality. This method achieves higher reconstruction accuracy under sparse sampling conditions, effectively restores image details and structures, and significantly reduces distortion and artifacts.

[0127] Step 108: When the loss value is within a preset range, a final photoacoustic image reconstruction model is obtained.

[0128] After calculating the loss value of the photoacoustic image reconstruction model to be trained, it can be determined whether the loss value is within a preset range. If so, it means that the photoacoustic image reconstruction model to be trained has converged, and at this time, the final photoacoustic image reconstruction model can be obtained.

[0129] The photoacoustic image reconstruction model trained through the above steps can be applied in the medical field. The specific implementation example can be shown as follows:

[0130] 1. Data acquisition: A photoacoustic imaging device equipped with sparse sampling function is used to scan the patient's liver area. In order to simulate the data sparsity in the real clinical environment, the acquisition device only uses a small number of detection elements and a limited viewing angle for data acquisition.

[0131] 2. Data processing: The collected sparse data is first processed by the AD-WaveNet model (i.e., the photoacoustic image reconstruction model provided in this embodiment). The model uses discrete wavelet transform (DWT) for multi-scale decomposition and optimizes the capture of details through the attention mechanism, thereby significantly reducing the computational complexity without sacrificing the reconstruction quality.

[0132] 3. Image reconstruction: The processed data is reconstructed through the inverse wavelet transform layer of the model to generate the final image. These images have high resolution and reduced artifacts, allowing doctors to more clearly observe subtle structural changes in the liver, which is crucial for early diagnosis of the disease.

[0133] The photoacoustic image reconstruction model training method provided in the embodiment of the present application obtains model training samples, and the model training samples include: sparse photoacoustic images and densely sampled photoacoustic images corresponding to the sparse photoacoustic images. The model training samples are input into the photoacoustic image reconstruction model to be trained, and the photoacoustic image reconstruction model to be trained includes: a discrete transformation layer, a high-frequency channel attention layer, and a low-frequency channel attention layer. The discrete transformation layer is called to perform a discrete two-dimensional wavelet transform on the sparse photoacoustic image to obtain a low-frequency feature map and a high-frequency feature map of the sparse photoacoustic image. The high-frequency channel attention layer is called to capture features in the vertical and horizontal directions of the high-frequency feature map to obtain a high-frequency attention feature map. The low-frequency channel attention layer is called to process the low-frequency feature map to obtain a low-frequency attention feature map. The high-frequency attention feature map, the low-frequency attention feature map, and the sparse photoacoustic image are image reconstructed to obtain a reconstructed image. Based on the reconstructed image and the densely sampled photoacoustic image, the loss value of the photoacoustic image reconstruction model to be trained is calculated. When the loss value is within a preset range, the final photoacoustic image reconstruction model is obtained. The embodiment of the present application is specifically designed to enhance the processing of different frequency subbands in the wavelet domain by deeply integrating wavelet decomposition with a customized attention mechanism. This synergy enables more efficient handling of sparse sampling challenges in PAT while capturing both low-frequency and high-frequency information with lower FLOPs and parameters, making it very suitable for deployment in resource-constrained scenarios. At the same time, the combination of discrete two-dimensional wavelet transform and attention mechanism can effectively improve the accuracy of sparsely sampled photoacoustic image reconstruction. By adopting multi-scale decomposition of the input image, the model effectively captures the global structure and local details, thereby better restoring fine features while maintaining overall structural consistency.

[0134] Reference Figure 4 , which shows a schematic diagram of the structure of a photoacoustic image reconstruction model training device provided in an embodiment of the present application, such as Figure 4 As shown, the photoacoustic image reconstruction model training device 400 may include the following modules:

[0135] A model sample acquisition module 410 is used to acquire model training samples, wherein the model training samples include: a sparse photoacoustic image and a densely sampled photoacoustic image corresponding to the sparse photoacoustic image;

[0136] A model sample input module 420 is used to input the model training sample into the photoacoustic image reconstruction model to be trained, wherein the photoacoustic image reconstruction model to be trained includes: a discrete transformation layer, a high-frequency channel attention layer, and a low-frequency channel attention layer;

[0137] A feature map acquisition module 430 is used to call the discrete transformation layer to perform a discrete two-dimensional wavelet transform on the sparse photoacoustic image to obtain a low-frequency feature map and a high-frequency feature map of the sparse photoacoustic image;

[0138] A high-frequency feature map acquisition module 440 is used to call the high-frequency channel attention layer to capture features of the high-frequency feature map in the vertical direction and the horizontal direction to obtain a high-frequency attention feature map;

[0139] A low-frequency feature map acquisition module 450 is used to call the low-frequency channel attention layer to process the low-frequency feature map to obtain a low-frequency attention feature map;

[0140] A reconstructed image acquisition module 460 is used to reconstruct the high-frequency attention feature map, the low-frequency attention feature map and the sparse photoacoustic image to obtain a reconstructed image;

[0141] A loss value calculation module 470, configured to calculate a loss value of the photoacoustic image reconstruction model to be trained based on the reconstructed image and the densely sampled photoacoustic image;

[0142] The photoacoustic image reconstruction model acquisition module 480 is used to obtain the final photoacoustic image reconstruction model when the loss value is within a preset range.

[0143] Optionally, the feature map acquisition module includes:

[0144] a component feature map acquisition unit, configured to call the discrete transformation layer to perform a discrete two-dimensional wavelet transform on the sparse photoacoustic image to obtain a low-frequency component feature map, a horizontal high-frequency component feature map, a vertical high-frequency component feature map, and a diagonal high-frequency component feature map of the sparse photoacoustic image;

[0145] The feature map acquisition unit is used to use the low-frequency component feature map as the low-frequency feature map, and use the horizontal high-frequency component feature map, the vertical high-frequency component feature map and the diagonal high-frequency component feature map as the high-frequency feature map.

[0146] Optionally, the high-frequency channel attention layer includes: a first convolution normalization activation layer, an enhanced axial attention mechanism layer, and a second convolution normalization activation layer.

[0147] The high-frequency feature map acquisition module includes:

[0148] A feature map acquisition unit, used to call the first convolution normalization activation layer to perform convolution, normalization and activation processing on the high-frequency feature map to obtain a feature map of the high-frequency feature map;

[0149] An enhanced axial feature map acquisition unit, used for calling the enhanced axial attention mechanism layer to perform enhanced axial attention processing on the feature map to obtain an enhanced axial attention feature map;

[0150] The high-frequency feature map acquisition unit is used to call the second convolution normalization activation layer to perform convolution, normalization and activation processing on the enhanced axial attention feature map to obtain the high-frequency attention feature map.

[0151] Optionally, the enhanced axial characteristic map acquisition unit includes:

[0152] A convolution feature map acquisition subunit, used to call the enhanced axial attention mechanism layer to perform convolution processing on the feature map to obtain a convolution feature map corresponding to the feature map;

[0153] A transformation feature map acquisition subunit, used for performing a linear transformation process on the convolution feature map to obtain a linear transformation feature map;

[0154] A feature map acquisition subunit, used to perform convolution processing on the linear transformation feature map to obtain a Query feature map, a Key feature map, and a Value feature map corresponding to the linear transformation feature map;

[0155] A Key feature map acquisition subunit is used to perform convolution and feature fusion processing on the Key feature map using two convolution kernels of opposite dimensions to obtain a fused Key feature map;

[0156] A Value feature map acquisition subunit is used to perform convolution and feature fusion processing on the Value feature map using two convolution kernels of opposite dimensions to obtain a fused Value feature map;

[0157] The cross-feature map acquisition subunit is used to process the Query feature map, the fused Key feature map, the fused Value feature map and the convolution feature map to obtain the enhanced axial attention feature map.

[0158] Optionally, the low-frequency channel attention layer includes: a first low-frequency attention module, a discrete transformation module, a second low-frequency attention module, a high-frequency attention module, an inverse discrete transformation module and a third low-frequency attention module.

[0159] The low-frequency feature map acquisition module includes:

[0160] A first feature map acquisition unit is used to call the first low-frequency attention module to process the low-frequency feature map using the MDTA attention mechanism to obtain a first attention feature map;

[0161] A feature map discrete transformation unit, used for calling the discrete transformation module to perform a discrete two-dimensional wavelet transform on the first attention feature map to obtain a sub-low-frequency feature map and a sub-high-frequency feature map;

[0162] A second feature map acquisition unit is used to call the second low-frequency attention module to process the sub-low-frequency feature map using the MDTA attention mechanism to obtain a second attention feature map;

[0163] A third feature map acquisition unit, used for calling the high-frequency attention module to process the sub-high-frequency feature map to obtain a third attention feature map;

[0164] A discrete feature map acquisition unit, configured to call the inverse discrete transformation module to perform an inverse discrete two-dimensional wavelet transform on the second attention feature map and the third attention feature map to obtain a discrete transformation feature map;

[0165] An updated feature map acquisition unit, used to connect the discrete transformation feature map and the low-frequency feature map in a skip connection manner to obtain an updated low-frequency feature map;

[0166] A low-frequency feature map acquisition unit, used for calling the third low-frequency attention module to process the updated low-frequency feature map using the MDTA attention mechanism to obtain the low-frequency attention feature map;

[0167] Among them, the network structure of the high-frequency attention module is the same as the network structure of the high-frequency channel attention layer.

[0168] Optionally, the reconstructed image acquisition module includes:

[0169] An inverse discrete feature map acquisition unit, used for performing an inverse discrete two-dimensional wavelet transform on the high-frequency attention feature map and the low-frequency attention feature map to obtain an inverse discrete transform feature map;

[0170] The reconstructed image acquisition unit is used to reconstruct the inverse discrete transformation feature map and the sparse photoacoustic image by using residual connection to obtain the reconstructed image.

[0171] The photoacoustic image reconstruction model training device provided in the embodiment of the present application obtains model training samples, and the model training samples include: sparse photoacoustic images and densely sampled photoacoustic images corresponding to the sparse photoacoustic images. The model training samples are input into the photoacoustic image reconstruction model to be trained, and the photoacoustic image reconstruction model to be trained includes: a discrete transformation layer, a high-frequency channel attention layer, and a low-frequency channel attention layer. The discrete transformation layer is called to perform a discrete two-dimensional wavelet transform on the sparse photoacoustic image to obtain a low-frequency feature map and a high-frequency feature map of the sparse photoacoustic image. The high-frequency channel attention layer is called to capture features in the vertical and horizontal directions of the high-frequency feature map to obtain a high-frequency attention feature map. The low-frequency channel attention layer is called to process the low-frequency feature map to obtain a low-frequency attention feature map. The high-frequency attention feature map, the low-frequency attention feature map, and the sparse photoacoustic image are image reconstructed to obtain a reconstructed image. Based on the reconstructed image and the densely sampled photoacoustic image, the loss value of the photoacoustic image reconstruction model to be trained is calculated. When the loss value is within a preset range, the final photoacoustic image reconstruction model is obtained. The embodiment of the present application is specifically designed to enhance the processing of different frequency subbands in the wavelet domain by deeply integrating wavelet decomposition with a customized attention mechanism. This synergy enables more efficient handling of sparse sampling challenges in PAT while capturing both low-frequency and high-frequency information with lower FLOPs and parameters, making it very suitable for deployment in resource-constrained scenarios. At the same time, the combination of discrete two-dimensional wavelet transform and attention mechanism can effectively improve the accuracy of sparsely sampled photoacoustic image reconstruction. By adopting multi-scale decomposition of the input image, the model effectively captures the global structure and local details, thereby better restoring fine features while maintaining overall structural consistency.

[0172] An embodiment of the present application also provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program implements the above-mentioned photoacoustic image reconstruction model training method when executed by the processor.

[0173] Figure 5 FIG. 5 is a schematic diagram showing the structure of an electronic device 500 according to an embodiment of the present invention. Figure 5 As shown, the electronic device 500 includes a central processing unit (CPU) 501, which can perform various appropriate actions and processes according to computer program instructions stored in a read-only memory (ROM) 502 or computer program instructions loaded from a storage unit 508 into a random access memory (RAM) 503. In RAM 503, various programs and data required for the operation of the electronic device 500 can also be stored. CPU 501, ROM 502 and RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0174] Multiple components in the electronic device 500 are connected to the I / O interface 505, including: an input unit 506, such as a keyboard, a mouse, a microphone, etc.; an output unit 507, such as various types of displays, speakers, etc.; a storage unit 508, such as a disk, an optical disk, etc.; and a communication unit 509, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 509 allows the electronic device 500 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0175] The various processes and processing described above may be performed by the processing unit 501. For example, the method of any of the above embodiments may be implemented as a computer software program, which is tangibly contained in a computer-readable medium, such as the storage unit 508. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 500 via the ROM 502 and / or the communication unit 509. When the computer program is loaded into the RAM 503 and executed by the CPU 501, one or more actions in the method described above may be performed.

[0176] In addition, an embodiment of the present application further provides a computer-readable storage medium having a computer program stored thereon, which implements the above-mentioned photoacoustic image reconstruction model training method when executed by a processor.

[0177] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0178] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, devices, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0179] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, terminals (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0180] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0181] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal so that a series of operating steps are executed on the computer or other programmable terminal to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable terminal. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0182] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present application.

[0183] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal. In the absence of further restrictions, the elements defined by the statement "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or terminal including the elements.

[0184] The above is a detailed introduction to a photoacoustic image reconstruction model training method, a photoacoustic image reconstruction model training device, an electronic device and a computer-readable storage medium provided by the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for a person skilled in the art, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A photoacoustic image reconstruction model training method, characterized in that: The method comprises: Acquire a model training sample, wherein the model training sample includes: a sparse photoacoustic image and a densely sampled photoacoustic image corresponding to the sparse photoacoustic image; Inputting the model training samples into a photoacoustic image reconstruction model to be trained, wherein the photoacoustic image reconstruction model to be trained comprises: a discrete transformation layer, a high-frequency channel attention layer, and a low-frequency channel attention layer; Calling the discrete transformation layer to perform discrete two-dimensional wavelet transformation on the sparse photoacoustic image to obtain a low-frequency feature map and a high-frequency feature map of the sparse photoacoustic image; Calling the high-frequency channel attention layer to capture features of the high-frequency feature map in vertical and horizontal directions to obtain a high-frequency attention feature map; Calling the low-frequency channel attention layer to process the low-frequency feature map to obtain a low-frequency attention feature map; Performing image reconstruction on the high-frequency attention feature map, the low-frequency attention feature map, and the sparse photoacoustic image to obtain a reconstructed image; Based on the reconstructed image and the densely sampled photoacoustic image, calculating a loss value of the photoacoustic image reconstruction model to be trained; When the loss value is within a preset range, a final photoacoustic image reconstruction model is obtained.

2. The method according to claim 1, characterized in that The calling of the discrete transformation layer to perform discrete two-dimensional wavelet transformation on the sparse photoacoustic image to obtain a low-frequency feature map and a high-frequency feature map of the sparse photoacoustic image includes: Calling the discrete transformation layer to perform discrete two-dimensional wavelet transform on the sparse photoacoustic image to obtain a low-frequency component feature map, a horizontal high-frequency component feature map, a vertical high-frequency component feature map, and a diagonal high-frequency component feature map of the sparse photoacoustic image; The low-frequency component feature map is used as the low-frequency feature map, and the horizontal high-frequency component feature map, the vertical high-frequency component feature map and the diagonal high-frequency component feature map are used as the high-frequency feature map.

3. The method according to claim 1, characterized in that The high-frequency channel attention layer includes: a first convolution normalized activation layer, an enhanced axial attention mechanism layer, and a second convolution normalized activation layer. The calling of the high-frequency channel attention layer to capture features of the high-frequency feature map in vertical and horizontal directions to obtain a high-frequency attention feature map includes: Calling the first convolution normalization activation layer to perform convolution, normalization and activation processing on the high-frequency feature map to obtain a feature map of the high-frequency feature map; Calling the enhanced axial attention mechanism layer to perform cross attention processing on the feature map to obtain an enhanced axial attention feature map; The second convolution normalization activation layer is called to perform convolution, normalization and activation processing on the enhanced axial attention feature map to obtain the high-frequency attention feature map.

4. The method according to claim 3, characterized in that The calling of the enhanced axial attention mechanism layer to perform enhanced axial attention processing on the feature map to obtain an enhanced axial attention feature map includes: Calling the enhanced axial attention mechanism layer to perform convolution processing on the feature map to obtain a convolution feature map corresponding to the feature map; Performing linear transformation processing on the convolution feature map to obtain a linear transformation feature map; Performing convolution processing on the linear transformation feature map to obtain a Query feature map, a Key feature map, and a Value feature map corresponding to the linear transformation feature map; Using two convolution kernels of opposite dimensions to perform convolution and feature fusion processing on the Key feature map to obtain a fused Key feature map; Using two convolution kernels of opposite dimensions to perform convolution and feature fusion processing on the Value feature map to obtain a fused Value feature map; The Query feature map, the fused Key feature map, the fused Value feature map and the convolution feature map are processed to obtain the enhanced axial attention feature map.

5. The method according to claim 1, characterized in that The low-frequency channel attention layer includes: a first low-frequency attention module, a discrete transformation module, a second low-frequency attention module, a high-frequency attention module, an inverse discrete transformation module and a third low-frequency attention module. The calling of the low-frequency channel attention layer to process the low-frequency feature map to obtain a low-frequency attention feature map includes: Calling the first low-frequency attention module to process the low-frequency feature map using the MDTA attention mechanism to obtain a first attention feature map; Calling the discrete transformation module to perform discrete two-dimensional wavelet transformation on the first attention feature map to obtain a sub-low-frequency feature map and a sub-high-frequency feature map; Calling the second low-frequency attention module to process the sub-low-frequency feature map using the MDTA attention mechanism to obtain a second attention feature map; Calling the high-frequency attention module to process the sub-high-frequency feature map to obtain a third attention feature map; Calling the inverse discrete transformation module to perform an inverse discrete two-dimensional wavelet transform on the second attention feature map and the third attention feature map to obtain a discrete transformation feature map; The discrete transformation feature map and the low-frequency feature map are connected by using a skip connection method to obtain an updated low-frequency feature map; Calling the third low-frequency attention module to process the updated low-frequency feature map using the MDTA attention mechanism to obtain the low-frequency attention feature map; Among them, the network structure of the high-frequency attention module is the same as the network structure of the high-frequency channel attention layer.

6. The method according to claim 1, characterized in that The performing image reconstruction on the high-frequency attention feature map, the low-frequency attention feature map and the sparse photoacoustic image to obtain a reconstructed image includes: Performing an inverse discrete two-dimensional wavelet transform on the high-frequency attention feature map and the low-frequency attention feature map to obtain an inverse discrete transform feature map; Residual connection is used to reconstruct the inverse discrete transformation feature map and the sparse photoacoustic image to obtain the reconstructed image.

7. A photoacoustic image reconstruction model training device, characterized in that: The device comprises: A model sample acquisition module, used to acquire model training samples, wherein the model training samples include: a sparse photoacoustic image and a densely sampled photoacoustic image corresponding to the sparse photoacoustic image; A model sample input module, used for inputting the model training sample into a photoacoustic image reconstruction model to be trained, wherein the photoacoustic image reconstruction model to be trained comprises: a discrete transformation layer, a high-frequency channel attention layer and a low-frequency channel attention layer; A feature map acquisition module, used for calling the discrete transformation layer to perform discrete two-dimensional wavelet transformation on the sparse photoacoustic image to obtain a low-frequency feature map and a high-frequency feature map of the sparse photoacoustic image; A high-frequency feature map acquisition module is used to call the high-frequency channel attention layer to capture features of the high-frequency feature map in the vertical and horizontal directions to obtain a high-frequency attention feature map; A low-frequency feature map acquisition module, used to call the low-frequency channel attention layer to process the low-frequency feature map to obtain a low-frequency attention feature map; A reconstructed image acquisition module, used to reconstruct the high-frequency attention feature map, the low-frequency attention feature map and the sparse photoacoustic image to obtain a reconstructed image; A loss value calculation module, used for calculating the loss value of the photoacoustic image reconstruction model to be trained based on the reconstructed image and the densely sampled photoacoustic image; The photoacoustic image reconstruction model acquisition module is used to obtain the final photoacoustic image reconstruction model when the loss value is within a preset range.

8. The device according to claim 7, characterized in that The feature map acquisition module comprises: a component feature map acquisition unit, configured to call the discrete transformation layer to perform a discrete two-dimensional wavelet transform on the sparse photoacoustic image to obtain a low-frequency component feature map, a horizontal high-frequency component feature map, a vertical high-frequency component feature map, and a diagonal high-frequency component feature map of the sparse photoacoustic image; The feature map acquisition unit is used to use the low-frequency component feature map as the low-frequency feature map, and use the horizontal high-frequency component feature map, the vertical high-frequency component feature map and the diagonal high-frequency component feature map as the high-frequency feature map.

9. An electronic device, characterized in that: include: A processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor implements the photoacoustic image reconstruction model training method according to any one of claims 1 to 6 when executing the program.

10. A computer-readable storage medium, characterized in that: When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the photoacoustic image reconstruction model training method according to any one of claims 1 to 6.

Images