Method, system, device and medium for suppressing artifacts in limited-viewing-angle photoacoustic images

Abstract

The present invention relates to the field of photoacoustic imaging technology, and discloses a method, system, device, and medium for suppressing artifacts in limited-view photoacoustic images. The method comprises: generating a reconstructed limited-view photoacoustic image and a corresponding full-view photoacoustic image through an image reconstruction algorithm; adding a sub-image extraction module to each level of a U-Net model except the first level to extract a sub-image sequence; optimizing artifact suppression model parameters and selecting an optimal artifact suppression model based on a loss function of a structural similarity index and a mean absolute error between the limited-view photoacoustic image and the corresponding full-view photoacoustic image; inputting a limited-view photoacoustic image constructed based on an actual acquired limited-view photoacoustic signal into a trained artifact suppression model, and outputting a post-processed image after artifact suppression. The present invention has good post-processing results for limited-view photoacoustic images, can effectively suppress artifacts, and restore some lost information in the image.

Description

Technical Field

[0001] The present invention relates to the technical field of photoacoustic imaging, and in particular to a method for suppressing photoacoustic image artifacts with a limited viewing angle. Background Art

[0002] Photoacoustic tomography (PAT) is a new, noninvasive medical imaging modality that combines high optical contrast with high acoustic penetration depth. PAT uses pulsed laser light to illuminate biological tissue, causing it to expand due to heat and generate ultrasonic waves that propagate outward. Using an ultrasound detector array to receive the photoacoustic wave signals, image reconstruction algorithms can be used to reconstruct the fine internal structure of living tissue.

[0003] Depending on the imaging dimension, photoacoustic imaging can be divided into two-dimensional photoacoustic imaging and three-dimensional photoacoustic imaging. Ideal two-dimensional photoacoustic imaging is to use a sufficiently dense full-view annular detection array to cover the imaging object to receive the photoacoustic signals emitted by the object in all directions within the plane. However, in reality, due to factors such as production cost and imaging time, the ideal imaging condition of full-view imaging may be difficult to achieve, so photoacoustic imaging can only be performed based on limited or sparse viewing angles. The photoacoustic image formed under a limited viewing angle may be distorted, producing redundant structures such as negative artifacts, streak artifacts, and splitting artifacts, which affect the imaging quality and spatial resolution and bring serious troubles to disease diagnosis. Therefore, suppressing artifacts in limited-view photoacoustic reconstruction images and recovering the structural information of the organism from its blurred images is a research field worthy of in-depth exploration.

[0004] Deep learning algorithms establish a mapping relationship between input and output based on multi-layer neural networks. Therefore, by optimizing a specific objective function, a post-processing network can be constructed to transform limited-view photoacoustic images into full-view photoacoustic images. This removes redundant artifacts from the image, restores detailed information, and achieves high-quality photoacoustic imaging using low-cost imaging equipment. The U-Net model, based on a multi-layer convolutional neural network, is currently one of the simplest and most effective image post-processing models.

[0005] The U-Net model can perform various post-processing operations on images, including image segmentation and artifact reduction. The U-Net model first geometrically increases the number of image channels through convolution and geometrically reduces the image size through pooling. This process is called encoding the image. The encoded image is then upsampled and convolved, and merged with the pre-encoded image of the same size to restore the image size and channel count. This process is called decoding the image. Finally, the decoded image undergoes convolution and nonlinear operations to produce the final output image.

[0006] The U-Net model is highly versatile and can handle a wide range of practical problems, but it does not necessarily interface seamlessly with each problem. For example, in the area of ​​artifact suppression in limited-view photoacoustic images, there are currently no patents specifically addressing this issue using the U-Net model. Regarding artifact suppression in other application areas, although there are some post-processing structure designs based on the U-Net model, some require integration with other algorithms, while others make significant changes to the U-Net model. The end result, while significantly improving reconstruction quality, loses the original rapid responsiveness of the U-Net model.

[0007] Specific application problems require specific analysis based on the specific application field. In artifact suppression of limited-view photoacoustic images, the characteristic that artifacts in the image are chaotic while the information is highly continuous can be used to achieve differentiated processing between information and artifacts. Summary of the Invention

[0008] To address the above technical problems, the present invention provides a method, system, device, and medium for artifact suppression in limited-view photoacoustic images. Based on a self-designed improved U-Net model structure, namely the Sub4-UNet model, the present invention performs artifact suppression on photoacoustic images under limited view and evaluates the impact of model complexity on image post-processing results.

[0009] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0010] In a first aspect, the present invention provides a method for suppressing artifacts in a limited-viewing angle photoacoustic image, comprising:

[0011] generating a photoacoustic image by an image reconstruction algorithm, wherein the photoacoustic image includes a limited-view photoacoustic image and a corresponding full-view photoacoustic image;

[0012] Constructing an artifact suppression model: Add a sub-image extraction module to each layer of the U-Net model except the first layer to extract a sub-image sequence with the same number of channels as the feature map output by the current layer's coding layer. Perform an information separation operation on the coding layer output: subtract the limited-view photoacoustic image from the feature map output by the first layer's coding layer, and subtract the sub-image sequence of the corresponding layer from the feature map output by the coding layer of each other layer to separate artifacts. Perform a dual-path merging operation at the decoding layer: Merge the upsampled feature map with the feature map output by the coding layer of the same layer and the limited-view photoacoustic image or sub-image sequence.

[0013] constructing a loss function based on the structural similarity index and mean absolute error of the limited-view photoacoustic image and the corresponding full-view photoacoustic image, optimizing the artifact suppression model parameters, and selecting the optimal artifact suppression model;

[0014] The limited-view photoacoustic image constructed based on the actually collected limited-view photoacoustic signal is input into the trained artifact suppression model, and the post-processed image after artifact suppression is output.

[0015] In one embodiment, generating a reconstructed limited-view photoacoustic image and a corresponding full-view photoacoustic image by an image reconstruction algorithm specifically includes:

[0016] Based on the k-Wave toolbox, a limited-view detector arrangement and a full-view detector arrangement are set for biological tissues. Under the same sound source and medium distribution, the forward propagation is simulated to obtain the limited-view photoacoustic signal and the full-view photoacoustic signal, respectively. Then, the image reconstruction algorithm is used for reverse solution to obtain the limited-view photoacoustic image reconstructed by the biological tissue and the corresponding full-view photoacoustic image, respectively.

[0017] In one embodiment, the sub-image extraction module is added to each level of the U-Net model except the first level, specifically including: the input of the coding layer of the first level is the limited-view photoacoustic image; the input of the coding layers of other levels is the feature map output by the coding layer of the previous level minus the sub-image sequence output by the sub-image extraction module of the corresponding level.

[0018] In one embodiment, extracting a sub-image sequence having the same number of channels as a feature image output by a current-level coding layer specifically includes:

[0019] The sub-image extraction process includes: dividing the image of each channel in the input image of the sub-image extraction module into multiple 2×2 blocks, extracting the pixel values ​​of the upper left corner, upper right corner, lower left corner, and lower right corner of each block, arranging them into four sub-images of half size, and placing the four sub-images into four channels; repeating the above process for the image of each channel in the input image to obtain a sub-image sequence of the input image;

[0020] The input image of the sub-image extraction module of the second level is the limited-view photoacoustic image, and the input image of the sub-image extraction modules of other levels is the sub-image sequence output by the sub-image extraction module of the previous level. Channel replication is used to ensure that the sub-image sequence has the same number of channels as the feature map output by the corresponding encoding layer.

[0021] In one embodiment, the channel replication is performed so that the sub-image sequence and the feature map output by the corresponding encoding layer have the same number of channels, specifically including:

[0022] When the number of channels of the sub-image sequence is less than the feature map output by the encoding layer of the current level, the sub-image sequence and the feature map output by the corresponding encoding layer are made to have the same number of channels by copying the sub-images of other channels of the sub-image sequence.

[0023] In one embodiment, constructing a loss function based on the structural similarity index and mean absolute error of the limited-view photoacoustic image and the corresponding full-view photoacoustic image, optimizing the artifact suppression model parameters, and selecting the optimal artifact suppression model specifically includes:

[0024] Loss Function ;

[0025] SSIM is the structural similarity index:

[0026] ;

[0027] in, and Limited-view photoacoustic imaging and full-view photoacoustic imaging The average value of the local area, and are the variances of the local regions of x and y, is the covariance of the local region of x and y, and the parameter ,parameter ,in is the upper bound of the range of pixel values ​​of the photoacoustic image;

[0028] is the mean absolute error:

[0029] ;

[0030] in, is the total number of pixels of the photoacoustic image, for The pixel value of the i-th pixel in , for The pixel value of the i-th pixel in .

[0031] In a second aspect, the present invention provides a limited-viewing-angle photoacoustic image artifact suppression system, comprising:

[0032] Dataset construction module: generates photoacoustic images through image reconstruction algorithm, wherein the photoacoustic images include limited-view photoacoustic images and corresponding full-view photoacoustic images;

[0033] Model construction module: A sub-image extraction module is added to each layer of the U-Net model except the first layer to extract a sub-image sequence with the same number of channels as the feature map output by the encoding layer of the current layer; an information separation operation is performed on the encoding layer output: the limited-view photoacoustic image is subtracted from the feature map output by the encoding layer of the first layer, and the sub-image sequence of the corresponding layer is subtracted from the feature map output by the encoding layer of each other layer to separate artifacts; a dual-path merging operation is performed at the decoding layer: the upsampled feature map is simultaneously merged with the feature map output by the encoding layer of the same layer, and the limited-view photoacoustic image or sub-image sequence;

[0034] Model training module: constructing a loss function based on the structural similarity index and mean absolute error of the limited-view photoacoustic image and the corresponding full-view photoacoustic image, optimizing the artifact suppression model parameters, and selecting the optimal artifact suppression model;

[0035] Prediction module: The limited-view photoacoustic image constructed based on the actual collected limited-view photoacoustic signal is input into the trained artifact suppression model, and the post-processed image after artifact suppression is output.

[0036] In one embodiment, in the model training module, the loss function is constructed based on the structural similarity index and mean absolute error of the limited-view photoacoustic image and the corresponding full-view photoacoustic image, the artifact suppression model parameters are optimized, and the optimal artifact suppression model is selected, which specifically includes:

[0037] Loss Function ;

[0038] SSIM is the structural similarity index:

[0039] ;

[0040] in, and Limited-view photoacoustic imaging and full-view photoacoustic imaging The average value of the local area, and are the variances of the local regions of x and y, is the covariance of the local region of x and y, and the parameter ,parameter ,in is the upper bound of the range of pixel values ​​of the photoacoustic image;

[0041] is the mean absolute error:

[0042] ;

[0043] Where n is the total number of pixels in the photoacoustic image, for The pixel value of the i-th pixel in , for The pixel value of the i-th pixel in .

[0044] In a third aspect, the present invention provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method of any one embodiment of the first aspect when executing the computer program.

[0045] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method of any one of the embodiments of the first aspect.

[0046] Compared with the prior art, the beneficial technical effects of the present invention are:

[0047] Building on the existing U-Net model, this invention preserves the original structure as much as possible to inherit the U-Net model's efficiency and speed. Furthermore, for the specific field of artifact suppression in photoacoustic images under limited viewing angles, based on the poor continuity of artifacts, the artifact-containing image is decomposed into a sequence of sub-images with essentially continuous information but multiple artifacts by segmenting the image and extracting pixels at each corner point. This allows for multiple separations of information from artifacts during image post-processing. Artifacts are then processed separately, and finally, effective information is added layer by layer to obtain the post-processed image. This organic combination of a general network structure and specific field requirements enables the constructed Sub4-UNet model to achieve more comprehensive post-processing results at a lower cost.

[0048] In terms of image reconstruction quality, the present invention has achieved good numerical results, both based on the PSNR indicator for measuring reconstruction error and the SSIM indicator for measuring the structural similarity between the reconstructed image and the original image. In particular, on the mouse embryo dataset with strong continuity, both PSNR and SSIM results reached excellent levels, indicating that the present invention can obtain high-quality reconstructed images under limited viewing angles.

[0049] In terms of image reconstruction time, due to the simple network structure and fewer parameters, the present invention can process limited-viewing angle photoacoustic images at an extremely fast speed of the order of 10 milliseconds, obtain high-quality post-processed images, and realize real-time imaging.

[0050] Overall, the Sub4-UNet model achieves excellent post-processing results for limited-view photoacoustic images, effectively suppressing artifacts and restoring some lost information. Furthermore, the Sub4-UNet model performs significantly better for continuous structures than for dispersed ones. Finally, the Sub4-UNet model also excels in reconstruction speed, making it suitable for image post-processing in real-time imaging. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 is a flow chart of a method in an embodiment of the present invention;

[0052] Figure 2 Schematic diagram of model training and model prediction in an embodiment of the present invention;

[0053] Figure 3 Schematic diagram of the working principle of the sub-graph extraction module in an embodiment of the present invention;

[0054] Figure 4 Schematic diagram of the structure of the Sub4-UNet model in an embodiment of the present invention;

[0055] Figure 5 Graphs showing the post-processing effects of Sub4-UNet models of different complexities on limited-view photoacoustic images of blood vessels according to an embodiment of the present invention;

[0056] Figure 6 This is a diagram showing the post-processing effect of the Sub4-UNet model of different complexities on the limited-angle photoacoustic image of mouse embryos in an embodiment of the present invention. DETAILED DESCRIPTION

[0057] A preferred embodiment of the present invention will be described in detail below with reference to the accompanying drawings.

[0058] like Figure 1 As shown, the present invention provides a method for suppressing artifacts in a limited-viewing angle photoacoustic image, comprising the following steps:

[0059] S1, generating a photoacoustic image by an image reconstruction algorithm, wherein the photoacoustic image includes a limited-view photoacoustic image and a corresponding full-view photoacoustic image;

[0060] S2, constructing an artifact suppression model: adding a sub-image extraction module to each layer of the U-Net model except the first layer to extract a sub-image sequence with the same number of channels as the feature map output by the encoding layer of the current layer; performing an information separation operation on the encoding layer output: subtracting the limited-view photoacoustic image from the feature map output by the encoding layer of the first layer, and subtracting the sub-image sequence of the corresponding layer from the feature map output by the encoding layer of each other layer to separate artifacts; performing a dual-path merging operation at the decoding layer: merging the upsampled feature map with the feature map output by the encoding layer of the same layer and the limited-view photoacoustic image or sub-image sequence;

[0061] S3, constructing a loss function based on the structural similarity index and mean absolute error of the limited-view photoacoustic image and the corresponding full-view photoacoustic image, optimizing the artifact suppression model parameters, and selecting the optimal artifact suppression model;

[0062] S4: inputting a limited-view photoacoustic image constructed based on the actually acquired limited-view photoacoustic signal into a trained artifact suppression model, and outputting a post-processed image after artifact suppression.

[0063] The operation process of the present invention is as follows Figure 2 As shown in Figure 2, it can be divided into two stages: model training and model prediction.

[0064] During the model training phase, based on the original biological tissue images, a limited-view and full-view photoacoustic imaging simulation model was first constructed using the k-Wave toolbox, with all other conditions identical, to facilitate forward propagation of the photoacoustic signal. After obtaining the photoacoustic signal, initial reconstructed photoacoustic images under limited and full-view conditions were generated using a traditional image reconstruction algorithm, serving as the training dataset. Next, the constructed training dataset was imported into the PyTorch deep learning framework to construct the artifact suppression model (Sub4-UNet model), also known as the post-processing network, described in this invention. The corresponding number of base channels was selected to adjust the model complexity, and a mapping relationship between the limited-view and full-view photoacoustic images was established. Finally, based on the objective function, all parameters in the artifact suppression model were iteratively optimized. The artifact suppression model with the optimal complexity was selected based on reconstruction quality and reconstruction time to serve as the final post-processing network for the photoacoustic image under limited-view conditions.

[0065] During the model prediction phase, a photoacoustic signal with a limited field of view under the same imaging conditions is first obtained using a photoacoustic imaging device. Next, a photoacoustic image with a limited field of view is obtained using a traditional image reconstruction algorithm. Finally, the image is fed into the optimized artifact suppression model to obtain a post-processed image with artifact suppression.

[0066] In one embodiment, generating a reconstructed limited-view photoacoustic image and a corresponding full-view photoacoustic image by an image reconstruction algorithm in step S1 specifically includes:

[0067] Based on the k-Wave toolbox, a limited-view detector arrangement and a full-view detector arrangement are set for biological tissues. Under the same sound source and medium distribution, the forward propagation is simulated to obtain the limited-view photoacoustic signal and the full-view photoacoustic signal, respectively. Then, the image reconstruction algorithm is used for reverse solution to obtain the limited-view photoacoustic image reconstructed by the biological tissue and the corresponding full-view photoacoustic image, respectively.

[0068] In the forward problem of photoacoustic signal propagation, the photoacoustic source is known to be Initial photoacoustic pressure at , then at any position at time t Photoacoustic pressure at for:

[0069] ;

[0070] in, The forward propagation problem in photoacoustic imaging is based on the above equation. The conditions such as the sound source, medium, and detector distribution are set to obtain the photoacoustic signal detected by each detector over a period of time.

[0071] The process of reconstructing the initial photoacoustic source image from the detected photoacoustic signal is called the inverse problem. Currently, there is no unified solution to this problem, but there are some traditional photoacoustic reconstruction algorithms, such as delayed summation (DAS), filtered back projection (FBP), series expansion (SE), time reversal (TR) and iterative reconstruction (IR).

[0072] By setting two detector arrangements, limited viewing angle and full viewing angle, under the same sound source and medium distribution, the photoacoustic images of biological tissues reconstructed at limited viewing angle and full viewing angle can be obtained by simulating forward propagation and solving the inverse problem, thereby constructing a data set.

[0073] In one embodiment, step S2 of adding a sub-image extraction module to each level of the U-Net model except the first level specifically includes: the input of the coding layer of the first level is the limited-view photoacoustic image; the input of the coding layers of other levels is the feature map output by the coding layer of the previous level minus the sub-image sequence output by the sub-image extraction module of the corresponding level.

[0074] In one embodiment, extracting a sub-image sequence having the same number of channels as the feature image output by the current level coding layer in step S2 specifically includes:

[0075] The sub-image extraction process includes: dividing the image of each channel in the input image of the sub-image extraction module into multiple 2×2 blocks, extracting the pixel values ​​of the upper left corner, upper right corner, lower left corner, and lower right corner of each block, arranging them into four sub-images of half size, and placing the four sub-images into four channels; repeating the above process for the image of each channel in the input image to obtain a sub-image sequence of the input image;

[0076] The input image of the sub-image extraction module of the second level is the limited-view photoacoustic image, and the input image of the sub-image extraction modules of other levels is the sub-image sequence output by the sub-image extraction module of the previous level. Channel replication is used to ensure that the sub-image sequence has the same number of channels as the feature map output by the corresponding encoding layer.

[0077] The sub-image sequence has the same amount of information as the input image, but with a reduced size and an increased number of channels. The size is halved, meaning that the width of the sub-image is halved compared to the width of the input image, and the height of the sub-image is halved compared to the height of the input image.

[0078] The artifact suppression model (Sub4-UNet model) of the present invention, while retaining the concise structure of the U-Net model as much as possible, adds a sub-image extraction module based on the characteristics of effective information and redundant artifacts. Its extraction mode is as follows: Figure 3 shown.

[0079] like Figure 4 As shown in , since the Sub4-UNet model has 4 layers, the above sub-image extraction operation only needs three iterations. In order to match the number of channels in each layer of the Sub4-UNet model, after obtaining the sub-images of each iteration, it is necessary to perform certain replication operations on the limited-view photoacoustic image and the sub-images of the three sequences based on the required number of channels. The final result is shown in Figure 4 As shown in the dotted rectangle box.

[0080] Taking an input image with a width and height of 256 pixels as an example, the network structure of the Sub4-UNet model using bilinear interpolation as the upsampling method is as follows: Figure 4 As shown, Figure 4 The numbers above each rectangle indicate the number of image channels, and the numbers below the left indicate the image size. Because the input image, output image, and each intermediate image are all square images with equal width and height, only one size is indicated. Furthermore, the limited-view photoacoustic image and each sequence subimage shown in the dashed rectangle on the left have the same size and number of channels as the rectangle to its left, so the two information are combined. Figure 4 The number of channels in the middle left corner of the top layer is called the base channel number ( Figure 4 The base number of channels in is 64), and this value can be changed, thus affecting the complexity of the model.

[0081] from Figure 4As can be seen from the figure, the Sub4-UNet model is essentially the same as the original U-Net model, but with two improvements. The first improvement is that after double convolution on the left, the result is subtracted from the limited-view photoacoustic image or the extracted sub-image sequence. In order to be consistent with the number of channels of the feature map output by the coding layer at the corresponding level, the limited-view photoacoustic image and sub-image sequence are channel-copied. This is done to separate information from artifacts. Because the information is highly connected, after sub-image extraction, each sub-image can still retain structural information, but the artifacts are basically chaotic. This operation facilitates subsequent direct artifact suppression. The second improvement is that after upsampling on the right, in addition to merging with the original pre-encoded image of the same size, it is also merged with the corresponding limited-view photoacoustic image or sub-image sequence. This is done to gradually restore effective structural information layer by layer after artifact suppression.

[0082] The U-Net model includes an encoding path and a decoding path. The encoding path includes multiple encoding layers, and the decoding path includes multiple decoding layers. The encoding layers correspond to the decoding layers one by one and have the same number.

[0083] In one embodiment, step S3 constructs a loss function based on the structural similarity index and mean absolute error of the limited-view photoacoustic image and the corresponding full-view photoacoustic image, optimizes the artifact suppression model parameters, and selects the optimal artifact suppression model, specifically including:

[0084] Loss Function ;

[0085] SSIM is the structural similarity index:

[0086] ;

[0087] in, and Limited-view photoacoustic imaging and full-view photoacoustic imaging The average value of the local area, and are the variances of the local regions of x and y, is the covariance of the local region of x and y, and the parameter ,parameter ,in is the upper bound of the range of pixel values ​​of the photoacoustic image;

[0088] is the mean absolute error:

[0089] ;

[0090] Where n is the total number of pixels in the photoacoustic image, for The pixel value of the i-th pixel in , for The pixel value of the i-th pixel in .

[0091] The artifact suppression model parameters in step S3 are the parameters of the U-Net model on which it is based.

[0092] The parameter iteration method used in model training is the stochastic gradient descent method with momentum, where the learning rate is 0.01, the momentum parameter is 0.9, and the decay coefficient is 1×10 -4 The batch size is 2, the number of iterations is 100, and the early stopping method is used during the iteration process. That is, if the validation set error does not decrease or increases instead of decreasing during training, the training is stopped immediately, and the model parameters that minimize the validation set error are selected as the final parameters. After the model training is completed, the present invention uses the PSNR and SSIM indicators to evaluate the image post-processing results. The PSNR calculation formula is as follows:

[0093] ;

[0094] Among them, MSE is the mean square error, and its expression is:

[0095] .

[0096] In addition, the running time of the post-processing network for a single image is used as an evaluation indicator of the model reconstruction speed.

[0097] Next, we will analyze and discuss the effectiveness of the Sub4-UNet model in suppressing artifacts in limited-view photoacoustic images, using two examples: blood vessels and mouse embryos. To analyze the differences in the network's processing of dispersed structures (such as blood vessels) and continuous structures (such as mouse embryos), both examples utilize a linear array for limited-view observation and an annular array for full-view observation. The linear array contains 431 detectors, while the annular array contains 512 detectors.

[0098] In terms of dataset construction, both examples first simulate forward photoacoustic propagation of the original image using the k-Wave toolbox. The filtered back projection (FBP) algorithm is then used to reconstruct the limited-view photoacoustic image and the corresponding full-view photoacoustic image. In the vascular example, the constructed dataset consists of 28 images, 19 of which serve as training data, 4 as validation data, and the remaining 5 as test data. In the mouse embryo example, the selected dataset consists of 2878 images. The present invention selects the first 500 images, of which 350 serve as the training set, 75 as the validation set, and the remaining 75 as the test set. Furthermore, the width and height of the images in the examples are each 256 pixels.

[0099] In the case of dispersed blood vessels, the Sub4-UNet model processes an image in the blood vessel test set as follows: Figure 5 shown. Figure 5 (a) and (d) represent the limited-view photoacoustic image and the full-view photoacoustic image, respectively. Figure 5 (b), (c), (e), and (f) represent the post-processing results of the Sub4-UNet model on limited-view photoacoustic images at different complexities. Figure 5 The number in the brackets after Sub4-UNet represents the number of basic channels of the Sub4-UNet model.

[0100] As can be seen, even with only eight base channels, the resulting images are significantly improved compared to the original images. All models essentially suppress artifacts in the limited-view photoacoustic images and recover relevant information. Furthermore, as the model complexity increases, the resulting post-processed images become increasingly similar to the full-view photoacoustic images, although this effect becomes less pronounced towards the end.

[0101] The post-processing results of the Sub4-UNet model with different basic channel numbers for vascular limited-view photoacoustic images are shown in Table 1, where the mean and standard deviation of each value are given.

[0102] Table 1 Post-processing results of Sub4-UNet models with different numbers of basic channels for vascular limited-view photoacoustic images

[0103]

[0104] It can be seen that with Figure 5Consistent with the qualitative results in

[15] , the post-processing performance of the Sub4-UNet model for limited-view photoacoustic images gradually improves with increasing model complexity (i.e., the number of base channels). Numerically, the SSIM results are acceptable, but the PSNR results are somewhat below expectations. Even with a base channel number of 64, the PSNR value is only close to 20. In terms of runtime, the post-processing time of the Sub4-UNet model is generally short, and increases roughly linearly with model complexity, making it suitable for fast, real-time image reconstruction.

[0105] In the continuous mouse embryo example, the post-processing results of the Sub4-UNet model on an image in the mouse embryo test set are as follows: Figure 6 shown. Figure 6 (a) and (d) represent the limited-view photoacoustic image and the full-view photoacoustic image, respectively. Figure 6 (b), (c), (e), and (f) represent the post-processing results of the Sub4-UNet model on limited-view photoacoustic images at different complexities.

[0106] from Figure 6 As can be seen from the figure, the four Sub4-UNet models with different complexities all effectively suppress artifacts in the limited-view photoacoustic images of mouse embryos to a certain extent. As the complexity of the model increases, the resulting images become increasingly close to the full-view photoacoustic images, especially the two sub-images with 32 and 64 as the basis channels, which are almost the same as the full-view photoacoustic images, indicating that the Sub4-UNet model performs extremely well in continuous structures.

[0107] The quantitative results of post-processing of mouse embryo limited-view photoacoustic images by the Sub4-UNet model with different base channel numbers are shown in Table 2:

[0108] Table 2 Post-processing results of Sub4-UNet models with different numbers of basic channels for limited-view photoacoustic images of mouse embryos

[0109]

[0110] It can be seen that the change trend of the post-processing results of the Sub4-UNet model for the limited-view photoacoustic images of mouse embryos is similar to that of the vascular images, and the running time is basically the same as before. Unlike the post-processing results of blood vessels, the PSNR and SSIM values ​​of the post-processing results of mouse embryos are significantly higher, which is also consistent with the Figure 6 The post-processing results of the limited-view photoacoustic image are almost consistent with the full-view results.

[0111] In summary, the Sub4-UNet model is particularly suitable for continuous structural images such as mouse embryos. However, in dispersed structural images such as blood vessels, the Sub4-UNet model also has a great effect on suppressing artifacts in reconstructed images under limited viewing angles.

[0112] Based on the U-Net model structure, the present invention constructs a Sub4-UNet model through a multi-layer nested subgraph extraction method and information and artifact separation operation based on the continuity difference between effective information and redundant artifacts. This model suppresses the related artifacts in limited-viewing angle photoacoustic images and restores the structural information in the image as much as possible, realizing the rapid acquisition and application of high-quality photoacoustic images at low cost.

[0113] The main innovations of the present invention include: the layer-by-layer separation of information and artifacts by the Sub4-UNet model, and the degree of influence of the network structure on the post-processing results of dispersed and continuous artifact-containing photoacoustic images.

[0114] Post-processing of photoacoustic images obtained under limited viewing angles can be performed through other more complex network structures, such as FD-UNet. It can also be combined with intelligent algorithms such as generative adversarial networks and diffusion models to change the network complexity or parameter training method to obtain better post-processing results of limited viewing angle photoacoustic images.

[0115] It should be understood that although the steps in the flowcharts of the accompanying drawings are shown in sequence as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least some of the steps in the flowcharts of the accompanying drawings may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times. The order of execution of these steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a portion of the steps or stages in other steps.

[0116] Based on the description of the above method embodiments, the present invention also provides a system. The system can be a system that uses the software (application), module, component, server, client, etc. of the method described in the embodiments of this specification and is combined with the necessary implementation hardware. Based on the same innovative concept, the system in one or more embodiments provided by the embodiments of the present disclosure is as described in the following embodiments. Since the implementation scheme and method for solving the problem of the system are similar, the implementation of the specific system of the embodiments of this specification can refer to the implementation of the aforementioned method, and the repeated parts will not be repeated. As used below, the term "module" or "module" refers to a combination of software and / or hardware that can realize the predetermined function. Although the system described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceived.

[0117] A limited-viewing-angle photoacoustic image artifact suppression system comprising:

[0118] Dataset construction module: Generates reconstructed limited-view photoacoustic images and corresponding full-view photoacoustic images through image reconstruction algorithms;

[0119] Model construction module: A sub-image extraction module is added to each layer of the U-Net model except the first layer to extract a sub-image sequence with the same number of channels as the feature map output by the encoding layer of the current layer; an information separation operation is performed on the encoding layer output: the limited-view photoacoustic image is subtracted from the feature map output by the encoding layer of the first layer, and the sub-image sequence of the corresponding layer is subtracted from the feature map output by the encoding layer of each other layer to separate artifacts; a dual-path merging operation is performed at the decoding layer: the upsampled feature map is simultaneously merged with the feature map output by the encoding layer of the same layer, and the limited-view photoacoustic image or sub-image sequence;

[0120] Model training module: constructing a loss function based on the structural similarity index and mean absolute error of the limited-view photoacoustic image and the corresponding full-view photoacoustic image, optimizing the artifact suppression model parameters, and selecting the optimal artifact suppression model;

[0121] Prediction module: The limited-view photoacoustic image constructed based on the actual collected limited-view photoacoustic signal is input into the trained artifact suppression model, and the post-processed image after artifact suppression is output.

[0122] In one embodiment, the present invention provides a computer device, which may be a server. The computer device includes a processor, a memory, and a network interface connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device is used to store data used in the above method. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, the above method is implemented.

[0123] In one embodiment, the present invention further provides a computer-readable storage medium, such as a memory, comprising instructions, wherein the instructions are executable by a processor to perform the above method. The storage medium may be a computer-readable storage medium, such as a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, or optical data storage device.

[0124] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0125] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above and that the invention can be embodied in other specific forms without departing from the spirit or essential characteristics of the invention. Therefore, the embodiments should be considered in all respects as illustrative and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. It is intended that all variations within the meaning and range of equivalents of the claims be embraced herein, and any reference signs in the claims should not be construed as limiting the claims to which they relate.

[0126] In addition, it should be understood that although this specification is described in terms of implementation methods, not every implementation method contains only one independent technical solution. This narrative method of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment can also be appropriately combined to form other implementation methods that can be understood by those skilled in the art.

Claims

1. A method for suppressing artifacts in limited-viewing angle photoacoustic images, characterized in that: include: generating a photoacoustic image by an image reconstruction algorithm, wherein the photoacoustic image includes a limited-view photoacoustic image and a corresponding full-view photoacoustic image; Constructing an artifact suppression model: Adding a sub-image extraction module to each layer of the U-Net model except the first layer extracts a sub-image sequence with the same number of channels as the feature map output by the current layer's coding layer. Performing an information separation operation on the coding layer output: Subtracting the limited-view photoacoustic image from the feature map output by the first layer's coding layer, and subtracting the sub-image sequence of the corresponding layer from the feature map output by each other layer's coding layer to separate artifacts. A dual-path merging operation is performed at the decoding layer: the upsampled feature map is simultaneously merged with the feature map output by the encoding layer at the same level, as well as the limited-view photoacoustic image or sub-image sequence; constructing a loss function based on the structural similarity index and mean absolute error of the limited-view photoacoustic image and the corresponding full-view photoacoustic image, optimizing the artifact suppression model parameters, and selecting the optimal artifact suppression model; The limited-view photoacoustic image constructed based on the actually collected limited-view photoacoustic signal is input into the trained artifact suppression model, and the post-processed image after artifact suppression is output.

2. The method for suppressing artifacts in a limited-viewing-angle photoacoustic image according to claim 1, wherein: The generating of the reconstructed limited-viewing angle photoacoustic image and the corresponding full-viewing angle photoacoustic image by the image reconstruction algorithm specifically includes: Based on the k-Wave toolbox, a limited-view detector arrangement and a full-view detector arrangement are set for biological tissues. Under the same sound source and medium distribution, the forward propagation is simulated to obtain the limited-view photoacoustic signal and the full-view photoacoustic signal, respectively. Then, the image reconstruction algorithm is used for reverse solution to obtain the limited-view photoacoustic image reconstructed by the biological tissue and the corresponding full-view photoacoustic image, respectively.

3. The method for suppressing artifacts in a limited-viewing angle photoacoustic image according to claim 1, wherein: The method of adding a sub-image extraction module to each level of the U-Net model except the first level specifically includes: the input of the coding layer of the first level is the limited-view photoacoustic image; the input of the coding layers of other levels is the feature map output by the coding layer of the previous level minus the sub-image sequence output by the sub-image extraction module of the corresponding level.

4. The method for suppressing artifacts in a limited-viewing angle photoacoustic image according to claim 1, wherein: The extracting of a sub-image sequence having the same number of channels as the feature image output by the current level encoding layer specifically includes: The sub-image extraction process includes: dividing the image of each channel in the input image of the sub-image extraction module into multiple 2×2 blocks, extracting the pixel values ​​of the upper left corner, upper right corner, lower left corner, and lower right corner of each block, arranging them into four sub-images of half size, and placing the four sub-images into four channels; repeating the above process for the image of each channel in the input image to obtain a sub-image sequence of the input image; The input image of the sub-image extraction module of the second level is the limited-view photoacoustic image, and the input image of the sub-image extraction modules of other levels is the sub-image sequence output by the sub-image extraction module of the previous level. Channel replication is used to ensure that the sub-image sequence has the same number of channels as the feature map output by the corresponding encoding layer.

5. The method for suppressing artifacts in a limited-viewing-angle photoacoustic image according to claim 4, characterized in that: The channel replication is performed so that the sub-image sequence and the feature map output by the corresponding encoding layer have the same number of channels, specifically including: When the number of channels of the sub-image sequence is less than the feature map output by the encoding layer of the current level, the sub-image sequence and the feature map output by the corresponding encoding layer are made to have the same number of channels by copying the sub-images of other channels of the sub-image sequence.

6. The method for suppressing artifacts in a limited-viewing angle photoacoustic image according to claim 1, wherein: The method further comprises: constructing a loss function based on the structural similarity index and the mean absolute error of the limited-view photoacoustic image and the corresponding full-view photoacoustic image, optimizing the artifact suppression model parameters, and selecting the optimal artifact suppression model. Loss Function ; SSIM is the structural similarity index: ； in, and Limited-view photoacoustic imaging and full-view photoacoustic imaging The average value of the local area, and are the variances of the local regions of x and y, is the covariance of the local region of x and y, and the parameter ,parameter ,in is the upper bound of the range of pixel values ​​of the photoacoustic image; is the mean absolute error: ； in, is the total number of pixels of the photoacoustic image, for The pixel value of the i-th pixel in , for The pixel value of the i-th pixel in .

7. A limited-viewing-angle photoacoustic image artifact suppression system, characterized in that: include: Dataset construction module: generates photoacoustic images through image reconstruction algorithm, wherein the photoacoustic images include limited-view photoacoustic images and corresponding full-view photoacoustic images; Model construction module: A sub-image extraction module is added to each layer of the U-Net model except the first layer to extract a sub-image sequence with the same number of channels as the feature map output by the encoding layer of the current layer. An information separation operation is performed on the encoding layer output: the limited-view photoacoustic image is subtracted from the feature map output by the first encoding layer, and the sub-image sequence of the corresponding layer is subtracted from the feature map output by the encoding layer of each other layer to separate artifacts. A dual-path merging operation is performed at the decoding layer: the upsampled feature map is simultaneously merged with the feature map output by the encoding layer at the same level, as well as the limited-view photoacoustic image or sub-image sequence; Model training module: constructing a loss function based on the structural similarity index and mean absolute error of the limited-view photoacoustic image and the corresponding full-view photoacoustic image, optimizing the artifact suppression model parameters, and selecting the optimal artifact suppression model; Prediction module: The limited-view photoacoustic image constructed based on the actual collected limited-view photoacoustic signal is input into the trained artifact suppression model, and the post-processed image after artifact suppression is output.

8. The limited-viewing-angle photoacoustic image artifact suppression system according to claim 7, characterized in that: In the model training module, the loss function is constructed based on the structural similarity index and mean absolute error of the limited-view photoacoustic image and the corresponding full-view photoacoustic image, the artifact suppression model parameters are optimized, and the optimal artifact suppression model is selected, which specifically includes: Loss Function ; SSIM is the structural similarity index: ； in, and Limited-view photoacoustic imaging and full-view photoacoustic imaging The average value of the local area, and are the variances of the local regions of x and y, is the covariance of the local region of x and y, and the parameter ,parameter ,in is the upper bound of the range of pixel values ​​of the photoacoustic image; is the mean absolute error: ； Where n is the total number of pixels of the photoacoustic image, for The pixel value of the i-th pixel in , for The pixel value of the i-th pixel in .

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images