Limited-view-angle photoacoustic image artifact suppression method, system and device and medium

Abstract

The invention relates to the technical field of photoacoustic imaging, and discloses a limited-view-angle photoacoustic image artifact suppression method, system and device and a medium. The method comprises the following steps: generating a reconstructed limited-view-angle photoacoustic image and a corresponding full-view-angle photoacoustic image through an image reconstruction algorithm; adding a sub-graph extraction module in each hierarchy except the first hierarchy of the U-Net model, and extracting a sub-graph sequence; optimizing artifact suppression model parameters based on the loss function of the structural similarity index and the average absolute value error of the limited view angle photoacoustic image and the corresponding full view angle photoacoustic image, and selecting an optimal artifact suppression model; and inputting a limited view angle photoacoustic image constructed based on an actually collected limited view angle photoacoustic signal into the trained artifact suppression model, and outputting a post-processing image after artifact suppression. The method has a good post-processing result for the photoacoustic image with the limited visual angle, artifacts can be basically and effectively inhibited, and part of loss information in the image is recovered.

Description

Technical Field

[0001] The present invention relates to the technical field of photoacoustic imaging, and particularly to a method for suppressing photoacoustic image artifacts with a limited view angle. Background Art

[0002] Photoacoustic Tomography (PAT) is a new non-invasive medical imaging modality that combines high optical contrast and high acoustic penetration depth. Photoacoustic imaging uses a pulsed laser to irradiate biological tissue, causing the tissue to heat up and expand, and generating ultrasonic waves that propagate outward. By using an ultrasonic detector array to receive the photoacoustic wave signals, the internal fine structure of living tissue can be reconstructed based on image reconstruction algorithms.

[0003] According to the different imaging dimensions, photoacoustic imaging can be divided into two-dimensional photoacoustic imaging and three-dimensional photoacoustic imaging. Ideal two-dimensional photoacoustic imaging uses a sufficiently dense full-view annular detection array to cover the imaging object to receive the photoacoustic signals emitted by the object in all directions within the plane. However, in actual situations, limited by factors such as manufacturing costs and imaging time, it may be difficult to achieve this ideal imaging condition of full-view imaging, and thus photoacoustic imaging can only be performed based on a limited view angle or a sparse view angle. The photoacoustic images formed under a limited view angle may be distorted, generating redundant structures such as negative value artifacts, stripe artifacts, and split artifacts, which affect the imaging quality and spatial resolution and cause serious troubles in disease diagnosis. Therefore, suppressing the artifacts in the photoacoustic reconstruction images with a limited view angle and recovering the biological structure information from its blurred images is a research field worthy of in-depth exploration.

[0004] Deep learning algorithms establish a mapping relationship between input and output based on multi-layer neural networks. Therefore, by optimizing a specific objective function, a post-processing network can be constructed between the limited-view photoacoustic images and the full-view photoacoustic images, thereby removing the redundant artifacts in the images, recovering the detailed information, and achieving high-quality photoacoustic imaging with low-cost imaging devices. The U-Net model based on multi-layer convolutional neural networks is one of the currently simple and effective image post-processing models.

[0005] The U-Net model can perform various post-processing operations on images, including image segmentation, artifact suppression, etc. The U-Net model first increases the number of image channels proportionally through convolution operations, and at the same time reduces the image size proportionally through pooling operations. This process is called the encoding operation of the image. The encoded image then undergoes upsampling and convolution operations, and is merged with the pre-encoded image of the same size to restore the image size and reduce the number of channels to the original. This process is called the decoding operation of the image. Finally, the decoded image undergoes convolution and non-linear operations to obtain the final output image.

[0006] The U-Net model is highly versatile and can handle various types of practical problems, but it may not necessarily be seamlessly integrated with each problem. For example, in the suppression of artifacts in limited-view photoacoustic images, there are currently no relevant patents that specifically address this issue by combining the U-Net model. In the suppression of artifacts in other application fields, although there are some post-processing structure designs based on the U-Net model, some need to be combined with other algorithms, and some make too many changes to the U-Net model. Although the final results have greatly improved in terms of reconstruction quality, the original fast responsiveness of the U-Net model is lost.

[0007] Specific application problems need to be analyzed specifically in combination with specific application fields. In the suppression of artifacts in limited-view photoacoustic images, the distinctiveness between information and artifacts can be achieved based on the characteristics that the artifacts in the image are chaotic while the information has strong continuity. Summary of the Invention

[0008] To solve the above technical problems, the present invention provides a method, system, device, and medium for suppressing artifacts in limited-view photoacoustic images. Based on the self-designed improved U-Net model structure, namely the Sub4-UNet model, the present invention performs artifact suppression processing on photoacoustic images under limited views and evaluates the influence degree of model complexity on the post-processing results of images.

[0009] To solve the above technical problems, the present invention adopts the following technical solutions: In a first aspect, the present invention provides a method for suppressing artifacts in limited-view photoacoustic images, including: Generating photoacoustic images through an image reconstruction algorithm, where the photoacoustic images include limited-view photoacoustic images and corresponding full-view photoacoustic images; Constructing an artifact suppression model: adding a sub-graph extraction module to each layer of the U-Net model except the first layer to extract a sub-graph sequence with the same number of channels as the feature map output by the encoding layer of the current layer; performing an information separation operation on the output of the encoding layer: subtracting the limited-view photoacoustic image from the feature map output by the encoding layer of the first layer, and subtracting the sub-graph sequence of the corresponding layer from the feature map output by the encoding layer of each other layer to separate the artifacts; performing a dual-path merging operation in the decoding layer: the upsampled feature map is merged with the feature map output by the encoding layer of the same layer, as well as the limited-view photoacoustic image or the sub-graph sequence; Constructing a loss function based on the structural similarity index and mean absolute error between the limited-view photoacoustic image and the corresponding full-view photoacoustic image, optimizing the parameters of the artifact suppression model, and selecting the optimal artifact suppression model; Inputting the limited-view photoacoustic image constructed based on the actually collected limited-view photoacoustic signals into the trained artifact suppression model, and outputting the post-processed image after artifact suppression.

[0010] In one embodiment, generating the reconstructed limited-view photoacoustic image and the corresponding full-view photoacoustic image through the image reconstruction algorithm specifically includes: Based on the k-Wave toolbox, set the detector arrangement for the limited view and the detector arrangement for the full view for the biological tissue. Under the same sound source and medium distribution, through forward propagation simulation, obtain the limited-view photoacoustic signal and the full-view photoacoustic signal respectively, and then perform backpropagation through the image reconstruction algorithm to obtain the reconstructed limited-view photoacoustic image of the biological tissue and the corresponding full-view photoacoustic image respectively.

[0011] In one embodiment, adding a subgraph extraction module to each layer of the U-Net model except the first layer specifically includes: the input of the encoding layer of the first layer is the limited-view photoacoustic image; the input of the encoding layers of other layers is the feature map output by the encoding layer of the previous layer minus the subgraph sequence output by the subgraph extraction module of the corresponding layer.

[0012] In one embodiment, extracting the subgraph sequence with the same number of channels as the feature map output by the encoding layer of the current layer specifically includes: The process of subgraph extraction includes: dividing the image of each channel in the input image of the subgraph extraction module into multiple 2×2 tiles, respectively taking out the pixel values in the upper left, upper right, lower left, and lower right corners of each tile, arranging them into four subgraphs with halved sizes, and putting the four subgraphs into four channels; repeating the above process for the image of each channel in the input image to obtain the subgraph sequence of the input image; The input image of the subgraph extraction module of the second layer is the limited-view photoacoustic image, and the input images of the subgraph extraction modules of other layers are the subgraph sequences output by the subgraph extraction modules of the previous layer, and the subgraph sequence is made to have the same number of channels as the feature map output by the corresponding encoding layer through channel replication.

[0013] In one embodiment, making the subgraph sequence have the same number of channels as the feature map output by the corresponding encoding layer through channel replication specifically includes: When the number of channels of the subgraph sequence is less than the number of channels of the feature map output by the encoding layer of the current layer, replicate the subgraphs of other channels of the subgraph sequence to make the subgraph sequence have the same number of channels as the feature map output by the corresponding encoding layer.

[0014] In one embodiment, constructing a loss function based on the structural similarity index and the mean absolute error between the limited-view photoacoustic image and the corresponding full-view photoacoustic image, optimizing the parameters of the artifact suppression model, and selecting the optimal artifact suppression model specifically includes: Loss function ; The SSIM is the Structural Similarity Index: ; Among them, and are respectively the average values of the local regions of the limited-view photoacoustic image and the full-view photoacoustic image ; and are respectively the variances of the local regions of x and y, is the covariance of the local regions of x and y, and the parameter , the parameter , where is the upper bound of the range of the pixel values of the photoacoustic image; is the Mean Absolute Error: ; Among them, is the total number of pixels of the photoacoustic image, is the pixel value of the i-th pixel in is the pixel value of the i-th pixel in

[0015] In a second aspect, the present invention provides a system for suppressing artifacts in limited-view photoacoustic images, including: Dataset construction module: generating a photoacoustic image through an image reconstruction algorithm, where the photoacoustic image includes a limited-view photoacoustic image and a corresponding full-view photoacoustic image; Model construction module: adding a subgraph extraction module in each layer of the U-Net model except the first layer to extract a subgraph sequence with the same number of channels as the feature map output by the encoding layer of the current layer; performing an information separation operation on the output of the encoding layer: subtracting the limited-view photoacoustic image from the feature map output by the encoding layer of the first layer, and subtracting the subgraph sequence of the corresponding layer from the feature map output by the encoding layer of each other layer to separate out the artifacts; performing a dual-path merging operation in the decoding layer: the upsampled feature map is merged with the feature map output by the encoding layer of the same layer, and the limited-view photoacoustic image or the subgraph sequence at the same time; Model training module: constructing a loss function based on the structural similarity index and the mean absolute error between the limited-view photoacoustic image and the corresponding full-view photoacoustic image, optimizing the parameters of the artifact suppression model, and selecting the optimal artifact suppression model; Prediction module: inputting the limited-view photoacoustic image constructed based on the actually collected limited-view photoacoustic signal into the trained artifact suppression model, and outputting the post-processed image after artifact suppression.

[0016] In one embodiment, in the model training module, a loss function is constructed based on the structural similarity index and the mean absolute error between the limited-view photoacoustic image and the corresponding full-view photoacoustic image to optimize the parameters of the artifact suppression model and select the optimal artifact suppression model. Specifically, it includes: Loss function ; SSIM is the structural similarity index: ; where, and are the average values of the local regions of the limited-view photoacoustic image and the full-view photoacoustic image respectively, and are the variances of the local regions of x and y respectively, is the covariance of the local regions of x and y, the parameter , the parameter , where is the upper bound of the range of the pixel values of the photoacoustic image; is the mean absolute error: ; where, n is the total number of pixels of the photoacoustic image, is the pixel value of the i-th pixel in is the pixel value of the i-th pixel in

[0017] In a third aspect, the present invention provides a computer device, including a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the steps of the method according to any one of the embodiments of the first aspect are implemented.

[0018] In a fourth aspect, the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method according to any one of the embodiments of the first aspect are implemented.

[0019] Compared with the prior art, the beneficial technical effects of the present invention are: Building on the existing U-Net model, this invention preserves the original structure as much as possible to inherit the U-Net model's efficiency and speed. Furthermore, for the specific field of artifact suppression in photoacoustic images under limited viewing angles, based on the poor continuity of artifacts, the artifact-containing image is decomposed into a sequence of sub-images with essentially continuous information but multiple artifacts by segmenting the image and extracting pixels at each corner point. This allows for multiple separations of information from artifacts during image post-processing. Artifacts are then processed separately, and finally, effective information is added layer by layer to obtain the post-processed image. This organic combination of a general network structure and specific field requirements enables the constructed Sub4-UNet model to achieve more comprehensive post-processing results at a lower cost.

[0020] In terms of image reconstruction quality, the present invention has achieved good numerical results, both based on the PSNR indicator for measuring reconstruction error and the SSIM indicator for measuring the structural similarity between the reconstructed image and the original image. In particular, on the mouse embryo dataset with strong continuity, both PSNR and SSIM results reached excellent levels, indicating that the present invention can obtain high-quality reconstructed images under limited viewing angles.

[0021] In terms of image reconstruction time, due to the simple network structure and fewer parameters, the present invention can process limited-viewing angle photoacoustic images at an extremely fast speed of the order of 10 milliseconds, obtain high-quality post-processed images, and realize real-time imaging.

[0022] Overall, the Sub4-UNet model achieves excellent post-processing results for limited-view photoacoustic images, effectively suppressing artifacts and restoring some lost information. Furthermore, the Sub4-UNet model performs significantly better for continuous structures than for dispersed ones. Finally, the Sub4-UNet model also excels in reconstruction speed, making it suitable for image post-processing in real-time imaging. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 is a flow chart of a method in an embodiment of the present invention; Figure 2 Schematic diagram of model training and model prediction in an embodiment of the present invention; Figure 3 Schematic diagram of the working principle of the sub-graph extraction module in an embodiment of the present invention; Figure 4 Schematic diagram of the structure of the Sub4-UNet model in an embodiment of the present invention; Figure 5 Graphs showing the post-processing effects of Sub4-UNet models of different complexities on limited-view photoacoustic images of blood vessels according to an embodiment of the present invention; Figure 6This is the post - processing effect diagram of the Sub4 - UNet model with different complexities in the embodiments of the present invention for the limited - view photoacoustic images of mouse embryos. Detailed implementation manners

[0024] The following is a detailed description of a preferred implementation manner of the present invention with reference to the accompanying drawings.

[0025] As Figure 1 shown, the present invention provides a method for suppressing artifacts in limited - view photoacoustic images, including the following steps: S1. Generate photoacoustic images through an image reconstruction algorithm, where the photoacoustic images include limited - view photoacoustic images and corresponding full - view photoacoustic images; S2. Construct an artifact suppression model: Add a sub - map extraction module to each layer of the U - Net model except the first layer to extract a sub - map sequence with the same number of channels as the feature map output by the encoding layer of the current layer; Perform an information separation operation on the output of the encoding layer: Subtract the limited - view photoacoustic image from the feature map output by the encoding layer of the first layer, and subtract the sub - map sequence of the corresponding layer from the feature map output by the encoding layer of each other layer to separate the artifacts; Perform a dual - path merging operation on the decoding layer: The up - sampled feature map is merged with the feature map output by the encoding layer of the same layer, as well as the limited - view photoacoustic image or the sub - map sequence; S3. Construct a loss function based on the structural similarity index and the mean absolute error between the limited - view photoacoustic image and the corresponding full - view photoacoustic image, optimize the parameters of the artifact suppression model, and select the optimal artifact suppression model; S4. Input the limited - view photoacoustic image constructed based on the actually collected limited - view photoacoustic signals into the trained artifact suppression model to output the post - processed image after artifact suppression.

[0026] The operation process of the present invention is as Figure 2 shown, which can be divided into two stages: model training and model prediction.

[0027] In the model training stage, first, based on the original biological tissue images, a finite-view and full-view photoacoustic imaging simulation model with all other conditions being exactly the same is constructed using the k-Wave toolbox to perform the forward propagation of photoacoustic signals. After obtaining the photoacoustic signals, the initial reconstructed photoacoustic images in the finite-view and full-view are obtained through traditional image reconstruction algorithms as the training dataset. Second, the constructed training dataset is imported into the PyTorch deep learning framework to build the artifact suppression model (Sub4-UNet model) in the present invention, which can also be called the post-processing network. Select the corresponding basic number of channels to adjust the model complexity and establish the mapping relationship between the finite-view photoacoustic image and the full-view photoacoustic image. Finally, based on the objective function, all the parameters in the artifact suppression model are iteratively optimized, and according to the reconstruction quality and reconstruction time, the artifact suppression model under the optimal complexity is selected as the final post-processing network for the photoacoustic image under finite-view conditions.

[0028] In the model prediction stage, first, the photoacoustic signals of the finite-view under the same imaging conditions are obtained through a photoacoustic imaging device. Second, the photoacoustic image in the finite-view is obtained through a traditional image reconstruction algorithm. Finally, the image is substituted into the artifact suppression model with optimized parameters to obtain the post-processed image with suppressed artifacts.

[0029] In one embodiment, generating the reconstructed finite-view photoacoustic image and the corresponding full-view photoacoustic image through the image reconstruction algorithm in step S1 specifically includes: Based on the k-Wave toolbox, the detector arrangement methods of the finite-view and the full-view are set for the biological tissue. Under the same sound source and medium distribution, through the simulation of forward propagation, the photoacoustic signals of the finite-view and the full-view are respectively obtained, and then through the image reconstruction algorithm for back-solving, the reconstructed finite-view photoacoustic image of the biological tissue and the corresponding full-view photoacoustic image are respectively obtained.

[0030] In the forward problem of photoacoustic signal propagation, given the initial photoacoustic pressure at the photoacoustic source , then the photoacoustic pressure at any position at time t is: ; where is the sound speed of the medium. The forward propagation problem in photoacoustic imaging is based on the above formula, and by setting conditions such as the sound source, medium, and detector distribution, the photoacoustic signals detected by each detector within a period of time are obtained.

[0031] The process of reconstructing the initial photoacoustic source image from the detected photoacoustic signal is called an inverse problem. Currently, there is no unified solution to this problem, but there are some traditional photoacoustic reconstruction algorithms, such as Delay and Sum (DAS), Filtered Back Projection (FBP), Series Expansion (SE), Time Reversal (TR), and Iterative Reconstruction (IR), etc.

[0032] By setting two detector arrangements, namely the limited-view and full-view arrangements, under the same sound source and medium distribution, the photoacoustic images reconstructed for biological tissues under the limited-view and full-view can be obtained through two steps: simulating forward propagation and solving the inverse problem, thereby constructing a dataset.

[0033] In one embodiment, adding a sub-graph extraction module to each layer of the U-Net model except the first layer in step S2 specifically includes: the input of the encoding layer of the first layer is the limited-view photoacoustic image; the input of the encoding layers of other layers is the feature map output by the encoding layer of the previous layer minus the sub-graph sequence output by the sub-graph extraction module of the corresponding layer.

[0034] In one embodiment, extracting a sub-graph sequence with the same number of channels as the feature map output by the encoding layer of the current layer in step S2 specifically includes: The process of sub-graph extraction includes: dividing the image of each channel in the input image of the sub-graph extraction module into multiple 2×2 tiles, respectively taking out the pixel values at the upper left, upper right, lower left, and lower right corners of each tile, arranging them into four sub-graphs with halved sizes, and putting the four sub-graphs into four channels; repeating the above process for the image of each channel in the input image to obtain the sub-graph sequence of the input image; The input image of the sub-graph extraction module of the second layer is the limited-view photoacoustic image, and the input images of the sub-graph extraction modules of other layers are the sub-graph sequences output by the sub-graph extraction modules of the previous layer, and the sub-graph sequences are made to have the same number of channels as the feature maps output by the corresponding encoding layers through channel replication.

[0035] The sub-graph sequence has the same amount of information as the input image, but with reduced size and increased number of channels. Reducing the size by half means that the width of the sub-graph is halved compared to the width of the input image, and the height of the sub-graph is halved compared to the height of the input image.

[0036] The artifact suppression model (Sub4-UNet model) of the present invention, while trying to retain the simple structure of the U-Net model, adds a sub-graph extraction module according to the characteristics of effective information and redundant artifacts, and its extraction mode is as Figure 3 shown.

[0037] As Figure 4As shown, since the Sub4-UNet model has 4 layers, the above sub-graph extraction operation only needs to be iterated three times. In order to match the number of channels in each layer of the Sub4-UNet model, after obtaining the sub-graphs of each iteration, it is also necessary to perform a certain replication operation on the limited-view photoacoustic image and the sub-graphs of the three sequences based on the required number of channels. The final result is as shown in Figure 4 the dashed rectangular box in.

[0038] Taking an input image with a width and height of 256 pixels as an example, the network structure of the Sub4-UNet model using bilinear interpolation as the upsampling method is as shown in Figure 4 shown, Figure 4 The numbers above each rectangle in are the number of image channels, and the numbers in the lower left corner are the image sizes. Since the input image, output image, and each intermediate image are all square images with equal width and height, only one size is marked. In addition, the sizes and channel numbers of the limited-view photoacoustic image and the sub-graphs of each sequence shown in the left dashed rectangular box are the same as those of the rectangle on its left, so the information of the two is combined into one place. Figure 4 The number of channels in the middle on the left of the topmost layer is called the base number of channels ( Figure 4 the base number of channels in is 64), and this value can be changed, thereby affecting the model complexity.

[0039] As can be seen from Figure 4 , the Sub4-UNet model is generally the same as the original U-Net model, but there are two improvements. The first improvement is that after double convolution on the left, the obtained result will subtract the limited-view photoacoustic image or the extracted sub-graph sequence. Among them, in order to be consistent with the number of channels of the feature map output by the encoding layer of the corresponding level, the limited-view photoacoustic image and the sub-graph sequence are copied in channels. This is to separate the information from the artifacts. Because the information has strong connectivity, after sub-graph extraction, each sub-graph can still retain the structural information, but the artifacts are basically chaotic, so such an operation can facilitate the subsequent direct suppression of the artifacts. The second improvement is that after upsampling on the right, in addition to merging with the pre-encoding image of the original size, it will also be merged with the corresponding limited-view photoacoustic image or sub-graph sequence. This is to gradually restore the effective structural information layer by layer after artifact suppression.

[0040] The U-Net model includes an encoding path and a decoding path. The encoding path includes multiple encoding layers, and the decoding path includes multiple decoding layers. The encoding layers and the decoding layers correspond one by one and have the same number.

[0041] In one embodiment, constructing a loss function based on the structural similarity index and the mean absolute error between the limited-view photoacoustic image and the corresponding full-view photoacoustic image in step S3, optimizing the parameters of the artifact suppression model, and selecting the optimal artifact suppression model specifically includes: Loss Function ; SSIM is the structural similarity index: ; in, and Limited-view photoacoustic imaging and full-view photoacoustic imaging The average value of the local area, and are the variances of the local regions of x and y, is the covariance of the local region of x and y, and the parameter ,parameter ,in is the upper bound of the range of pixel values of the photoacoustic image; is the mean absolute error: ; Where n is the total number of pixels of the photoacoustic image, for The pixel value of the i-th pixel in , for The pixel value of the i-th pixel in .

[0042] The artifact suppression model parameters in step S3 are the parameters of the U-Net model on which it is based.

[0043] The parameter iteration method used in model training is the stochastic gradient descent method with momentum, where the learning rate is 0.01, the momentum parameter is 0.9, and the decay coefficient is 1×10 -4 The batch size is 2, the number of iterations is 100, and the early stopping method is used during the iteration process. That is, if the validation set error does not decrease or increases instead of decreasing during training, the training is stopped immediately, and the model parameters that minimize the validation set error are selected as the final parameters. After the model training is completed, the present invention uses the PSNR and SSIM indicators to evaluate the image post-processing results. The PSNR calculation formula is as follows: ; Among them, MSE is the mean square error, and its expression is: .

[0044] In addition, the running time of the post-processing network for a single image is used as an evaluation indicator of the model reconstruction speed.

[0045] Next, based on two examples of blood vessels and mouse embryos, the present invention will analyze and discuss the effect of the Sub4-UNet model on suppressing artifacts in limited-view photoacoustic images. To analyze the differences in the processing results of the network for dispersed structures (such as blood vessels) and continuous structures (such as mouse embryos), both examples use a linear array as the observation array for limited views and a circular array as the observation array for full views. Among them, the linear array contains 431 detectors, and the circular array contains 512 detectors.

[0046] In terms of dataset construction, both examples first perform forward photoacoustic propagation simulation on the original images based on the k-Wave toolbox, and obtain the reconstructed limited-view photoacoustic images and corresponding full-view photoacoustic images based on the filtered back-projection (FBP) algorithm. In the blood vessel example, the constructed dataset has a total of 28 images, of which 19 are used as the training dataset, 4 are used as the validation dataset, and the remaining 5 are used as the test dataset. In the mouse embryo example, the selected dataset has a total of 2878 images. The present invention selects the first 500 images, of which 350 are used as the training set, 75 are used as the validation set, and the remaining 75 are used as the test set. In addition, the width and height of the images in the examples are each 256 pixels.

[0047] In the dispersed blood vessel example, the processing results of the Sub4-UNet model for a certain image in the blood vessel test set are as Figure 5 shown. Figure 5 In (a) and (d) of [], they represent the limited-view photoacoustic image and the full-view photoacoustic image respectively, Figure 5 In (b), (c), (e), and (f) of [], they represent the post-processing results of the Sub4-UNet model for the limited-view photoacoustic image under different complexities respectively, Figure 5 The numbers in the brackets after Sub4-UNet in [] represent the basic number of channels of the Sub4-UNet model.

[0048] It can be seen that even with only 8 basic channels, the obtained images are greatly improved compared with the original images. All models basically suppress the artifacts in the limited-view photoacoustic images and recover the relevant information. Moreover, as the complexity of the model increases, the obtained post-processed images are getting closer and closer to the full-view photoacoustic images, although this approaching effect becomes less and less obvious later.

[0049] The post-processing results of the Sub4-UNet model for the limited-view photoacoustic images of blood vessels under different basic numbers of channels are shown in Table 1, where each value gives its mean and standard deviation.

[0050] Table 1 Post-processing results of the Sub4-UNet model for the limited-view photoacoustic images of blood vessels under different basic numbers of channels

[0051] It can be seen that, consistent with the qualitative results in Figure 5 , as the model complexity, i.e., the number of base channels, increases, the post - processing effect of the Sub4 - UNet model on limited - view photoacoustic images is gradually improved. From the specific numerical values, the numerical results of SSIM are acceptable, but the numerical results of PSNR are somewhat lower than expected. Even when 64 is finally used as the number of base channels, the value of PSNR is only close to 20. In terms of running time, the post - processing time of the Sub4 - UNet model is generally short and basically shows a linear growth trend with the increase of model complexity, which can be used for fast real - time image reconstruction.

[0052] In the continuous mouse embryo example, the post - processing results of a certain image in the mouse embryo test set by the Sub4 - UNet model are as Figure 6 shown. Figure 6 In Figure 6 , (a) and (d) represent the limited - view photoacoustic image and the full - view photoacoustic image respectively, and in Figure 6 , (b), (c), (e), and (f) represent the post - processing results of the Sub4 - UNet model on the limited - view photoacoustic image under different complexities respectively.

[0053] The quantitative results of the post - processing of the Sub4 - UNet model with different numbers of base channels on the limited - view photoacoustic image of the mouse embryo are shown in Table 2: Table 2 Post - processing results of the Sub4 - UNet model with different numbers of base channels on the limited - view photoacoustic image of the mouse embryo

[0054] It can be seen that the changing trend of the post - processing results of the Sub4 - UNet model on the limited - view photoacoustic image of the mouse embryo is similar to that of the blood vessel image, and the running time is basically the same as before. Different from the post - processing results of blood vessels, the PSNR and SSIM values of the post - processing results of the mouse embryo are significantly higher, which also corresponds to the fact that the post - processing results of the limited - view photoacoustic image in Figure 6 are almost the same as the full - view results.

[0055] In summary, the Sub4-UNet model is particularly suitable for continuous structural images such as mouse embryos. However, in dispersed structural images such as blood vessels, the Sub4-UNet model also has a great effect on suppressing artifacts in reconstructed images under limited viewing angles.

[0056] Based on the U-Net model structure, the present invention constructs a Sub4-UNet model through a multi-layer nested subgraph extraction method and information and artifact separation operation based on the continuity difference between effective information and redundant artifacts. This model suppresses the related artifacts in limited-viewing angle photoacoustic images and restores the structural information in the image as much as possible, realizing the rapid acquisition and application of high-quality photoacoustic images at low cost.

[0057] The main innovations of the present invention include: the layer-by-layer separation of information and artifacts by the Sub4-UNet model, and the degree of influence of the network structure on the post-processing results of dispersed and continuous artifact-containing photoacoustic images.

[0058] Post-processing of photoacoustic images obtained under limited viewing angles can be performed through other more complex network structures, such as FD-UNet. It can also be combined with intelligent algorithms such as generative adversarial networks and diffusion models to change the network complexity or parameter training method to obtain better post-processing results of limited viewing angle photoacoustic images.

[0059] It should be understood that although the steps in the flowcharts of the accompanying drawings are shown in sequence as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least some of the steps in the flowcharts of the accompanying drawings may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times. The order of execution of these steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a portion of the steps or stages in other steps.

[0060] Based on the description of the above method embodiments, the present invention also provides a system. The system may be a software (application), module, component, server, client, etc. that uses the method described in the embodiments of this specification and combines the necessary implementation hardware. Based on the same innovative concept, the systems in one or more embodiments provided by the present disclosure are as described in the following embodiments. Since the implementation solutions for the system to solve problems are similar to those of the method, the implementation of the specific system in the embodiments of this specification can refer to the implementation of the foregoing method, and repeated parts will not be elaborated. As used hereinafter, the term "module" or "modular unit" is a combination of software and / or hardware that can implement a predetermined function. Although the systems described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware is also possible and contemplated.

[0061] A limited-view photoacoustic image artifact suppression system, comprising: A dataset construction module: generating a reconstructed limited-view photoacoustic image and a corresponding full-view photoacoustic image through an image reconstruction algorithm; A model construction module: adding a sub-graph extraction module to each layer of the U-Net model except the first layer, and extracting a sequence of sub-graphs having the same number of channels as the feature map output by the encoding layer of the current layer; performing an information separation operation on the output of the encoding layer: subtracting the limited-view photoacoustic image from the feature map output by the encoding layer of the first layer, and subtracting the sequence of sub-graphs of the corresponding layer from the feature map output by the encoding layer of each other layer to separate the artifacts; performing a dual-path merging operation on the decoding layer: the upsampled feature map is merged with the feature map output by the encoding layer of the same layer, as well as the limited-view photoacoustic image or the sequence of sub-graphs; A model training module: constructing a loss function based on the structural similarity index and the mean absolute error between the limited-view photoacoustic image and the corresponding full-view photoacoustic image, optimizing the parameters of the artifact suppression model, and selecting the optimal artifact suppression model; A prediction module: inputting the limited-view photoacoustic image constructed based on the actually collected limited-view photoacoustic signals into the trained artifact suppression model, and outputting a post-processed image after artifact suppression.

[0062] In one embodiment, the present invention provides a computer device, which may be a server. The computer device includes a processor, a memory, and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store the data used in the above method. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the above method is implemented.

[0063] In one embodiment, the present invention also provides a computer-readable storage medium including instructions, such as a memory including instructions, and the above instructions can be executed by a processor to complete the above method. The storage medium may be a computer-readable storage medium. For example, the computer-readable storage medium may be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device, etc.

[0064] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.

[0065] For those skilled in the art, it is obvious that the present invention is not limited to the details of the above exemplary embodiments, and without departing from the spirit or basic characteristics of the present invention, the present invention can be implemented in other specific forms. Therefore, in any aspect, the embodiments should be regarded as exemplary and non-limiting. The scope of the present invention is defined by the appended claims rather than the above description. Therefore, it is intended to embrace all changes within the meaning and scope of the equivalent elements of the claims in the present invention, and any reference signs in the claims should not be regarded as limiting the claimed rights involved.

[0066] In addition, it should be understood that although this specification is described according to embodiments, not every embodiment only contains an independent technical solution. This narrative way of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A method for suppressing artifacts in limited-view photoacoustic images, characterized in that Including: Generating a photoacoustic image through an image reconstruction algorithm, where the photoacoustic image includes a limited-view photoacoustic image and a corresponding full-view photoacoustic image; Constructing an artifact suppression model: adding a sub-image extraction module to each layer of the U-Net model except the first layer to extract a sub-image sequence with the same number of channels as the feature map output by the encoding layer of the current layer; performing an information separation operation on the output of the encoding layer: subtracting the limited-view photoacoustic image from the feature map output by the encoding layer of the first layer, and subtracting the sub-image sequence of the corresponding layer from the feature map output by the encoding layer of each other layer to separate the artifacts; Performing a dual-path merging operation in the decoding layer: the upsampled feature map is merged with the feature map output by the encoding layer of the same layer, as well as the limited-view photoacoustic image or the sub-image sequence; Constructing a loss function based on the structural similarity index and the mean absolute error between the limited-view photoacoustic image and the corresponding full-view photoacoustic image, optimizing the parameters of the artifact suppression model, and selecting the optimal artifact suppression model; Inputting the limited-view photoacoustic image constructed based on the actually collected limited-view photoacoustic signals into the trained artifact suppression model to output a post-processed image after artifact suppression.

2. The method for suppressing artifacts in a limited-view photoacoustic image according to claim 1, wherein The generating of the reconstructed limited-view photoacoustic image and the corresponding full-view photoacoustic image through the image reconstruction algorithm specifically includes: Based on the k-Wave toolbox, setting the detector arrangement methods for the limited view and the full view for the biological tissue. Under the same sound source and medium distribution, by simulating the forward propagation, the limited-view photoacoustic signals and the full-view photoacoustic signals are respectively obtained, and then through the image reconstruction algorithm for back-propagation solution, the reconstructed limited-view photoacoustic image of the biological tissue and the corresponding full-view photoacoustic image are respectively obtained.

3. A method for suppressing artifacts in limited-view photoacoustic images according to claim 1, characterized in that, The adding of the sub-image extraction module to each layer of the U-Net model except the first layer specifically includes: the input of the encoding layer of the first layer is the limited-view photoacoustic image; the input of the encoding layer of other layers is the feature map output by the encoding layer of the previous layer minus the sub-image sequence output by the sub-image extraction module of the corresponding layer.

4. A method for suppressing artifacts in limited-view photoacoustic images according to claim 1, characterized in that, The extracting of the sub-image sequence with the same number of channels as the feature map output by the encoding layer of the current layer specifically includes: The process of sub-image extraction includes: dividing the image of each channel in the input image of the sub-image extraction module into multiple 2×2 tiles, respectively taking out the pixel values in the upper left, upper right, lower left, and lower right corners of each tile, arranging them into four sub-images with halved sizes, and putting the four sub-images into four channels; repeating the above process for the image of each channel in the input image to obtain the sub-image sequence of the input image; The input image of the sub-image extraction module of the second layer is the limited-view photoacoustic image, and the input image of the sub-image extraction module of other layers is the sub-image sequence output by the sub-image extraction module of the previous layer, and the sub-image sequence is made to have the same number of channels as the feature map output by the corresponding encoding layer through channel replication.

5. A method for suppressing artifacts in limited-view photoacoustic images according to claim 4, characterized in that, The making of the sub-image sequence have the same number of channels as the feature map output by the corresponding encoding layer through channel replication specifically includes: When the number of channels of the sub - graph sequence is less than the number of channels of the feature map output by the encoding layer of the current level, duplicate the sub - graphs of other channels of the sub - graph sequence to make the sub - graph sequence have the same number of channels as the feature map output by the corresponding encoding layer.

6. A method for suppressing artifacts in limited-view photoacoustic images according to claim 1, characterized in that, Constructing a loss function based on the structural similarity index and the mean absolute error between the limited - view photoacoustic image and the corresponding full - view photoacoustic image, optimizing the parameters of the artifact suppression model, and selecting the optimal artifact suppression model specifically include: Loss function ; SSIM is the structural similarity index: ； Among them, and are the average values of local regions of the limited-view photoacoustic image and the full-view photoacoustic image respectively, and are the variances of local regions of x and y respectively, is the covariance of local regions of x and y, and the parameter , the parameter , where is the upper bound of the range of pixel values of the photoacoustic image; is the mean absolute error: ； Among them, is the total number of pixels in the photoacoustic image, is the pixel value of the i-th pixel in is the pixel value of the i-th pixel in 7. A limited-view photoacoustic image artifact suppression system, characterized in that, including: Dataset construction module: Generate photoacoustic images through an image reconstruction algorithm, where the photoacoustic images include limited - view photoacoustic images and the corresponding full - view photoacoustic images; Model construction module: Add a sub - graph extraction module to each level of the U - Net model except the first level to extract a sub - graph sequence with the same number of channels as the feature map output by the encoding layer of the current level; perform information separation operations on the output of the encoding layer: subtract the limited - view photoacoustic image from the feature map output by the encoding layer of the first level, and subtract the sub - graph sequence of the corresponding level from the feature map output by the encoding layer of each other level to separate the artifacts; Perform a dual - path merging operation in the decoding layer: The up - sampled feature map is merged with the feature map output by the encoding layer of the same level, as well as the limited - view photoacoustic image or the sub - graph sequence; Model training module: Construct a loss function based on the structural similarity index and the mean absolute error between the limited - view photoacoustic image and the corresponding full - view photoacoustic image, optimize the parameters of the artifact suppression model, and select the optimal artifact suppression model; Prediction module: Input the limited - view photoacoustic image constructed based on the actually collected limited - view photoacoustic signals into the trained artifact suppression model, and output the post - processed image after artifact suppression.

8. A limited perspective photoacoustic image artifact suppression system according to claim 7, characterized in that In the model training module, constructing a loss function based on the structural similarity index and the mean absolute error between the limited - view photoacoustic image and the corresponding full - view photoacoustic image, optimizing the parameters of the artifact suppression model, and selecting the optimal artifact suppression model specifically include: Loss function ; SSIM is the structural similarity index: ； Among them, and are the average values of local regions of the limited-view photoacoustic image and the full-view photoacoustic image respectively, and are the variances of local regions of x and y respectively, is the covariance of local regions of x and y, and the parameter , the parameter , where is the upper bound of the range of pixel values of the photoacoustic image; is the mean absolute error: ； where n is the total number of pixels in the photoacoustic image, is the pixel value of the i-th pixel in is the pixel value of the i-th pixel in.

9. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the method according to any one of claims 1 to 6.

Images