Limited view angle photoacoustic image reconstruction method based on self-supervised neural representation

Abstract

The invention discloses a limited view angle photoacoustic image reconstruction method based on self-supervised neural representation. The method comprises the following steps: establishing a self-supervised neural network based on a multi-layer perceptron; outputting the coordinates of the sampling points in the two-dimensional space coordinates in the image plane; acquiring photoacoustic signal data corresponding to the sampling points under the limited visual angle of the ultrasonic sensor, and converting the photoacoustic signal data into image pixel values corresponding to the two-dimensional space coordinate points in the image plane; converting coordinate points in the two-dimensional space coordinates and image pixel values at the corresponding coordinate points into coded high-dimensional features; outputting an image intensity prediction value corresponding to the two-dimensional space coordinates according to the coded high-dimensional features; parameters of the multiple perceptron layers are optimized, and image intensity predicted values corresponding to the two-dimensional space coordinates are obtained again; and reconstructing a photoacoustic image according to the two-dimensional space coordinates. The method reconstructs a high-quality photoacoustic image from a common limited view angle.

Description

Technical Field

[0001] The present invention relates to the technical field of image reconstruction, and in particular to a limited-viewing angle photoacoustic image reconstruction method based on self-supervised neural representation. Background Art

[0002] As an emerging non-invasive medical imaging technology, photoacoustic tomography has received widespread attention in recent years and has shown promising preclinical and clinical applications. This technology combines the high contrast of optical imaging and the deep penetration of ultrasound imaging through the photoacoustic effect, making it a very promising imaging method. Laser pulses stimulate biological tissues to emit photoacoustic signals, which are then detected by ultrasound sensors. The intensity and characteristics of the photoacoustic signal are directly related to the light absorption of the tissue, which is closely related to the physiological and pathological state of the tissue. By reconstructing the photoacoustic signal, the distribution of radiation absorption within the tissue can be depicted, providing valuable information for the medical field.

[0003] However, in practical applications, it is often difficult to fully surround the target tissue or organ, so a limited angle array must be used, which may result in the loss of important information. Currently, in experimental limited angle settings, the detection angles are usually 60°, 90°, 135°, and 180°, such as Figure 1 shown.

[0004] Solving the problem of photoacoustic tomography signal reconstruction under limited viewing angle detection space sampling has become a focus of current research. Traditional PAT reconstruction algorithms can be roughly divided into two categories: linear reconstruction methods and model-based reconstruction methods. Linear reconstruction methods, such as back-projection and time reversal, are computationally efficient but often generate images with distortion and artifacts under limited viewing angle conditions. In contrast, model-based reconstruction methods rely on optimization iteration strategies to minimize the difference between the predicted signal estimated by the photoacoustic forward model and the measured signal. In order to achieve accurate reconstruction, model-based algorithms require accurate model matrices and appropriate prior knowledge. Although the incorporation of prior knowledge helps to reduce artifacts and significantly improve the quality of reconstruction, there is still a lack of effective regularization terms to fully describe the reconstruction results, which may cause suboptimal solutions. In addition, model-based reconstruction algorithms are time-consuming, and the choice of regularization term weights has a significant impact on the reconstruction results, limiting the performance of this method.

[0005] Currently, the most advanced method to solve the limited perspective problem is to use CNN deep learning methods to reconstruct limited perspective images, but this type of method requires a large number of corresponding images for training, which greatly reduces the applicability of this method.

[0006] Therefore, it is necessary to provide a limited-view photoacoustic image reconstruction method based on self-supervised neural representation to solve the above problems. Summary of the invention

[0007] The technical problem to be solved by the present invention is to provide a limited-viewing angle photoacoustic image reconstruction method based on self-supervised neural representation, which does not require a large amount of external data training and can reconstruct high-quality, artifact-free photoacoustic images from various common limited viewing angles.

[0008] The technical solution adopted by the present invention to solve the above technical problems is to provide a limited-viewing angle photoacoustic image reconstruction method based on self-supervised neural representation, comprising the following steps:

[0009] Establishing a self-supervised neural network based on a multi-layer perceptron, wherein the self-supervised neural network includes an input layer, a position encoding layer and a multi-layer perceptron layer;

[0010] Inputting the two-dimensional spatial coordinates of the image plane into the input layer to obtain the coordinates of the sampling point in the two-dimensional spatial coordinates in the image plane;

[0011] Acquire photoacoustic signal data corresponding to the sampling point under a limited viewing angle of the ultrasonic sensor, and convert the photoacoustic signal data into image pixel values ​​corresponding to the two-dimensional spatial coordinate points in the image plane according to the coordinates of the sampling point;

[0012] The position encoding layer converts the coordinate points in the two-dimensional space coordinates and the image pixel values ​​at the corresponding coordinate points into encoded high-dimensional features;

[0013] The multi-layer perceptron layer outputs an image intensity prediction value corresponding to the two-dimensional space coordinate according to the encoded high-dimensional features;

[0014] The image intensity prediction value at the coordinate point is optimized by the forward propagation model and the photoacoustic signal data obtained by the ultrasonic sensor, the parameters of the multi-layer perceptron layer are optimized, and the image intensity prediction value corresponding to the two-dimensional space coordinate is re-obtained;

[0015] Reconstruct the photoacoustic image based on the two-dimensional spatial coordinates.

[0016] Optionally, the photoacoustic signal data under a limited viewing angle is expressed as:

[0017]

[0018] Among them, K is the number of sensors, M is the number of sampling points of each sensor, and y s is the sampling value at the sampling point of each sensor;

[0019] y s Reshape into a (K×M)×1 column vector; convert the photoacoustic signal data into coordinate points in two-dimensional space coordinates and the image pixel values ​​at the corresponding coordinate points;

[0020] The desired photoacoustic image x is represented as a continuous function parameterized by a neural network with multiple layers of perceptrons:

[0021] I=f Θ (p)

[0022]

[0023] Where Θ represents the trainable parameters of the multi-layer perceptron layer, f Θ represents the implicit function of the multi-layer perceptron layer, p represents any two-dimensional spatial coordinate point in the image plane, and I is the image intensity prediction value corresponding to the position p in the photoacoustic image Z; Represents the image plane.

[0024] Optionally, the position encoding layer transforms the two-dimensional space coordinates into a higher-dimensional space by Fourier feature mapping The two-dimensional space coordinate points and the image pixel values ​​of the coordinate points are converted into encoded high-dimensional feature vectors, so that the multi-layer perceptron layer can capture high-frequency image features. The transformation function is expressed as:

[0025] γ(p)=[cos(2πBp),sin(2πBp)] T

[0026] Among them, γ(p) represents the Space to Fourier eigenmapping function of space; p represents any two-dimensional spatial coordinate in the image plane; B represents a row vector of (n×1), in the two-dimensional case B is an n*2 matrix, in the three-dimensional case it is an n*3 vector, where the values ​​are generated by random numbers that conform to the normal distribution (N~(0,1)), n is a natural number, and T represents the transpose of the matrix.

[0027] Optionally, a back-propagation gradient descent algorithm is used to optimize the parameters of the multi-layer perceptron layer. The image intensity prediction value at the coordinate point represents the predicted photoacoustic signal data after being calculated by the forward propagation model. The objective function is expressed as:

[0028]

[0029] in, is the objective function; is the predicted photoacoustic signal data; s To show the photoacoustic signal data actually collected; The loss function is used to measure the predicted photoacoustic signal data and the photoacoustic signal data y actually collected s The difference between them, A is the forward propagation model matrix calculated by S-wave;

[0030] Minimize the objective function to make the function value converge, and obtain the implicit function f of the image intensity prediction value close to the image pixel value Θ .

[0031] Optionally, all two-dimensional spatial coordinates are input to the multi-layer perceptron layer, and the implicit function f Θ The image intensity prediction value of the two-dimensional space coordinates is calculated to reconstruct the photoacoustic image x.

[0032] Optionally, the multi-layer perceptron layer uses an Adam optimizer to minimize the objective function; the hyperparameter of the Adam optimizer is set to β 1 =0.9,β 2 =0.999,∈=10 -8 , the initial learning rate is set to 10 -4 .

[0033] Optionally, the multilayer perceptron layer includes a first hidden layer, a second hidden layer and an output layer arranged in sequence, the first hidden layer and the second hidden layer each have 256 neurons, and the first hidden layer and the second hidden layer are both configured with a ReLU activation function; the output layer is configured with a Sigmoid activation function.

[0034] Compared with the prior art, the technical solution of the embodiment of the present invention has the following beneficial effects:

[0035] The limited-viewing angle photoacoustic image reconstruction method based on self-supervised neural representation provided by the present invention converts the coordinate points in the two-dimensional space coordinates and the image pixel values ​​at the corresponding coordinate points into high-frequency image features that can be captured by the multi-layer perceptron layer through a position encoding layer, processes the encoded high-dimensional features through the multi-layer perceptron layer, and finally outputs the image intensity prediction value of the corresponding coordinate, so as to realize the inference of high-quality photoacoustic images from the photoacoustic signal data obtained from a limited viewing angle.

[0036] Furthermore, the image intensity prediction value at the coordinate point is optimized with the photoacoustic signal data at the coordinate point after passing through the forward propagation model, so as to optimize the parameters of the multi-layer perception machine layer, reduce the error and improve the accuracy of the image intensity prediction value. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following is a brief introduction to the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are some embodiments of the present invention, not all embodiments. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor.

[0038] Figure 1Common imaging images of existing photoacoustic tomography, including 360° full-range tomography (green), and representative limited viewing angles: 180° (purple), 135° (orange), 90° (dark red), and 60° (blue);

[0039] Figure 2 Flow chart of a limited-viewing angle photoacoustic image reconstruction method based on self-supervised neural representation in an embodiment of the present invention;

[0040] Figure 3 It is an architecture diagram of a limited-viewing angle photoacoustic image reconstruction method based on self-supervised neural representation in an embodiment of the present invention;

[0041] Figure 4 It is a schematic diagram of a limited-viewing angle photoacoustic image reconstruction method based on self-supervised neural representation in an embodiment of the present invention;

[0042] Figure 5 a is the original image of sample 1 at a limited viewing angle of 180° in Example 1 of the present invention;

[0043] Figure 5 b is an image reconstructed by the existing delayed superposition algorithm of the sample 1 at a limited viewing angle of 180° in Example 1 of the present invention;

[0044] Figure 5 c is an image reconstructed by the existing image reconstruction method based on the model algorithm at a limited viewing angle of 180° of the sample 1 in Example 1 of the present invention;

[0045] Figure 5 d is an image reconstructed by the limited viewing angle photoacoustic image reconstruction method based on self-supervised neural representation of sample 1 at a limited viewing angle of 180° in Example 1 of the present invention;

[0046] Figure 6 a is the original image of sample 2 at a limited viewing angle of 180° in Example 1 of the present invention;

[0047] Figure 6 b is an image reconstructed by the existing delayed superposition algorithm of the sample 2 at a limited viewing angle of 180° in Example 1 of the present invention;

[0048] Figure 6 c is an image reconstructed by the existing image reconstruction method based on the model algorithm at a limited viewing angle of 180° for the sample 2 in Example 1 of the present invention;

[0049] Figure 6 d is an image reconstructed by the limited viewing angle photoacoustic image reconstruction method based on self-supervised neural representation of sample 2 at a limited viewing angle of 180° in Example 1 of the present invention;

[0050] Figure 7 a is the original image of sample 3 at a limited viewing angle of 180° in Example 1 of the present invention;

[0051] Figure 7 b is an image reconstructed by the existing delayed superposition algorithm at a limited viewing angle of 180° of the sample 3 in Example 1 of the present invention;

[0052] Figure 7 c is an image reconstructed by the existing image reconstruction method based on the model algorithm at a limited viewing angle of 180° for the sample 3 in Example 1 of the present invention;

[0053] Figure 7 d is an image reconstructed by the limited viewing angle photoacoustic image reconstruction method based on self-supervised neural representation of sample 3 at a limited viewing angle of 180° in Example 1 of the present invention;

[0054] Figure 8 a is the original image of sample 1 at a limited viewing angle of 135° in Example 2 of the present invention;

[0055] Figure 8 b is an image reconstructed by the existing delayed superposition algorithm at a limited viewing angle of 135° for sample 1 in Example 2;

[0056] Figure 8 c is an image reconstructed by the existing image reconstruction method based on the model algorithm at a limited viewing angle of 135° for the sample 1 in Example 2 of the present invention;

[0057] Figure 8 d is an image reconstructed by the limited viewing angle photoacoustic image reconstruction method based on self-supervised neural representation of sample 1 at a limited viewing angle of 135° in Example 2 of the present invention;

[0058] Fig. 9 a is the original image of sample 2 at a limited viewing angle of 135° in Example 2 of the present invention;

[0059] Fig. 9 b is an image reconstructed by the existing delayed superposition algorithm at a limited viewing angle of 135° for sample 2 in Example 2 of the present invention;

[0060] Fig. 9 c is an image reconstructed by the existing image reconstruction method based on the model algorithm at a limited viewing angle of 135° for the sample 2 in Example 2 of the present invention;

[0061] Fig. 9 d is an image reconstructed by the limited viewing angle photoacoustic image reconstruction method based on self-supervised neural representation of sample 2 at a limited viewing angle of 135° in Example 2 of the present invention;

[0062] Fig.10 a is the original image of sample 3 at a limited viewing angle of 135° in Example 2 of the present invention;

[0063] Fig.10 b is an image reconstructed by the existing delayed superposition algorithm at a limited viewing angle of 135° for sample 3 in Example 2 of the present invention;

[0064] Fig.10 c is an image reconstructed by the existing image reconstruction method based on the model algorithm at a limited viewing angle of 135° for sample 3 in Example 2 of the present invention;

[0065] Fig.10 d is an image reconstructed by the limited viewing angle photoacoustic image reconstruction method based on self-supervised neural representation of sample 3 at a limited viewing angle of 135° in Example 2 of the present invention;

[0066] Fig.11 a is the original image of sample 1 at a limited viewing angle of 90° in Example 3 of the present invention;

[0067] Fig.11 b is an image reconstructed by the existing delayed superposition algorithm of the sample 1 at a limited viewing angle of 90° in Example 3 of the present invention;

[0068] Fig.11 c is an image reconstructed by the existing image reconstruction method based on a model algorithm at a limited viewing angle of 90° for sample 1 in Example 3 of the present invention;

[0069] Fig.11 d is an image reconstructed by the limited viewing angle photoacoustic image reconstruction method based on self-supervised neural representation of sample 1 at a limited viewing angle of 90° in Example 3 of the present invention;

[0070] Fig.12 a is the original image of sample 2 at a limited viewing angle of 90° in Example 3 of the present invention;

[0071] Fig.12 b is an image reconstructed by the existing delayed superposition algorithm at a limited viewing angle of 90° for sample 2 in Example 3 of the present invention;

[0072] Fig.12 c is an image reconstructed by the existing image reconstruction method based on the model algorithm at a limited viewing angle of 90° for the sample 2 in Example 3 of the present invention;

[0073] Fig.12 d is an image reconstructed by the limited viewing angle photoacoustic image reconstruction method based on self-supervised neural representation of sample 2 at a limited viewing angle of 90° in Example 3 of the present invention;

[0074] Fig.13 a is the original image of sample 3 at a limited viewing angle of 90° in Example 3 of the present invention;

[0075] Fig.13 b is an image reconstructed by the existing delayed superposition algorithm at a limited viewing angle of 90° for sample 3 in Example 3 of the present invention;

[0076] Fig.13 c is an image reconstructed by the existing image reconstruction method based on a model algorithm at a limited viewing angle of 90° for sample 3 in Example 3 of the present invention;

[0077] Fig.13 d is an image reconstructed by the limited viewing angle photoacoustic image reconstruction method based on self-supervised neural representation of sample 3 at a limited viewing angle of 90° in Example 3 of the present invention;

[0078] Fig.14 a is the original image of sample 1 at a limited viewing angle of 60° in Example 4 of the present invention;

[0079] Fig.14 b is an image reconstructed by the existing delayed superposition algorithm of the sample 1 at a limited viewing angle of 60° in Example 4 of the present invention;

[0080] Fig.14 c is an image reconstructed by the existing image reconstruction method based on the model algorithm of the sample 1 at a limited viewing angle of 60° in Example 4 of the present invention;

[0081] Fig.14 d is an image reconstructed by the limited viewing angle photoacoustic image reconstruction method based on self-supervised neural representation of sample 1 at a limited viewing angle of 60° in Example 4 of the present invention;

[0082] Fig.15 a is the original image of sample 2 at a limited viewing angle of 60° in Example 4 of the present invention;

[0083] Fig.15 b is an image reconstructed by the existing delayed superposition algorithm at a limited viewing angle of 60° for sample 2 in Example 4 of the present invention;

[0084] Fig.15 c is an image reconstructed by the existing image reconstruction method based on the model algorithm at a limited viewing angle of 60° for the sample 2 in Example 4 of the present invention;

[0085] Fig.15 d is an image reconstructed by the limited viewing angle photoacoustic image reconstruction method based on self-supervised neural representation of sample 2 at a limited viewing angle of 60° in Example 4 of the present invention;

[0086] Fig.16 a is the original image of sample 3 at a limited viewing angle of 60° in Example 4 of the present invention;

[0087] Fig.16 b is an image reconstructed by the existing delayed superposition algorithm at a limited viewing angle of 60° for sample 3 in Example 4 of the present invention;

[0088] Fig.16 c is an image reconstructed by the existing image reconstruction method based on the model algorithm at a limited viewing angle of 60° for the sample 3 in Example 3 of the present invention;

[0089] Fig.16 d is an image reconstructed by the limited viewing angle photoacoustic image reconstruction method based on self-supervised neural representation of sample 3 at a limited viewing angle of 60° in Example 4 of the present invention;

[0090] Fig.17 This is a schematic diagram of sparse sampling of leaf veins in an embodiment of the present invention;

[0091] Fig.18 This is a diagram of the image reconstruction architecture of leaf vein sparse viewing angle photoacoustic microscopy imaging in an embodiment of the present invention;

[0092] Fig.19 a is a full sampling image of leaf veins in an embodiment of the present invention;

[0093] Fig.19 b is a 1 / 16 sampling image of leaf veins in an embodiment of the present invention;

[0094] Fig.19 c is a reconstructed image of leaf veins at a 1 / 16 sampling sparse viewing angle in an embodiment of the present invention;

[0095] Fig. 20 a is another leaf vein full sampling image in an embodiment of the present invention;

[0096] Fig. 20 b is another leaf vein 1 / 4 sampling image in an embodiment of the present invention;

[0097] Fig. 20 c is a reconstructed image of another leaf vein under a 1 / 4 sampling sparse viewing angle in an embodiment of the present invention. DETAILED DESCRIPTION

[0098] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0099] The technical solution of the present invention is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0100] Based on the problems existing in the prior art, an embodiment of the present invention provides a limited-viewing angle photoacoustic image reconstruction method based on self-supervised neural representation, which does not require a large amount of external data training and can reconstruct high-quality, artifact-free photoacoustic images from various common limited viewing angles.

[0101] Figure 2 Flow chart of a limited-viewing angle photoacoustic image reconstruction method based on self-supervised neural representation in an embodiment of the present invention; Figure 3 It is an architecture diagram of a limited-viewing angle photoacoustic image reconstruction method based on self-supervised neural representation in an embodiment of the present invention; Figure 4 Schematic diagram of a limited-viewing angle photoacoustic image reconstruction method based on self-supervised neural representation in an embodiment of the present invention.

[0102] View now Figures 2 to 4 The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0103] Figure 1 This is a flow chart of a limited-viewing angle photoacoustic image reconstruction method based on self-supervised neural representation in an embodiment of the present invention.

[0104] See also Figure 1 The limited-viewing angle photoacoustic image reconstruction method based on self-supervised neural representation of an embodiment of the present invention comprises the following steps:

[0105] S101: Establish a self-supervised neural network based on a multi-layer perceptron, where the self-supervised neural network includes an input layer, a position encoding layer, and a multi-layer perceptron layer;

[0106] S102: Input the two-dimensional spatial coordinates of the image plane into the input layer to obtain the coordinates of the sampling point in the two-dimensional spatial coordinates in the image plane;

[0107] S103: Acquire photoacoustic signal data corresponding to the sampling point under the limited viewing angle of the ultrasonic sensor, and convert the photoacoustic signal data into image pixel values ​​corresponding to the two-dimensional space coordinate points in the image plane according to the coordinates of the sampling points;

[0108] S104: The position encoding layer converts the coordinate points in the two-dimensional space coordinates and the image pixel values ​​at the corresponding coordinate points into encoded high-dimensional features;

[0109] S105: The multi-layer perceptron layer outputs an image intensity prediction value corresponding to the two-dimensional space coordinate according to the encoded high-dimensional features;

[0110] S106: performing optimization calculation on the image intensity prediction value at the coordinate point through the forward propagation model and the photoacoustic signal data acquired by the ultrasonic sensor, optimizing the parameters of the multi-layer perceptron layer, and re-obtaining the image intensity prediction value corresponding to the two-dimensional space coordinate;

[0111] S107: Reconstructing the photoacoustic image according to the two-dimensional spatial coordinates.

[0112] In some embodiments, the photoacoustic signal data under a limited viewing angle is represented as:

[0113]

[0114] Among them, K is the number of sensors, M is the number of sampling points of each sensor, and y s is the sampling value at the sampling point of each sensor;

[0115] y s Reshape into a (K×M)×1 column vector; convert the photoacoustic signal data into coordinate points in two-dimensional space coordinates and the image pixel values ​​at the corresponding coordinate points.

[0116] The desired photoacoustic image x is represented as a continuous function parameterized by a neural network with multiple layers of perceptrons:

[0117] I=f Θ (p)

[0118]

[0119] Where Θ represents the trainable parameters of the multi-layer perceptron layer, f Θ represents the implicit function of the multi-layer perceptron layer, p represents any two-dimensional spatial coordinate point in the image plane, and I is the image intensity prediction value corresponding to the position p in the photoacoustic image Z; Represents the image plane.

[0120] In some embodiments, the position encoding layer transforms the two-dimensional space coordinates into a higher-dimensional space by Fourier feature mapping. The two-dimensional space coordinate points and the image pixel values ​​of the coordinate points are converted into encoded high-dimensional feature vectors, so that the multi-layer perceptron layer can capture high-frequency image features. The transformation function is expressed as:

[0121] γ(p)=[cos(2πBp),sin(2πBp)] T

[0122] Among them, γ(p) represents the Space to Fourier eigenmapping function of space; p represents any two-dimensional spatial coordinate in the image plane; B represents a row vector of (n×11), where the value is generated by random numbers that conform to the normal distribution (N~(0,1)), n is a natural number, and T represents the transpose of the matrix. p is a value of 1*1. Specifically, B is an n*2 matrix in the two-dimensional case and an n*3 vector in the three-dimensional case. The corresponding p for the two-dimensional case is (2×1), and then the matrix multiplication is performed, that is, B×p, (n*2)×(2*1)=(n*1) to achieve dimensionality increase in sequence.

[0123] In some embodiments, a back-propagation gradient descent algorithm is used to optimize the parameters of the multi-layer perceptron layer. The image intensity prediction value at the coordinate point represents the predicted photoacoustic signal data after being calculated by the forward propagation model. The objective function is expressed as:

[0124]

[0125] in, is the objective function; is the predicted photoacoustic signal data; s To show the photoacoustic signal data actually collected; The loss function is used to measure the predicted photoacoustic signal data and the photoacoustic signal data y actually collected s The difference between them, A is the forward propagation model matrix calculated by S-wave;

[0126] Using S-wave, the pixel values ​​of the pixels in the image are set to 1 in turn to obtain the standard data of the corresponding points, and then all the obtained standard data are spliced ​​to form a forward propagation matrix A. Specifically, the initial pressure distribution is regarded as a collection of single pixels. By obtaining the standard sensor data from a single pixel in advance, the phase and amplitude of the sensor data of a specific pixel can be easily manipulated using loop and multiplication operators. The effectiveness of this method is verified by the optimization-based reconstruction algorithm.

[0127] Minimize the objective function to make the function value converge, and obtain the implicit function f of the image intensity prediction value close to the image pixel value Θ .

[0128] In the specific implementation, all two-dimensional space coordinates are input into the multi-layer perceptron layer, and the implicit function f Θ The image intensity prediction values ​​of the two-dimensional spatial coordinates are calculated to reconstruct the photoacoustic image x.

[0129] In some embodiments, the multi-layer perceptron layer uses the Adam optimizer to minimize the objective function; the hyperparameter of the Adam optimizer is set to β 1 =0.9,β 2 =0.999,∈=10 -8 , the initial learning rate is set to 10 -4 .

[0130] In some embodiments, the multilayer perceptron layer includes a first hidden layer, a second hidden layer and an output layer arranged in sequence, the first hidden layer and the second hidden layer each have 256 neurons, and the first hidden layer and the second hidden layer are both configured with a ReLU activation function; the output layer is configured with a Sigmoid activation function.

[0131] In order to evaluate the performance of the limited-viewing angle photoacoustic image reconstruction method based on self-supervised neural representation in an embodiment of the present invention, an implementation test was conducted to compare the performance of the present method with the existing image reconstruction method based on the delayed superposition algorithm and the image reconstruction method based on the model algorithm in terms of reconstruction quality and computational efficiency.

[0132] Three samples were selected for image reconstruction under various common limited perspectives. Sample 1 was a basic geometric figure. The basic reconstruction ability of the method was verified by using basic figures without complex structures. Sample 2 was a simple vascular network biological tissue. The performance of the method in dealing with biological tissues with certain structural characteristics was verified by using relatively simple and clear vascular network biological tissues. Sample 3 was a complex vascular network biological tissue with richer and more complex vascular distribution, in order to test whether the method could achieve high-quality reconstruction when dealing with highly complex biological tissue structures.

[0133] Embodiment 1:

[0134] Under a limited viewing angle of 180°, image reconstruction experiments were performed on sample 1, sample 2, and sample 3 respectively.

[0135] The image reconstruction comparison results are as follows Figure 5-Figure 7 As shown, it can be seen from the image reconstruction results that, whether it is a basic graphic or a complex biological tissue, the image reconstruction method of the embodiment of the present invention can reconstruct a high-quality photoacoustic image close to the original image under a limited viewing angle of 180°, and the reconstruction quality is better than the existing image reconstruction method based on the delayed superposition algorithm and the image reconstruction method based on the model algorithm.

[0136] Embodiment 2:

[0137] At a limited viewing angle of 135°, image reconstruction experiments were performed on sample 1, sample 2, and sample 3, respectively.

[0138] The image reconstruction comparison results are as follows Figure 8-Figure 10 As shown, it can be seen from the image reconstruction results that, whether it is a basic graphic or a complex biological tissue, the image reconstruction method of the embodiment of the present invention can reconstruct a high-quality photoacoustic image close to the original image under a limited viewing angle of 135°, and the reconstruction quality is better than the existing image reconstruction method of the delayed superposition algorithm and the image reconstruction method based on the model algorithm.

[0139] Embodiment 3:

[0140] Under a limited viewing angle of 90°, image reconstruction experiments were performed on sample 1, sample 2, and sample 3 respectively.

[0141] The image reconstruction comparison results are as follows Figure 11-13 As shown, it can be seen from the image reconstruction results that, whether it is a basic graphic or a complex biological tissue, the image reconstruction method of the embodiment of the present invention can reconstruct a high-quality photoacoustic image close to the original image under a limited viewing angle of 90°, and the reconstruction quality is better than the existing image reconstruction method of the delayed superposition algorithm and the image reconstruction method based on the model algorithm.

[0142] Embodiment 4:

[0143] Under a limited viewing angle of 60°, image reconstruction experiments were performed on sample 1, sample 2, and sample 3 respectively.

[0144] The image reconstruction comparison results are as follows Figure 14-16 As shown, it can be seen from the image reconstruction results that, whether it is a basic graphic or a complex biological tissue, the image reconstruction method of the embodiment of the present invention can reconstruct a high-quality photoacoustic image close to the original image under a limited viewing angle of 60°, and the reconstruction quality is better than the existing image reconstruction method of the delayed superposition algorithm and the image reconstruction method based on the model algorithm.

[0145] The quantitative evaluation results of image reconstruction in Examples 1 to 4 are summarized in Table 1 below:

[0146] Table 1 Summary of quantitative evaluation results

[0147]

[0148] The values ​​in the above table are Ssim / Psnr, where Ssim refers to structural similarity, which is an indicator for measuring the similarity between two images. Here, it indicates the structural similarity between the reconstructed image and the original image. The larger the value, the higher the similarity with the original image. Psnr refers to peak signal-to-noise ratio, which is an objective evaluation method based on pixel values ​​and is used to measure the quality difference between a digital image or video and its original image or video. Here, it indicates the quality difference between the reconstructed image and the original image. The closer the value is to 1, the smaller the quality difference with the original image.

[0149] It can be seen from Table 1 that under limited viewing angle, the limited viewing angle photoacoustic image reconstruction method based on self-supervised neural representation of the embodiment of the present invention far exceeds the traditional method in terms of Ssim and Psnr indicators; the image reconstruction quality is greatly improved.

[0150] The method of the embodiment of the present invention is also applicable to sparse view reconstruction images. Conventional photoacoustic microscopy (PAM) usually requires a large number of samples through raster scanning, which leads to extended data acquisition time, increased system complexity and increased costs. Sparse view reconstruction of photoacoustic microscopy significantly reduces the sampling data through subsampling, thereby speeding up data acquisition, reducing the time and cost required for the imaging process, and providing a valuable solution for photoacoustic microscopy imaging. Implicit neural representation is used for sparse view image reconstruction of photoacoustic microscopy imaging to achieve high-quality imaging and reduce the amount of data sampling without the need for a lot of training.

[0151] See also Fig.17 , 268 original full-scan raw data images of leaf veins were obtained from a public photoacoustic microscopy imaging database, and sparse photoacoustic microscopy imaging data were created by downsampling these full-scan data, generating low-sampled images with fewer pixels and shorter imaging time. These low-sampled images, such as 4x downsampling or 16x downsampling, were used as input to the implicit neural representation (INR) model.

[0152] See also Fig.18 In the implicit neural representation model, the sparse photoacoustic microscopy imaging data is assumed to be a continuous implicit voxel function, denoted as where x = (x, y, z) represents the spatial coordinates of any 3D voxel in the image, and I represents the voxel intensity at position x; a multilayer perceptron network with Fourier feature maps is used to approximate the function from a sparse image. Performing maximum amplitude projection (MAP) along the z direction on the dense 3D photoacoustic microscopy imaging data to obtain 2D images, and comparing these images with the maximum amplitude projection images of the original dense 3D photoacoustic microscopy imaging data along the z direction; obtaining an implicit function of the maximum amplitude projection image of the 3D photoacoustic microscopy imaging data approximating the maximum amplitude projection image of the original dense 3D photoacoustic microscopy imaging data A robust estimate of the continuous function is obtained, and then the photoacoustic microscopy image is reconstructed.

[0153] The reconstruction results are as follows Fig.19 and Fig. 20 As shown, under sparse sampling of 1 / 16 and 1 / 4, the photoacoustic microscopy imaging images of leaf veins were reconstructed to obtain reconstructed images close to full sampling.

[0154] To summarize, the limited-viewing angle photoacoustic image reconstruction method based on self-supervised neural representation in the embodiment of the present invention, the limited-viewing angle photoacoustic image reconstruction method based on self-supervised neural representation provided by the present invention, converts the coordinate points in the two-dimensional space coordinates and the image pixel values ​​at the corresponding coordinate points into high-frequency image features that can be captured by the multi-layer perceptron layer through the position encoding layer, processes the encoded high-dimensional features through the multi-layer perceptron layer, and finally outputs the image intensity prediction value of the corresponding coordinate, so as to realize the inference of high-quality photoacoustic images from the photoacoustic signal data obtained from a limited viewing angle.

[0155] Furthermore, the image intensity prediction value at the coordinate point is combined with the photoacoustic signal data at the coordinate point through a forward propagation model to optimize the parameters of the multi-layer perception machine layer, reduce the error, and improve the accuracy of the image intensity prediction value.

[0156] Although the present invention has been disclosed as above with preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art may make some modifications and improvements without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention shall be based on the definition of the claims.

Claims

1. A limited-view photoacoustic image reconstruction method based on self-supervised neural representation, characterized in that: The steps include: Establishing a self-supervised neural network based on a multi-layer perceptron, wherein the self-supervised neural network includes an input layer, a position encoding layer and a multi-layer perceptron layer; Inputting the two-dimensional spatial coordinates of the image plane into the input layer to obtain the coordinates of the sampling point in the two-dimensional spatial coordinates in the image plane; Acquire photoacoustic signal data corresponding to the sampling point under a limited viewing angle of the ultrasonic sensor, and convert the photoacoustic signal data into image pixel values ​​corresponding to the two-dimensional spatial coordinate points in the image plane according to the coordinates of the sampling point; The position encoding layer converts the coordinate points in the two-dimensional space coordinates and the image pixel values ​​at the corresponding coordinate points into encoded high-dimensional features; The multi-layer perceptron layer outputs an image intensity prediction value corresponding to the two-dimensional space coordinate according to the encoded high-dimensional features; The image intensity prediction value at the coordinate point is optimized by the forward propagation model and the photoacoustic signal data obtained by the ultrasonic sensor, the parameters of the multi-layer perceptron layer are optimized, and the image intensity prediction value corresponding to the two-dimensional space coordinate is re-obtained; Reconstructing the photoacoustic image according to the two-dimensional spatial coordinates; The photoacoustic signal data under limited viewing angle is expressed as: Among them, K is the number of sensors, M is the number of sampling points of each sensor, and y s is the sampling value at the sampling point of each sensor; y s Reshape into a (K×M)×1 column vector; convert the photoacoustic signal data into coordinate points in two-dimensional space coordinates and the image pixel values ​​at the corresponding coordinate points; The desired photoacoustic image x is represented as a continuous function parameterized by a neural network with multiple layers of perceptrons: I=f Θ (p) Where Θ represents the trainable parameters of the multi-layer perceptron layer, f Θ represents the implicit function of the multi-layer perceptron layer, p represents any two-dimensional spatial coordinate point in the image plane, and I is the image intensity prediction value corresponding to the position p in the photoacoustic image Z; represents the image plane; The position encoding layer transforms the two-dimensional space coordinates into a higher-dimensional space through Fourier feature mapping. The two-dimensional space coordinate points and the image pixel values ​​of the coordinate points are converted into encoded high-dimensional feature vectors, so that the multi-layer perceptron layer can capture high-frequency image features. The transformation function is expressed as: γ(p)=[cos(2πBp),sin(2πBp)] T Among them, γ(p) represents the Space to Fourier eigenmapping function of space; p represents any two-dimensional spatial coordinate in the image plane; B represents a row vector of (n×1), in the two-dimensional case B is an n*2 matrix, in the three-dimensional case it is an n*3 vector, where the values ​​are generated by random numbers that conform to the normal distribution (N~(0,1)), n is a natural number, and T represents the transpose of the matrix.

2. The limited-view photoacoustic image reconstruction method based on self-supervised neural representation as claimed in claim 1 is characterized in that a back-propagation gradient descent algorithm is used to optimize the parameters of the multi-layer perceptron layer, and the image intensity prediction value at the coordinate point represents the predicted photoacoustic signal data after being calculated by a forward propagation model, and the objective function is expressed as: in, is the objective function; is the predicted photoacoustic signal data; s To show the photoacoustic signal data actually collected; The loss function is used to measure the predicted photoacoustic signal data and the photoacoustic signal data y actually collected s The difference between them, A is the forward propagation model matrix calculated by S-wave; Minimize the objective function to make the function value converge, and obtain the implicit function f of the image intensity prediction value close to the image pixel value Θ .

3. The limited-view photoacoustic image reconstruction method based on self-supervised neural representation according to claim 2, characterized in that: All two-dimensional space coordinates are input into the multi-layer perceptron layer, and the implicit function f Θ The image intensity prediction value of the two-dimensional space coordinates is calculated to reconstruct the photoacoustic image x.

4. The limited-view photoacoustic image reconstruction method based on self-supervised neural representation according to claim 2, characterized in that: The multi-layer perceptron layer uses the Adam optimizer to minimize the objective function; the hyperparameters of the Adam optimizer are set to β1=0.9, β2=0.999, ∈=10 -8 , the initial learning rate is set to 10 -4 .

5. The limited-viewing angle photoacoustic image reconstruction method based on self-supervised neural representation according to claim 1, characterized in that: The multilayer perceptron layer includes a first hidden layer, a second hidden layer and an output layer which are arranged in sequence, the first hidden layer and the second hidden layer each have 256 neurons, the first hidden layer and the second hidden layer are both configured with a ReLU activation function; the output layer is configured with a Sigmoid activation function.

Images