Sparse regularization photoacoustic imaging unmixing method and system based on end member automatic extraction
By employing an automatic endmember extraction and sparse regularization method for photoacoustic imaging unmixing, the problems of insufficient spectral information and overfitting in photoacoustic imaging unmixing algorithms are solved, achieving high-precision separation of biological tissue components without prior knowledge and improving the accuracy and reliability of the unmixing results.
Patent Information
- Application Number
- CN202511329655.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-09-17
- Publication Date
- 2026-01-27
AI Technical Summary
Existing photoacoustic imaging unmixing algorithms suffer from insufficient spectral information, require prior knowledge, and are prone to overfitting when processing large-scale datasets, making it difficult to effectively analyze multiple components in biological tissues.
A sparse regularization method based on automatic endmember extraction is adopted. The endmembers in the photoacoustic image are extracted by finding the pixels of convex hull expansion through orthogonal projection. Sparse regularization constraints are introduced to minimize the objective function to solve the abundance distribution of each component in biological tissue, reduce the dependence on prior knowledge and avoid overfitting.
Without prior knowledge, it can accurately separate different chemical components in biological tissues, improve the accuracy and reliability of unmixing results, cover the spectral information of common biological chromophores, reduce errors caused by manual selection, and improve the accuracy and anti-overfitting ability of imaging unmixing.
Smart Images
Figure CN121414616A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of photoacoustic image demixing, and in particular to a sparse regularized photoacoustic imaging demixing method and system based on automatic endmember extraction. Background Technology
[0002] Photoacoustic imaging is an advanced, non-destructive biomedical imaging technique that combines the advantages of both optical and acoustic imaging. It leverages the high selectivity of pure optical imaging and the deep penetration of pure ultrasound imaging, exhibiting unique imaging characteristics. This technology effectively overcomes the problem of severe light scattering in biological tissues, achieving high-resolution and high-contrast imaging of deep tissues in living organisms. It can also quantitatively analyze changes in various physiological parameters of tissues, thus enabling functional imaging.
[0003] Photoacoustic detection technology holds the potential to reveal tissue characteristics within organisms through multi-wavelength measurements. However, in most cases, the signals received by photoacoustic detection are the result of the combined effects of multiple biological chromophores. Given the complexity of biological tissue composition, different components may be treated as part of a single continuum in photoacoustic imaging, making the application of photoacoustic technology to the detection of biological tissue health a challenging task.
[0004] Despite the complexity of biological tissue composition, high-precision photoacoustic imaging unmixing algorithms can still resolve the contribution of individual components from multi-wavelength photoacoustic imaging. Common unmixing algorithms include linear least squares, blind source separation, and deep learning-based methods. While linear least squares offers high accuracy, it requires prior knowledge of the optical absorption spectra of each component, which is often difficult to achieve in vivo applications. Blind source separation methods can automatically detect chromophores within tissues without prior knowledge. However, in most cases, blind source analysis requires data to satisfy certain assumptions. In practical applications, most photoacoustic data may not strictly conform to these basic assumptions. Furthermore, with the widespread application of deep learning in network image processing, some deep learning-based photoacoustic imaging unmixing algorithms have gradually developed. However, these methods require large amounts of sample data to train the network, and currently, there is a lack of publicly available datasets related to photoacoustic imaging.
[0005] CN118655090A discloses a method, apparatus, storage medium, and electronic device for photoacoustic imaging unmixing. When irradiating a sample to be unmixed with multiple wavelengths of raw laser light, for each wavelength of raw laser light, a first light energy distribution of the sample is determined from several output ports. This first light energy distribution characterizes the light energy distribution of points within the sample. Based on the first light energy distribution and the first photoacoustic image of the sample at that wavelength, a second photoacoustic image at that wavelength can be determined. Thus, the sample can be unmixed according to the second photoacoustic image corresponding to each wavelength of raw laser light. This method considers the different light energy distributions within the sample under irradiation with different wavelengths of laser light, thereby improving the accuracy of multi-wavelength photoacoustic imaging unmixing. However, this method uses a limited number of wavelengths, resulting in insufficient spectral information, making it difficult to cover common biological chromophores, and it requires prior knowledge.
[0006] Current photoacoustic imaging unmixing algorithms suffer from drawbacks such as insufficient spectral information, the need for prior knowledge, and a tendency to overfit when processing large-scale datasets. Summary of the Invention
[0007] The purpose of this invention is to provide a sparse regularized photoacoustic imaging demixing method and system based on automatic endmember extraction.
[0008] The objective of this invention can be achieved through the following technical solutions:
[0009] A sparse regularized photoacoustic imaging unmixing method based on automatic endmember extraction includes the following steps:
[0010] Acquire multi-wavelength photoacoustic signals and reconstruct photoacoustic images;
[0011] Based on photoacoustic images, pixels that cause the convex hull to expand continuously are found through orthogonal projection. Endmembers in the photoacoustic images are extracted and smoothed to obtain endmember curves.
[0012] The extracted endmember curves are used as prior knowledge. Sparse regularization constraints are introduced into the objective function of photoacoustic imaging unmixing. The abundance distribution of each component in biological tissue is solved by minimizing the objective function, and the unmixing result is obtained.
[0013] The method for extracting endmembers from photoacoustic images is as follows:
[0014] Project all pixels in the photoacoustic image onto a random direction, and identify the pixel with the maximum projection value as the initial endmember.
[0015] Calculate the vector orthogonal to the space spanned by the currently extracted endmembers as the projection vector, project the remaining pixels onto the projection vector, identify the pixel with the largest projection value as the endmember extracted in the current iteration, and repeat this process until all endmembers are identified. In the first iteration, the vector orthogonal to the initial endmember is used as the projection vector.
[0016] The method for determining whether all end-members have been identified is: the number of extracted end-members has reached the preset number.
[0017] The minimization objective function is expressed as:
[0018]
[0019] Where X represents photoacoustic data at a certain wavelength; A is an endmember matrix composed of different endmembers; and S is an abundance matrix. Let ||||1 represent the square of the Frobenius norm, ||||1 represent the L1 norm, i.e., the introduced sparse regularization constraint; λ is the penalty coefficient.
[0020] The multi-wavelength photoacoustic signals are acquired by photoacoustic imaging equipment, which includes linear array tomography, ring array tomography, and multi-wavelength photoacoustic microscopy.
[0021] A sparse regularized photoacoustic imaging demixing system based on automatic endmember extraction, comprising:
[0022] Signal acquisition module: used to acquire multi-wavelength photoacoustic signals and reconstruct photoacoustic images;
[0023] Endmember extraction module: Based on photoacoustic images, it finds pixels that cause the convex hull to expand continuously through orthogonal projection, extracts endmembers in the photoacoustic images, and performs smoothing processing to obtain endmember curves;
[0024] Unmixing module: It is used to take the extracted endmember curves as prior knowledge, introduce sparse regularization constraints into the objective function of photoacoustic imaging unmixing, and solve the abundance distribution of each component in biological tissue by minimizing the objective function to obtain the unmixing result.
[0025] The method for extracting endmembers by the endmember extraction module is as follows:
[0026] Project all pixels in the photoacoustic image onto a random direction, and identify the pixel with the maximum projection value as the initial endmember.
[0027] Calculate the vector orthogonal to the space spanned by the currently extracted endmembers as the projection vector, project the remaining pixels onto the projection vector, identify the pixel with the largest projection value as the endmember extracted in the current iteration, and repeat this process until all endmembers are identified. In the first iteration, the vector orthogonal to the initial endmember is used as the projection vector.
[0028] The method for determining whether all end-members have been identified is: the number of extracted end-members has reached the preset number.
[0029] The minimization objective function is expressed as:
[0030]
[0031] Where X represents photoacoustic data at a certain wavelength; A is an endmember matrix composed of different endmembers; and S is an abundance matrix. Let ||||1 represent the square of the Frobenius norm, ||||1 represent the L1 norm, i.e., the introduced sparse regularization constraint; λ is the penalty coefficient.
[0032] The signal acquisition module acquires multi-wavelength photoacoustic signals through photoacoustic imaging equipment, which includes linear array tomography, ring array tomography, and multi-wavelength photoacoustic microscopy.
[0033] Compared with the prior art, the present invention has the following beneficial effects:
[0034] (1) No prior knowledge required: Compared with the traditional least squares unmixing algorithm, the endmember extraction method based on vertex component analysis proposed in this invention does not require accurate acquisition of the light absorption coefficient of each substance as prior knowledge, which lowers the application threshold of the algorithm and improves its applicability in actual biomedical imaging.
[0035] (2) Strong resistance to overfitting: The unmixing algorithm with sparse regularization is introduced. By adding the L1 norm regularization term to the objective function, the overfitting phenomenon that occurs when processing large-scale datasets is effectively avoided, and the accuracy and reliability of the unmixing results are improved.
[0036] (3) High reliability: This invention can fully cover the absorption peak range of common biological chromophores (oxyhemoglobin, deoxyhemoglobin, lipids and water) in the 690nm-2000nm wavelength band (including the wavelength range of near-infrared I and II regions), thus possessing more sufficient spectral information. Furthermore, the method of automatically extracting endmember curves avoids the subjective bias caused by manual selection and initially reduces the error that may be caused by the laser scattering coefficient and absorption coefficient of different wavelengths, thereby improving the accuracy and reliability of the unmixing results. Attached Figure Description
[0037] Figure 1 This is a flowchart of the method of the present invention;
[0038] Figure 2 This is a schematic diagram of a disk model, a cylindrical phantom, and a photoacoustic imaging system in one embodiment.
[0039] Figure 3This is the endmember extraction result of the disk model in one embodiment;
[0040] Figure 4 This is the demixing result of the disk model in one embodiment;
[0041] Figure 5 This is the demixing and quantization result of the disk model in one embodiment;
[0042] Figure 6 This is the end-member extraction result of a circular tube phantom in one embodiment;
[0043] Figure 7 This is the demixing result of a circular tube phantom in one embodiment. Detailed Implementation
[0044] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. These embodiments are based on the technical solution of the present invention and provide detailed implementation methods and specific operating procedures. However, the scope of protection of the present invention is not limited to the following embodiments.
[0045] Example 1
[0046] This embodiment provides a sparse regularized photoacoustic imaging demixing method based on automatic endmember extraction, such as... Figure 1 As shown, it includes the following steps:
[0047] S1 acquires multi-wavelength photoacoustic signals and reconstructs photoacoustic images.
[0048] In this embodiment, photoacoustic imaging equipment is used to acquire multi-wavelength photoacoustic signals for application scenarios such as phantoms, animals, and humans. The photoacoustic imaging equipment includes, but is not limited to, linear array tomography, ring array tomography, and multi-wavelength photoacoustic microscopy equipment.
[0049] Specifically, this embodiment includes two parts: a simulation example and a photoacoustic imaging experiment example. In the simulation example, the k-Wave toolbox is used to perform numerical simulation of the photoacoustic signal, including the forward propagation and backward reconstruction process of the photoacoustic signal, and to acquire the simulated photoacoustic signal of the disk model. A photoacoustic imaging device (inVision 256-TF instrument) is used to acquire multi-wavelength photoacoustic signals from the cylindrical phantom. Specific regions of the photoacoustic signal are then selected.
[0050] The k-Wave toolbox is an open-source MATLAB toolbox developed by Bradley Treeby et al., specifically designed for simulating the propagation of sound waves in one-dimensional, two-dimensional, or three-dimensional space.
[0051] The disc model is 4mm in diameter and consists of 16 discs of the same size. Different rows represent different substances, and the numbers above indicate different concentration levels.
[0052] In the photoacoustic imaging example, the photoacoustic imaging device was the inVision 256-TF, equipped with 256 sensors uniformly arranged across a 310° imaging field of view, a center frequency of 5.0 MHz, and a bandwidth greater than 55%. The device features 360° light source coverage (frequency 10-20 Hz, pulse energy up to 100 mJ). Experiments were conducted in the wavelength range of 690 to 950 nm, with a spectral resolution of 5 nm, and each wavelength was measured an average of 10 times. During scanning, cross-sectional scans were performed from near to far with a step size of 1 mm, and the entire imaging process took approximately 10 minutes. Image reconstruction was performed using a back-projection algorithm. The reconstructed image had a field of view of 20 mm, a cross-sectional resolution of 72 μm, and low-frequency and high-frequency cutoff settings for the filters were 50 kHz to 6.5 MHz.
[0053] like Figure 2 As shown, the circular tube model consists of a 5% agarose matrix with four embedded plastic tubes. To simulate the acoustic scattering characteristics of biological tissue, a certain amount of coffee creamer was added to the agarose matrix as a scatterer. Different concentrations of blood and fat were filled into the four plastic tubes, respectively. During the experiment, it was crucial to ensure that air was eliminated from the plastic tubes as much as possible. The presence of air bubbles could cause reflection and refraction of the laser beam within the tubes, which would adversely affect the imaging quality of the phantom. The blood used was sterile defibrinated pig blood, diluted with an appropriate amount of water to prepare blood solutions of different concentrations. Specifically, the undiluted blood concentration was used as a baseline (defined as 1x concentration), and further diluted to concentrations of 0.75x and 0.5x. For the fat portion, pork lard was used, pre-treated using a meat grinder to facilitate filling into the plastic tubes, thus simulating the optical and acoustic properties of adipose tissue.
[0054] The specific region selection involved manually marking the main imaging area using the `drawfreehand` function in MATLAB software. This step aimed to reduce the impact of artifacts from non-imaging areas on the imaging results.
[0055] S2, based on photoacoustic images, finds pixels that cause the convex hull to expand continuously through orthogonal projection, extracts endmembers in the photoacoustic images, and performs smoothing processing to obtain endmember curves.
[0056] S21, Extract endmembers
[0057] S211, Project all pixels in the photoacoustic image onto a random direction, and identify the pixel with the maximum projection value as the initial endmember;
[0058] S212, calculate the vector orthogonal to the space spanned by the currently extracted endmembers as the projection vector, project the remaining pixels onto the projection vector, identify the pixel with the maximum projection value as the endmember extracted in the current iteration, repeat the process until all endmembers are identified, wherein in the first iteration, the vector orthogonal to the initial endmember is used as the projection vector.
[0059] In this embodiment, the method for determining that all end-members have been identified is: the number of extracted end-members has reached the preset number.
[0060] S22, Smoothing
[0061] Specifically, a moving average method was used as a smoothing technique, with a smoothing window size of 5 data points. This helps improve the accuracy of endmember extraction and provides a more robust data foundation for subsequent unmixing analysis.
[0062] S3 uses the extracted endmember curves as prior knowledge and introduces sparse regularization constraints into the objective function of photoacoustic imaging unmixing. The abundance distribution of each component in biological tissue is solved by minimizing the objective function to obtain the unmixing result.
[0063] The objective function to be minimized is expressed as:
[0064]
[0065] Where X represents photoacoustic data at a certain wavelength; A is an endmember matrix composed of different endmembers; and S is an abundance matrix. Let ||||1 represent the square of the Frobenius norm, ||||1 represent the L1 norm, i.e., the introduced sparse regularization constraint; λ is the penalty coefficient.
[0066] This optimization problem can be solved using gradient descent. In each iteration, the gradient of the loss function with respect to the weights is first calculated, and then the weights are updated based on the gradient. The specific update formula is as follows:
[0067]
[0068] Where η is the learning rate, which determines the speed of parameter updates; A i Let represent the endmember matrix obtained in the i-th iteration.
[0069] To verify the results of this invention, this embodiment simulates the original photoacoustic signal and initial pressure distribution of a disk model. The photoacoustic signal of the circular tube model is acquired using an inVision 256-TF device, and a back-projection algorithm is used to reconstruct the acquired photoacoustic signal. The simulation experiment uses 256 sensors configured across the entire field of view. These sensors are evenly distributed around the two models on a 3mm radius circle, with the sensor center frequencies and bandwidths set to default. The grid spatial distribution is set to a 12.8×12.8cm square area, with Nx and Ny set to 1280, and dx and dy set to 0.1mm, completely covering the disk model. Photoacoustic signals are acquired every 10nm within the wavelength range of 690nm to 950nm. All experimental procedures are implemented using the k-Wave toolbox in MATLAB.
[0070] Appendix Figure 3-7 The endmember extraction and demixing results for the disk model and the circular tube phantom are shown. Figure 3 and Figure 6 The end-member extraction results show that the end-member extraction of this invention can accurately separate different chemical components in immiscible substances. Figure 4 , 5 The demixing results of 7 show that the sparse regularization demixing algorithm has high accuracy.
[0071] Example 2
[0072] This embodiment provides a sparse regularized photoacoustic imaging demixing system based on automatic endmember extraction, used to implement the method of Embodiment 1. The system includes:
[0073] Signal acquisition module: used to acquire multi-wavelength photoacoustic signals and reconstruct photoacoustic images;
[0074] Endmember extraction module: Based on photoacoustic images, it finds pixels that cause the convex hull to expand continuously through orthogonal projection, extracts endmembers in the photoacoustic images, and performs smoothing processing to obtain endmember curves;
[0075] Unmixing module: It is used to take the extracted endmember curves as prior knowledge, introduce sparse regularization constraints into the objective function of photoacoustic imaging unmixing, and solve the abundance distribution of each component in biological tissue by minimizing the objective function to obtain the unmixing result.
[0076] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the described module can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.
[0077] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.
Claims
1. A sparse regularized photoacoustic imaging unmixing method based on automatic endmember extraction, characterized in that, Includes the following steps: Acquire multi-wavelength photoacoustic signals and reconstruct photoacoustic images; Based on photoacoustic images, pixels that cause the convex hull to expand continuously are found through orthogonal projection. Endmembers in the photoacoustic images are extracted and smoothed to obtain endmember curves. The extracted endmember curves are used as prior knowledge. Sparse regularization constraints are introduced into the objective function of photoacoustic imaging unmixing. The abundance distribution of each component in biological tissue is solved by minimizing the objective function, and the unmixing result is obtained.
2. The sparse regularized photoacoustic imaging unmixing method based on automatic endmember extraction according to claim 1, characterized in that, The method for extracting endmembers from photoacoustic images is as follows: Project all pixels in the photoacoustic image onto a random direction, and identify the pixel with the maximum projection value as the initial endmember. Calculate the vector orthogonal to the space spanned by the currently extracted endmembers as the projection vector, project the remaining pixels onto the projection vector, identify the pixel with the largest projection value as the endmember extracted in the current iteration, and repeat this process until all endmembers are identified. In the first iteration, the vector orthogonal to the initial endmember is used as the projection vector.
3. The sparse regularized photoacoustic imaging demixing method based on automatic endmember extraction according to claim 2, characterized in that, The method for determining whether all end-members have been identified is: the number of extracted end-members has reached the preset number.
4. The sparse regularized photoacoustic imaging demixing method based on automatic endmember extraction according to claim 1, characterized in that, The minimization objective function is expressed as: Where X represents photoacoustic data at a certain wavelength; A is an endmember matrix composed of different endmembers; and S is an abundance matrix. Let ||||1 represent the square of the Frobenius norm, ||||1 represent the L1 norm, i.e., the introduced sparse regularization constraint; λ is the penalty coefficient.
5. The sparse regularized photoacoustic imaging unmixing method based on automatic endmember extraction according to claim 1, characterized in that, The multi-wavelength photoacoustic signals are acquired by photoacoustic imaging equipment, which includes linear array tomography, ring array tomography, and multi-wavelength photoacoustic microscopy.
6. A sparse regularized photoacoustic imaging demixing system based on automatic endmember extraction, characterized in that, include: Signal acquisition module: used to acquire multi-wavelength photoacoustic signals and reconstruct photoacoustic images; Endmember extraction module: Based on photoacoustic images, it finds pixels that cause the convex hull to expand continuously through orthogonal projection, extracts endmembers in the photoacoustic images, and performs smoothing processing to obtain endmember curves; Unmixing module: It is used to take the extracted endmember curves as prior knowledge, introduce sparse regularization constraints into the objective function of photoacoustic imaging unmixing, and solve the abundance distribution of each component in biological tissue by minimizing the objective function to obtain the unmixing result.
7. A sparse regularized photoacoustic imaging demixing system based on automatic endmember extraction according to claim 6, characterized in that, The method for extracting endmembers by the endmember extraction module is as follows: Project all pixels in the photoacoustic image onto a random direction, and identify the pixel with the maximum projection value as the initial endmember. Calculate the vector orthogonal to the space spanned by the currently extracted endmembers as the projection vector, project the remaining pixels onto the projection vector, identify the pixel with the largest projection value as the endmember extracted in the current iteration, and repeat this process until all endmembers are identified. In the first iteration, the vector orthogonal to the initial endmember is used as the projection vector.
8. A sparse regularized photoacoustic imaging demixing system based on automatic endmember extraction according to claim 7, characterized in that, The method for determining whether all end-members have been identified is: the number of extracted end-members has reached the preset number.
9. A sparse regularized photoacoustic imaging demixing system based on automatic endmember extraction according to claim 6, characterized in that, The minimization objective function is expressed as: Where X represents photoacoustic data at a certain wavelength; A is an endmember matrix composed of different endmembers; and S is an abundance matrix. Let ||||1 represent the square of the Frobenius norm, ||||1 represent the L1 norm, i.e., the introduced sparse regularization constraint; λ is the penalty coefficient.
10. A sparse regularized photoacoustic imaging demixing system based on automatic endmember extraction according to claim 6, characterized in that, The signal acquisition module acquires multi-wavelength photoacoustic signals through photoacoustic imaging equipment, which includes linear array tomography, ring array tomography, and multi-wavelength photoacoustic microscopy.