Dust-covered target libsspectrum effectiveness identification method based on phate dimension reduction

Abstract

The application discloses a dust surface target LIBS spectrum effectiveness identification method based on PHATE dimension reduction, combines a PHATE (affinity-based trajectory embedding heat diffusion potential) algorithm and a K-means algorithm, and realizes accurate identification of dust surface target LIBS spectrum effectiveness. The application comprises two stages of feature dimension reduction and cluster analysis: first, based on the PHATE algorithm, high-dimensional LIBS spectrum data is transformed into low-dimensional feature vectors, while retaining the global structure of the data, the core features of the spectrum are extracted; then, based on the K-means algorithm, cluster analysis is performed on the low-dimensional spectrum feature vectors, so as to distinguish three different spectrum data, namely dust spectrum, dust-matrix transition spectrum and matrix spectrum. The application has the advantages of simple and efficient data processing, high spectrum effectiveness identification accuracy and strong robustness, helps to screen out truly effective target spectrum from the dust surface target LIBS spectrum, and is suitable for fast analysis of LIBS spectrum collected in non-clean scenes such as field exploration and deep space exploration.

Description

Technical Field

[0001] This invention relates to the field of laser spectral analysis technology, specifically a method for identifying the effectiveness of LIBS spectra of dust-covered targets based on PHATE dimensionality reduction. This method utilizes the PHATE algorithm for feature dimensionality reduction and the K-means algorithm for clustering, which can effectively distinguish LIBS spectra of dust-covered targets and preliminarily identify whether there are mineral components of research value in the target samples under dust coverage. This allows for accurate and efficient screening of potentially high-value spectra from a large number of LIBS spectra for further chemical composition analysis. Background Technology

[0002] Laser-induced breakdown spectroscopy (LIBS) is a chemical composition detection technique based on laser-induced plasma radiation. It boasts advantages such as rapid response, minimal sample damage, simultaneous multi-element analysis, and remote detection, making it widely applicable in environmental monitoring, industrial inspection, and planetary exploration. Its remote detection capability, in particular, has made LIBS crucial for planetary exploration. Currently, LIBS technology has been successfully applied in three Mars exploration missions. NASA's Curiosity and Perseverance rovers, and my country's Zhurong rover, all carried scientific payloads equipped with LIBS systems: the ChemCam, SuperCam, and the Martian Surface Composition Analyzer (MarSCoDe). Using LIBS technology, these scientific payloads can analyze the chemical elemental composition of Martian soil, rocks, and other materials.

[0003] However, compared to the clean environment of a conventional laboratory, the Martian atmosphere is filled with particulate matter with a particle size on the order of micrometers, namely Martian dust. This Martian dust covers the surface of LIBS target materials such as rocks, and the thickness of the dust layer can range from micrometers to meters, as detailed in reference [1]. Although high-energy laser pulses can ablate some of the dust on the target surface during LIBS detection, they cannot completely remove it, as detailed in references [2,3]. This means that the LIBS spectrum contains the compositional information of the dust to a certain extent, which has a significant interfering effect on the analysis of the actual target material composition.

[0004] In order to reduce the negative impact of dust on the target surface on the analysis of Mars LIBS spectral data, the international ChemCam and SuperCam payload teams adopted the method of treating the first 5 LIBS spectra obtained from each probe as spectra originating from dust and excluding these spectral data from the total dataset for subsequent analysis. However, in fact, the 6th and subsequent LIBS spectra obtained from each probe may still contain dust composition information, as detailed in reference [4]. Therefore, the above-mentioned overly simplistic discrimination method cannot accurately determine the validity of the LIBS spectra, that is, whether the spectra originate from the real substrate material beneath the dust layer.

[0005] To more accurately and reliably determine the effectiveness of LIBS spectra and identify whether the spectral data originates from surface dust or the target substrate material, it is necessary to develop more reasonable spectral data clustering or classification methods. However, there are currently no research reports on this issue both domestically and internationally.

[0006] When clustering or classifying LIBS spectral data from different sources, the simplest and most direct way is to process the entire LIBS spectrum directly. However, a single LIBS spectrum can typically contain thousands of data points (each raw LIBS spectrum acquired by MarSCoDe contains 5400 data points). Directly analyzing the entire spectrum is not only very time-consuming, but also increases the difficulty of clustering or classification due to the presence of a large number of redundant features.

[0007] To preserve the main features of thousands of data points in each LIBS spectrum and remove redundant features, feature reduction processing can be performed on the spectral data first, thereby reducing the difficulty of analysis and saving computation time. In this model of dimensionality reduction before analysis, the rationality of the feature reduction algorithm is one of the most critical factors affecting the accuracy of clustering or classification results. Currently, commonly used feature reduction algorithms in data analysis include Principal Component Analysis (PCA) and t-Random Neighbor Embedding (t-SNE), whose principles and disadvantages are as follows:

[0008] For the PCA algorithm, LIBS spectra are typically transformed into a set of linearly uncorrelated eigenvectors through orthogonal transformation. Features appearing earlier in the eigenvectors exhibit higher dissimilarity. Selecting the first N features (depending on the situation) as the new features achieves feature reduction while preserving most of the information in the LIBS spectra. PCA is simple and feasible, and can well preserve the global structure of the data. However, regarding the interpretability of the dimensionality reduction results, the meaning of the principal component feature dimensions in PCA is ambiguous, resulting in poor interpretability. Regarding the preservation of original information, the standard for PCA dimensionality reduction is to select the principal components that maximize the variance of the original data on the new coordinate axes. However, features with small variance are not necessarily unimportant. This standard inevitably leads to the loss of some effective information during the dimensionality reduction process, resulting in information loss. Furthermore, PCA is based on the linear assumption, so it may be difficult to extract the main features of data with non-linear structures. Finally, regarding the robustness of the dimensionality reduction process, PCA may be overloaded by fluctuations or outliers in the dataset and cannot accurately reflect the local structure of the data.

[0009] The t-SNE algorithm typically achieves dimensionality reduction by mapping the similarity between points in a high-dimensional space to a probability distribution in a low-dimensional space, and then minimizing the difference between the distributions in the two spaces. t-SNE performs well in visualizing high-dimensional data and is excellent at preserving local structures within the high-dimensional data; similar data points cluster together in the reduced low-dimensional space, forming distinct classes. However, t-SNE has high computational complexity, especially when dealing with large-scale datasets, requiring a significant amount of time to complete the dimensionality reduction task. Furthermore, using t-SNE requires trying different initialization points and running the algorithm multiple times to select the best result, thus eliminating the influence of local optima. Additionally, t-SNE exhibits poor robustness during dimensionality reduction, with stringent requirements on hyperparameter settings such as the learning rate and number of iterations; improper settings can lead to poor dimensionality reduction or even failure. Finally, regarding data structure information, t-SNE can only preserve local structural information and cannot effectively retain global structural information.

[0010] References

[0011] [1] Newman CE, et al. 7.24-Martian Dust / / SHRODER J F. Treatise onGeomorphology (Second Edition). Oxford; Academic Press. 2022: 637-66.

[0012] [2]SKSharma,et al.Combined remote LIBS and Raman spectroscopy at8.6m of sulfur-containing minerals,and minerals coated with hematite orcovered with basaltic dust.Spectrochimica Acta Part A 68(2007)1036–1045.

[0013] [3]Vasily N. Lednev, et al. Combining Raman and laser induced breakdownspectroscopy by double pulse lasing. Analytical and Bioanalytical Chemistry410 (2018) 277–286.

[0014] [4] Roger C. Wiens, et al. The ChemCam Instrument Suite on the MarsScience Laboratory (MSL) Rover: Body Unit and Combined System Tests. SpaceScience Reviews 170 (2012) 167–227. Summary of the Invention

[0015] To address the aforementioned background and shortcomings of existing technologies, this invention proposes a PHATE-based dimensionality reduction method for identifying the validity of LIBS spectra of dust-covered targets. This method effectively distinguishes between the spectra of surface-covered dust and the spectra of the covered scientific targets during in-situ LIBS exploration on Mars. This method leverages the unique advantages of PHATE in feature reduction and information extraction, applying it to the effective feature extraction of Martian LIBS spectra. Simultaneously, it utilizes the efficient and rapid clustering characteristics of the K-means algorithm, applying it to the clustering analysis of low-dimensional feature vectors. This enables the effective differentiation of three different types of spectral data: dust spectra, dust-substrate transition spectra, and substrate spectra. This provides crucial technical support and a practical solution for the subsequent efficient identification and screening of valid spectra, thereby accurately analyzing the true Martian environment LIBS spectra of dust-covered samples.

[0016] The technical solution of the present invention is as follows:

[0017] The overall process of this technical solution can be divided into six major steps, as shown in the appendix to the instruction manual. Figure 1As shown. The details are as follows:

[0018] S1. Preliminary Preparation: Prepare the detection sample for laser-induced breakdown spectroscopy (LIBS). The sample is a target with a dust-covered surface, prepared through a specific process, i.e., a dust-coated target, hereinafter referred to as the target. The names and composition information of the substrate material and surface dust material for each target are recorded. The target preparation process includes two parts. First, based on the compaction density ρ of the dust material and the area S of the sheet, the dust mass M corresponding to a dust sheet of thickness D is calculated. d The mass to be weighed is M d The dust material is spread evenly in the tablet press mold; then the base material is placed on top of the powder surface and spread evenly. Boric acid powder is placed on the outside of the mold to coat the dust material and the base material. The pressure is set to 238 MPa, the compression time is 1.5 minutes, the mold diameter is 40 mm, and the tablet press is started to complete the target preparation. When preparing each target, the mass of dust, matrix, and boric acid powder should be kept as consistent as possible with the previous target to ensure good consistency in the shape, size, and weight of the prepared targets.

[0019] After pressing, check whether the target surface is completely covered by the dust layer. If the check is correct, put it into a sample bag for storage. Name the sample as "Dust Name" + "Target Substrate Name" and record the chemical composition information of the dust and target substances.

[0020] S2. LIBS spectra of the targets are acquired using a LIBS spectral detection device. During the spectral acquisition of each target, the same location on the target must be continuously excited by laser pulses at least P times to ensure that the dust covering layer on the target surface is broken down and the spectrum of the substrate material can be acquired. The LIBS spectra acquired in the experiment are defined as the original spectral dataset. During the experiment, a LIBS device with high laser energy and longitudinal profile capability should be selected to continuously excite the targets using LIBS, ensuring that the laser pulse energy is sufficient to break down the dust layer at the excited location on the target surface after at least P continuous excitations. During the spectral acquisition process, the changes in elemental characteristic peaks should be monitored to ensure that the surface dust covering layer can be broken down and the LIBS spectrum of the target substrate material can be acquired.

[0021] S3. Preprocessing the LIBS spectra in the raw spectral dataset, where the preprocessing steps generally include dark background subtraction, noise filtering, background baseline removal, wavelength calibration, invalid pixel screening, channel splicing and per-channel spectral normalization. The dark background subtraction operation refers to subtracting the dark background spectrum from the raw LIBS spectrum to obtain the effective spectrum, where the dark background spectrum refers to the spectrum of the spectrometer response when there is no light signal acquired under the same instrument parameters as those for collecting the raw spectrum, which is generally obtained by averaging 60 spectra; the noise filtering operation refers to removing Gaussian noise in the effective spectrum using an adaptive wavelet denoising method to obtain the denoised spectrum; the background baseline removal operation refers to removing the continuous baseline in the denoised spectrum using an asymmetric least squares baseline correction method; the wavelength calibration refers to converting the pixel serial number of the spectrometer into a wavelength value through a multivariate quadratic fitting method; the invalid pixel screening refers to removing the pixel response values of each band of the LIBS spectrum that exceed the wavelength range; the channel splicing refers to splicing the LIBS spectra of multiple bands that have undergone invalid pixel screening into a whole spectrum in the order of wavelength; the per-channel spectral normalization refers to mapping the LIBS spectrum of each channel to the [0,1] interval respectively, so that the sum of the LIBS spectral intensity values in each single channel is 1, and the sum of the intensity values of the entire LIBS spectrum is 3. The preprocessed LIBS spectra are defined as the dust-covered target LIBS spectral dataset.

[0022] S4. In the dust-covered target LIBS spectral dataset, let the number of data points contained in each spectrum be M, which is defined as the spectral dimension; sequentially perform feature dimension reduction operation on all LIBS spectra of each target to reduce the spectral dimension from M to N, where N < M, and the specific value of N is set according to the specific dataset and actual requirements. The feature dimension reduction operation is performed through the affinity-based trajectory embedding heat diffusion potential algorithm PHATE. After the feature dimension reduction operation, each LIBS spectrum is correspondingly converted into an N-dimensional feature vector. As a nonlinear dimension reduction algorithm, the PHATE algorithm is often used to process high-dimensional data to extract its low-dimensional representation, and is especially good at revealing the continuous structure of data (such as development trajectory). For the feature dimension reduction process, the input is all LIBS spectra of a single target, and the output is a matrix composed of low-dimensional feature vectors corresponding to all spectra of the target, with a size of Q×N, where Q is the number of LIBS spectra of the target, and N is the number of features after dimension reduction, which is determined according to the actual situation. The relevant parameters of the PHATE algorithm include ndim, k, t_max and mds_methods. ndim represents the feature dimension of the output vector, which determines the number of features retained after dimension reduction; k represents the number of neighbor points considered when constructing the graph, this variable affects local relationship modeling and can be adjusted according to data density; t_max represents the maximum value of the optimal time step for finding the Markov diffusion process; mds_methods represents the multidimensional scaling analysis method used to map the diffusion potential energy to the low-dimensional space.

[0023] When adjusting the parameters of the PHATE algorithm, a comprehensive consideration of various factors is necessary. The default value of ndim is 2, which outputs a two-dimensional feature vector for two-dimensional visualization analysis. If the analysis results are unsatisfactory, it can be set to a higher dimension (such as 3 or 5). The default value of k is 5. If the input data is sparse, the value of k should be increased appropriately. If the input data is dense, the value of k should be decreased appropriately. t_max determines the maximum number of time steps in the Markov diffusion process. Setting it to a smaller value will encourage the algorithm to pay more attention to the local information of the input data, while setting it to a larger value will pay more attention to the global structure. mds_methods affects the calculation method of the final low-dimensional feature vector. The default is to use metric multidimensional scaling analysis (Metric MDS) for calculation. The other two calculation methods are classical multidimensional scaling analysis (Classical MDS) and non-metric multidimensional scaling analysis (Non-metric MDS), which are suitable for scenarios where the data strictly satisfy the Euclidean distance condition and scenarios where the data has a non-linear relationship, respectively.

[0024] S5. Clustering is performed sequentially on all N-dimensional feature vectors of each target, dividing the feature vectors into three categories: surface dust layer spectrum, dust-matrix transition zone spectrum, and matrix spectrum, respectively. This clustering operation is performed using the K-means clustering algorithm. As a commonly used unsupervised learning clustering algorithm, K-means is widely used due to its efficiency and simplicity. Its core objective is to divide the input data into t clusters, making samples within the same cluster as similar (closely packed) as possible, while samples between different clusters are as different (separated) as possible. For the clustering process, the input is all N-dimensional feature vectors and the number of categories corresponding to the LIBS spectrum of a single target, and the output is the category corresponding to each LIBS spectrum. The relevant parameters of the K-means algorithm include n_clusters, Distance, MaxIter, and Start. Here, n_clusters represents the number of clusters in the data, i.e., the number of categories formed after clustering; Distance refers to the calculation method of the distance metric between the cluster centroid and the sample point in the feature space; MaxIter refers to the maximum number of iterations the algorithm can run; and Start refers to the method for selecting the initial cluster centroid position.

[0025] When adjusting the parameters of the K-means algorithm, a comprehensive consideration of various factors is necessary. `n_clusters` determines the number of clusters, directly affecting the clustering results. It needs to be determined based on prior knowledge of the input data and the specific application scenario before clustering. `Distance` defaults to Euclidean distance, but sometimes other distance metrics are more suitable for data characteristics. When using Euclidean distance, the data can be standardized or scaled to fit the assumptions of Euclidean clustering. `MaxIter` determines the maximum number of iterations per run, affecting the speed and effectiveness of the algorithm's convergence. For large-scale datasets or scenarios that may require more time to converge, it can be appropriately increased. If the data volume is small, the number of iterations can be appropriately reduced to accelerate computation. `Start` determines the location of the initial cluster centroids. By default, a heuristic method is used to repeatedly calculate and find the initial cluster centroids, which can converge to the optimum faster. Other methods include randomly selecting the initial cluster centroids from the dataset.

[0026] S6. Evaluate the execution effect of steps S4 and S5 based on spectral similarity index, and optimize the relevant parameters of the algorithms used in the feature reduction and clustering processes. This invention introduces three evaluation metrics for assessing the accuracy of spectral effectiveness identification: silhouette coefficient S, spectral angle mapping θ, and cluster centroid distance and d. The silhouette coefficient S serves as the core metric for evaluating the performance of the feature reduction and clustering algorithms, while the other two metrics, θ and d, are auxiliary metrics. The silhouette coefficient S is directly normalized to obtain S0. norm The spectral angle plotting is obtained by taking the reciprocal of θ and normalizing it. The cluster centroid distance and d are directly normalized to obtain d. norm , will S norm , and d norm Summing gives Eval norm , with Eval norm The evaluation metric used in subsequent parameter optimization processes is the grid optimization algorithm. In optimizing the parameters in steps S4 and S5, this algorithm sets the grid value range and grid point spacing for each parameter to be optimized, tries every possible parameter combination, and calculates the aforementioned evaluation metric, Eval. norm This continues until all possible combinations of parameters are exhausted, ultimately selecting the one that allows Eval to be satisfied. norm The scheme with the largest parameter combination is taken as the final scheme, thereby completing the relevant parameter optimization.

[0027] The core indicator, silhouette coefficient S, can characterize whether each feature point has been correctly assigned to the corresponding cluster. Therefore, when the algorithm and its parameters used in the clustering process are consistent, silhouette coefficient S can clearly reflect the performance of the feature dimensionality reduction algorithm, that is, whether the feature points of the same category of LIBS spectra can be tightly clustered after feature dimensionality reduction, while the feature points of different categories of LIBS spectra can be significantly separated. Silhouette coefficient S considers both the compactness within the cluster and the separation between the clusters. The specific calculation method is shown in formula (1):

[0028]

[0029] Where S(i) is the silhouette coefficient of the i-th feature point; a(i) represents the average distance between the i-th feature point and other feature points in its cluster, called "cohesion," reflecting the compactness within the cluster; b(i) represents the average distance between the i-th feature point and the feature points of the nearest non-cluster, called "dispersion," reflecting the separation between clusters. When the silhouette coefficient is close to 1, it means that the i-th feature point is compact within its cluster and well separated from other clusters; when the silhouette coefficient is close to 0, it means that the feature point is at the cluster boundary, or the distance between points within a cluster is comparable to the distance between clusters; when the silhouette coefficient is close to -1, it means that the feature point may have been incorrectly assigned to the wrong cluster. To evaluate the effectiveness of the entire feature dimensionality reduction process, the average silhouette coefficient of all feature points is usually calculated, and this average value is used as the silhouette coefficient of the entire feature dimensionality reduction process for subsequent performance evaluation.

[0030] The auxiliary index spectral angle mapping θ is a common method for spectral similarity analysis. This method evaluates the similarity between two spectra by calculating the angle between the sample spectrum and the reference spectrum. To verify the feature reduction performance of PHATE and the clustering performance of the K-means algorithm, this invention uses the LIBS spectrum corresponding to the feature point closest to the centroid of each cluster as the reference spectrum of that cluster, and the remaining spectra corresponding to the feature points of that cluster as the sample spectra. The angle between each sample spectrum and the reference spectrum is calculated in turn and averaged. Thus, the similarity of the LIBS spectra corresponding to the cluster is used to judge the quality of feature reduction and clustering. The specific calculation method is shown in formula (2):

[0031]

[0032] Among them, X * For the reference spectrum, X i The sample spectrum.

[0033] The auxiliary index cluster centroid distance and d are evaluation indicators for the accuracy of LIBS spectral identification of dust-coated targets proposed in this invention. This index calculates the sum of the Euclidean distances between the centroid positions of the clusters corresponding to the matrix LIBS spectra and the centroid positions of the clusters corresponding to the dust LIBS spectra and the dust-matrix transition zone LIBS spectra. The specific calculation method is shown in formula (3):

[0034]

[0035] Where d represents the cluster centroid distance and L subtrate L dust L dust-subtrate These represent the centroid position vectors of the corresponding clusters in the matrix LIBS spectrum, the dust LIBS spectrum, and the dust-matrix transition region LIBS spectrum, respectively.

[0036] The mechanism of action of this invention is as follows:

[0037] This invention applies the PHATE algorithm to dimensionality reduction of LIBS data for dust-covered targets. The PHATE algorithm encodes local data information through local similarity and uses latent distance to encode global relationships within the data. Finally, it embeds the latent distance information into a low-dimensional dataset for visualization, ultimately achieving data dimensionality reduction. In practical LIBS data dimensionality reduction, the PHATE algorithm effectively overcomes the shortcomings of conventional feature reduction algorithms such as PCA and t-SNE. By denoising the spectral data, it accurately captures the true structure of high-dimensional data, thus providing robust and clear visualization of the spectral data and achieving dimensionality reduction. Applying the PHATE algorithm to the LIBS spectral identification task, the visualization results show a significant improvement in data clustering compared to the two conventional algorithms mentioned above, exhibiting higher spectral discriminative power.

[0038] Beneficial effects

[0039] Compared with the prior art, the advantages of the present invention are:

[0040] Compared with schemes that directly identify the validity of the entire LIBS spectrum without using feature dimensionality reduction algorithms, the PHATE feature dimensionality reduction algorithm used in this invention can significantly reduce the feature dimension, effectively reduce the impact of noise or redundant feature information in the spectrum on subsequent clustering results, thereby improving clustering accuracy, computation speed, and interpretability of results.

[0041] Second, compared with LIBS spectral validity identification schemes that use PCA as a feature dimensionality reduction algorithm, the PHATE algorithm used in this invention can more accurately capture the true structure of high-dimensional LIBS data, resulting in superior visualization and interpretability. Furthermore, unlike the PCA algorithm, which focuses only on the global structure of the data, the PHATE algorithm comprehensively preserves both the global and local structures, enabling it to identify nonlinear effects in LIBS data. This leads to more accurate results and is almost unaffected by fluctuations in the dataset, unlike the PCA algorithm.

[0042] Third, compared with LIBS spectral validity identification schemes that use the t-SNE algorithm as a feature dimensionality reduction algorithm, the PHATE algorithm used in this invention requires relatively fewer computational resources and has a shorter running time when processing LIBS data, which can significantly save time costs. Furthermore, unlike the t-SNE algorithm, which only focuses on local structural information of the data, the PHATE algorithm can effectively acquire and retain both local and global structural information of the data, and is almost unaffected by hyperparameter limitations and local optima.

[0043] In summary, this invention uses the PHATE algorithm as a feature dimensionality reduction method for identifying the spectral effectiveness of dust-coated LIBS targets. It also combines this with the K-means algorithm to distinguish whether the target of LIBS laser pulse ablation is the matrix sample or the simulated Martian dust covering the surface. The PHATE algorithm effectively transforms high-dimensional LIBS spectral data into low-dimensional feature vectors, extracting core spectral features while preserving the global data structure. The K-means algorithm effectively performs cluster analysis on the low-dimensional spectral feature vectors, thereby distinguishing between three different types of spectral data: dust spectra, dust-matrix transition spectra, and matrix spectra, achieving accurate identification of the LIBS spectral effectiveness of dust-coated targets. The advantages of this invention include: simple and efficient data processing, high degree of feature dimensionality reduction, complete data structure information, high accuracy of spectral validity identification results, strong robustness, and strong interpretability. It helps to screen out truly effective target spectra from dust-covered target LIBS spectra, better meeting the application needs of LIBS spectral validity identification of non-clean surface samples. It is suitable for rapid analysis of LIBS spectra collected in non-clean environments such as field exploration and deep space exploration, and therefore has significant value in the field of laser spectral analysis technology. Attached Figure Description

[0044] Figure 1 This is a schematic diagram of the overall process of the technical solution.

[0045] Figure 2 Comparison of LIBS spectra at lines 1, 40, 80, 120, and 160 in the continuously excited LIBS spectrum of the "dolomite powder + basalt" target.

[0046] Figure 3The images show the dimensionality reduction results of the PHATE algorithm and two other common dimensionality reduction algorithms for the LIBS spectrum of dust-covered targets. (a) Dimensionality reduction image based on the PHATE algorithm, (b) Dimensionality reduction image based on the PCA algorithm, and (c) Dimensionality reduction image based on the t-SNE algorithm. The numbers in the color bar on the right represent the spectral sequence numbers. Detailed Implementation

[0047] The following is a specific experimental case to illustrate the application process of the method described in the invention content section:

[0048] 1. In this example, the detection sample prepared for laser-induced breakdown spectroscopy (LIBS) is a dust-coated target prepared through a specific process, hereinafter referred to as the target. The dust material is JMSS-1 model simulated Martian soil, with a composition and content basically consistent with Martian surface dust. The base material selected includes 10 potentially hydrous minerals from the Martian surface: chlorite, serpentine, calcite, talc, zeolite, muscovite, sodium montmorillonite, magnesite, kaolinite, and kaolinite. When preparing the dust-coated target, firstly, calculate the mass of dust required to press a 0.6mm dust sheet based on the compaction density of the dust material. Weigh the corresponding mass of dust material and spread it evenly in the tablet press mold. Then, place the base material on the powder plane and spread it evenly. Put boric acid powder on the outside and top of the mold to coat the dust material and the base material. Start the fully automatic fluorescent tablet press to press the powder sample into a cake shape. The parameters are set as follows: upper pressure limit 30T, lower pressure limit 27T, mold size 40mm, mold pressure 238MPa, and pressing time 2 minutes. After pressing, check whether the target surface is completely covered by the dust layer. After confirming that there are no errors, make the corresponding label. The naming format is "dust name" + "target base material name", for example, "JMSS-1+chlorite".

[0049] 2. Acquire LIBS spectral data of the target using a LIBS spectral detection device. In this example, the prepared target was placed inside a vacuum chamber simulating the Martian atmospheric environment, and LIBS spectra were acquired in this environment. The detection device was a ground backup of the Mars Surface Composition Analyzer (MarSCoDe) payload, with a LIBS detection distance of 2 m, a laser single-pulse energy of approximately 9 mJ, and a laser wavelength of 1064.4 nm. One sampling location was selected for each sample, and 180 valid LIBS spectra were continuously acquired at each location, totaling 1800 spectra. These spectra constituted the original spectral dataset. (See attached instruction manual.) Figure 2The changes in the intensity of elemental characteristic peaks in the 240-340 nm band of the 1st, 40th, 80th, 120th, and 160th LIBS spectra in a basalt sample covered with dolomite powder are presented. The results show that with increasing excitation count, the intensity of the characteristic peaks representing Mg and Ca, which are associated with dolomite powder, gradually decreases, while the characteristic peaks representing Si, Fe, and Al, which are associated with basalt sample, appear and gradually increase in intensity. This indicates that the dust cover layer was penetrated, and the underlying substrate material was excited by LIBS, consistent with the experimental hypothesis.

[0050] 3. The LIBS spectra in the original spectral dataset are preprocessed, including dark background subtraction, noise filtering, background baseline removal, wavelength calibration, invalid pixel screening, channel stitching, and channel-specific spectral normalization. The preprocessed LIBS spectra are defined as the dust-covered target LIBS spectral dataset, and each LIBS spectrum can be represented as a 1×4506 vector.

[0051] 4. The entire LIBS spectrum of each target is sequentially subjected to feature dimensionality reduction to obtain a two-dimensional feature vector of the target LIBS spectrum. This feature dimensionality reduction process is performed using the affinity-based trajectory embedding thermal diffusion potential algorithm PHATE. According to the invention description, after comprehensively adjusting the relevant parameters of the PHATE algorithm and performing parameter optimization in step 6, ndim is set to 2, i.e., outputting two-dimensional visual features; k is set to 5, i.e., the number of neighbor points considered when constructing the graph is 5; t_max is set to 100, i.e., the maximum number of steps in finding the optimal time step for the Markov diffusion process is no more than 100; and mds_methods is set to 'MetricMDS', i.e., using metric multidimensional scaling analysis to map the diffusion potential energy to a low-dimensional space and output a two-dimensional visual feature vector.

[0052] After setting the algorithm parameters, the entire LIBS spectrum of each target is sequentially fed into the PHATE algorithm for training and feature dimensionality reduction. The final output is a two-dimensional visualization feature vector corresponding to the LIBS spectrum. This vector can be used to intuitively observe the differences between the surface dust layer spectrum, the dust-matrix transition zone spectrum, and the matrix spectrum, and can also be used for subsequent cluster analysis. To demonstrate the superiority of the PHATE algorithm in feature dimensionality reduction, two other feature dimensionality reduction algorithms were also used to perform feature dimensionality reduction on the dust-covered target LIBS spectrum dataset, and the results were compared with those of the PHATE algorithm. In this example, the three feature dimensionality reduction algorithms are the PHATE algorithm, the PCA algorithm, and the t-SNE algorithm. The feature dimensionality reduction results for the entire LIBS spectrum of the "JMSS-1+ chlorite" target are shown in the appendix of the manual. Figure 3As shown in the figure, the 2D visualization features reduced by the PHATE algorithm exhibit clear class boundaries, easily separating the surface dust layer spectrum, the dust-matrix transition region spectrum, and the matrix spectrum. The other two feature reduction algorithms output overlapping or mixed 2D visualization feature distributions, making it difficult to distinguish clear boundaries between different categories or clusters through simple visual observation. They primarily reflect the continuous variation of feature points and do not well preserve the boundary information of the original high-dimensional spectral data. After completing the cluster analysis, we will further compare the results of the three feature reduction algorithms from multiple evaluation metrics.

[0053] 5. Clustering is performed sequentially on all two-dimensional feature vectors of each target, dividing the feature vectors into three categories, representing that the LIBS spectrum corresponding to the feature vector belongs to the surface dust layer spectrum, the dust-matrix transition zone spectrum, and the matrix spectrum, respectively. The clustering operation is performed based on the K-means clustering algorithm. As described in the invention, after comprehensively adjusting the relevant parameters of the K-means algorithm and performing parameter optimization in step 6, n_clusters is set to 3, indicating that all two-dimensional feature vectors of each target need to be divided into three categories; Distance is set to 'sqeuclidean', indicating that the K-means algorithm uses Euclidean distance to represent the distance between feature points and the centroid of each cluster when calculating the distance; MaxIter is set to 100, indicating that the maximum number of iterations per run is 100, which is sufficient for the clustering task of 180 feature points each time; Start is set to 'plus', which uses a heuristic method to repeatedly calculate and find the initial cluster centroid, and can converge to the optimum faster.

[0054] After setting the algorithm parameters, all two-dimensional feature vectors of each target were sequentially fed into the K-menas algorithm for training and cluster analysis, ultimately obtaining the centroid position of each cluster and the cluster category corresponding to each two-dimensional feature vector. Based on the clustering results, it is possible to identify whether each LIBS spectrum belongs to the surface dust layer spectrum, the dust-matrix transition zone spectrum, or the matrix spectrum during continuous excitation, providing an effective technical solution for subsequent spectral identification and screening of potential Martian minerals of interest. We further evaluated the results of dimensionality reduction and cluster analysis of the LIBS spectral features for each target in detail, based on specific indicators.

[0055] 6. Based on relevant indicators, evaluate the execution effect of steps 4 and 5, and comprehensively optimize the relevant parameters of the algorithms used in the feature reduction and clustering processes as described in the invention, ultimately obtaining the optimal feature reduction and clustering analysis results. The main evaluation indicators include the silhouette coefficient S, spectral angle mapping θ, and cluster centroid distance and d. The silhouette coefficient S serves as the core indicator for evaluating the performance of the algorithms used in the feature reduction and clustering processes, while the other two indicators are auxiliary indicators. The parameter optimization scheme is a grid optimization algorithm. In this example, we used the PHATE algorithm, PCA algorithm, and t-SNE algorithm to perform feature reduction on the LIBS spectra of 10 dust-covered targets, and used the K-means algorithm to perform cluster analysis on the two-dimensional feature vectors obtained after dimensionality reduction, obtaining the spectral category to which each LIBS spectrum belongs. To evaluate the performance of different feature reduction algorithms and K-means algorithm combinations for feature reduction and clustering analysis, we calculated the above three evaluation indicators for the optimal feature reduction and clustering analysis results for each dust-covered target, as described in S6 of the invention.

[0056] The core metric, silhouette coefficient S, characterizes whether each feature point has been correctly assigned to its corresponding cluster. Therefore, when the clustering algorithm and its parameters are consistent, the silhouette coefficient S clearly reflects the performance of the feature reduction algorithm, i.e., whether the feature reduction algorithm can tightly aggregate feature points of the same category of LIBS spectra while significantly separating feature points of different categories of LIBS spectra. In the calculation process, we calculated and averaged the silhouette coefficients of all feature points obtained by a certain feature reduction algorithm for each target. It was found that the average silhouette coefficient of the two-dimensional feature points obtained by the PHATE algorithm for the LIBS spectra of 10 dust-covered targets reached 0.78, while the average silhouette coefficients of the two-dimensional feature points obtained by the PCA and t-SNE algorithms were 0.69 and 0.72, respectively, both lower than that of the PHATE algorithm. Furthermore, the maximum silhouette coefficient of the two-dimensional feature points obtained by the PHATE algorithm was 0.89, and the minimum was 0.71, exhibiting the highest silhouette coefficient on most targets, only matching the other two algorithms on a few targets. This indicates that the feature dimensionality reduction algorithm based on the PHATE algorithm can better preserve the boundary information of the target LIBS spectrum after dimensionality reduction, which facilitates the K-means algorithm in distinguishing spectral categories. Table 1 lists the silhouette coefficients S of the LIBS spectrum of each dust-covered target after dimensionality reduction by the three feature dimensionality reduction algorithms and clustering by the K-means algorithm.

[0057] Table 1

[0058] Sample Name <![CDATA[S PHATE ]]> <![CDATA[S PCA ]]> <![CDATA[S t-SNE ]]> JMSS-1+ Chlorite 0.79 0.56 0.66 JMSS-1+ Serpentine 0.78 0.66 0.72 JMSS-1+ Calcite 0.79 0.68 0.73 JMSS-1+ Talc 0.79 0.68 0.70 JMSS-1+ Zeolite 0.77 0.72 0.78 JMSS-1+Mica 0.89 0.75 0.64 JMSS-1+ Sodium Montmorillonite 0.73 0.73 0.80 JMSS-1+ Magnesite 0.74 0.75 0.68 JMSS-1+ Kaolinite 0.80 0.63 0.69 JMSS-1+ Kaolin 0.71 0.69 0.76 average 0.78 0.69 0.72

[0059] The auxiliary index spectral angle mapping θ focuses on analyzing the effectiveness of feature dimensionality reduction and clustering analysis at the LIBS spectral level. First, the LIBS spectrum corresponding to the feature point closest to the centroid of each category is used as the reference spectrum for that category. The average spectral angle mapping of other LIBS spectra in that category compared to the reference spectrum is then used as the spectral angle mapping for that category. This process is repeated for all three categories, and the average is taken to obtain the similarity of spectra within each category of the target's LIBS spectrum. The results show that after dimensionality reduction using three feature dimensionality reduction algorithms and K-means clustering, the spectral angle mapping index of each combination method is basically similar. This indicates that the dimensionality reduction and clustering results obtained by the three combination methods can approximately represent the category information of the original LIBS spectrum. Table 2 lists the average spectral angle mapping θ of the LIBS spectrum for each dust-covered target.

[0060] Table 2

[0061] Sample Name <![CDATA[θ PHATE ]]> <![CDATA[θ PCA ]]> <![CDATA[θ t-SNE ]]> JMSS-1+ Chlorite 5.98 6.90 6.12 JMSS-1+ Serpentine 6.52 6.51 6.44 JMSS-1+ Calcite 6.68 6.47 6.07 JMSS-1+ Talc 5.61 6.29 5.49 JMSS-1+ Zeolite 7.83 6.74 6.29 JMSS-1+Mica 8.81 8.54 8.55 JMSS-1+ Sodium Montmorillonite 6.21 6.41 5.90 JMSS-1+ Magnesite 8.75 8.65 7.67 JMSS-1+ Kaolinite 7.05 6.85 7.02 JMSS-1+ Kaolin 6.97 6.12 6.85 average 7.04 6.95 6.64

[0062] The auxiliary indices, cluster centroid distance and d, are used to evaluate the separation between the matrix spectral category and the other two categories. The sum of the distances between the centroids of the matrix spectral category and the centroids of the other two categories is calculated to assess whether the feature reduction algorithm adequately separates the LIBS spectra of the three categories. After clustering, the average cluster centroid distances of the 10 targets obtained by the three feature reduction algorithms were 4.49, 4.47, and 4.79, respectively. The PHATE algorithm showed relatively high cluster centroid distances across all 10 targets, while the PCA and t-SNE algorithms obtained significantly lower results on "JMSS-1+chlorite" and "JMSS-1+calcite," significantly lower than the PHATE algorithm. This indicates that after dimensionality reduction using the PHATE algorithm and subsequent K-means clustering, the clusters representing matrix spectral categories show significant differences from those representing dust and dust-target transition region spectral categories. Furthermore, similar results were obtained across different target LIBS spectra, with no outliers observed, demonstrating the strong robustness of the PHATE algorithm. Table 3 lists the cluster centroid distances and d for each dust-coated target's LIBS spectra after dimensionality reduction using the three feature reduction algorithms and K-means clustering.

[0063] Table 3

[0064] Sample Name <![CDATA[d PHATE ]]> <![CDATA[d PCA ]]> <![CDATA[d t-SNE ]]> JMSS-1+ Chlorite 4.46 3.68 3.60 JMSS-1+ Serpentine 4.53 4.22 4.44 JMSS-1+ Calcite 4.36 4.05 4.08 JMSS-1+ Talc 4.77 4.86 4.16 JMSS-1+ Zeolite 4.39 5.82 5.04 JMSS-1+Mica 4.60 4.52 5.65 JMSS-1+ Sodium Montmorillonite 4.43 4.29 5.28 JMSS-1+ Magnesite 4.37 4.48 5.34 JMSS-1+ Kaolinite 4.74 4.25 4.97 JMSS-1+ Kaolin 4.24 4.50 5.33 average 4.49 4.47 4.79

[0065] The results presented in this example demonstrate that, in terms of quantitative evaluation metrics for feature reduction and clustering effects, the PHATE algorithm outperforms both the PCA and t-SNE algorithms for the core metric, the silhouette coefficient S. For auxiliary metrics such as spectral angle mapping θ and cluster centroid distance d, the performance of the PHATE algorithm is similar to that of the PCA and t-SNE algorithms, but the PHATE algorithm exhibits stronger robustness. Regarding the intuitive visualization of clustering effects, the two-dimensional visualized feature points obtained based on the PHATE feature reduction algorithm show significant intra-class compactness and inter-class separation, effectively preserving important global features and boundary information of the high-dimensional LIBS spectrum, resulting in highly interpretable results. In summary, this invention offers advantages such as high accuracy, strong robustness, and strong interpretability in the application scenario of identifying the effectiveness of LIBS spectra of dust-covered targets, making it of significant value for analyzing LIBS spectral data from Mars exploration.

[0066] The above specific embodiments are merely explanations of the present invention and are not intended to limit the present invention. After reading this specification, those skilled in the art can make modifications to these embodiments without contributing any inventive step, but as long as they are within the scope of the claims of the present invention, they are protected by patent law.

Claims

1. A method for identifying the effectiveness of LIBS spectra of dust-coated targets based on PHATE dimension reduction, characterized in that... comprising the following steps: S1. Pre-preparation: Prepare detection samples for laser-induced breakdown spectroscopy (LIBS), where the samples are targets with dust covered on the surface prepared through a specific process, i.e., dust-covered surface targets, hereinafter referred to as "targets" for short; count and record the names and composition information of the base material and the surface dust material of each target; S2. Collect the LIBS spectra of the targets using a LIBS spectral detection device. During the spectral collection process for each target, the same position of the target needs to be continuously excited more than P times by laser pulses, so as to ensure that the dust covering layer on the target surface is broken through and the spectrum of the base material can be collected. The LIBS spectra collected through experiments are defined as the original spectral dataset; S3. Preprocess the LIBS spectra in the original spectral dataset. The preprocessing steps generally include dark background subtraction, noise filtering, background baseline removal, wavelength calibration, invalid pixel screening, channel splicing, and per-channel spectral normalization. The preprocessed LIBS spectra are defined as the dust-covered surface target LIBS spectral dataset; S4. In the dust-covered surface target LIBS spectral dataset, set the number of data points contained in each spectrum as M, and define M as the spectral dimension; sequentially perform feature dimension reduction operation on all LIBS spectra of each target, to reduce the spectral dimension from M to N, where N<M, and the specific value of N is set according to the specific dataset and actual requirements. The feature dimension reduction operation is performed through the affinity-based trajectory embedding heat diffusion potential algorithm PHATE; after the feature dimension reduction operation, each LIBS spectrum is correspondingly converted into an N-dimensional feature vector; S5. Sequentially perform clustering operation on all N-dimensional feature vectors of each target, and divide the feature vectors into three categories, which respectively indicate that the LIBS spectrum corresponding to the feature vector belongs to the surface dust layer spectrum, the dust-matrix transition region spectrum, and the matrix spectrum. The clustering operation is performed through the K-means clustering algorithm; S6. Evaluate the execution effect of steps S4 and S5 based on the spectral similarity index, and optimize the relevant parameters of the algorithms used in the feature dimension reduction process and the clustering process.

2. The method for identifying the effectiveness of LIBS spectra of dust-coated targets based on PHATE dimensionality reduction as described in claim 1, characterized in that: In step S1, the target preparation process includes two parts. First, based on the compaction density ρ of the dust material and the area S of the sheet, the dust mass M corresponding to the dust sheet with thickness D is calculated. d The mass to be weighed is M d The dust material is first spread evenly in the tablet press mold; then the base material is placed on the powder plane and spread evenly. Boric acid powder is placed on the outside of the mold to coat the dust material and the base material. The tablet press is then started to complete the preparation of the target.

3. The method for identifying the effectiveness of LIBS spectra of dust-coated targets based on PHATE dimensionality reduction as described in claim 1, characterized in that: In step S2, when performing spectral collection with the LIBS spectral detection device, it is necessary to ensure that the laser pulse energy is sufficient to break through the dust layer at the excited position on the target surface after more than P times of continuous excitation.

4. The method for identifying the effectiveness of LIBS spectra of dust-coated targets based on PHATE dimensionality reduction as described in claim 1, characterized in that: In step S3, when preprocessing the LIBS spectra in the original spectral dataset, the dark background subtraction operation refers to subtracting the dark background spectrum from the original LIBS spectrum to obtain an effective spectrum, where the dark background spectrum refers to the spectrum of the spectrometer's response when there is no light signal, acquired under the same instrument parameters as those for collecting the original spectrum; The noise filtering operation refers to removing Gaussian noise in the effective spectrum by using an adaptive wavelet denoising method, so as to obtain a denoised spectrum; The background baseline removal operation refers to removing the continuous baseline in the denoised spectrum by using an asymmetric least squares baseline correction method; Wavelength calibration refers to converting the spectrometer pixel number into wavelength value through multivariate quadratic fitting; invalid pixel removal refers to removing the pixel response value of each band of the LIBS spectrum that exceeds the wavelength range; channel splicing refers to splicing multiple bands of LIBS spectrum that have been removed by invalid pixel removal into a whole line in wavelength order; channel spectrum normalization refers to mapping the intensity value of each channel of LIBS spectrum to the [0,1] interval, so that the sum of the intensity values ​​of the LIBS spectrum in each single channel is 1.

5. The method for identifying the LIBS spectral effectiveness of dust-coated targets based on PHATE dimensionality reduction as described in claim 1, characterized in that: In step S4, a feature dimensionality reduction algorithm based on PHATE is designed, and the relevant parameters of the PHATE algorithm are comprehensively adjusted, including ndim, k, t_max, and mds_methods. Here, ndim represents the feature dimension of the output vector; k represents the number of nearest neighbors considered when constructing the graph, which affects the modeling of local relationships and can be adjusted according to the data density; t_max represents the maximum value of the optimal time step for finding the Markov diffusion process; and mds_methods represents the multidimensional scaling analysis method used to map the diffusion potential energy to a low-dimensional space. For the feature dimensionality reduction process, the input is the total LIBS spectrum of a single target, and the output is a matrix composed of low-dimensional feature vectors corresponding to the total spectrum of the target, with a size of Q×N, where Q is the number of LIBS spectra of the target, and N is the number of features after dimensionality reduction.

6. The method for identifying the effectiveness of LIBS spectra of dust-coated targets based on PHATE dimensionality reduction as described in claim 1, characterized in that: In step S5, a K-means-based clustering algorithm is designed, and relevant parameters of the K-means algorithm are comprehensively adjusted, including n_clusters, Distance, MaxIter, and Start. Here, n_clusters represents the number of clusters in the data, that is, the number of categories formed after clustering; Distance refers to the calculation method of the distance metric between the cluster centroid and the sample point in the feature space; MaxIter refers to the maximum number of iterations of the algorithm; and Start refers to the method for selecting the initial cluster centroid position. For the clustering process, the input is all N-dimensional feature vectors and the number of categories corresponding to a single target LIBS spectrum, and the output is the category corresponding to each LIBS spectrum.

7. The method for identifying the LIBS spectral effectiveness of dust-coated targets based on PHATE dimensionality reduction as described in claim 1, characterized in that: In step S6, three evaluation metrics are introduced to assess the accuracy of spectral effectiveness identification: silhouette coefficient S, spectral angle mapping θ, and cluster centroid distance and d. Silhouette coefficient S serves as the core metric for evaluating the performance of feature dimensionality reduction and clustering algorithms, while the other two metrics, θ and d, are auxiliary metrics. The three metrics are normalized and then summed to obtain Eval. norm , with Eval norm The evaluation metric used in subsequent parameter optimization processes is the grid optimization algorithm. In optimizing the parameters in steps S4 and S5, this algorithm sets the grid value range and grid point spacing for each parameter to be optimized, tries every possible parameter combination, and calculates the aforementioned evaluation metric, Eval. norm This continues until all possible combinations of parameters are exhausted, ultimately selecting the one that allows Eval to be satisfied. norm The scheme with the largest parameter combination is taken as the final scheme, thereby completing the relevant parameter optimization.

Images