Multi-modal noise reduction evaluation model based on density sensitive clustering

Abstract

The invention discloses a multi-modal noise reduction evaluation model based on density sensitive clustering, and the model proposes a bidirectional optimization framework of dynamic coupling unsupervised feature extraction and supervised dimension reduction for the dimension reduction demand of high-dimensional data before the high-dimensional data is input into a machine learning model. A topological feedback channel between a high-dimensional space and a low-dimensional manifold is constructed, and a comprehensive scoring system of six types of orthogonalization evaluation indexes is combined, so that automatic selection of a dimension reduction algorithm is realized. The method comprises the following steps: designing a cross-space structure fidelity verification mechanism based on bidirectional topological mapping, a dynamic range calibration and robust normalization system of a multi-modal index, an adaptive search and quality constraint engine of a density clustering parameter, and a composite scoring function design oriented to resource optimization. The method is suitable for a high-noise multi-modal data scene formed by multi-dimensional data including spectral wavelength, intensity, pulse width and the like and different mathematical quantities, all dimensions of the multi-dimensional data can be associated with different databases, and potential association of statistical or physical significance exists among the data.

Description

Technical Field

[0001] This invention relates to the field of data dimensionality reduction and machine learning preprocessing technology, specifically to a multimodal noise reduction evaluation model based on density-sensitive clustering. Background Technology

[0002] In machine learning-driven analysis systems, dimensionality reduction preprocessing of high-dimensional data is a core step in balancing model complexity and computational efficiency. Its essence lies in using mathematical mapping to remove redundant noise and capture essential features, thereby constructing a compact and information-preserving low-dimensional representation for downstream tasks. Taking Laser-Induced Breakdown Spectroscopy (LIBS) as an example, this technique uses ultrashort pulse lasers to excite samples and generate transient plasmas. The radiation spectrum includes continuous background radiation (originating from plasma bremsstrahlung and recombination radiation), discrete characteristic peaks (corresponding to elemental characteristic emission lines), and instrument noise (such as detector dark current and optical system scattering). With spectrometer resolution breaking through the micrometer band and covering the near-infrared, a single spectrum can encompass tens of thousands of wavelength channels, forming an ultra-high-dimensional sparse matrix. When such data is directly input into machine learning models for qualitative identification of material composition or quantitative concentration inversion, it not only faces the risk of overfitting due to the curse of dimensionality but also suffers from significantly reduced model convergence speed and generalization ability due to noise and collinearity. Therefore, constructing a dimensionality-reduced model adapted to the spectral characteristics has become a key technical bottleneck for improving the accuracy and real-time performance of LIBS analysis.

[0003] The composition of multidimensional data must meet the following scientific characteristics: (1) each dimension can represent different mathematical or physical quantities; (2) there is statistical correlation or physical constraint relationship between dimensions; (3) different dimensional subsets can be mapped to heterogeneous databases (such as the spectral wavelength dimension corresponding to the atomic emission spectrum library, and the time-domain pulse width dimension corresponding to the laser energy feature library). The current mainstream dimensionality reduction algorithms are based on different mathematical paradigms and optimization objectives, and their applicability is highly dependent on the inherent structure of the data and the requirements of the analysis task. However, the strong background fluctuations, elemental spectral line overlap and matrix effects of LIBS spectra make the feature extraction effects of different dimensionality reduction models show significant differences; inappropriate dimensionality reduction strategies may lead to the loss of key fingerprint information (such as PCA ignoring nonlinear features) or noise amplification (such as Autoencoder overfitting local noise), which in turn causes systematic bias in downstream classifiers.

[0004] Traditional solutions rely on empirical trial and error, sequentially applying multiple dimensionality reduction models to generate feature subsets and comparing their performance on machine learning tasks. For basic qualitative analysis tasks (such as binary classification), the computational cost is acceptable when using lightweight models (such as logistic regression and K-nearest neighbors) combined with a single dimensionality reduction method. However, for high-precision quantitative analysis or complex multi-classification scenarios, it is usually necessary to introduce deep neural network architectures (such as residual networks and attention mechanism models) and couple them with intelligent optimization algorithms (such as Bayesian optimization, genetic algorithms, particle swarm optimization, and sparrow search) for hyperparameter search, while also adding network search or cross-validation to improve model generalization. While this process can significantly improve prediction accuracy, the computation time for a single model training and optimization cycle can be several hours. If it is necessary to evaluate the performance differences of multiple dimensionality reduction methods, the overall computational cost will increase linearly or even exponentially with the number of dimensionality reduction algorithms. For example, in LIBS qualitative classification tasks, a single batch of data covers tens of thousands of spectra. Performing full-process optimization and validation for each dimensionality reduction method would severely limit real-time analysis capabilities due to the high time cost. Even more critically, implicit noise introduced into the spectral data by environmental disturbances or instrument drift may further cause the optimality of different dimensionality reduction methods to dynamically change with the dataset. Therefore, establishing a fast and adaptive dimensionality reduction algorithm evaluation and optimization mechanism during the preprocessing stage to reduce redundant computational resource consumption and improve detection efficiency has become a key scientific problem in optimizing high-dimensional data analysis workflows. Summary of the Invention

[0005] The purpose of this invention is to address the aforementioned problems by providing a multimodal noise reduction evaluation model based on density-sensitive clustering. This model balances cluster quality verification, supervision information utilization, and topology preservation in a multi-dimensional evaluation process, thereby solving the problems of fragmented evaluation criteria and wasted computational resources in the selection of dimensionality reduction algorithms for high-dimensional data and achieving automated optimization of dimensionality reduction strategies for different datasets.

[0006] The technical solution of the present invention is as follows: A multimodal noise reduction evaluation model based on density-sensitive clustering includes the following steps: High-dimensional data preprocessing and topological benchmark construction: Standardize the original high-dimensional dataset to eliminate differences in feature dimensions and construct the topological benchmark of the original high-dimensional space. Multimodal dimensionality reduction and dynamic parameter search: It iterates through a variety of preset dimensionality reduction algorithms that need to be evaluated, loads the dimensionality reduction data files generated by the dimensionality reduction algorithms, obtains the normalized matrix of each dimensionality reduction algorithm, and uses the DBSCAN algorithm to perform dynamic parameter search on each normalized matrix to obtain pseudo-label vectors; Density clustering and multimodal index calculation, based on generated pseudo-labels, comprehensively evaluate the dimensionality reduction effect from three dimensions: unsupervised clustering quality, supervised consistency, and topology preservation through six types of orthogonalization indices; Dynamic normalization and weighted scoring are used to examine a multimodal index set through six types of orthogonal indicators. A comprehensive scoring function is constructed through the coupled operation of the directional correction matrix, the dynamic range normalization matrix and the weight matrix. Results storage and automated selection: The complete evaluation data is stored in a structured file, and the dimensionality reduction method with the highest score is selected as the strategy based on the weight vector and the overall score ranking.

[0007] Furthermore, the dynamic normalization and weighted scoring includes the following: Set directional focus correction matrix Its diagonal elements This represents the directional correction factor for each indicator, with the negative indicator DB having a correction factor of -1 and the other positive indicators having a correction factor of 1; the directional correction focusing matrix will convert the original indicator vector... Converting to a unified, positively additive form, and then unifying the direction of all indicators through directional correction: , , Based on historical data distribution, a dynamic normalized parameter matrix is ​​constructed. row vector Indicates the first Normalization range of each indicator: , in, The metric is based on the number of nearest neighbors. Stratified percentiles were calculated; the normalization function was set to a piecewise linear mapping. , For the DB metric, its normalization requires direction calculation: , Set weight vector ,satisfy ; The comprehensive scoring function achieves a linear combination of normalized indicators and weights through matrix multiplication, where... The normalized index vector: .

[0008] Furthermore, the multimodal dimensionality reduction and dynamic parameter search include the following: Iterate through the preset dimensionality reduction algorithms to be evaluated, load the dimensionality reduction data files generated by the algorithms, and output the low-dimensional feature matrix of each algorithm. Standardization is performed again to eliminate scale differences, i.e., the mean is calculated for each column of features. and standard deviation The standardized matrix is ​​obtained. ; The standardized low-dimensional feature matrix, together with the true labels, constitutes the evaluation sample set. , For the sample The status indicates, For the corresponding supervision labels: , The DBSCAN algorithm is used for each Perform dynamic parameter search; let the distance matrix in the reduced-dimensional space be... Its elements ; Neighborhood radius parameter of DBSCAN Generate within the interval using linear interpolation One, that is Minimum number of samples Select a fixed value; For each dimensionality reduction method, the parameter combination Performing DBSCAN clustering yields pseudo-label vectors. Furthermore, noise points are recoded: if a sample is labeled as noise, it is assigned to a newly added independent cluster, thus ensuring that all samples participate in subsequent evaluations. , Introduce clustering validity constraint: only if the number of pseudo-label clusters Furthermore, if the proportion of non-noise samples is greater than 70%, the evaluation result of this parameter combination is retained; otherwise, it is marked as invalid.

[0009] Furthermore, the density clustering and multimodal index calculation includes the following: For the unsupervised dimension, the Silhouette coefficient, Davidsonburg index (DB), Calinski-Harabasz index (CH) and Dunn index (DNN index) are selected to quantify intra-class compactness, inter-cluster similarity, variance ratio, and extreme split robustness, respectively, ensuring multi-angle coverage of the clustering structure. For the supervised dimension, standardized mutual information (NMI) is used to measure the statistical dependence between the dimensionality reduction features and the true labels, enhancing the adaptability to downstream classification tasks. For the topological dimension, a unique Trust index is created to verify the fidelity of local structures by comparing the k-nearest neighbor overlap between the original space and the dimensionality reduction space.

[0010] Furthermore, the high-dimensional data preprocessing and topology benchmark construction include the following: The original high-dimensional dataset is standardized and preprocessed to eliminate the impact of differences in feature dimensions on the dimensionality reduction effect. , For sample size, The original dimension is used to obtain the normalized feature matrix through standardization. ; Construct a topological benchmark for the original high-dimensional space; the construction of the topological benchmark includes cross-dimensional correlation verification: by calculating the mutual information matrix of different dimensional subsets, the nonlinear dependencies between multidimensional data are quantified as an additional verification benchmark for the topological fidelity after dimensionality reduction. High-dimensional data includes heterogeneous dimensions, and its topological structure reflects the nonlinear relationships between different dimensions; the dimensionality reduction process preserves the physical meaning of the relationships in the low-dimensional space through mathematical mapping: calculating the standardized Euclidean distance matrix between samples. Its elements This serves as a reference for subsequent evaluation of the topological fidelity of the dimensionality reduction space; for mapping results generated by various dimensionality reduction algorithms, a unified output dimension is forcibly imposed. Dimensionality is used to eliminate the interference of dimensional differences on clustering evaluation.

[0011] Furthermore, the dimensionality reduction algorithm includes Principal Component Analysis (PCA), Kernel Principal Component Analysis (KPCA), Multidimensional Scaling (MDS), Non-negative Matrix Factorization (NMF), and Autoencoder.

[0012] Furthermore, the normalized matrix is ​​obtained through... The normalization method yielded the following results: , , .

[0013] Furthermore, the dimension The value range is set based on popular embedding theory. By limiting the dimensions to 50, the distance dilation effect in high-dimensional spaces is avoided. hour, Approximates the intrinsic dimension of the data, satisfying the minimum distortion constraint of local isometry; when At time, fixed It can suppress dimensional disasters.

[0014] Furthermore, the DBSCN neighborhood radius parameter The selected range is [2.0-5.0]. The wide-range search adapts to the changes in manifold density caused by different dimensionality reduction algorithms; the neighborhood radius parameter of DBSCN selects the number of generators within a specified range. =20; Sets the minimum number of samples as a global fixed parameter. ; through fixing With dynamic search .

[0015] Furthermore, the Trust metric is based on the number of nearest neighbors. Stratified percentile calculation, because the calculation of the Trust metric depends on Nearest neighbor number, different The value will cause scale differences in its numerical distribution, therefore, the selected value is... ; The design principle of the weighting system is determined through a multi-objective optimization framework. To emphasize the core role of unsupervised clustering quality and reflect the balance between intra-cluster compactness and inter-cluster separation, the weighting system is determined... The value is 0.35; to highlight the key role of topological fidelity, the consistency of the nearest neighbor structure in high-dimensional and low-dimensional spaces is quantified, and the value is determined. The value is set to 0.30; to emphasize the importance of consistent supervision of labels while avoiding overfitting to known labels, the value is set to... The value is 0.10; to supplement the evaluation of inter-cluster similarity, variance ratio, and minimum distance ratio, auxiliary evaluation indicators are set. , , .

[0016] In this technical solution, a multimodal evaluation system is constructed based on three core dimensions: unsupervised clustering quality, consistency of supervised information, and preservation of topology.

[0017] This technical solution focuses on the inherent separability of the dimensionality-reduced data to improve the quality of unsupervised clustering. It utilizes the Silhouette coefficient, Davies-Bouldin index (DB), Calinski-Harabasz index (CH), and Dunn index. Silhouette and DB characterize cluster density from the perspectives of intra- and inter-class balance and cluster similarity, respectively. CH quantifies spatial separation through variance ratio, and Dunn focuses on the ratio of minimum inter-cluster distance to maximum intra-cluster diameter. All four factors collectively reflect the global rationality and local stability of clustering results in unsupervised scenarios.

[0018] In this technical solution, to ensure the consistency of supervision information, the statistical dependence between the dimensionality reduction features and the true labels is measured by standardized mutual information (NMI). The closer the value is to 1, the stronger the fit between the low-dimensional space and the supervision task. This helps to verify the ability of the dimensionality reduction results to retain known classification priors and provides interpretability assurance for downstream classification tasks.

[0019] In this technical solution, regarding topology preservation: it is independently supported by the originality trust index. By quantifying the overlap between high-dimensional and low-dimensional space neighbors with a fixed number of nearest neighbors k_values=4, it directly evaluates the ability of the dimensionality reduction algorithm to preserve the original data manifold structure. The closer the value is to 1, the more it indicates that the key local topological relationships (such as cluster boundaries and density distribution) have not been destroyed, thus avoiding semantic distortion caused by excessive compression.

[0020] Compared with existing technologies, the advantages of this invention are: 1. A cross-spatial structure fidelity verification mechanism based on bidirectional topological mapping: Traditional dimensionality reduction methods rely on distance metrics or reconstruction error assessments in a single space, making it difficult to quantify the topological consistency of the original high-dimensional popular data in low-dimensional embeddings. This application innovatively introduces a nearest-neighbor structure overlap verification index by constructing a bidirectional feedback channel between the high-dimensional space and the low-dimensional popular data, directly measuring the ability to preserve local cluster structures (such as density distribution and boundary morphology) during dimensionality reduction. This mechanism effectively avoids semantic distortion caused by feature compression through cross-spatial topological alignment, and is particularly suitable for scenarios with complex noise, such as plasma spectroscopy, significantly improving the generalization of downstream classification models. 2. Dynamic range calibration and robust normalization system for multimodal indicators: Traditional evaluation methods are difficult to directly integrate multi-dimensional evaluation results due to differences in indicator dimensions, resulting in a scoring system biased towards a single characteristic. This application proposes a dynamic range calibration strategy based on historical data distribution. For heterogeneous indicators such as unsupervised clustering quality, supervised consistency, and topological fidelity, adaptive percentile normalization is applied to eliminate scale differences and outlier interference. Through a hierarchical calibration mechanism, the stability and comparability of scores in multimodal data scenarios such as high-noise signals are ensured, avoiding subjective bias from manually set thresholds. 3. Adaptive Search and Quality Constraint Engine for Density Clustering Parameters: To address the issue of evaluation result fluctuations caused by the sensitivity of density clustering parameters, this application adopts a strategy that combines fixed core parameters with dynamic search domain radius. It traverses the parameter space under a unified density scale and automatically filters low-quality clustering results (such as insufficient clusters or excessive noise). Compared with the full parameter optimization of traditional network search, this mechanism focuses on evaluating the feature extraction capability of the dimensionality reduction algorithm itself by constraining the clustering validity conditions, thus significantly improving the evaluation efficiency. 4. Design of a composite scoring function for dimensionality resource optimization: Traditional methods rely on empirical weight allocation or single index optimization, which makes it difficult to balance the multi-objective requirements of clustering quality, supervision adaptation, and structure fidelity. This application uses a pre-set weight system to dynamically normalize and weight-fuse multi-modal indicators to form an interpretable comprehensive score. This is directly related to the core objective of dimensionality reduction, and prioritizes lightweight high-scoring algorithms in resource-constrained scenarios, avoiding full-process verification of inefficient dimensionality reduction strategies, reducing computational overhead, and supporting weight fine-tuning to adapt to different data characteristics. Attached Figure Description

[0021] Figure 1 This is a flowchart of a multimodal noise reduction evaluation model based on density-sensitive clustering.

[0022] Figure 2 The images show the raw spectral signals collected from ten different types of experimental samples.

[0023] Figure 3 Radius parameters for each model in different domains Below is a chart showing the fluctuation of the total scores for the six different indicators.

[0024] Figure 4 This is a graph showing the accuracy curves of each model's output test set after dimensionality reduction processing using SVM. Detailed Implementation

[0025] It should be noted that relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0026] The features and performance of the present invention will be further described in detail below with reference to embodiments.

[0027] Please see Figure 1-4 A multimodal noise reduction evaluation model based on density-sensitive clustering, such as Figure 1 As shown, it includes the following steps: High-dimensional data preprocessing and topological benchmark construction: Standardize the original high-dimensional dataset to eliminate differences in feature dimensions and construct the topological benchmark of the original high-dimensional space. Multimodal dimensionality reduction and dynamic parameter search: It iterates through a variety of preset dimensionality reduction algorithms that need to be evaluated, loads the dimensionality reduction data files generated by the dimensionality reduction algorithms, obtains the normalized matrix of each dimensionality reduction algorithm, and uses the DBSCAN algorithm to perform dynamic parameter search on each normalized matrix to obtain pseudo-label vectors; Density clustering and multimodal index calculation, based on generated pseudo-labels, comprehensively evaluate the dimensionality reduction effect from three dimensions: unsupervised clustering quality, supervised consistency, and topology preservation through six types of orthogonalization indices; Dynamic normalization and weighted scoring are used to examine a multimodal index set through six types of orthogonal indicators. A comprehensive scoring function is constructed through the coupled operation of the directional correction matrix, the dynamic range normalization matrix and the weight matrix. Results storage and automated selection: The complete evaluation data is stored in a structured file, and the dimensionality reduction method with the highest score is selected as the strategy based on the weight vector and the overall score ranking.

[0028] High-dimensional data preprocessing and topology benchmark construction include the following: The original high-dimensional dataset is standardized and preprocessed to eliminate the impact of differences in feature dimensions on the dimensionality reduction effect. , For sample size, The original dimension is used to obtain the normalized feature matrix through standardization. Normalized matrix through The normalization method yielded the following results: , , .

[0029] Construct a topological benchmark for the original high-dimensional space; the construction of the topological benchmark includes cross-dimensional correlation verification: by calculating the mutual information matrix of different dimensional subsets, the nonlinear dependencies between multidimensional data are quantified as an additional verification benchmark for the topological fidelity after dimensionality reduction. High-dimensional data includes heterogeneous dimensions (such as spectral wavelength, time-domain statistics, and frequency-domain energy), and its topological structure reflects the nonlinear correlation between different dimensions (e.g., through mutual information or manifold distance metrics). The dimensionality reduction process preserves the physical meaning of the correlation in the low-dimensional space through mathematical mapping (e.g., elemental fingerprints correspond to spectral peaks, and process parameters correspond to time-frequency features): calculating the standardized Euclidean distance matrix between samples. Its elements This serves as a reference for subsequent evaluation of the topological fidelity of the dimensionality reduction space; for mapping results generated by various dimensionality reduction algorithms, a unified output dimension is forcibly imposed. Dimensionality reduction aims to eliminate the interference of dimensionality differences on clustering evaluation. Dimensionality setting is based on a balance between the estimation of intrinsic data dimensionality and computational efficiency. The essence of dimensionality reduction lies in removing redundant dimensions through mathematical mapping, while preserving the statistical correlations and physical meanings between key dimensions (such as spectral peak intensity ratio reflecting elemental concentration relationships, and time-frequency features relating to process parameters). This allows the distance metric in the low-dimensional space to be interpreted as a compressed representation of the physical relationships in the original high-dimensional space: the d-dimensional space can both suppress the high-dimensional distance inflation effect (avoiding the curse of dimensionality) and retain sufficient discriminative information to meet subsequent clustering and classification needs. Standardization ensures that the feature scales output by different dimensionality reduction methods are consistent, avoiding distance metric bias caused by differences in numerical ranges.

[0030] Dimension The value range is set based on popular embedding theory. By limiting the dimensions to 50, the distance dilation effect in high-dimensional spaces is avoided. hour, Approximates the intrinsic dimension of the data, satisfying the minimum distortion constraint of local isometry; when At time, fixed It can suppress dimensional disasters.

[0031] Multimodal dimensionality reduction and dynamic parameter search include the following: After completing the high-dimensional data standardization and topological benchmark construction, the various dimensionality reduction algorithms to be evaluated are iterated through, and the dimensionality reduction data files generated by the algorithms are loaded. The low-dimensional feature matrix output by each algorithm is also considered. Standardization is performed again to eliminate scale differences, i.e., the mean is calculated for each column of features. and standard deviation The standardized matrix is ​​obtained. Dimensionality reduction algorithms include Principal Component Analysis (PCA), Kernel Principal Component Analysis (KPCA), Multidimensional Scaling (MDS), Non-negative Matrix Factorization (NMF), and Autoencoder.

[0032] The standardized low-dimensional feature matrix, together with the true labels, constitutes the evaluation sample set. , For the sample The status indicates, For the corresponding supervision labels: , To quantify the adaptability of the dimensionality reduction results in density clustering, the DBSCAN algorithm is used for each... Perform dynamic parameter search; let the distance matrix in the reduced-dimensional space be... Its elements ; Neighborhood radius parameter of DBSCAN Generate within the interval using linear interpolation One, that is Minimum number of samples Use fixed values ​​to ensure that the clustering process of all dimensionality reduction methods is performed at the same density scale, thus avoiding evaluation bias caused by parameter differences.

[0033] For each dimensionality reduction method, the parameter combination Performing DBSCAN clustering yields pseudo-label vectors. Furthermore, noise points are recoded: if a sample is labeled as noise, it is assigned to a newly added independent cluster, thus ensuring that all samples participate in subsequent evaluations. , Introduce clustering validity constraint: only if the number of pseudo-label clusters Furthermore, if the proportion of non-noise samples is greater than 70%, the evaluation result of this parameter combination is retained; otherwise, it is marked as invalid. This mechanism filters out low-quality clusters and avoids invalid parameters interfering with the calculation of the comprehensive score.

[0034] DBSCN neighborhood radius parameter The selected range is [2.0-5.0]. The wide-range search adapts to the changes in manifold density caused by different dimensionality reduction algorithms; the neighborhood radius parameter of DBSCN selects the number of generators within a specified range. =20; Sets the minimum number of samples as a global fixed parameter. ; through fixing With dynamic search .

[0035] Density clustering and multimodal index calculation include the following: For the unsupervised dimension, the Silhouette coefficient, Davidsonburg index (DB), Calinski-Harabasz index (CH) based on logarithmic transformation, and Dunn index are selected to quantify intra-cluster compactness, inter-cluster similarity, variance ratio, and extreme split robustness, respectively, ensuring multi-angle coverage of the clustering structure. For the supervised dimension, standardized mutual information (NMI) is used to measure the statistical dependence between dimensionality-reduced features and true labels, enhancing adaptability to downstream classification tasks. For the topological dimension, a unique Trust index is used to verify the fidelity of local structures by comparing the k-nearest neighbor overlap between the original space and the dimensionality-reduced space. The design of the six indices follows the principle of positive interaction and complementarity to avoid information redundancy.

[0036] Dynamic normalization and weighted scoring include the following: Set directional focus correction matrix Its diagonal elements This represents the directional correction factor for each indicator, with the negative indicator DB having a correction factor of -1 and the other positive indicators having a correction factor of 1; the directional correction focusing matrix will convert the original indicator vector... Converting to a unified, positively additive form, and through directional correction, unifying the optimal direction of all indicators to "the larger the value, the better," eliminates the impact of directional differences in indicators on the scoring: , , Based on historical data distribution, a dynamic normalized parameter matrix is ​​constructed. row vector Indicates the first Normalization range of each indicator: , in, The metric is based on the number of nearest neighbors. Stratified percentile calculation; avoid due to Scale bias introduced by value variations. The normalization function is set as a piecewise linear mapping: , For the DB metric, its normalization requires direction calculation: , Set weight vector ,satisfy ; The comprehensive scoring function achieves a linear combination of normalized indicators and weights through matrix multiplication, where... The normalized index vector: .

[0037] The Trust index is based on the number of nearest neighbors. Stratified percentile calculation, because the calculation of the Trust metric depends on Nearest neighbor number, different The value will cause scale differences in its numerical distribution, therefore, the selected value is... ; The design principle of the weighting system is determined through a multi-objective optimization framework. To emphasize the core role of unsupervised clustering quality and reflect the balance between intra-cluster compactness and inter-cluster separation, the weighting system is determined... The value is 0.35; to highlight the key role of topological fidelity, the consistency of the nearest neighbor structure in high-dimensional and low-dimensional spaces is quantified, and the value is determined. The value is set to 0.30; to emphasize the importance of consistent supervision of labels while avoiding overfitting to known labels, the value is set to... The value is 0.10; to supplement the evaluation of inter-cluster similarity, variance ratio, and minimum distance ratio, auxiliary evaluation indicators are set. , , .

[0038] Results storage and automation options include: After calculating the comprehensive score for all dimensionality reduction methods and parameter combinations, complete evaluation data (including original index values, normalized scores, and weighted scores for each method under different EPS, such as Silhouette, DB, NMI, CH, Dunn, Trust indices, and final scores) is stored in a structured file to ensure the traceability and scalability of the evaluation results. Finally, based on the preset weight vector and comprehensive score ranking, the dimensionality reduction method and parameter combination with the highest score is selected as the optimal strategy, providing low-dimensional feature input for downstream machine learning tasks, realizing data-driven optimization and fully automated decision-making for dimensionality reduction algorithms.

[0039] In another embodiment, a data acquisition optical path based on coplanar non-collinear three-filament coupling breakdown was constructed. Ten different grades and types of plastics were selected for spectral acquisition. The acquired spectral information was used as input data to verify the dimensionality reduction scheme selection capability of a density-sensitive clustering-based multimodal noise reduction evaluation model. The steps are as follows: Step 1: Construct a system optical path based on coplanar non-collinear three-filament coupling breakdown, and perform breakdown on 10 different plastic samples (sample selection is shown in Table 1). Collect spectral information from different types of plastic samples to construct the initial high-dimensional input data. The spectrometer in the optical system acquired 22,940 dimensions of data in the wavelength range of 200-900 nm (e.g., ...). Figure 2 As shown), 500 different locations were selected for breakdown testing of each sample, therefore... Although this embodiment uses the spectral wavelength dimension as an example, the method is also applicable to other multidimensional data structures (such as data that simultaneously includes spectral intensity, pulse width time-domain statistics, frequency domain wavelet coefficients, etc.). In this embodiment, at the database association level, the spectral wavelength dimension is mapped to the NIST atomic spectral database, the intensity dimension is associated with the plasma radiation model library, and the pulse width dimension is connected to the laser process parameter library. The physical constraints between dimensions are verified through covariance matrix analysis.

[0040] Table 1. Selection of Experimental Samples

[0041] Preferably, the 10 different plastic samples selected are ethylene-vinyl acetate copolymer (EVA), polyvinyl chloride (PVC), polystyrene (PS), polypropylene (PP), polymethyl methacrylate (PMMA), polyethylene terephthalate (PET), polycarbonate (PC), nylon (PA), low-density vinyl chloride (LDPE), and high-density polyethylene (HDPE).

[0042] Step 2: After completing the high-dimensional data standardization and topology benchmark construction, iterate through the various preset dimensionality reduction algorithms to be evaluated and load the dimensionality reduction data files they generate. Select 5 different dimensionality reduction models to... The data undergoes dimensionality reduction to ensure consistent dimensionality across different dimensionality reduction models and generate dimensionality-reduced mappings. The reduced data is then standardized again to eliminate scale differences and exported as a document. A multimodal denoising evaluation model based on density-sensitive clustering (DR-FAST) is constructed. The DBSCAN algorithm is used to perform dynamic parameter search on the data corresponding to each dimensionality reduction scheme, setting the neighborhood radius parameter. The search range is [3-5], and 20 experimental values ​​are generated by linear interpolation within this range, with a fixed minimum sample size. .

[0043] For each combination of parameters for the dimensionality reduction method, DBSCAN clustering is performed to obtain pseudo-label vectors, and noise points are re-encoded. At the same time, clustering validity constraints are introduced, and the different output data obtained under different dimensionality reduction models are evaluated by the DR-FAST model.

[0044] Preferably, the five different dimensionality reduction schemes used are Principal Component Analysis (PCA), Kernel Principal Component Analysis (KPCA), Multidimensional Scaling (MDS), Non-negative Matrix Factorization (NMF), and Autoencoder.

[0045] Preferred, After five different dimensionality reduction schemes, the data dimension was reduced from 22940 to 10. .

[0046] Step 3, Preset Based on pseudo-labels generated by dynamic parameter search in the dimensionality reduction space, the dimensionality reduction effect is comprehensively evaluated from three orthogonal dimensions: unsupervised clustering quality (profile coefficient, DB, CH, Dunn), supervised information consistency (NMI), and topology preservation (Trust) through six orthogonalization indicators.

[0047] Step 4: The multimodal index set generated based on the above steps Construct the direction correction matrix Dynamic range normalized matrix Weight matrix Different calculations will result in different corresponding solutions. The scores for each indicator are shown below. Different parameters are calculated by coupling these indicators. The final overall score under the combination .

[0048] Step 5: After calculating the comprehensive score for all dimensionality reduction methods and parameter combinations, store the complete evaluation data, including the original index values, normalized scores, and weighted scores, in a structured file to ensure the traceability and scalability of the evaluation results. Finally, based on the preset weight vector and comprehensive score ranking, select the dimensionality reduction method and parameter combination with the highest score as the optimal strategy. This provides low-dimensional feature input for downstream machine learning tasks, realizing data-driven optimization and fully automated decision-making for the dimensionality reduction algorithm.

[0049] like Figure 3 As shown in Table 2, the final output is a table of overall performance evaluation results for the DBSCN multimodal denoising assessment model. The performance ranking of each dimensionality reduction method is PCA > NMF > Autoencoder > MDS > KPCA. This ranking profoundly reflects the coupling relationship between algorithm characteristics and plasma spectral data structure.

[0050] Table 2 Optimal neighborhood radius parameters for each model Corresponding indicator evaluation

[0051] PCA The highest score of 0.869 was achieved at 4.579, thanks to its ability to globally preserve the features of the continuous spectrum through linear orthogonal decomposition. It effectively captures clustering patterns along the principal component direction even with a large neighborhood radius. Meanwhile, the stable Trust index of 0.078 indicates its reliability in preserving high-dimensional nearest neighbor structures. Although NMF... =5.0 reached a peak of 0.696, but although its nonnegativity constraint fits the nonnegativity of spectral intensity, its linear decomposition is insufficient to express nonlinear features, resulting in limited improvements in NMI (0.031) and Trust (0.079). Autoencoder in At a score of 5.0, the Silhouette score jumped to 0.876, but the Trust score of only 0.015 revealed local topological distortion caused by overfitting noise in the neural network, reflecting the vulnerability of deep learning models to high-noise spectra in the absence of prior constraints. The low scores of MDS and KPCA stem from the defects in their core mechanisms—the distance preservation assumption of MDS is difficult to maintain in high-dimensional spectral manifolds, while the kernel function selection of KPCA is mismatched with the sparsity of plasma features, resulting in a simultaneous deterioration of clustering index and structure fidelity.

[0052] It is worth noting that PCA, NMF, and Autoencoder are at their maximum The high score of DBSCAN at 5.0 is essentially due to its large neighborhood radius, which mitigates local density fluctuations in the feature space after dimensionality reduction, especially for uniform low-dimensional embeddings generated by linear methods such as PCA. It can cover the complete clustering pattern along the principal component direction, but we need to be wary of evaluation bias caused by excessive cluster merging. This phenomenon reveals the need for synergistic optimization of dimensionality reduction algorithms and density clustering parameters: linear methods are better suited to wide parameter searches due to their structural stability, while nonlinear methods require fine-tuning to avoid topological distortion.

[0053] To verify the reliability of the DR-FAST framework, support vector machine (SVM) was used to verify the adaptability of the dimensionality reduction results to downstream tasks. A linear kernel SVM model (penalty coefficient C=1.5) was selected, and the 10-dimensional features output by PCA, KPCA, MDS, NMF, and Autoencoder were used as inputs. The training set, validation set, and test set were divided in a 7:2:1 ratio.

[0054] Experimental results show (e.g.) Figure 4As shown in the figure, PCA significantly outperformed the others with a test set accuracy of 88.7%, while NMF, Autoencoder, MDS, and KPCA achieved accuracies of 69.8%, 57.6%, 52.5%, and 49.4%, respectively. The ranking is completely consistent with the DR-FAST pre-evaluation results (PCA > NMF > Autoencoder > MDS > KPCA), confirming the effectiveness of the framework's evaluation. This performance difference stems from the inherent compatibility between dimensionality reduction algorithms and the characteristics of plasma spectral data: PCA's linear orthogonal decomposition naturally fits the global structure of the continuous spectral spectrum, and its principal component directions' ability to preserve energy distribution features supports the clear delineation of SVM classification boundaries, with a validation set deviation of only 2.1% and optimal generalization; although NMF matches spectral intensity characteristics through non-negative constraints, its linear basis combination struggles to capture the non-local correlation features of plasma pulse peaks, leading to overfitting on the validation set (72.5% in the training set vs. 69.8% in the test set), exposing the model's bottleneck in capturing complex patterns; the Autoencoder, due to the deviation between the unsupervised reconstruction target and the classification task, introduces high-frequency noise perturbations in its nonlinear embedding space (pre-evaluation Trust=0.015), coupled with the expressive limitations of the SVM linear kernel, blurring the classification boundaries; MDS's global distance preservation assumption conflicts with the local sparsity of the high-dimensional spectral manifold, while KPCA's RBF kernel function causes dimensionality collapse under sparse distribution of plasma features (pre-evaluation NMI=0.007), ultimately leading to a collapse in classification performance.

[0055] Experimental results further reveal that the consistency between the dimensionality reduction algorithm and the downstream model's objectives is the core of the performance leap—the linear orthogonality of PCA and the geometric separability of SVM form a positive synergy, while nonlinear methods such as Autoencoder require a deep classifier to unleash their potential. This highlights the engineering value of the DR-FAST framework in screening suitable dimensionality reduction strategies during the preprocessing stage.

[0056] The embodiments described above merely illustrate specific implementation methods of this application, and while the descriptions are detailed and specific, they should not be construed as limiting the scope of protection of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the technical solution of this application, and these modifications and improvements all fall within the scope of protection of this application.

Claims

1. A multimodal noise reduction evaluation model based on density-sensitive clustering, characterized in that, Includes the following steps: High-dimensional data preprocessing and topological benchmark construction: Standardize the original high-dimensional dataset to eliminate differences in feature dimensions and construct the topological benchmark of the original high-dimensional space. Multimodal dimensionality reduction and dynamic parameter search: It iterates through a variety of preset dimensionality reduction algorithms that need to be evaluated, loads the dimensionality reduction data files generated by the dimensionality reduction algorithms, obtains the normalized matrix of each dimensionality reduction algorithm, and uses the DBSCAN algorithm to perform dynamic parameter search on each normalized matrix to obtain pseudo-label vectors; Density clustering and multimodal index calculation, based on generated pseudo-labels, comprehensively evaluate the dimensionality reduction effect from three dimensions: unsupervised clustering quality, supervised consistency, and topology preservation through six types of orthogonalization indices; Dynamic normalization and weighted scoring are used to examine a multimodal index set through six types of orthogonal indicators. A comprehensive scoring function is constructed through the coupled operation of the directional correction matrix, the dynamic range normalization matrix and the weight matrix. Results storage and automated selection: The complete evaluation data is stored in a structured file, and the dimensionality reduction method with the highest score is selected as the strategy based on the weight vector and the overall score ranking.

2. The multimodal noise reduction evaluation model based on density-sensitive clustering according to claim 1, characterized in that, The dynamic normalization and weighted scoring includes the following: Set directional focus correction matrix Its diagonal elements This represents the directional correction factor for each indicator, with the negative indicator DB having a correction factor of -1 and the other positive indicators having a correction factor of 1; the directional correction focusing matrix will convert the original indicator vector... Converting to a unified, positively additive form, and then unifying the direction of all indicators through directional correction: ， , Based on historical data distribution, a dynamic normalized parameter matrix is ​​constructed. row vector Indicates the first Normalization range of each indicator: ， in, The metric is based on the number of nearest neighbors. Stratified percentiles were calculated; the normalization function was set to a piecewise linear mapping. ， For the DB metric, its normalization requires direction calculation: ， Set weight vector ,satisfy ; The comprehensive scoring function achieves a linear combination of normalized indicators and weights through matrix multiplication, where... The normalized index vector: 。 3. The multimodal noise reduction evaluation model based on density-sensitive clustering according to claim 2, characterized in that, The multimodal dimensionality reduction and dynamic parameter search include the following: Iterate through the preset dimensionality reduction algorithms to be evaluated, load the dimensionality reduction data files generated by the algorithms, and output the low-dimensional feature matrix of each algorithm. Standardization is performed again to eliminate scale differences, i.e., the mean is calculated for each column of features. and standard deviation The standardized matrix is ​​obtained. ; The standardized low-dimensional feature matrix, together with the true labels, constitutes the evaluation sample set. , For the sample The status indicates, For the corresponding supervision labels: ， The DBSCAN algorithm is used for each Perform dynamic parameter search; let the distance matrix in the reduced-dimensional space be... Its elements ; Neighborhood radius parameter of DBSCAN Generate within the interval using linear interpolation. One, that is Minimum number of samples Select a fixed value; For each dimensionality reduction method, the parameter combination Performing DBSCAN clustering yields pseudo-label vectors. Furthermore, noise points are recoded: if a sample is labeled as noise, it is assigned to a newly added independent cluster, thus ensuring that all samples participate in subsequent evaluations. ， Introduce clustering validity constraint: only if the number of pseudo-label clusters Furthermore, if the proportion of non-noise samples is greater than 70%, the evaluation result of this parameter combination is retained; otherwise, it is marked as invalid.

4. The multimodal noise reduction evaluation model based on density-sensitive clustering according to claim 3, characterized in that, The density clustering and multimodal index calculation includes the following: For the unsupervised dimension, the Silhouette coefficient, Davidsonburg index (DB), Calinski-Harabasz index (CH) and Dunn index (DNN index) are selected to quantify intra-class compactness, inter-cluster similarity, variance ratio, and extreme split robustness, respectively, ensuring multi-angle coverage of the clustering structure. For the supervised dimension, standardized mutual information (NMI) is used to measure the statistical dependence between the dimensionality reduction features and the true labels, enhancing the adaptability to downstream classification tasks. For the topological dimension, a unique Trust index is created to verify the fidelity of local structures by comparing the k-nearest neighbor overlap between the original space and the dimensionality reduction space.

5. The multimodal noise reduction evaluation model based on density-sensitive clustering according to claim 4, characterized in that, The high-dimensional data preprocessing and topology benchmark construction include the following: The original high-dimensional dataset is standardized and preprocessed to eliminate the impact of differences in feature dimensions on the dimensionality reduction effect. , For sample size, The original dimension is used to obtain the normalized feature matrix through standardization. ; Construct a topological benchmark for the original high-dimensional space; the construction of the topological benchmark includes cross-dimensional correlation verification: by calculating the mutual information matrix of different dimensional subsets, the nonlinear dependencies between multidimensional data are quantified as an additional verification benchmark for the topological fidelity after dimensionality reduction. High-dimensional data includes heterogeneous dimensions, and its topological structure reflects the nonlinear relationships between different dimensions; the dimensionality reduction process preserves the physical meaning of the relationships in the low-dimensional space through mathematical mapping: calculating the standardized Euclidean distance matrix between samples. Its elements This serves as a reference for subsequent evaluation of the topological fidelity of the dimensionality reduction space; for mapping results generated by various dimensionality reduction algorithms, a unified output dimension is forcibly imposed. Dimensionality is used to eliminate the interference of dimensional differences on clustering evaluation.

6. The multimodal noise reduction evaluation model based on density-sensitive clustering according to claim 3, characterized in that, The dimensionality reduction algorithms include Principal Component Analysis (PCA), Kernel Principal Component Analysis (KPCA), Multidimensional Scaling (MDS), Non-negative Matrix Factorization (NMF), and Autoencoder.

7. The multimodal noise reduction evaluation model based on density-sensitive clustering according to claim 5, characterized in that, The normalized matrix is ​​obtained through The normalization method yielded the following results: ， ， 。 8. The multimodal noise reduction evaluation model based on density-sensitive clustering according to claim 5, characterized in that, The dimension The value range is set based on popular embedding theory. By limiting the dimensions to 50, the distance dilation effect in high-dimensional spaces is avoided. hour, Approximates the intrinsic dimension of the data, satisfying the minimum distortion constraint of local isometry; when At time, fixed It can suppress the dimensional catastrophe.

9. A multimodal noise reduction evaluation model based on density-sensitive clustering according to claim 3, characterized in that, The DBSCN neighborhood radius parameter The selected range is [2.0-5.0]. The wide-range search adapts to the changes in manifold density caused by different dimensionality reduction algorithms; the neighborhood radius parameter of DBSCN selects the number of generated elements within a specified range. =20; Sets the minimum number of samples as a global fixed parameter. ; through fixing With dynamic search .

10. A multimodal noise reduction evaluation model based on density-sensitive clustering according to claim 2, characterized in that, The Trust metric is based on the number of nearest neighbors. Stratified percentile calculation, because the calculation of the Trust metric depends on Nearest neighbor number, different The value will cause scale differences in its numerical distribution, therefore, the selected value is... ; The design principle of the weighting system is determined through a multi-objective optimization framework. To emphasize the core role of unsupervised clustering quality and reflect the balance between intra-cluster compactness and inter-cluster separation, the weighting system is determined... The value is 0.35; to highlight the key role of topological fidelity, the consistency of the nearest neighbor structure in high-dimensional and low-dimensional spaces is quantified, and the value is determined. The value is set to 0.30; to emphasize the importance of consistent supervision of labels while avoiding overfitting to known labels, the value is set to... The value is 0.10; to supplement the evaluation of inter-cluster similarity, variance ratio, and minimum distance ratio, auxiliary evaluation indicators are set. , , .

Images