Clustering method, device, equipment, medium and program product

Abstract

The embodiment of the invention provides a clustering method and device, a medium and a program product. The method comprises the following steps: acquiring target data containing noise; filtering the target data to form an effective sample set; performing standardization processing on the effective sample set, and performing multi-scale noise filtering to reduce dispersive noise interference to obtain noise-reduced data; the comprehensive density score of each sample is calculated for the denoised data based on a density fusion model, the density fusion model accelerates neighbor search by constructing an index structure, the local density is calculated by fusing the spatial distance and the feature similarity, and the comprehensive density score is obtained by combining global distribution feature correction; clustering candidate centers are screened in combination with an adaptive threshold, and an initial clustering center set is obtained after redundant candidate centers are removed; and distributing the effective samples to a cluster corresponding to the nearest initial clustering center, iteratively updating the clustering center, and carrying out secondary distribution on the samples which are low in score and conform to a spatial proximity condition to generate a final clustering result.

Description

Technical Field

[0001] This application relates to the field of computers, and more specifically to a clustering method, apparatus, medium, and program product. Background Technology

[0002] Clustering analysis, a core technology in unsupervised learning, automatically identifies the potential distribution structure of data, grouping samples with similar features into the same cluster and samples with different features into different clusters. It has been widely applied in many important fields such as data mining (e.g., anomaly detection in financial transactions), computer vision (e.g., image texture clustering), and industrial 3D inspection (e.g., point cloud dimension analysis of parts). However, with the popularization of data acquisition technologies and the expansion of application scenarios, the actual data to be clustered inevitably contains various types of noise. These noises mainly originate from equipment acquisition errors, environmental interference, and data transmission losses, and include types such as Gaussian noise, salt-and-pepper noise, and random discrete noise, typically accounting for 5%-30%. This type of noise severely damages the authenticity of the data distribution, leading to a significant decrease in the accuracy and stability of traditional clustering algorithms, becoming a key bottleneck limiting the large-scale implementation of clustering technology in industrial scenarios.

[0003] There is an urgent need for a clustering method that can comprehensively filter various types of noise in order to address the many shortcomings of existing technologies in noisy environments and improve the accuracy, stability and practicality of clustering results. Summary of the Invention

[0004] The purpose of this application is to provide a clustering method, apparatus, device, medium, and program product that can reduce noise interference in clustering to a certain extent and improve clustering accuracy.

[0005] A first aspect of this application provides a clustering method, the method comprising: Acquire noisy target data; Extreme value filtering is performed on the target data to remove samples that significantly deviate from the data distribution, forming an effective sample set; The effective sample set is standardized to eliminate feature scale differences, and then multi-scale noise filtering is performed to reduce scattered noise interference, resulting in denoised data. The comprehensive density score of each sample is calculated based on the density fusion model. The density fusion model accelerates nearest neighbor search by constructing an index structure, calculates local density by fusing spatial distance and feature similarity, and obtains the comprehensive density score by combining global distribution features for correction. By combining adaptive threshold screening to select cluster candidate centers and removing redundant candidate centers, an initial set of cluster centers is obtained. Valid samples are assigned to the clusters corresponding to the nearest initial cluster centers. The cluster centers are iteratively updated, and samples with low scores but meeting the spatial proximity condition are reassigned to generate the final clustering result.

[0006] A second aspect of this application provides a clustering apparatus, the clustering apparatus comprising: The first acquisition module is used to acquire target data containing noise. The first filtering module is used to perform extreme value filtering on the target data to remove samples that deviate significantly from the data distribution, thereby forming an effective sample set; The second filtering module is used to standardize the effective sample set to eliminate feature scale differences, and then perform multi-scale noise filtering to reduce scattered noise interference, so as to obtain denoised data. The first fusion module is used to calculate the comprehensive density score of each sample in the denoised data based on the density fusion model. The density fusion model accelerates the nearest neighbor search by constructing an index structure, calculates the local density by fusing spatial distance and feature similarity, and obtains the comprehensive density score by combining global distribution features for correction. The first elimination module is used to filter cluster candidate centers by combining adaptive thresholds, and obtain an initial cluster center set after eliminating redundant candidate centers; The first iteration module is used to assign valid samples to the clusters corresponding to the nearest initial cluster centers, iteratively update the cluster centers, and perform secondary assignment on samples with low scores but meeting the spatial proximity condition to generate the final clustering result.

[0007] A third aspect of this application provides an electronic device including a processor and a memory, the memory storing a program or instructions executable on the processor, the program or instructions, when executed by the processor, implementing the steps of the clustering method as described in the first aspect.

[0008] A fourth aspect of this application provides a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the clustering method as described in the first aspect.

[0009] A fifth aspect of this application provides a chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run a program or instructions to implement the steps of the clustering method as described in the first aspect.

[0010] A sixth aspect of this application provides a computer program product stored in a storage medium, which is executed by at least one processor to implement the steps of the clustering method as described in the first aspect.

[0011] In this embodiment, the present invention acquires target data containing various common types of noise without prior noise classification and screening, directly adapting to multiple practical application scenarios and overcoming the limitation of traditional algorithms that can only handle a single type of noise. By removing samples that significantly deviate from the data distribution to form an effective sample set, the interference of extreme outliers on the subsequent clustering process can be reduced from the source, avoiding the problems of false deletion of effective samples or residual extreme noise caused by traditional single filtering methods. Standardizing the effective sample set eliminates scale differences between different feature dimensions, ensuring that each feature has a fair weight in density calculation. At the same time, multi-scale noise filtering further reduces the interference of scattered noise, optimizes the data distribution, and provides high-quality input for subsequent density evaluation, solving the deficiency of traditional single filtering in comprehensively dealing with scattered noise. When calculating the comprehensive density score based on the density fusion model, an index structure is constructed to accelerate nearest neighbor search, improving the efficiency of obtaining nearest neighbor sets. Simultaneously, spatial distance and feature similarity are fused to calculate local density, and global distribution features are used for correction. This ensures that the density assessment both reflects local sample correlations and adapts to the overall data distribution, overcoming the bias in high-dimensional data density assessment caused by traditional methods relying solely on a single distance metric. It can accurately distinguish between low-density effective samples and noise. Furthermore, adaptive thresholding for selecting cluster candidate centers eliminates the need for manually preset parameters. The judgment criteria can be dynamically adjusted based on the data's characteristics, adapting to the differences in different datasets. Redundant candidate centers are eliminated to prevent excessive clustering of initial cluster centers, making the initial center distribution more closely resemble the true cluster structure of the data. This solves the problems of poor adaptability of traditional fixed thresholds and cluster partitioning offset caused by random initial center selection. After assigning valid samples to the clusters corresponding to the nearest initial cluster centers, the cluster centers are iteratively updated to eliminate interference from residual noise in center localization. Simultaneously, samples with low scores but meeting the spatial proximity condition are reassigned. This ensures the stability of the cluster centers and avoids the erroneous deletion of low-density valid samples at the cluster edges, significantly improving the accuracy and completeness of the clustering results. It solves the problems of poor robustness and large fluctuations in clustering results inherent in traditional algorithms. The overall technical solution, through a collaborative design across the entire process, systematically addresses the core shortcomings of traditional clustering algorithms, such as incomplete noise filtering, one-sided density assessment, poor robustness, and insufficient scenario adaptability. It can be widely applied to clustering tasks in data mining, computer vision, industrial inspection, and other fields. Attached Figure Description

[0012] Figure 1 This is a flowchart illustrating the clustering method provided in an embodiment of this application; Figure 2 This is a schematic diagram of the hardware structure of an electronic device that implements an embodiment of this application; Figure 3 This is a schematic diagram of another hardware structure of an electronic device that implements an embodiment of this application. Detailed Implementation

[0013] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0014] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0015] The clustering method, apparatus, device, storage medium, chip, and computer program product provided in this application can effectively solve the above-mentioned technical problems. The following, in conjunction with the accompanying drawings, provides a detailed description of the clustering method, apparatus, device, storage medium, chip, and computer program product provided in this application through specific embodiments and application scenarios.

[0016] like Figure 1 As shown, Figure 1 This is a flowchart illustrating the clustering method provided in this application embodiment. The clustering method can be applied to electronic devices, such as computers, smartphones, tablets, wearable smart devices, etc., or servers in distributed systems, cloud servers, intelligent cloud computing servers with artificial intelligence technology, or intelligent cloud hosts. It can also be applied to clustering devices, Figure 2 or Figure 3 For details regarding the electronic devices shown, please refer to the relevant descriptions below. Figure 1 The clustering method includes the following steps S11 to S16: S11, acquire target data containing noise; S12, Perform extreme value filtering on the target data to remove samples that deviate significantly from the data distribution, forming an effective sample set; S13, standardize the effective sample set to eliminate feature scale differences, and then perform multi-scale noise filtering to reduce scattered noise interference to obtain denoised data; S14. Calculate the comprehensive density score of each sample in the denoised data based on the density fusion model. The density fusion model accelerates nearest neighbor search by constructing an index structure, calculates local density by fusing spatial distance and feature similarity, and obtains the comprehensive density score by combining global distribution features for correction. S15, Combine adaptive threshold to filter cluster candidate centers, and get the initial cluster center set after removing redundant candidate centers; S16 assigns valid samples to the clusters corresponding to the nearest initial cluster centers, iteratively updates the cluster centers, and performs secondary assignment on samples with low scores but meeting the spatial proximity condition to generate the final clustering result.

[0017] Noisy target data is a common object of processing in cluster analysis. It usually includes interference data caused by equipment acquisition errors, environmental interference, data transmission loss, etc. The noise types cover a variety of common types and need to be processed in a targeted manner before clustering. In this application, there is no need to classify and screen the noise types in advance. It can be directly adapted to data to be clustered in multiple scenarios such as data mining, computer vision, and industrial inspection, and provides a widely applicable basis for subsequent multi-stage noise filtering.

[0018] Extreme value filtering is a common noise filtering method in data preprocessing. It refers to the use of statistical methods to identify and remove extreme outliers that deviate significantly from the overall data distribution, thereby reducing the interference of extreme values ​​on subsequent data processing and clustering results. In this application, the extreme value interval is determined by calculating the first quartile and the third quartile of each feature dimension of the target data. Samples outside the interval are marked as samples to be removed. At the same time, the samples within the interval are verified a second time to remove scattered noise with excessive deviation between feature values ​​and dimension means, thus forming a high-quality and effective sample set.

[0019] Standardization is a preprocessing operation that eliminates differences in feature scales. It refers to converting feature data with different dimensions and scales into a dimensionless uniform numerical range to ensure that each feature has a fair weight in subsequent calculations. In this application, the Z-score formula is used to calculate the standardized value. Potential noise samples with absolute values ​​exceeding the standard after standardization are relabeled and removed. After updating the statistical parameters, a second standardization is performed to further optimize the data distribution.

[0020] Multi-scale noise filtering is a collaborative noise reduction method for dispersed noise. It refers to combining two filtering dimensions, time domain and frequency domain, to process different types of dispersed noise in a targeted manner, avoiding the limitations of a single filtering method. In this application, moving average filtering is used for time domain filtering to smooth high-frequency impulse noise, and wavelet decomposition is used for frequency domain filtering to remove high-frequency noise and retain low-frequency core signals. Finally, the results of the two filtering methods are fused to obtain the noise-reduced data.

[0021] The density fusion model is the core model for improving the accuracy of density assessment. It refers to the integration of multiple density-related assessment indicators and the construction of a comprehensive scoring system by combining local and global distribution characteristics, which is suitable for the density calculation needs of high-dimensional or complex data. In this application, a Kd-tree index is constructed to accelerate nearest neighbor search, Euclidean distance and cosine similarity are integrated to calculate local density, and then the global density weight is calculated by combining the global maximum local density and the dispersion coefficient. Finally, the product of local density and global weight is used as the comprehensive density score.

[0022] Adaptive threshold is a key parameter used to distinguish between valid samples and noise and to screen candidate cluster centers. It refers to a judgment boundary that is dynamically adjusted according to the characteristics of the data itself without manual preset, and adapts to the differences of different datasets. In this application, the initial threshold is calculated based on the overall variance of the target data, the historical maximum variance and the noise ratio, and the final threshold is determined through iterative calibration to ensure that the threshold adapts to the variance characteristics and noise ratio of the current dataset.

[0023] Redundant candidate center removal is an operation to optimize the distribution of initial cluster centers. It refers to removing overly clustered redundant centers by judging the spatial distance relationship between candidate centers, so as to avoid cluster division shift due to center clustering. In this application, samples whose distance from the retained candidate centers is less than a preset multiple of the average distance of candidate centers are removed to obtain a reasonably distributed and representative initial cluster center set.

[0024] Sample allocation is the process of completing the initial cluster division, which refers to assigning the pre-processed valid samples to the clusters represented by the corresponding cluster centers according to preset rules, laying the foundation for subsequent optimization. In this application, each valid sample is assigned to the cluster corresponding to the nearest initial cluster center to ensure the rationality of the initial cluster division.

[0025] Iteratively updating cluster centers is an optimization operation to improve the accuracy of cluster division. It refers to gradually eliminating noise interference by adjusting the position of cluster centers multiple times, so that the cluster centers converge to the true high-density region. In this application, low-scoring samples with a comprehensive density score below the adaptive threshold are excluded in each iteration, and the mean of the remaining samples is calculated as the new cluster center. The iteration stops when the change in the center is less than the preset convergence threshold or the preset upper limit of the number of iterations is reached.

[0026] Secondary sample allocation is an operation to ensure the integrity of clustering results. It refers to reclassifying samples that are initially judged to have low scores but meet specific spatial conditions to avoid the deletion of valid samples due to a single scoring standard. In this application, samples with a comprehensive density score below the threshold but whose distance from the center of a cluster is less than a preset multiple of the cluster radius are reassigned to the corresponding cluster based on spatial proximity and cluster distribution characteristics.

[0027] For example, in an industrial 3D parts inspection scenario, 3D point cloud target data containing random discrete noise is acquired. The noise sources are scanning equipment vibration and dust interference in the air. First, the quartiles of each feature dimension are calculated to determine the extreme value intervals. Extreme outliers outside the intervals are removed. Then, the samples within the intervals are re-verified to remove scattered noise where the feature values ​​deviate excessively from the dimension means. The Z-score formula is used to standardize the effective samples. Abnormal samples after standardization are removed, and the parameters are updated for a second standardization. Moving average filtering is used to smooth high-frequency impulse noise in the time domain, and wavelet decomposition is used to remove high-frequency noise in the frequency domain. The results of the two filtering methods are fused to obtain the denoised point cloud data. A Kd-tree index is constructed by sorting the variance of the feature dimensions, the number of nearest neighbors is dynamically adjusted, and Euclidean distance and cosine similarity are fused to calculate the local density. The global density weight is obtained by combining the global maximum local density and the dispersion coefficient, and then a comprehensive density score is generated. An initial threshold is calculated based on the overall variance of the point cloud data, the maximum variance of historical data of the same type, and the noise ratio. After iterative calibration, the final adaptive threshold is determined. Samples with a comprehensive density score not lower than the threshold are selected as candidate centers. Redundant centers whose distance from the retained candidate centers is less than a preset multiple of the average distance are removed to obtain an initial cluster center set. Valid point cloud samples are assigned to the clusters corresponding to the initial cluster centers according to the principle of closest proximity. After excluding low-scoring samples in the cluster, the average value is taken to update the cluster centers. The iteration continues until the change in the center is less than a preset convergence threshold. Edge point cloud samples with a comprehensive density score lower than the threshold but whose distance from the center of a cluster is less than a preset multiple of the radius of that cluster are reassigned to that cluster, and finally, accurate 3D part point cloud clustering results are generated.

[0028] In this embodiment, a multi-stage noise filtering mechanism is used to comprehensively filter out extreme values ​​and scattered noise, avoiding the accidental deletion of valid samples and reducing the risk of residual noise. By fusing spatial distance and feature similarity through a density fusion model and combining it with global distribution feature correction, the accuracy of density assessment of high-dimensional 3D point cloud data is improved, and low-density valid edge points can be accurately distinguished from noise. The adaptive threshold does not require manual preset and is adapted to the variance characteristics and noise ratio of 3D point cloud data through iterative calibration, improving the adaptability and accuracy of noise judgment. The combination of redundant candidate center elimination and iterative update of cluster centers effectively avoids cluster division offset caused by initial center aggregation and noise interference, and improves the stability of cluster center positioning. The sample secondary allocation mechanism recovers low-density valid samples at the cluster edges, ensuring the integrity and comprehensiveness of the clustering results.

[0029] In some optional implementations, the extreme value filtering determines the extreme value range by calculating the first quartile and the third quartile of each feature dimension of the target data, and marks samples outside the extreme value range as samples to be removed.

[0030] The first and third quartiles are fundamental parameters in statistics used to describe data distribution. They refer to the characteristic values ​​that occupy specific percentage positions after the data is sorted in ascending order. They are often used to construct the statistical boundaries of data distribution. The extreme value range is a reasonable range of data derived from these two quartiles and the interquartile range. Its core function is to initially identify and isolate extreme outliers that deviate significantly from the main distribution of the data, and reduce the interference of extreme values ​​on subsequent data processing.

[0031] In this application, the first and third quartiles of each feature dimension of the target data are calculated, and the difference between them (interquartile range) is used to construct an extreme value interval. Samples outside the interval are directly marked as samples to be removed, thus achieving rapid filtering of extreme outliers. This design does not require manual preset of fixed thresholds, and determines the screening boundary entirely based on the data's own distribution characteristics, adapting to the distribution differences of different feature dimensions, and avoiding the limitations of traditional fixed thresholds in adapting to multi-dimensional data.

[0032] For example, in an industrial 3D parts inspection scenario, after acquiring 3D point cloud target data containing random discrete noise, the first and third quartiles of each dimension are calculated for the spatial coordinates (x, y, z axes) and surface curvature of the point cloud. Based on the interquartile range of each dimension, an extreme value range for the corresponding dimension is constructed, and samples exceeding the range (such as background points of the workbench accidentally captured during scanning or coordinate anomalies caused by equipment failure) are marked as samples to be removed. Subsequently, the samples within the range are further verified to form a valid sample set, providing high-quality input for subsequent standardization, multi-scale noise filtering, and other steps.

[0033] In this embodiment, the combination of quartiles and extreme value intervals enables accurate identification of extreme outliers, preventing extreme noise from directly entering subsequent processing. The extreme value intervals are dynamically generated based on the data's own distribution, adapting to the distribution differences of various feature dimensions of 3D point clouds, eliminating the need for manual parameter adjustment and improving the automation level of the algorithm. While quickly filtering out extreme outliers, it retains potential effective samples within the interval (such as low-density points on the edges of parts), reserving space for subsequent refined noise filtering and balancing noise removal efficiency with effective sample retention rate.

[0034] In some optional implementations, the extreme value filtering further includes performing a secondary verification on the samples within the extreme value range, removing samples whose feature values ​​deviate from the mean of the dimension by more than a preset standard deviation multiple, to form the effective sample set.

[0035] The deviation of eigenvalues ​​from the mean of a dimension is an indicator that measures the degree to which a sample deviates from the overall average level of the data in a specific feature dimension. It reflects the dispersion of the sample in that dimension and is the core basis for identifying dispersed noise within an interval. The preset standard deviation multiple is an anomaly judgment benchmark set based on the normal distribution characteristics of the data. By quantifying the correlation between deviation and the degree of data dispersion, samples that exceed the reasonable fluctuation range are screened out.

[0036] In this application, after determining the extreme value range by quartiles and removing samples outside the range, for the scattered noise that may still remain within the range, the deviation of the feature value of each sample in each feature dimension from the mean of that dimension is further calculated. The deviation is compared with a preset standard deviation multiple, and samples with deviations exceeding the standard are removed. This achieves fine filtering of data within the extreme value range, avoids scattered noise from being unidentified and entering the subsequent processing flow, and further purifies the effective sample set.

[0037] For example, in an industrial 3D parts inspection scenario, after acquiring 3D point cloud target data containing random discrete noise, extreme value intervals are first constructed using the quartiles of each feature dimension. Background points on the scanning workbench and coordinate anomalies caused by equipment malfunctions outside the intervals are then removed. Subsequently, the point cloud samples within the intervals are subjected to secondary verification. For the z-axis height feature dimension of the point cloud, the deviation between the z-axis feature value of each sample and the mean of that dimension is calculated. If the deviation exceeds a preset standard deviation multiple, it is determined to be dispersed noise (such as height offset points caused by dust interference during the scanning process) and removed. Finally, a high-quality effective sample set is formed, providing clean data input for subsequent standardization processing and density calculation.

[0038] In this embodiment, secondary verification within the extreme value range accurately identifies and eliminates scattered noise hidden within reasonable ranges, overcoming the limitation of single extreme value range filtering which can only handle extreme outliers. Judgment is based on the deviation between feature values ​​and dimensional mean, which aligns with the data's own distribution characteristics and avoids misjudgment and deletion of low-density effective samples at cluster edges. The secondary verification and extreme value range filtering form a collaborative hierarchical filtering mechanism, ensuring both the comprehensiveness of noise filtering and improving the purity of the effective sample set, laying a solid foundation for the accuracy of subsequent processing steps.

[0039] In some optional implementations, the multi-scale noise filtering includes performing time-domain filtering using moving average filtering and frequency-domain filtering using wavelet decomposition, and fusing the results of time-domain filtering and frequency-domain filtering to obtain the noise-reduced data.

[0040] Moving average filtering is a commonly used smoothing technique in time-domain signal processing. It uses a sliding window to cover the data sequence and calculates the average value of samples within the window to replace the current sample value. Its core function is to suppress sudden high-frequency impulse noise, reduce transient interference in the data, and make the signal trend more stable. Wavelet decomposition is a multi-scale analysis method in the frequency domain that can decompose the original signal into low-frequency approximate components and high-frequency detail components. The low-frequency components carry the core features of the data, while the high-frequency components are mostly noise, which can be removed through thresholding. The fusion of time-domain and frequency-domain filtering results combines the advantages of both filtering dimensions. By integrating the processing results through weighted averaging or feature concatenation, it avoids the limitation of a single filter being able to deal with only a certain type of noise and ensures the comprehensiveness of the noise reduction effect.

[0041] In this application, multi-scale noise filtering serves as a secondary noise reduction step after extreme value filtering. First, moving average filtering is used to process the standardized effective samples in the time domain to smooth high-frequency impulse noise generated by equipment acquisition errors and transient interference. Then, multi-layer wavelet decomposition is used for frequency domain analysis to separate and remove high-frequency detail components containing scattered noise, while retaining low-frequency components carrying core features. Finally, the smoothed data after time-domain filtering is fused with the core signal reconstructed by frequency-domain filtering to obtain the noise-reduced data, thus thoroughly reducing the interference of scattered noise on subsequent density calculations.

[0042] For example, in an industrial 3D parts inspection scenario, after extreme value filtering, secondary verification, and standardization, 3D point cloud data still retains high-frequency pulse noise caused by equipment vibration and dispersed high-frequency noise caused by dust interference. At this point, a moving average filter is first used to smooth the spatial coordinate features of the 3D point cloud in the temporal domain by setting an appropriate window size, attenuating the coordinate fluctuations caused by instantaneous pulse noise. Then, multi-level wavelet decomposition is performed on the time-domain filtered point cloud data, splitting the signal into low-frequency approximate components and high-frequency detail components. After removing the high-frequency components containing dispersed noise, a low-frequency signal that retains the core geometric features of the part is reconstructed. Finally, the time-domain smoothed point cloud data is fused with the frequency-domain reconstructed signal to obtain denoised, clean point cloud data, providing high-quality input for the accurate calculation of the subsequent density fusion model.

[0043] In this embodiment, comprehensive filtering of impulse noise and dispersed high-frequency noise is achieved through coordinated filtering in the time and frequency domains, overcoming the limitation of traditional single filtering which can only handle a certain type of noise. The fusion of moving average filtering and wavelet decomposition effectively removes noise while preserving the geometric structural features of the 3D point cloud to the greatest extent, avoiding distortion of the effective signal during the denoising process. The noise residue of the data after denoising is significantly reduced, providing a more accurate input basis for subsequent density calculation and cluster center location, indirectly improving the accuracy of the final clustering results. Adapting to the noise characteristics of unstructured data such as 3D point clouds further expands the applicability of the algorithm in industrial inspection scenarios.

[0044] In some optional implementations, the standardization process uses a preset standardization method to calculate the standardized value, and samples whose absolute value after standardization exceeds a preset threshold are re-marked as samples to be removed. After updating the mean and standard deviation of the effective samples, a second standardization is performed.

[0045] In some optional implementations, the preset standardization method is Z-score standardization.

[0046] Pre-standardization is a common method for unifying feature scales in data preprocessing. It refers to converting feature data with different dimensions and numerical ranges into dimensionless values ​​using fixed mathematical formulas. The core purpose is to eliminate the interference of feature scale differences on subsequent calculations and ensure that each feature has a fair weight in the calculation. Pre-threshold is an anomaly judgment boundary set based on the normal distribution characteristics of the data. It is used to screen potential noise samples that still deviate significantly from the main distribution of the data after standardization. Secondary standardization is to recalculate the statistical parameters of the effective samples after removing outlier samples and perform standardization again to further optimize the data distribution.

[0047] In this application, the standardization process first uses a preset standardization method to calculate the initial standardized values ​​of the effective sample set. Samples whose absolute values ​​after standardization exceed a preset threshold are marked as samples to be removed and then removed. Then, based on the remaining effective samples, the statistical parameters such as the mean and standard deviation are updated. Finally, a second standardization is performed to obtain standardized data with uniform scale and less residual noise. This avoids parameter deviations caused by initial noise interference from single standardization and provides high-quality input for subsequent density calculations.

[0048] For example, in an industrial 3D parts inspection scenario, after the 3D point cloud is filtered for extreme values ​​and undergoes secondary verification within the interval to obtain a valid sample set, it enters the standardization process. For the feature dimensions of the point cloud with different dimensions such as spatial coordinates (x, y, z axes) and surface curvature, a preset standardization method is used to calculate the initial standardized value of each sample. If the absolute value of the standardized value of a sample's z-axis exceeds a preset threshold, it is judged as potential noise (such as height anomalies caused by dust interference) and removed. Then, the mean and standard deviation of the remaining valid samples are updated, and secondary standardization is performed on all samples based on the new statistical parameters. Finally, standardized point cloud data with consistent scale of each feature dimension and data distribution that is more in line with the real features is obtained, laying the foundation for distance and similarity calculation in the subsequent density fusion model.

[0049] In this embodiment, a pre-defined standardization method is used to eliminate the dimensional differences of different feature dimensions of 3D point clouds, ensuring that features such as spatial coordinates and surface curvature have balanced weights in density calculation and avoiding calculation deviations caused by dimensional differences. By combining pre-defined threshold screening with secondary standardization, abnormal samples remaining after the initial standardization are further eliminated, optimizing the data distribution pattern. During secondary standardization, statistical parameters are updated to avoid interference from initial noise on the standardization results, improving the accuracy and robustness of data standardization. The purified data after standardization provides a consistent input basis for the accurate calculation of the subsequent density fusion model, indirectly improving the reliability of cluster center location and final cluster division.

[0050] Z-score normalization is a classic dimensionless method in data preprocessing. Its core principle is to map feature data at different scales to a uniform distribution interval using a mathematical formula. The formula is:

[0051] in These are the original eigenvalues. The sample mean. The standard deviation of the sample serves to eliminate differences in dimensions, ensuring that different features have fair weights in subsequent calculations; secondary standardization is performed after the initial standardization to remove outliers and update statistical parameters. and The standardized optimization process is executed again to correct the interference of initial noise on statistical parameters and further refine the data distribution.

[0052] In this application, the standardization process first uses the Z-score formula to calculate the initial standardized value for the effective sample set, then filters out potential noise samples whose absolute value after standardization exceeds a preset threshold and removes them; subsequently, the mean is recalculated based on the remaining effective samples. with standard deviation The updated parameters are substituted into the Z-score formula to perform secondary standardization, ultimately obtaining standardized data with uniform scale and less residual noise, ensuring that the application of the formula and data purification form a closed loop.

[0053] For example, in an industrial 3D parts inspection scenario, after the 3D point cloud undergoes extreme value filtering and secondary verification within the interval to obtain a valid sample set, it enters the standardization process. For feature dimensions of different dimensions such as spatial coordinates (x, y, z axes) and surface curvature of the point cloud, the Z-score formula is used. Calculate the initial standardized value for each sample, where These are the original measured values ​​for each feature. This represents the mean of the effective samples under this feature dimension. The standard deviation is used to determine the corresponding standard deviation; if the absolute value of the standardized z-axis value of a sample exceeds a preset threshold, it is identified as potential noise (such as height outliers caused by dust interference) and removed; then the remaining valid samples are updated. and The new parameters are substituted into the Z-score formula again to complete the secondary standardization, and finally standardized point cloud data with consistent scale of each feature dimension and data distribution that better fits the real geometric features of the parts are obtained. This lays the foundation for the accurate calculation of Euclidean distance and cosine similarity in the subsequent density fusion model.

[0054] In this embodiment, by applying the Z-score formula precisely, the dimensional differences between different feature dimensions of the 3D point cloud are strictly eliminated, ensuring that features such as spatial coordinates and surface curvature have balanced weights in density calculation, thus avoiding calculation deviations caused by dimensional differences. The initial standardization, combined with preset threshold filtering, further eliminates hidden outliers, reducing noise interference with data distribution. The second standardization updates... and Furthermore, the Z-score formula was reused to dynamically correct the statistical parameter bias caused by initial noise, thereby improving the accuracy and robustness of the standardization results. The standardized clean data provided a consistent input basis for the accurate calculation of the subsequent density fusion model, indirectly improving the reliability of cluster center location and final cluster division.

[0055] In some optional implementations, when constructing the index structure, the features are sorted by variance of the feature dimension, and the feature with the largest variance is selected as the segmentation dimension. The index structure is a K-dimensional tree.

[0056] K-dimensional trees are commonly used data indexing structures in high-dimensional spaces. Their core function is to accelerate nearest neighbor search by recursively partitioning the data space and establishing hierarchical indexes, thereby reducing the amount of traversal computation in high-dimensional data scenarios. This is a key technology for improving the efficiency of large-scale data processing. Selecting the splitting dimension by sorting the feature dimension variance is a design idea based on optimizing the index structure according to the data distribution characteristics. By prioritizing the splitting of the feature dimension with the largest variance, the data space partitioning can be made more discriminative, further improving the accuracy and efficiency of nearest neighbor search.

[0057] In this application, the index structure adopts a K-dimensional tree. During construction, the variance of each feature dimension of the target data is first calculated. After sorting by variance, the feature with the largest variance is selected as the segmentation dimension first. Then, the data space is recursively partitioned to complete the K-dimensional tree construction. This design addresses the multi-feature dimension characteristics of high-dimensional data such as 3D point clouds and images, avoiding the low indexing efficiency problem caused by traditional fixed segmentation dimensions, and providing efficient support for nearest neighbor search in subsequent density calculation.

[0058] For example, in an industrial 3D part inspection scenario, 3D point cloud data contains multiple feature dimensions such as x, y, z-axis spatial coordinates and surface curvature. When constructing a K-tree, the variance of each feature dimension is first calculated. If the variance of the z-axis coordinate is the largest (corresponding to the most significant fluctuation in the height direction of the part), then the z-axis is used as the initial dividing axis to divide the point cloud data into upper and lower subspaces. Subsequently, within each subspace, the variance of the remaining feature dimensions is recalculated, and the feature with the largest variance is selected as the next level dividing axis. This process is recursively executed until the K-tree construction is completed. When the density calculation requires obtaining the nearest neighbor set of each sample, the neighboring samples can be quickly located through this K-tree index without traversing all the point cloud data.

[0059] In this embodiment, the feature with the largest variance is preferentially selected as the segmentation dimension, making the spatial partitioning of the K-tree more consistent with the data distribution characteristics of 3D point clouds, significantly reducing invalid traversals during nearest neighbor search and improving search efficiency. The K-tree index structure effectively adapts to the high-dimensional characteristics of 3D point clouds, avoiding the impact of the curse of dimensionality on nearest neighbor search in high-dimensional data scenarios. Accelerating nearest neighbor search through indexing provides efficient data support for the local density calculation of the subsequent density fusion model, indirectly improving the processing speed of the overall clustering algorithm. Compared with the traditional unoptimized index construction method, this design significantly reduces the time consumption of nearest neighbor search for large-scale 3D point cloud data, adapting to the large-scale data processing needs in industrial scenarios.

[0060] In some alternative implementations, the number of neighbors is dynamically adjusted during nearest neighbor search, increasing the number of neighbors in data-dense regions and decreasing the number of neighbors in data-sparse regions.

[0061] The nearest neighbor number is a core parameter in nearest neighbor search, referring to the number of neighboring samples that each sample needs to be associated with. It is used to reflect the data distribution characteristics through local sample association and is the basis for density calculation. Dynamically adjusting the nearest neighbor number is an optimization strategy that adapts and adjusts this number according to the density differences of the data spatial distribution. The core is to avoid the limitations of a fixed nearest neighbor number in missing associated samples in dense areas and mixing in noisy samples in sparse areas, thereby improving the targeting and effectiveness of the nearest neighbor set.

[0062] In this application, the nearest neighbor search is based on the constructed Kd-tree index. The spatial structure of the Kd-tree recursively divides the data density distribution: the number of nearest neighbors is increased in dense data regions (samples are highly clustered in space) to ensure that more comprehensive local relationships are captured; the number of nearest neighbors is decreased in sparse data regions (samples are scattered in space) to avoid noise samples from being mixed into the nearest neighbor set, ensuring that the nearest neighbor relationship matches the true distribution of the samples, and providing accurate nearest neighbor data support for the subsequent density fusion model.

[0063] For example, in an industrial 3D parts inspection scenario, after multi-scale noise filtering and standardization, the 3D point cloud is used to construct a Kd-tree index based on feature dimension variance sorting. The density distribution is identified through the spatial partitioning results of the Kd-tree: point cloud samples in the main area of ​​the part are clustered (data-dense areas), and the number of neighbors is increased during nearest neighbor search to fully capture the local correlation of samples in the main area; point cloud samples at the edges and corners of the part are scattered (data-sparse areas), and the number of neighbors is reduced during nearest neighbor search to avoid including noise samples formed by dust in the air into the nearest neighbor set; subsequently, based on this dynamically adjusted nearest neighbor set, Euclidean distance and cosine similarity are fused to calculate the local density to ensure that the density assessment fits the true distribution characteristics of the 3D point cloud.

[0064] In this embodiment, by dynamically adjusting the number of nearest neighbors, the nearest neighbor set covers more comprehensive associated samples in data-dense areas, avoiding deviations in local density calculations caused by insufficient association; in data-sparse areas, core associated samples are accurately selected to avoid noise interference affecting the accuracy of density assessment; the dynamic adjustment strategy is deeply adapted to the density recognition capability of the Kd-tree index, adapting to the distribution differences of different regions of 3D point clouds without manual intervention, thus improving the automation and intelligence of the algorithm; the accurate nearest neighbor set provides high-quality input for subsequent local density calculations and global density weight correction, indirectly improving the reliability of the comprehensive density score and the accuracy of the clustering results.

[0065] In some optional implementations, when calculating local density by fusing spatial distance and feature similarity, the contribution ratio of the two is adjusted by a weighting coefficient, wherein the spatial distance is Euclidean distance and the feature similarity is cosine similarity.

[0066] Euclidean distance is a classic metric for measuring the spatial differences between samples in a high-dimensional space. It reflects the physical distribution correlation by calculating the straight-line distance between sample feature vectors and is a core indicator for representing spatial proximity in density calculation. Cosine similarity is a metric for measuring the directional consistency of sample feature vectors. It reflects the semantic or geometric alignment of features by calculating the cosine of the angle between vectors and is a key indicator for representing feature correlation in density calculation. Weighting coefficients are parameters that adjust the contribution ratio of the two metrics in density calculation. By quantifying the weights, density assessment can both fit the spatial distribution and take into account feature consistency, avoiding the one-sidedness of a single indicator.

[0067] In this application, when calculating local density by fusing spatial distance and feature similarity, Euclidean distance is explicitly used for spatial distance and cosine similarity is used for feature similarity, with preset weighting coefficients. and The contribution ratio of the two is adjusted. Specifically, a nonlinear fusion formula is used to integrate the distance difference corresponding to Euclidean distance and the direction difference corresponding to cosine similarity, where... Used to control the decay rate of Euclidean distance, enhancing the positive contribution of spatially neighboring samples to density; This is used to adjust the influence weight of cosine similarity, highlight the correlation between samples with consistent feature orientation, and ultimately achieve a dual-dimensional density evaluation of "spatial location and feature orientation".

[0068] For example, in an industrial 3D part inspection scenario, after multi-scale noise filtering and standardization, the 3D point cloud is used to obtain the dynamic nearest neighbor set for each sample based on a Kd-tree index. When calculating the local density, the spatial position difference between the sample and each sample in the nearest neighbor set is first quantified by Euclidean distance (e.g., the 3D coordinate distance between a point on the part surface and its nearest neighbor points), and then the cosine similarity is used to characterize the consistency of the normal vector directions of the two (e.g., the degree of matching of the normal vector directions of points on the flat surface of the part). By setting... The parameter controls the attenuation intensity of the Euclidean distance, causing the contribution of distant samples to the density to decrease rapidly. (Setting...) The parameters adjust the weight of cosine similarity to ensure that samples with the same normal vector direction (such as parts in the same plane) receive a higher density contribution. Finally, the local density value of each sample is calculated through a nonlinear fusion formula, which provides a basis for subsequent global density weight correction.

[0069] In this embodiment, the fusion of Euclidean distance and cosine similarity overcomes the limitations of traditional density assessments that rely solely on a single distance metric. This approach captures both the spatial distribution characteristics of the samples and the consistency of feature orientations; weighting coefficients... and The flexible adjustment allows density calculation to adapt to the characteristics of unstructured data such as 3D point clouds, enabling accurate density assessment in different areas of the part (such as flat surfaces and corners); the dual-dimensional density calculation can accurately distinguish between "low-density effective edge points" and "noise points", avoiding the problem of traditional algorithms misjudging effective edge samples as noise; the fused local density provides a reliable basis for comprehensive density scoring, indirectly improving the accuracy of subsequent cluster center location and cluster division.

[0070] In some optional implementations, the calculation of the local density employs a nonlinear fusion method, which fuses the distance difference corresponding to Euclidean distance with the directional difference corresponding to cosine similarity.

[0071] Nonlinear fusion is a commonly used integration method in the field of data fusion. It refers to the weighted integration of evaluation indicators of different dimensions through nonlinear functions (such as exponential functions). Compared with linear fusion, it can more flexibly highlight the contribution of key indicators and suppress the interference of secondary factors. Its core function is to adapt to the complex distribution characteristics of data and make the fusion results more in line with the actual correlation patterns.

[0072] In this application, the nonlinear fusion of local density calculations is achieved through an exponential function, specifically using the formula...

[0073] in The distance difference is represented by Euclidean distance. Corresponding to the directional differences represented by cosine similarity, and The weighting coefficients are used to significantly amplify the contribution of close-range, highly similar samples to local density through the non-linear decay characteristics of the exponential function, while the contribution of distant, low-similarity samples decays rapidly. This achieves precise coupling between distance differences and directional differences, avoiding the rigidity of index weights caused by linear fusion.

[0074] For example, in an industrial 3D parts inspection scenario, after multi-scale noise filtering and standardization, the 3D point cloud is used to obtain the dynamic nearest neighbor set for each sample based on the Kd-tree index. When calculating the local density, the Euclidean distance (e.g., the straight-line distance between a point on the part surface and its nearest neighbor) and cosine similarity (e.g., the degree of coincidence of their normal vector directions) between each sample and each sample in the nearest neighbor set are substituted into the above nonlinear fusion formula. For two points on a flat surface of a part, the Euclidean distance is relatively close and the normal vector directions are consistent ( Approaching 1), substituting into the formula yields a large exponential function value, which significantly contributes to the local density; while the effective point on the edge of the part and the distant noise point have a large Euclidean distance and a large difference in normal vector direction ( The exponential function value approaches 0, which has almost no impact on the local density calculation, and finally yields the local density value for each sample.

[0075] In this embodiment, a deep coupling of Euclidean distance and cosine similarity is achieved through nonlinear fusion of exponential functions, which is more adaptable to the complex distribution characteristics of 3D point clouds compared to linear fusion. The nonlinear decay characteristic effectively amplifies the density contribution of key samples while suppressing the interference of distant, low-similarity noise samples, thus improving the accuracy of local density calculation. This fusion method can accurately characterize the density correlation of samples in different regions (flat surfaces, corners) of the part, avoiding density estimation bias caused by single-dimensional or linear fusion. The accurate local density provides a reliable foundation for subsequent global density weight correction and comprehensive density scoring, indirectly improving the accuracy of cluster center localization and final cluster division.

[0076] In some optional implementations, the global distribution features include the global maximum local density and the global dispersion coefficient within a preset range. The global density weight is calculated using the global maximum local density and the global dispersion coefficient, and the comprehensive density score is the product of the local density and the global density weight.

[0077] The global maximum local density is the maximum value among all sample local density values. It is the core parameter for establishing a global density evaluation benchmark, used to quantify the upper limit of density in the dataset and provide a reference for global calibration of local density. The global dispersion coefficient is an adjustment parameter that adapts to the differences in density distribution across different datasets. By setting a reasonable range, it balances the impact of data dispersion on density scores and avoids the limitations of adaptation caused by a single benchmark. The global density weight is a correction factor obtained based on the ratio between local density and global maximum local density, combined with the global dispersion coefficient. Its core function is to incorporate local density into the evaluation from a global distribution perspective, eliminating the interference of differences in the absolute values ​​of density across different datasets. The comprehensive density score is the product of local density and global density weight, realizing the fusion of local sample correlation features and global data distribution features, making the density evaluation more consistent with the overall data characteristics.

[0078] In this application, the global distribution feature explicitly includes the global maximum local density and a global dispersion coefficient within a preset range: first, the global maximum local density is obtained by traversing the local density values ​​of all samples; then, based on the dispersion of the dataset, a global dispersion coefficient within a preset range is set to balance the density differences of datasets with different distribution patterns; the global density weight is expressed by the formula...

[0079] Calculation, where For the local density of a single sample, For the global maximum local density, The global dispersion coefficient is used; the final comprehensive density score is the product of the local density and the global density weight, which not only preserves the correlation information between local samples, but also adapts to the overall data distribution through global calibration, thus avoiding the one-sidedness of local density assessment.

[0080] For example, in an industrial 3D parts inspection scenario, after multi-scale noise filtering and standardization, the local density of each sample is calculated based on the Kd-tree index and dynamic nearest neighbor count of the 3D point cloud. Then, iterate through the local density values ​​of all samples to obtain the global maximum local density. Based on the dispersion of the 3D point cloud data (e.g., dense in the main body area and sparse in the edge area), a global dispersion coefficient is set. ; through formula Calculate the global density weight for each sample, where the high-density samples of the main body of the part... near The corresponding global density weight is high, and the effective samples at the edge are low density. Although small, reasonable weights can still be obtained by considering the global distribution characteristics; finally, local density is used. With global density weights The product of these factors serves as the comprehensive density score, providing a basis for subsequent adaptive threshold screening of candidate centers.

[0081] In this embodiment, a unified global density benchmark is established by using the global maximum local density, making the density scores of samples from different regions comparable and avoiding misjudgments caused by differences in the absolute values ​​of local density. The global dispersion coefficient adapts to the discrete characteristics of 3D point cloud data, making the density weight calibration more consistent with the actual data distribution and improving the adaptability of different datasets. The comprehensive density score integrates local correlation and global distribution features, effectively distinguishing low-density effective samples and noise at the edges of 3D point clouds, making up for the limitations of single local density evaluation. The correction mechanism of global density weight reduces the one-sidedness of local density calculation, making the density score more reflective of the true importance of samples in the overall data, providing an accurate basis for subsequent candidate center screening and cluster optimization.

[0082] In some optional implementations, the adaptive threshold is calculated based on the overall variance of the target data, the historical maximum variance, and the noise ratio to determine the final threshold after iterative calibration.

[0083] The overall variance of the target data is a statistical indicator describing the dispersion of feature values ​​of all samples in the current dataset. It reflects the overall fluctuation of the data distribution and is a core parameter for measuring data stability. The historical maximum variance is the upper limit of the variance of similar datasets recorded by the system, providing a cross-dataset reference benchmark for the current threshold calculation. The noise ratio is the estimated proportion of noisy samples in the current data, used to quantify the intensity of noise interference. The initial threshold formula is a linear calculation model that integrates the above three parameters, transforming data fluctuation, historical benchmark, and noise intensity into an initial judgment boundary through weighting coefficients. Iterative calibration is a feedback adjustment mechanism based on the noise recognition accuracy, improving judgment accuracy by gradually optimizing the threshold. The stopping condition is a constraint rule for terminating the iteration, avoiding infinite loops or overcalibration. The threshold constraint limits the reasonable range of the final threshold, preventing threshold failure in extreme cases.

[0084] In this application, the adaptive threshold calculation first obtains the overall variance of the target data by weighted averaging the variances of each feature dimension, retrieves the historical maximum variance of similar datasets stored in the system, and combines it with the current data noise ratio obtained by the pre-analysis module, then substitutes it into the formula.

[0085] Calculate the initial threshold; then perform noise identification based on the current threshold. If the accuracy does not reach the preset standard and the number of iterations does not exceed the upper limit, adjust the threshold by scaling and repeat the identification process until the stopping condition is met. Finally, constrain the threshold to a reasonable range of 0.12-0.48 to ensure that the threshold is adapted to the data characteristics and noise interference intensity.

[0086] For example, in an industrial 3D parts inspection scenario, 3D point cloud data, after undergoing multi-scale noise filtering and standardization, enters the adaptive threshold calculation stage. First, the variances of feature dimensions such as point cloud spatial coordinates and surface curvature are calculated, and then a weighted average is used to obtain the overall variance of the target data. ; Retrieve the historical maximum variance of 3D point clouds of similar automotive parts in the system. Pre-analysis determines the proportion of noise in the current point cloud caused by dust interference and equipment vibration. Substitute the three factors into the initial threshold formula to calculate the initial threshold; then, based on this threshold, candidate centers are screened and noise is identified. If the initial noise identification accuracy is 78% (less than 85%), the threshold is adjusted proportionally and calculated again. After three iterations, the accuracy reaches 87%, meeting the stopping condition. The final threshold is locked at 0.25 (within the 0.12-0.48 constraint range), providing a precise judgment boundary for subsequent candidate center screening.

[0087] In this embodiment, the initial threshold is calculated by fusing the overall variance of the target data, the historical maximum variance, and the noise ratio. This allows the threshold to adapt to both current data fluctuations and benchmark data of the same type, avoiding threshold deviation caused by a single parameter. The iterative calibration mechanism uses accuracy as feedback to gradually optimize the threshold, significantly improving the accuracy of distinguishing between noise and valid samples. The stopping condition balances calibration effect and computational efficiency, avoiding excessive iteration and resource consumption. The threshold constraint ensures the effectiveness of the threshold in extreme data scenarios, preventing the deletion of valid samples due to an excessively high threshold or the retention of noise due to an excessively low threshold. The entire process does not require manual threshold preset, achieving adaptive adaptation to datasets with different noise ratios and variances, improving the versatility and automation of the algorithm in industrial inspection scenarios.

[0088] In some optional implementations, the stopping condition for the iterative calibration is that the noise recognition accuracy reaches a preset standard or a preset number of iterations, and the iterative calibration process gradually adjusts the threshold by scaling up.

[0089] The stopping condition for iterative calibration is the core constraint for balancing the effectiveness of threshold optimization and computational efficiency. It refers to terminating the threshold adjustment process when the noise recognition accuracy reaches a preset standard (to ensure the accuracy of threshold determination) or the number of iterations reaches a preset upper limit (to avoid infinite loops that consume resources). The scaling method is the specific execution method of iterative calibration. By gradually adjusting the threshold size according to a fixed ratio, the threshold steadily approaches the optimal solution, avoiding calibration failure caused by large fluctuations.

[0090] In this application, iterative calibration uses noise recognition accuracy and iteration count as dual stopping constraints: first, noise recognition is performed based on an initial threshold. If the accuracy does not meet the preset standard and the iteration count does not exceed the upper limit, the threshold is finely adjusted by scaling (e.g., by enlarging or reducing the current threshold by a fixed ratio), and the noise recognition and threshold adjustment process is repeated. Once the accuracy meets the standard or the iteration count reaches the upper limit, calibration is stopped immediately to ensure that the threshold meets the judgment accuracy requirements without wasting computational resources.

[0091] For example, in an industrial 3D part inspection scenario, after the 3D point cloud data is density-calculated to obtain a comprehensive density score, an initial threshold is calculated based on the overall variance of the target data, the historical maximum variance, and the noise ratio. This threshold is used to initially screen candidate centers and identify noise. If the noise identification accuracy is 76% (not reaching the preset 85% standard) and the current iteration count is only 1 (not exceeding the 5-iteration limit), the threshold is adjusted by scaling (e.g., fine-tuning by 1.05 times). After the second iteration, the accuracy improves to 82%, but still does not meet the standard, so the scaling adjustment continues. After the third iteration, the accuracy reaches 86%, meeting the preset standard, triggering a stop condition, terminating the iteration calibration, and determining the final threshold. If the accuracy still does not meet the standard after the first 5 iterations, the calibration will also be terminated according to the stop condition to avoid over-iteration.

[0092] In this embodiment, the dual stopping condition ensures both the accuracy of threshold calibration (meeting the accuracy target) and controls the computational cost (maximum number of iterations), avoiding the problems of insufficient accuracy or low efficiency caused by a single condition. The scaling method makes the threshold adjustment smooth and controllable, avoiding the threshold deviation from the optimal range caused by large fluctuations, and ensuring that each calibration approaches the accuracy direction. This design is adapted to the noise fluctuation characteristics of 3D point cloud data. Under different noise ratios and different variance scenarios, it can efficiently find the appropriate threshold, further improving the reliability of the adaptive threshold and the engineering practicality of the algorithm.

[0093] In some optional implementations, redundancy removal of candidate centers is achieved by iteratively comparing the distances between candidate centers, removing samples whose distance from the retained candidate centers is less than a preset multiple of the average distance of candidate centers.

[0094] The distance between candidate centers is a core indicator for measuring the spatial distribution correlation of different candidate centers, reflecting the degree of dispersion of potential cluster cores. The average distance of candidate centers with a preset multiple is a statistical benchmark calculated based on the distance between all candidate centers. It is used to define the judgment boundary of excessive clustering. Its core function is to avoid the initial cluster centers from clustering in local space and to ensure the rationality of the center distribution.

[0095] In this application, the redundancy removal of candidate centers is achieved through an iterative comparison mechanism: first, samples with a comprehensive density score not lower than an adaptive threshold are selected as candidate centers; then, the average distance between all retained candidate centers is calculated; and finally, the distance between each candidate center and the retained centers is checked one by one. If the distance is less than the average distance of a preset multiple, it is determined to be a redundant candidate center and removed. Finally, a uniformly distributed and representative initial cluster center set is obtained, avoiding cluster partitioning offset caused by initial center aggregation.

[0096] For example, in an industrial 3D parts inspection scenario, the 3D point cloud is used to calculate a comprehensive density score, and combined with an adaptive threshold, a batch of candidate centers (mostly point cloud samples from high-density areas of the main body of the part and key structures) are selected. Then, the spatial distance between all candidate centers is calculated, and the average value is taken as the benchmark. If a candidate center is too close to the already retained core candidate center (less than a preset multiple of the average distance), it means that the two represent the same local cluster structure, and the candidate center is a redundant sample (such as two adjacent high-density points on the flat surface of the part), and it is removed. After multiple rounds of iterative comparison, the final retained initial cluster centers cover the main body area, corners, holes and other key structures of the part, and are evenly distributed in space without excessive aggregation.

[0097] In this embodiment, by iteratively comparing the distances between candidate centers, redundant samples with local spatial clustering are effectively eliminated, avoiding the repetition or offset of cluster division caused by the "clustering" of initial cluster centers. This solves the problem of random and uneven distribution of initial center selection in traditional algorithms. The preset multiple average distance determination method dynamically defines the redundancy boundary based on the distribution characteristics of the candidate centers themselves, without the need for manual intervention, thus improving the automation level of the algorithm. The optimized initial cluster center set is more in line with the real cluster structure of 3D point clouds, providing a precise basis for subsequent sample allocation and iterative updates of cluster centers, indirectly reducing the risk of fluctuation in the final clustering results, and reducing the waste of computing resources caused by redundant centers.

[0098] In some alternative implementations, when iteratively updating cluster centers, samples with an intra-cluster comprehensive density score lower than the adaptive threshold are first excluded, and then the mean of the remaining samples is calculated as the new cluster centers.

[0099] Iteratively updating cluster centers is a key step that involves repeatedly optimizing and adjusting the center positions to gradually align the cluster centers with the core of the actual data clusters. The core logic is to first remove low-quality samples remaining in the clusters, and then recalculate the centers based on the clean samples to avoid noise interfering with the center positioning. The stopping condition is a constraint rule that balances clustering accuracy and computational efficiency, and can be flexibly adapted to different data scale scenarios by adjusting the convergence accuracy or the upper limit of the number of iterations.

[0100] In this application, when iteratively updating the cluster centers, samples with a comprehensive density score within the cluster that is not lower than the adaptive threshold are first selected, and residual noise samples with low scores are excluded. Then, the mean of each feature dimension of the remaining valid samples (such as the mean of x, y, and z axis coordinates of 3D point cloud) is calculated, and the mean is used as the new cluster center. The stopping condition is set to "the change in cluster centers is less than the preset convergence threshold" or "the preset upper limit of the number of iterations is reached". When the target data sample size exceeds the preset scale (such as 100,000 samples), the upper limit of the number of iterations is used first to avoid excessive computation time due to the pursuit of ultimate convergence under large-scale data.

[0101] For example, in an industrial 3D parts inspection scenario, after the 3D point cloud is initially clustered, it enters the cluster center iterative update stage. For the point cloud samples in each cluster, residual noise points (such as low-scoring points caused by dust interference) with a comprehensive density score lower than the adaptive threshold are first removed, and only high-scoring valid part points are retained. Based on these valid points, the mean values ​​of feature dimensions such as x, y, z axis coordinates and surface curvature are calculated respectively, and the point corresponding to the mean value is used as the new cluster center. After each update, the change between the new center and the previous center is calculated. If the change is less than the preset convergence threshold (such as 1e-4), the iteration stops. If the sample size exceeds 100,000, even if the change in the center does not reach the convergence threshold, the update is terminated when the iteration reaches the upper limit of 200 times to ensure computational efficiency.

[0102] In this embodiment, low-scoring samples within clusters are excluded during iterative updates, effectively eliminating the interference of residual noise on center localization. This allows the cluster centers to gradually converge to the actual high-density areas of the parts, significantly improving center localization accuracy compared to iterative methods without noise removal. The dual-stop condition design ensures clustering accuracy for small-scale data (meeting the convergence threshold) while adapting to the processing efficiency of large-scale 3D point cloud data (upper limit of iterations), avoiding excessive iteration and resource consumption. The method of updating centers based on the mean of effective samples aligns with the geometric distribution characteristics of 3D point clouds, ensuring that the centers always represent the core features of the clusters. This provides a stable foundation for the final accurate cluster division and further reduces the risk of fluctuations in clustering results.

[0103] In some optional implementations, the stopping condition for the iterative update of the cluster centers is that the change in the number of centers is less than a preset convergence threshold or that a preset upper limit for the number of iterations is reached. When the sample size exceeds a preset scale, the upper limit for the number of iterations is used.

[0104] The stopping condition for cluster center iteration is a dual constraint that balances clustering accuracy and computational efficiency. The core is to determine the degree of convergence by whether the "center change amount meets the standard" and to control computational costs by whether the "iteration count has reached its limit," especially optimizing efficiency priorities for large-scale data scenarios. The center change amount is the sum of the differences in the mean values ​​of each feature dimension of the cluster centers between two adjacent iterations. The preset convergence threshold is a minimum value (e.g., 1e-4) set based on the accuracy requirements of industrial scenarios to ensure sufficiently accurate center positioning. The preset upper limit for the number of iterations is the maximum number of iterations determined by hardware computing power and data processing timeliness requirements (e.g., 200 times). When the sample size exceeds the preset scale (e.g., 100,000), the iteration is stopped first by triggering the upper limit, avoiding a surge in time consumption due to the pursuit of ultimate convergence in large-scale data scenarios.

[0105] In this application, the execution logic of the stopping condition is deeply integrated with the iterative update process: after each exclusion of low-scoring residual noise samples within the cluster and calculation of new cluster centers, the change between the new center and the center in the previous round is automatically calculated; if the change is less than the preset convergence threshold, it means that the center has stabilized in the real high-density region, and the iteration stops directly; if the change does not meet the threshold, but the current iteration count has reached the upper limit, the update is also terminated; when the target data sample size exceeds the preset scale (e.g., 150,000), the change judgment is skipped, and the upper limit of the iteration count is used directly as the only stopping condition, balancing the accuracy and timeliness of large-scale data processing.

[0106] For example, in an industrial 3D part inspection scenario, the 3D point cloud sample size is 120,000 (exceeding the preset scale of 100,000). After initial cluster allocation, it enters the cluster center iterative update stage: each iteration first removes residual noise points (such as low-scoring points caused by dust interference) with a comprehensive density score lower than the adaptive threshold. Based on the remaining valid points, the mean values ​​of the x, y, z axis coordinates and surface curvature are calculated to obtain new cluster centers. When iterating to the 150th iteration, the center change is still 2e-4 (greater than the preset convergence threshold of 1e-4), but because the sample size exceeds the preset scale, it continues to iterate to the upper limit of 200 iterations and then stops updating. If the sample size is 80,000 (not exceeding the preset scale), it continues to iterate until the center change drops to 8e-5 (less than 1e-4). At this time, the center has stably conformed to the real geometric core of the part, triggering the stop condition.

[0107] In this embodiment, the dual-stop condition design ensures both the clustering accuracy of small-scale 3D point cloud data (ensuring precise center positioning through a convergence threshold) and the processing efficiency of large-scale data (controlling time consumption through an upper limit on the number of iterations), avoiding the problems of insufficient accuracy or low efficiency caused by traditional single-stop conditions. When the sample size exceeds the preset scale, the upper limit on the number of iterations is used first, perfectly adapting to the actual needs of industrial inspection scenarios where large-scale data requires rapid results. The preset convergence threshold and the upper limit on the number of iterations can be flexibly adjusted according to the inspection accuracy requirements of different parts, improving the engineering practicality of the algorithm. The coordination between the stop condition and the removal of low-scoring samples in the iterative update ensures the stability of center convergence and avoids the waste of computing resources caused by excessive iteration, indirectly improving the industrial adaptability of the overall clustering process.

[0108] In some optional implementations, the secondary allocation of samples involves redistributing samples whose overall density score is below a threshold but whose distance from the center of a cluster is less than a preset multiple of the cluster radius. The allocation is determined by combining the spatial proximity of the sample to the cluster center and the distribution characteristics of the cluster.

[0109] Secondary sample allocation is the final step in cluster optimization. The core logic is to re-determine the affiliation of samples whose "overall density score is lower than the adaptive threshold but is spatially close to a certain cluster". This avoids misjudging low-density effective samples such as cluster edges and around holes as noise due to a single density score standard, and ensures the integrity and geometric consistency of the clustering results.

[0110] In this application, the secondary allocation relies on a dual judgment of "density score and spatial proximity": first, samples whose comprehensive density score does not reach the adaptive threshold are screened out; then, the spatial distance between each sample to be allocated and all cluster centers is calculated, and the radius of each cluster is determined (such as the average distance of all samples in the cluster to the center); if the distance between the sample to be allocated and the center of a cluster is less than a preset multiple of the cluster radius, and the normal vector and geometric features of the sample match the samples in the cluster, then it is reassigned to that cluster, thus achieving the effective recovery of samples that "do not meet the density standard but have clear spatial affiliation".

[0111] For example, in an industrial 3D part inspection scenario, after the cluster centers of the 3D point cloud are iteratively updated, some point cloud samples on the edges of parts and inside holes are still not assigned to any cluster due to their low density and overall density score being below the adaptive threshold. At this time, a secondary allocation process is initiated: first, the radius of each cluster is calculated (e.g., the radius of the main cluster of the part is the average distance from the point cloud in that cluster to the center); then, the unassigned samples are checked one by one. If the distance between an edge point and the center of the main cluster is less than a preset multiple of the cluster radius, and its normal vector direction is consistent with the angle between the normal vectors of the samples in the main cluster (fitting the geometric structure of the part surface), then the point is determined to be a valid edge point of the part and is reassigned to the main cluster. Similarly, low-density points around holes, if they are close to the cluster corresponding to the hole, are also assigned to the main cluster through the same logic.

[0112] In this embodiment, the secondary allocation mechanism accurately recovers low-density effective samples in special regions such as cluster edges and holes in the 3D point cloud, avoiding the loss of effective samples caused by the "one-size-fits-all" density scoring of traditional algorithms. The dual judgment of spatial proximity and geometric feature fit ensures the accuracy of secondary allocation and prevents irrelevant noise from being mistakenly included in the cluster. This design forms a closed loop with the process of iteratively updating the cluster center. Iterative optimization ensures the accurate positioning of the cluster center, and secondary allocation completes the edge structure of the cluster. This makes the final clustering result of the 3D part point cloud completely restore the geometric shape of the part, improving the reliability of size measurement and structure recognition in industrial inspection.

[0113] In some alternative implementations, density calculations are optimized for GPU parallelism, using a 3D mesh thread model designed based on the CUDA architecture, where each kernel function independently calculates the Euclidean distance and cosine similarity of sample pairs.

[0114] In some optional implementations, the number of threads in each thread block of the three-dimensional mesh thread model is configured according to the GPU hardware characteristics, and the dimension of the three-dimensional mesh is dynamically adjusted according to the number of samples and GPU hardware resources.

[0115] Among them, GPU parallel optimization refers to using the multi-core parallel computing capabilities of graphics processors to accelerate the execution of high-density computing tasks. In general fields, it is often used for matrix operations and distance calculations of large-scale data. In this application, Euclidean distance and cosine similarity calculations in the density calculation stage are specifically optimized, and the core relies on the CUDA architecture to realize task splitting and parallel execution.

[0116] Among them, the CUDA architecture is a parallel computing platform and programming model launched by NVIDIA. In general knowledge, the serial task of the CPU can be decomposed into multiple independent subtasks and distributed to multiple threads of the GPU for parallel processing. In this application, a three-dimensional mesh thread model is designed through this architecture, so that each thread independently carries a set of sample pairs of computing tasks, avoiding the efficiency bottleneck of serial traversal.

[0117] The three-dimensional grid thread model refers to the organization of parallel computing tasks into a three-dimensional grid and thread blocks. In general, the dimensions are configured through the gridDim and blockDim parameters to achieve a precise mapping between tasks and threads. In this application, the dimensions are split according to the computational task amount of "total number of samples × number of nearest neighbors", and the thread index directly corresponds to "sample". - Neighbors The coordinates are used to ensure that there are no conflicts in the calculation.

[0118] Kernel functions are parallel functions executed on the GPU in the CUDA architecture. Generally, each thread can independently call a kernel function to process a specific subtask. In this application, each kernel function is specifically responsible for a set of "samples". - Neighbors The calculation logic for Euclidean distance and cosine similarity is directly related to the core parameters in the density formula.

[0119] Among them, GPU shared memory is a high-speed cache space on the GPU chip. In general, it is used to cache frequently accessed data and reduce the read latency of global memory. In this application, feature data of neighboring samples is specifically cached to avoid multiple threads repeatedly reading the same neighboring feature and improve data access efficiency.

[0120] For example, in an industrial 3D parts inspection scenario, the number of 3D point cloud samples to be processed is 100,000, the number of dynamic nearest neighbors for each sample is k=10, and the total computational task reaches 1 million sets of "samples". - Neighbors The Euclidean distance and cosine similarity calculation for the sample is as follows: First, a 3D mesh thread model is configured based on the CUDA architecture, with each thread block containing 256 threads (GPU hardware-friendly configuration). The 3D mesh dimension is 32×32×4 (total number of blocks 4096, covering all computational tasks). The thread index is mapped to the sample index through "blockIdx.z×blockDim.x×blockDim.y+blockIdx.y×blockDim.x+blockIdx.x×blockDim.x+threadIdx.x". - Neighbors The coordinates are used to ensure that each thread processes a pair independently; when the kernel function is called, the thread first reads the standardized sample from the GPU shared memory. and neighbors The feature data (frequently accessed nearest neighbor features cached in shared memory) is then used to independently calculate Euclidean distance (supplemented square root of the squared differences of three-dimensional coordinates) and cosine similarity (vector dot product / modulus product, adding 1e-6 to avoid division by zero). The calculation results are stored in distMatrix and cosSimMatrix matrices. After all threads have completed the calculation, the intermediate results of local density are summarized through GPU parallel reduction operations. Finally, the time taken for density matrix calculation is reduced from 2 hours for CPU single thread to within 30 minutes, improving efficiency by more than 3 times.

[0121] In this embodiment, the CUDA-based 3D mesh thread model is perfectly adapted to the "sample-nearest neighbor" pairing computation scenario. The task is evenly split, avoiding thread idleness and fully leveraging the multi-core parallel advantages of the GPU to solve the efficiency bottleneck of large-scale data density computation. Each kernel function independently calculates the core parameters of the sample pair, ensuring that the computation process is independent and conflict-free, accurately matching the core logic of the density fusion model and not affecting the accuracy of density evaluation. The GPU shared memory caches the features of nearest neighbor samples, significantly reducing the latency of repeated global memory reads, breaking through the limitations of video memory bandwidth, and further improving computational efficiency. The entire optimization scheme does not require changing the core logic of density computation, but only achieves efficiency improvement through hardware parallelism and memory access optimization, ensuring that the clustering accuracy is consistent with the unoptimized scheme. It is perfectly adapted to scenarios such as industrial 3D part inspection and image clustering for large-scale real-time data processing, meeting the timeliness requirements of large-scale data processing.

[0122] In some optional implementations, the GPU parallel optimization also includes caching neighboring sample features through GPU shared memory, wherein the shared memory employs a block loading strategy to load neighboring sample features into the shared memory to improve access efficiency.

[0123] Among them, GPU shared memory cache nearest neighbor sample features refers to the use of high-speed shared memory on the GPU chip to temporarily store frequently accessed nearest neighbor sample feature data in density calculation. Its core function in general domains is to reduce repeated reads of low-speed global memory and lower data access latency. This application specifically addresses "samples". and Neighbor In the pairing computation scenario, cache nearest neighbor samples. The feature data avoids multiple threads repeatedly reading the same nearest neighbor feature, providing high-speed data support for parallel computation of kernel functions.

[0124] The block loading strategy refers to a storage management method that loads data to be cached into shared memory in batches according to a preset block size. In general knowledge, the block size can be dynamically adjusted according to the hardware cache capacity and data access frequency to avoid memory overflow caused by loading too much data at once. In this application, the actual capacity of GPU shared memory (such as the capacity limit of a single block of shared memory) and the feature dimension of neighboring samples are combined to split the features of neighboring samples into fixed block sizes and load them into shared memory in batches to ensure maximum cache utilization.

[0125] In this context, loading adjacent sample features into shared memory refers to preferentially loading the features of samples that are spatially or index-wise adjacent to each other into contiguous shared memory addresses. In general domains, continuous access to adjacent data can reduce memory access conflicts and improve memory bandwidth utilization. In this application, combining the characteristics of Kd-tree nearest neighbor search (where the nearest neighbor sample index is continuous), each sample... of Each neighboring feature is treated as a data block and loaded into a contiguous address space of shared memory to adapt to the continuous access mode of GPU threads.

[0126] For example, in an industrial 3D part inspection scenario, processing 100,000 samples of 3D point cloud data with k=10 (dynamic nearest neighbor count), where each sample has an 8-dimensional feature dimension, and the GPU shared memory block capacity supports storing 32 8-dimensional feature data: First, the block size of the block loading strategy is determined to be 32 adjacent sample features (adapting to shared memory capacity and thread access efficiency), based on the nearest neighbor list output by the Kd-tree (for each sample... 10 neighbors (index continuous distribution), each sample The corresponding 10 adjacent features are packaged into a data block; before the kernel function executes, the 0th thread of each thread block is responsible for loading the corresponding data block into the contiguous address space of shared memory in batches. After loading, the syncthreads() function is used to achieve thread synchronization, ensuring that all threads can access the valid cached data; subsequently, each thread calculates the "sample". Neighbor When calculating the Euclidean distance and cosine similarity, the nearest neighbors are directly read from contiguous addresses in shared memory. This solution utilizes feature data without requiring repeated access to global memory. Traditional shared memory caching methods without block loading are prone to cache overflow due to loading too much data at once, or access conflicts due to discontinuous data addresses. In contrast, this solution improves the shared memory access hit rate to over 90% and reduces data read latency by 40% through block loading and contiguous storage of adjacent features. Combined with a 3D mesh thread model, the final density matrix calculation efficiency is further improved by 15%-20% compared to the basic GPU acceleration solution, and the calculation time for 1 million pairs is further reduced from 30 minutes to within 25 minutes.

[0127] In this embodiment, the block loading strategy precisely adapts to the hardware capacity limitations of GPU shared memory, avoiding data overflow issues and ensuring the stable operation of the caching mechanism. The continuous loading method of adjacent sample features matches the memory access mode of GPU threads, reducing access conflicts and significantly improving memory bandwidth utilization. The collaborative design of shared memory caching, block loading, and adjacent feature loading further reduces the data access latency of kernel function parallel computation, breaking through the GPU memory bandwidth bottleneck. This optimization, together with the previously mentioned 3D mesh thread model and independent kernel function computation, forms a complete GPU parallel optimization system. Without changing the core logic of density calculation and clustering accuracy, it further consolidates the technical effect of improving density matrix calculation efficiency by more than 3 times, and is more suitable for the real-time clustering needs of large-scale 3D point cloud and image data with more than 100,000 samples, meeting the goal of efficient processing of large-scale data.

[0128] For example, in an industrial 3D parts inspection scenario, the target to be processed is 3D point cloud data of an engine block that has been preprocessed with "extreme value filtering (IQR+3σ) and multi-scale noise filtering (moving average filtering with window 4 and 3-layer wavelet decomposition)". The total sample size is 1 million, and 8-dimensional features are extracted from each sample and then normalized twice by Z-score (after updating). =0.2、 =0.8 (samples with an absolute value > 3 after standardization have been removed), dynamic nearest neighbor number Adjust by data density (close-range dense areas) =15, long-distance sparse region =5, overall average =10), the compatible GPU hardware parameters are: 4096 CUDA cores per card, a maximum of 256 threads per thread block, and a single block shared memory capacity of 48KB (can store 64 8-dimensional feature data, each 8-dimensional feature occupies 4×8=32 bytes, 64×32=2048 bytes < 48KB, with reserved redundant space to avoid overflow). The technical solution is as follows: First, the core configuration for GPU parallel optimization is implemented, and a 3D mesh thread model is designed based on the CUDA architecture: The boundaries of the parallel tasks are first defined, with a total computational workload of 1 million samples × =10=10 million sets of "samples" - Neighbors The calculation of Euclidean distance and cosine similarity is performed, with each task being independent and undependent, perfectly suited to the parallel characteristics of GPUs. The number of threads per thread block is configured according to GPU hardware characteristics, with each block having 256 threads. This avoids computational anomalies caused by exceeding hardware limitations and maximizes CUDA core utilization. The 3D grid dimension is then dynamically adjusted. The total number of blocks required is approximately 10 million ÷ 256 ≈ 39062.5, rounded up to 39063 blocks. Considering the distribution characteristics of GPU computing units, this is divided into a 3D grid dimension: gridDim.x = 64, gridDim.y = 64, gridDim.z = 10 (64 × 64 × 10 = 40960 blocks, covering 39063 blocks). Excess blocks are returned directly after checking if "threadId ≥ 10 million" using the thread index, avoiding wasted computing resources. The thread index is precisely mapped to the "sample" using a formula. - Neighbors The coordinates of the thread index are calculated using the following formula:

[0129] The mapping relationship between the sample index and the nearest neighbor index is as follows: , ,

[0130] ( To construct the Kd-tree nearest neighbor search results sorted by feature dimension variance, the feature with the largest variance in the z-axis coordinate is preferentially selected as the segmentation dimension, and the nearest neighbor sample index is continuous.

[0131] Then, parallel computation of the kernel function is initiated: each thread independently calls the kernel function, synchronously calculating the Euclidean distance and cosine similarity. The kernel function calculation logic strictly follows the requirements of the density formula: the Euclidean distance is obtained by summing the square differences and then taking the square root, and its mathematical expression is:

[0132] in For the sample The 8-dimensional standardized features For the sample The 8-dimensional standardized features; Cosine similarity is calculated by "vector dot product ÷ (product of magnitudes + ... To obtain (avoiding division by zero), first define the sample feature magnitude: ,

[0133] The formula for calculating cosine similarity is:

[0134] The calculation results are stored in distMatrix (1 million × 10) and cosSimMatrix (1 million × 10) respectively, providing a basis for subsequent local density ρ_i calculation.

[0135] Simultaneously, GPU shared memory optimization is performed: the kernel function declares the shared memory space "shared float sharedFeatures

[64] [8]" using the shared keyword (to adapt to the block loading requirements of 64 8-dimensional features), and adopts a block loading strategy—combining the continuity of the Kd-tree nearest neighbor index, the features of the nearest neighbor samples are split into data blocks of 64 adjacent samples (to adapt to the shared memory capacity and thread access efficiency), and the 0th thread of each thread block is responsible for block loading: first, the samples are read from global memory. corresponding Each adjacent feature data block is loaded in batches into a contiguous address space of shared memory. After loading, the __syncthreads() function is called to achieve thread synchronization, ensuring that all threads can access the valid cached data. After synchronization, threads read samples from the contiguous address space of shared memory. and neighbors Feature data is used to avoid repeated access to global memory. For In scenarios where neighboring sample features are frequently accessed, this strategy loads adjacent sample features continuously into shared memory, adapting to the continuous access mode of GPU threads and reducing memory access conflicts.

[0136] After the calculation is completed, the intermediate local density results are summarized through GPU parallel reduction operations, and each sample is... of Substituting the results of the nearest neighbor calculations into the local density formula

[0137] The optimized parameters are α=0.8 and β=0.2. For the sample of The nearest neighbor set can be used to quickly generate the local density matrix using this formula.

[0138] According to statistics from hardware monitoring tools, the shared memory access hit rate has increased to 94%, and the data read latency has been reduced by 52% compared to the basic GPU solution with no block loading and non-contiguous storage; the dynamic adaptation of the 3D mesh has enabled the thread utilization rate to reach 98%, with no obvious thread idle phenomenon.

[0139] Compared to traditional solutions: CPU single-threaded processing of 10 million pairs takes 2 hours, basic GPU-accelerated solutions (without dynamic adjustment of 3D mesh and no shared memory block loading) take 35 minutes, while this solution takes only 18 minutes to process 10 million pairs and generate the local density matrix. The density calculation efficiency is 6.7 times higher than CPU single-threaded processing and 48% higher than basic GPU solutions. Moreover, because the core logic of density calculation is not changed (integrating Euclidean distance and cosine similarity), the clustering purity is 28% higher than traditional solutions, perfectly solving the technical defects of low efficiency in large-scale data processing.

[0140] In this embodiment, the 3D mesh thread model dynamically adjusts its dimensions according to the number of GPU hardware cores and the workload, which not only adapts to hardware resource limitations but also ensures conflict-free computation through precise thread index mapping, fully leveraging the advantages of GPU multi-core parallelism. Each kernel function independently calculates the core parameters of the sample pair, and the computational logic is completely aligned with the density fusion model, ensuring that the accuracy of density evaluation is not affected by parallel optimization. The block loading strategy of shared memory precisely adapts to the hardware cache capacity, and the continuous storage of adjacent sample features further improves memory bandwidth utilization, breaking through the bottleneck of video memory bandwidth. The complete GPU parallel optimization system constructed by the three collaboratively not only meets the real-time processing requirements of large-scale data at the level of 1 million samples but also dynamically adjusts the configuration according to different GPU hardware resources (such as higher video memory capacity and more computing units) and data density scenarios, possessing extremely strong hardware adaptability and scenario scalability. This solution forms a closed loop with multi-stage noise filtering and density fusion model, improving processing efficiency while ensuring clustering accuracy in noisy scenarios, perfectly fitting application scenarios with high real-time requirements such as industrial 3D part inspection.

[0141] In some optional implementations, when the target data is image data, pixel or texture features are first extracted, and principal component analysis is used to reduce the dimensionality while retaining principal components with a preset variance percentage.

[0142] Extracting pixel or texture features refers to obtaining feature information that represents the essential attributes of an image from image data. In general, pixel features are the original pixel grayscale values ​​or RGB values ​​of the image. Texture features are parameters (such as contrast, correlation, and entropy) that represent the texture structure of an image, extracted through algorithms such as gray-level co-occurrence matrix and LBP (Local Binary Pattern). The core of these features is to capture the differences and correlations between images. In this application, for noisy image data (such as images of the surface of industrial parts and satellite remote sensing images), texture features are extracted first (taking into account both detail and noise resistance). If the image texture is not obvious, pixel features are extracted instead, providing basic data for subsequent dimensionality reduction and density calculation, and avoiding redundant interference caused by directly using the original image data.

[0143] Principal component analysis (PCA) dimensionality reduction refers to a statistical dimensionality reduction method that projects high-dimensional features onto a low-dimensional space through linear transformation. In general knowledge, the core is to calculate the eigenvalues ​​and eigenvectors of the feature covariance matrix, and select the principal components with the largest variance to construct a low-dimensional feature space, thereby eliminating feature redundancy and reducing computational complexity. In this application, we specifically target high-dimensional pixel / texture features extracted from images (usually with tens to hundreds of dimensions), and achieve dimensionality reduction through eigenvalue decomposition of the covariance matrix. This solves the problems of low density calculation efficiency and large clustering bias caused by high-dimensional features, and adapts to the computational requirements of density fusion models.

[0144] Among them, retaining the principal components with a preset variance percentage refers to the strategy of selecting principal components by cumulative contribution rate. In general fields, the preset variance percentage is usually 80%-99% to balance dimensionality reduction efficiency and information fidelity, and to avoid excessive dimensionality reduction that leads to the loss of key information. In this application, the preset variance percentage is clearly 95%. By selecting principal components with a cumulative contribution rate of 95%, the core discriminative information of the image (such as key features of the surface texture of the part and the main structural information of the image) is fully preserved while significantly compressing the feature dimension, ensuring the accuracy of subsequent density evaluation and clustering results.

[0145] For example, in the scenario of clustering detection of industrial parts surface images, the target data is an image of the surface of an automobile engine block containing salt-and-pepper noise. Clustering is needed to distinguish between qualified parts and parts with excessive surface scratches. First, the image is preprocessed (denoising and smoothing). Texture features are extracted using the gray-level co-occurrence matrix algorithm, including four types of features: contrast, correlation, energy, and entropy. Each type of feature is derived into 25 dimensions, resulting in a total of 100-dimensional high-dimensional texture features. Then, principal component analysis is performed to reduce the dimensionality of the 100-dimensional features: the feature covariance matrix is ​​calculated, and the corresponding eigenvalues ​​and eigenvectors are obtained. After sorting the eigenvalues ​​from largest to smallest, the cumulative contribution rate is calculated, and the top 20 principal components with a cumulative contribution rate of 95% are selected, compressing the 100-dimensional high-dimensional features into 20-dimensional low-dimensional features. The dimensionality-reduced features retain the core information of the cylinder block surface texture (such as the uniform texture features of qualified parts and the irregular texture features of scratched parts). Then, they are substituted into the density fusion model to calculate the comprehensive density score. Combined with adaptive thresholding and clustering optimization process, the accurate clustering of qualified parts and scratched parts is finally achieved.

[0146] In this embodiment, principal component analysis (PCA) effectively compresses high-dimensional image features into a low-dimensional space, avoiding the waste of computational resources and low clustering efficiency caused by the curse of dimensionality. This reduces the time complexity of density calculation by an order of magnitude, decreasing computation time by more than 40% compared to the non-PCA approach. Retaining 95% variance in principal components ensures that core image discrimination information is not lost, preventing misclassification of valid samples due to excessive feature compression, and maintaining image clustering accuracy above 88% in noisy environments. For noisy image data, texture features are extracted and combined with PCA to reduce the amplification effect of noise in high-dimensional features, further improving the purity and stability of the clustering results. This adaptation scheme enables the invention to accurately handle unstructured image data, overcoming the limitations of traditional clustering algorithms in adapting to high-dimensional image data. It significantly expands the algorithm's engineering application scope in fields such as industrial image detection and computer vision texture clustering, meeting the needs of multimodal data clustering adaptation.

[0147] In some optional implementations, when the target data is a 3D point cloud, the normal vector angle parameter is introduced in the local density calculation and the corresponding weight coefficient is configured, and the index structure is constructed in layers according to the acquisition distance.

[0148] The normal vector angle parameter is a physical quantity in 3D point cloud data that characterizes the consistency of local surface geometry of adjacent samples. In general, the cosine value is obtained by calculating the dot product of the normal vectors of two sample surfaces, and then the angle is determined. Its core purpose is to quantify changes in local surface curvature (e.g., the angle on a flat surface approaches 0°, while at corners it shows a gradient change). In this application, this parameter is used as an additional constraint term for calculating the local density of 3D point clouds, and is obtained through dot product operations. and neighbors The angle between the normal vectors It incorporates density formulas to capture the geometric structure features of 3D point clouds, distinguishing between spatially adjacent but geometrically inconsistent noise and spatially adjacent and geometrically consistent valid samples.

[0149] The corresponding weighting coefficient is a calibration factor that adjusts the contribution of specific parameters in the computational model. Generally, it needs to be dynamically configured based on the importance of the parameters and data characteristics to ensure the model adapts to the target scenario. In this application, the weighting coefficient is... The value range is clearly defined as 0.2-0.4. Its core function is to balance the contribution ratio of the angle between the normal vectors, the Euclidean distance, and the cosine similarity, so as to make the geometric structure consistent. Smaller samples receive higher density weights, and samples with large geometric differences ( The contribution of large noise samples is suppressed.

[0150] The index structure constructed according to the acquisition distance refers to the optimization method of dividing the data into layers based on the spatial distance during data acquisition and constructing an index independently at each layer. In general fields, this method is used to solve the problem of search efficiency imbalance of a single index in scenarios with uneven data distribution (such as dense in some areas and sparse in some areas). The index structure described in this application is a Kd-tree, which is divided into a near layer (distance < 5cm, dense point cloud), a middle layer (5-10cm), and a far layer (> 10cm, sparse point cloud) according to the acquisition distance between the acquisition device and the target part. Within each layer, the features are sorted according to the variance of the feature dimension, and the feature with the largest variance (such as spatial coordinates and surface curvature) is selected first as the segmentation dimension. The Kd-tree is constructed independently to adapt to the differences in point cloud distribution in different distance areas.

[0151] For example, in an industrial 3D parts inspection scenario, the target data is a 3D point cloud of an engine block containing discrete noise generated by equipment vibration. The distance between the acquisition device and the part ranges from 3 to 12 cm. First, a point cloud normal vector estimation algorithm (based on an initial Kd-tree to obtain the normal vector of each sample) is used. The nearest neighbor set (derived by combining the spatial relationship of nearest neighbor points) is used to obtain the surface normal vector of each sample, and the sample is calculated. and neighbors The angle between the normal vectors (Smooth surface sample) ≈0°, sample at the corner The noise sample exhibits a gradient change of 30°-60°. (Irregular fluctuations); then, based on the acquisition distance, the point cloud is divided into near-range layer (3-5cm), middle layer (5-10cm), and far-range layer (10-12cm). Within each layer, the variance of features such as spatial coordinates and surface curvature is calculated. The feature with the largest variance (e.g., the near-range layer has the largest variance in the z-axis coordinate, and the far-range layer has the largest variance in the x-axis coordinate) is selected as the segmentation dimension, and a hierarchical Kd-tree is independently constructed. When calculating local density, Substituting j into the adjusted density formula

[0152] Configuration =0.3 (adapting to the importance of geometric features in this scenario), and quickly obtain the nearest neighbor set through a hierarchical Kd-tree to complete the density calculation. Finally, effective samples at complex geometric structures such as part corners and holes are determined by... Regularity and density scores were accurately clustered, while discrete noise samples were... Irregular patterns and low density scores are filtered out, and the nearest neighbor search speed is improved by 35% compared to a single Kd-tree.

[0153] In this embodiment, the introduction of the normal vector angle parameter makes the local density calculation no longer solely dependent on spatial proximity and feature similarity, but rather deeply adapts to the geometric structural characteristics of 3D point clouds. This significantly improves the clustering accuracy of complex structural point clouds (such as corners and holes in parts), increasing the accuracy of distinguishing noise from valid samples by 22%; the corresponding weighting coefficients (0.2-0.4) can be dynamically adjusted according to the geometric complexity of 3D point clouds, balancing the contributions of geometric features and spatial and feature dimensions, avoiding the bias of single dimension dominating density calculation; Kd-tree is constructed in layers according to the acquisition distance, adapting to the density / sparseness differences of point clouds in different distance areas, solving the problem of low search efficiency of single Kd-tree in distant sparse areas, and significantly optimizing the nearest neighbor search efficiency; this scheme is deeply compatible with the scene characteristics of 3D point clouds, effectively making up for the defects of traditional clustering algorithms that ignore 3D point cloud geometric information and have poor index adaptability, meeting the requirements of accurate adaptation of 3D point cloud clustering, and expanding the application scope of the algorithm in the field of industrial 3D inspection.

[0154] The following explanation uses images as an example.

[0155] For example, in a textile industry fabric texture quality grading scenario, the target data consists of 1 million grayscale images of pure cotton fabric surfaces continuously collected from a production line. Each image has a resolution of 1280×960 pixels. Due to factors such as dust in the production environment, aging imaging equipment, and transmission link interference, the images contain 17% mixed noise (6% extreme bright / dark spot noise caused by equipment failure, 5% high-frequency pulse noise generated by equipment vibration during the acquisition process, and 6% salt-and-pepper noise caused by environmental dust). The detection objective is to automatically distinguish between three categories using a clustering algorithm: "flawless uniform texture fabric," "fabric fabric with slight yarn misalignment," and "fabric fabric with severe skipped stitches / holes," providing a basis for subsequent automated quality grading and sorting. The specific implementation process is as follows: First, a data preprocessing process is executed. This step integrates multi-stage noise filtering and multi-scale noise reduction optimization to address the shortcomings of existing technologies in "incomplete noise filtering" at the source. The first step is extreme value filtering. For the texture features extracted from each image (the feature extraction process will be explained in detail later), the first quartile Q1 and the third quartile Q3 of each dimension of the 128-dimensional texture features are calculated. Linear interpolation is used to determine the specific values ​​of Q1 and Q3. For example, for the "texture contrast" dimension, Q1=0.32 and Q3=0.86, an extreme value interval [Q1-1.5×(Q3-Q1), Q3+1.5×(Q3-Q1)] is constructed, that is, [0.32-1.5×0.86]. [54, 0.86 + 1.5 × 0.54] = [-0.49, 1.67], the samples outside the marked interval are extreme noise samples to be removed. These samples mainly correspond to extreme bright / dark spots in the image caused by equipment failure, accounting for about 6% of the total samples; then, a second fine-tuning filter is performed on the samples to be verified within the interval, calculating the deviation of each sample's feature value from the mean of that feature dimension, and removing scattered noise samples with "absolute deviation > 3 times the standard deviation", such as the mean of the "texture entropy" dimension. =1.25, standard deviation =0.31, and samples with an absolute deviation value >0.93 were judged as scattered noise. These samples mainly correspond to local noise caused by slight dust interference. About 2.3% of the samples were removed. Finally, through the collaborative mechanism of "coarse screening + fine screening", 917,000 effective image samples were initially retained. The false deletion rate of effective samples was controlled within 3%, which is far better than the 8% false deletion rate of the traditional single IQR filtering scheme.

[0156] The second step involves multi-scale noise filtering. For the high-frequency impulse noise and salt-and-pepper noise remaining after extreme value filtering, a combined time-domain and frequency-domain filtering approach is used. In the time-domain filtering, a moving average filter with a window size of 4 is selected (adapting to the frequency characteristics of impulse noise in this scenario; the size can be dynamically adjusted to 5 in noise-dense areas and 3 in sparse areas). The average grayscale value of four adjacent pixels is calculated point-by-point through a sliding window to smooth the high-frequency impulse noise generated by device vibration. For example, in an image, three consecutive pixel grayscale value jumps caused by device shaking (from 120 to 200 and then back to 130) are corrected to 150, 157, and 163 after moving average filtering, effectively attenuating the instantaneous fluctuations. In the frequency-domain filtering, a 3-layer db4 wavelet decomposition is used (the optimal decomposition layer verified experimentally; 2-layer decomposition is incomplete in noise reduction, and 4-layer decomposition easily loses texture details), to refine the image signal. The process is decomposed into one layer of low-frequency approximation coefficients and three layers of high-frequency detail coefficients. High-frequency detail coefficients containing salt-and-pepper noise are removed through soft thresholding, leaving only the low-frequency approximation coefficients that characterize the core features of the fabric texture. Finally, based on the proportions of the two types of noise (45% impulse noise and 55% high-frequency noise), the smoothed image after temporal filtering and the reconstructed image after frequency filtering are weighted and averaged with a weight of 4.5:5.5. The result is a high-purity denoised image of 892,000 pixels, with the noise residue rate reduced to 4.1%, which is significantly lower than the 12% residue rate of the traditional single filtering scheme. The continuity and detail features of the fabric texture are fully preserved, such as the interlacing structure of the yarn and the slightly misaligned texture direction.

[0157] After data denoising is completed, effective sample standardization is performed, and the standardized values ​​are calculated using the Z-score formula:

[0158] in These are 128-dimensional original texture feature values. The feature mean of 892,000 valid samples. This represents the standard deviation. For example, the original feature values ​​for the "texture correlation" dimension range from 0.1 to 0.9. =0.52、 =0.18, after standardization, the feature value of this dimension is mapped to the interval [-2.33, 2.11]; then, samples with an absolute value > 3 after standardization are relabeled as samples to be removed. This part of the samples is about 15,000 images, mainly extreme noise images that still remain after denoising. After removal, the data is updated. =0.19、 =0.76, and a second standardization was performed on the remaining 877,000 samples to obtain the standardized feature data. This completely eliminated the dimensional differences between different feature dimensions (such as mapping the 0-2 range of "texture contrast" and the 0-10 range of "texture entropy" to similar magnitudes), avoiding interference from feature scale differences in subsequent density calculations and ensuring that each feature has a fair contribution weight in density evaluation.

[0159] To address the curse of dimensionality issue that high-dimensional features in image data can easily lead to, feature adaptation optimization—principal component analysis—is performed for dimensionality reduction. First, the covariance matrix is ​​calculated for the 128-dimensional standardized texture features of 877,000 samples (extracted through the gray-level co-occurrence matrix, including four types of features: contrast, correlation, energy, and entropy, with each type of feature having 32 dimensions). This matrix has a dimension of 128×128, and its element calculation formula is as follows:

[0160] in, For the sample size, For the first The first sample 3D eigenvalues For the first The mean of the 128 eigenvalues ​​is obtained through eigenvalue decomposition. ≥ ≥ ≥ The cumulative contribution rate is calculated by sorting the corresponding eigenvectors in descending order of eigenvalues.

[0161] For example, the eigenvalues ​​of the first 10 principal components are 32.6, 28.9, 25.3, 21.7, 18.5, 15.2, 12.8, 10.5, 8.3, and 6.7, respectively, with a cumulative contribution rate of 62.3%. The cumulative contribution rate of the first 26 principal components reaches 95.1%, meeting the requirement of retaining principal components with a variance ratio of 95%. Therefore, the first 26 principal components are selected to construct a low-dimensional feature space, compressing the 128-dimensional high-dimensional features into 26-dimensional low-dimensional features. The dimensionality-reduced features eliminate feature redundancy (such as merging highly correlated horizontal and vertical texture contrasts into one principal component) while fully preserving the key discriminative information of fabric texture (such as the uniform texture features of flawless fabric, the local texture shift features of slightly misaligned fabric, and the texture breakage features of severely flawed fabric). The visualization results of the dimensionality-reduced features show that the cluster boundaries of the three types of fabric are clearer than the original high-dimensional features, and the computational complexity is reduced by 79.7%, laying the foundation for efficient processing of large-scale data in the future.

[0162] The density calculation process integrates a density fusion model with GPU parallel optimization, addressing the shortcomings of existing technologies such as one-sided density assessment and low efficiency in large-scale processing. First, a Kd-tree index is constructed to accelerate nearest neighbor search. The data is sorted by variance of the 26-dimensional principal component features, prioritizing the "defect edge contrast principal component" with the largest variance as the segmentation dimension, and dynamically adjusted accordingly. Value (area of ​​densely textured, flawless fabric) =15, sparse textured flaw area =6, overall average =10), quickly obtain each sample Nearest neighbor set. The construction of the Kd-tree index improves the efficiency of nearest neighbor search by 13 times compared to brute-force traversal. For example, for 877,000 samples, brute-force traversal requires calculating the distance of each sample to all other samples one by one, while Kd-tree limits the search scope to a local subspace through spatial partitioning, reducing the search time for a single sample's nearest neighbor from 12ms to 0.9ms.

[0163] Subsequently, GPU parallel optimization was performed. The adapted GPU hardware parameters were: 5120 CUDA cores per card, a maximum of 256 threads per thread block, and a shared memory capacity of 64KB per block (capable of storing 32 26-dimensional feature data, each 26-dimensional feature occupying 4×26=104 bytes, 32×104=3328 bytes < 64KB, with sufficient redundancy space reserved to avoid overflow). The first step was to configure the 3D mesh thread model and design parallel task partitioning based on the CUDA architecture: the total computation task was 877,000 samples × average k = 10 = 8,770,000 sets of "samples". Neighbor The Euclidean distance and cosine similarity calculations are performed independently for each task, fully adapting to the parallel characteristics of GPUs. Based on GPU hardware-friendly configuration, the number of threads per block is determined to be 256, resulting in a total block requirement of approximately 34,258 blocks (8.77 million ÷ 256). Considering the distribution characteristics of GPU computing units, this is divided into a three-dimensional grid dimension: gridDim.x = 64, gridDim.y = 64, gridDim.z = 8 (64 × 64 × 8 = 32,768 blocks; supplementing with gridDim.z = 9 results in a total of 36,864 blocks, covering 34,258 blocks). Excess blocks are directly returned after checking if "threadId ≥ 8.77 million" using the thread index, avoiding wasted computing resources. The thread index is calculated using the formula...

[0164] Precisely mapped to "sample" - Neighbors The coordinates of “, where: , ,

[0165] ( The Kd-tree nearest neighbor search results are constructed by sorting by the variance of the feature dimension, and the nearest neighbor sample index is continuous.

[0166] The second step involves initiating parallel computation of the kernel function. Each thread independently calls the kernel function, synchronously calculating the Euclidean distance and cosine similarity. The kernel function calculation logic strictly adheres to the density formula requirements: the Euclidean distance is obtained by "summarizing the square root of the differences in squares," i.e.

[0167] in , Samples and neighbors The Principal component characteristics; Cosine similarity is obtained by the following formula:

[0168] The calculation results are stored in distMatrix (877,000 × 10) and cosSimMatrix (877,000 × 10) respectively, for subsequent local density calculations. Computation provides the foundation.

[0169] To address the bottleneck of GPU global memory access latency, shared memory optimization is implemented: the kernel function declares a shared memory space "shared float shared Features

[32]

[26] " using the shared keyword (to adapt to the block loading requirements of 32 26-dimensional features), and adopts a block loading strategy—combining the continuity of the Kd-tree nearest neighbor index, the nearest sample features are split into data blocks of 32 adjacent samples (to adapt to the shared memory capacity and thread access efficiency), and the 0th thread of each thread block is responsible for block loading: first, samples are read from global memory. corresponding Each adjacent feature data block is loaded in batches into a contiguous address space of shared memory. After loading, the __syncthreads() function is called to achieve thread synchronization, ensuring that all threads can access the valid cached data. After synchronization, threads read samples from the contiguous address space of shared memory. and neighbors By utilizing characteristic data, repeated access to global memory is avoided. According to statistics from hardware monitoring tools, the shared memory access hit rate has increased to 93.6%, and the data read latency has been reduced by 51% compared to the basic GPU solution of "no block loading + non-contiguous storage", which significantly breaks through the limitations of video memory bandwidth.

[0170] Density fusion calculations strictly follow the density model, firstly through the local density formula.

[0171] Calculate the local density for each sample, where =0.8 (the optimal parameter obtained from the experiment, which controls the decay rate of the Euclidean distance). =0.2 (adjusting the contribution ratio of cosine similarity). For example, a flawless fabric sample. nearest neighbor samples All samples are flawless with similar textures. Smaller (average 0.32). Approaching 1 (average 0.96), after substituting into the formula... =9.14; Slightly misaligned fabric sample Average 0.57, Average 0.88 ≈10×exp(-0.8×0.3249-0.2×0.12)=10×exp(-0.2599- 0.024)=10×0.759=7.59; The sample of the severely defective fabric... Average 1.23, Average 0.45 ≈10×exp(-0.8×1.5129-0.2×0.55)=10×exp(-1.2103-0.11)=10×0.267=2.67. The local density differences among the three classes of samples are significant, providing a clear basis for subsequent clustering. Then, the global density weights are calculated to obtain the global maximum local density for all samples. =11.8, configure global dispersion coefficient =1.0 (adapting to the discrete nature of data in this scenario), through Calculate the global density weights, and the final comprehensive density score is: This results in flawless samples with high local density and a dominant global distribution receiving higher overall scores (average 8.52), while severely flawed samples receive the lowest overall scores (average 2.43), further amplifying the discriminative power of different categories of samples.

[0172] The adaptive threshold determination process addresses the shortcomings of existing technologies in terms of poor threshold adaptability. First, an initial threshold is calculated using a multi-factor fusion formula:

[0173] in The total variance of the current 26 principal component features is 0.35 (the weighted average of the variances of each feature dimension is 0.35). The variance is 0.48, which is the largest variance in historical datasets of similar fabrics. Given the noise percentage of the current data (4.1%), substituting it into the formula, we get the initial threshold T = 0.2 + 0.3 × (0.35 / 0.48) - 0.05 × (4.1% / 30%) ≈ 0.2 + 0.3 × 0.729 - 0.05 × 0.137 ≈ 0.2 + 0.219 - 0.007 ≈ 0.412. Subsequently, iterative calibration is performed, calculating the noise identification accuracy after each iteration. The accuracy of the first iteration is 82.3% (below the 85% threshold), so the threshold is adjusted to 0.398. The accuracy of the second iteration increases to 87.6%, meeting the preset conditions. The final threshold is determined to be 0.398 (within the reasonable range of 0.12-0.48 required by experimental experience). This adaptive threshold outperforms the traditional fixed threshold scheme. In test scenarios where the noise percentage increases from 5% to 25%, the noise detection accuracy remains stable at 86%-93%, while the accuracy of the traditional fixed threshold scheme fluctuates by 35%, significantly improving the robustness of the algorithm under different noise scenarios.

[0174] The clustering result generation process employs an optimized workflow of "candidate center screening + iterative update + secondary allocation" to further improve the accuracy and stability of the clustering results.

[0175] The first step is to screen the initial cluster centers: 68,000 samples with a comprehensive density score ≥ 0.398 are marked as candidate centers. The average distance between candidate centers is calculated to be 2.1 (feature space distance). Redundant samples with a distance < 2 × 2.1 = 4.2 from the retained candidate centers are removed to avoid excessive clustering of cluster centers. Finally, a set of 23,000 initial cluster centers is obtained, which is 66.2% less than the original number of candidate centers, ensuring that the initial centers have reasonable spatial distribution and representative density.

[0176] The second step is initial cluster assignment: Based on the Euclidean distance metric, each valid sample is assigned to the cluster corresponding to the nearest initial center, and is initially divided into three categories: "flawless uniform texture cluster", "slight yarn misalignment cluster" and "severe skipped stitch / hole cluster". The initial assignment accuracy rate is 89.7%.

[0177] The third step is iterative updating of cluster centers: In each iteration, low-scoring samples (noise residue) with a comprehensive density score below 0.398 within the cluster are excluded, and the mean value is used to update the centers. The stopping condition is set to "center change < 1e-4" or "150 iterations" (the current sample size of 877,000 is below 100,000, and the upper limit of the number of iterations has not been triggered). For example, the initial 26-dimensional principal component feature mean of the centers of the "flawless uniform texture cluster" is [0.82, 0.76, ..., 0.12]. After the first iteration, 12,000 low-scoring samples (mainly slightly misplaced samples) within the cluster are removed, and the centers are updated to [0.85, 0.79, ..., 0.11], with a center change of 0.032. After 86 iterations, the center change drops to 9.7 × 10⁻⁶. (<1e-4), the iteration terminates. At this point, the cluster center has converged to the true core region of each category, effectively reducing the interference of noise on the center localization.

[0178] The fourth step, secondary allocation, involved reassigning 37,000 samples with a comprehensive density score below 0.398 but whose distance from the center of a cluster was less than 1.2 times the cluster radius. Of these, 32,000 samples were reassigned to the "slight yarn misalignment cluster" because their distance from the center was less than 1.2 × 1.8 = 2.16 (the cluster radius is 1.8). These samples were primarily marginal valid samples with minor texture misalignment and temporarily low density scores, thus avoiding the erroneous deletion of valid samples due to a single density score. After secondary allocation, the final clustering accuracy improved to 94.3%, a 4.6% improvement over the initial allocation.

[0179] To verify the superiority of the technical solution of this invention, multiple sets of comparative experiments were conducted: the traditional K-Means algorithm (including single IQR outlier filtering) achieved a clustering accuracy of 76.2% on this dataset, with a processing time of 145 minutes; the DBSCAN density clustering algorithm (with fixed ε and MinPts parameters) achieved an accuracy of 81.5%, with a processing time of 128 minutes; the unoptimized density clustering algorithm (relying only on Euclidean distance, without GPU parallelism) achieved an accuracy of 85.3%, with a processing time of 92 minutes; while the solution of this invention achieved a clustering accuracy of 94.3%, with a processing time of only 23 minutes. The density calculation efficiency is 6.3 times higher than that of a single CPU thread and 4.2 times higher than that of a basic GPU acceleration solution. At the same time, the effective sample deletion rate is reduced to below 3%, and the noise residual rate is controlled at 4.1%, perfectly solving the four core defects of existing technologies: "incomplete noise filtering, one-sided density evaluation, poor robustness, and low efficiency in large-scale processing".

[0180] In this embodiment, multi-stage noise filtering and secondary standardization ensure feature purity from the data source, PCA dimensionality reduction effectively solves the curse of dimensionality for high-dimensional image features, Kd-tree indexing and GPU parallel optimization (dynamic configuration of 3D mesh + shared memory block loading) break through the efficiency bottleneck of large-scale data, density fusion model and adaptive threshold ensure clustering accuracy in noisy environments, and clustering optimization process further improves result stability. It not only addresses the core defects of existing technologies but also expands application scenarios through image data adaptation, perfectly aligning with the core goals of improving noise robustness, efficiency, and scenario adaptability. It provides an efficient and accurate automated solution for textile fabric texture quality grading and can be transferred to other image clustering scenarios such as industrial part surface defect detection and satellite remote sensing image classification, possessing broad engineering application value.

[0181] The clustering method provided in this application can be executed by a clustering device. This application uses an example of a clustering device executing the clustering method to illustrate the clustering device provided in this application.

[0182] The clustering device includes: The first acquisition module is used to acquire target data containing noise. The first filtering module is used to perform extreme value filtering on the target data to remove samples that deviate significantly from the data distribution, thereby forming an effective sample set; The second filtering module is used to standardize the effective sample set to eliminate feature scale differences, and then perform multi-scale noise filtering to reduce scattered noise interference, so as to obtain denoised data. The first fusion module is used to calculate the comprehensive density score of each sample in the denoised data based on the density fusion model. The density fusion model accelerates the nearest neighbor search by constructing an index structure, calculates the local density by fusing spatial distance and feature similarity, and obtains the comprehensive density score by combining global distribution features for correction. The first elimination module is used to filter cluster candidate centers by combining adaptive thresholds, and obtain an initial cluster center set after eliminating redundant candidate centers; The first iteration module is used to assign valid samples to the clusters corresponding to the nearest initial cluster centers, iteratively update the cluster centers, and perform secondary assignment on samples with low scores but meeting the spatial proximity condition to generate the final clustering result.

[0183] In some optional implementations, the extreme value filtering determines the extreme value range by calculating the first quartile and the third quartile of each feature dimension of the target data, and marks samples outside the extreme value range as samples to be removed.

[0184] In some optional implementations, the extreme value filtering further includes performing a secondary verification on the samples within the extreme value range, removing samples whose feature values ​​deviate from the mean of the dimension by more than a preset standard deviation multiple, to form the effective sample set.

[0185] In some optional implementations, the multi-scale noise filtering includes performing time-domain filtering using moving average filtering and frequency-domain filtering using wavelet decomposition, and fusing the results of time-domain filtering and frequency-domain filtering to obtain the noise-reduced data.

[0186] In some optional implementations, the standardization process uses a preset standardization method to calculate the standardized value, and samples whose absolute value after standardization exceeds a preset threshold are re-marked as samples to be removed. After updating the mean and standard deviation of the effective samples, a second standardization is performed.

[0187] In some optional implementations, the preset standardization method is Z-score standardization.

[0188] In some optional implementations, when constructing the index structure, the features are sorted by variance of the feature dimension, and the feature with the largest variance is selected as the segmentation dimension. The index structure is a K-dimensional tree.

[0189] In some alternative implementations, the number of neighbors is dynamically adjusted during nearest neighbor search, increasing the number of neighbors in data-dense regions and decreasing the number of neighbors in data-sparse regions.

[0190] In some optional implementations, when calculating local density by fusing spatial distance and feature similarity, the contribution ratio of the two is adjusted by a weighting coefficient, wherein the spatial distance is Euclidean distance and the feature similarity is cosine similarity.

[0191] In some optional implementations, the calculation of the local density employs a nonlinear fusion method, which fuses the distance difference corresponding to Euclidean distance with the directional difference corresponding to cosine similarity.

[0192] In some optional implementations, the global distribution features include the global maximum local density and the global dispersion coefficient within a preset range. The global density weight is calculated using the global maximum local density and the global dispersion coefficient, and the comprehensive density score is the product of the local density and the global density weight.

[0193] In some optional implementations, the adaptive threshold is calculated based on the overall variance of the target data, the historical maximum variance, and the noise ratio to determine the final threshold after iterative calibration.

[0194] In some optional implementations, the stopping condition for the iterative calibration is that the noise recognition accuracy reaches a preset standard or a preset number of iterations, and the iterative calibration process gradually adjusts the threshold by scaling up.

[0195] In some optional implementations, redundancy removal of candidate centers is achieved by iteratively comparing the distances between candidate centers, removing samples whose distance from the retained candidate centers is less than a preset multiple of the average distance of candidate centers.

[0196] In some alternative implementations, when iteratively updating cluster centers, samples with an intra-cluster comprehensive density score lower than the adaptive threshold are first excluded, and then the mean of the remaining samples is calculated as the new cluster centers.

[0197] In some optional implementations, the stopping condition for the iterative update of the cluster centers is that the change in the number of centers is less than a preset convergence threshold or that a preset upper limit for the number of iterations is reached. When the sample size exceeds a preset scale, the upper limit for the number of iterations is used.

[0198] In some optional implementations, the secondary allocation of samples involves redistributing samples whose overall density score is below a threshold but whose distance from the center of a cluster is less than a preset multiple of the cluster radius. The allocation is determined by combining the spatial proximity of the sample to the cluster center and the distribution characteristics of the cluster.

[0199] In some alternative implementations, density calculations are optimized for GPU parallelism, using a 3D mesh thread model designed based on the CUDA architecture, where each kernel function independently calculates the Euclidean distance and cosine similarity of sample pairs.

[0200] In some optional implementations, the number of threads in each thread block of the three-dimensional mesh thread model is configured according to the GPU hardware characteristics, and the dimension of the three-dimensional mesh is dynamically adjusted according to the number of samples and GPU hardware resources.

[0201] In some optional implementations, the GPU parallel optimization also includes caching neighboring sample features through GPU shared memory, wherein the shared memory employs a block loading strategy to load neighboring sample features into the shared memory to improve access efficiency.

[0202] In some optional implementations, when the target data is image data, pixel or texture features are first extracted, and principal component analysis is used to reduce the dimensionality while retaining principal components with a preset variance percentage.

[0203] In some optional implementations, when the target data is a 3D point cloud, the normal vector angle parameter is introduced in the local density calculation and the corresponding weight coefficient is configured, and the index structure is constructed in layers according to the acquisition distance.

[0204] The clustering device 90 in this application embodiment can be an electronic device or a component within an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television (TV), ATM, or self-service machine, etc. This application embodiment does not specifically limit the specific implementation.

[0205] The clustering device in this application embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit it.

[0206] The clustering device provided in this application embodiment can achieve... Figure 1 The various processes implemented in the method implementation examples will not be described again here to avoid repetition.

[0207] In some alternative implementations, such as Figure 2 As shown, this application embodiment also provides an electronic device 1300, including a processor 1301 and a memory 1302. The memory 1302 stores a program or instructions that can run on the processor 1301. When the program or instructions are executed by the processor 1301, they implement the various steps of the above-described clustering method embodiment and can achieve the same technical effect. To avoid repetition, they will not be described again here.

[0208] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0209] Figure 3 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of this application.

[0210] The electronic device 170 includes, but is not limited to, components such as: a radio frequency unit 1701, a network module 1702, an audio output unit 1703, an input unit 1704, a sensor 1705, a display unit 1706, a user input unit 1707, an interface unit 1708, a memory 1709, and a processor 17010. Those skilled in the art will understand that the electronic device 170 may also include a power supply (such as a battery) for powering the various components. The power supply can be logically connected to the processor 17010 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. Figure 3 The electronic device structure shown does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0211] The processor 17010 is used for: Acquire noisy target data; Extreme value filtering is performed on the target data to remove samples that significantly deviate from the data distribution, forming an effective sample set; The effective sample set is standardized to eliminate feature scale differences, and then multi-scale noise filtering is performed to reduce scattered noise interference, resulting in denoised data. The comprehensive density score of each sample is calculated based on the density fusion model. The density fusion model accelerates nearest neighbor search by constructing an index structure, calculates local density by fusing spatial distance and feature similarity, and obtains the comprehensive density score by combining global distribution features for correction. By combining adaptive threshold screening to select cluster candidate centers and removing redundant candidate centers, an initial set of cluster centers is obtained. Valid samples are assigned to the clusters corresponding to the nearest initial cluster centers. The cluster centers are iteratively updated, and samples with low scores but meeting the spatial proximity condition are reassigned to generate the final clustering result.

[0212] It should be understood that, in this embodiment, the input unit 1704 may include a graphics processing unit (GPU) 17041 and a microphone 17042. The GPU 17041 processes image data of still images or videos obtained by an image capture device (such as a camera) in video capture mode or image capture mode. The display unit 1706 may include a display panel 17061, which may be configured in the form of a liquid crystal display, an organic light-emitting diode, or the like. The user input unit 1707 includes at least one of a touch panel 17071 and other input devices 17072. The touch panel 17071 is also called a touch screen. The touch panel 17071 may include a touch detection device and a touch controller. Other input devices 17072 may include, but are not limited to, physical keyboards, function keys (such as volume control buttons, power buttons, etc.), trackballs, mice, and joysticks, which will not be described in detail here.

[0213] The memory 1709 can be used to store software programs and various data. The memory 1709 may primarily include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area may store the operating system, application programs or instructions required for at least one function (such as sound playback, image playback, etc.). Furthermore, the memory 1709 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM). The memory 1709 in the embodiments of this application includes, but is not limited to, these and any other suitable types of memory.

[0214] Processor 17010 may include one or more processing units; optionally, processor 17010 integrates an application processor and a modem processor, wherein the application processor mainly handles operations involving the operating system, user interface, and applications, and the modem processor mainly handles wireless communication signals, such as a baseband processor. It is understood that the aforementioned modem processor may also not be integrated into processor 17010.

[0215] Each of the above product embodiments can implement the various processes of the above clustering method embodiments through its own processor, and can achieve the same technical effect. To avoid repetition, they will not be described in detail.

[0216] This application also provides a readable storage medium storing a program or instructions. When executed by a processor, the program or instructions implement the various processes of the above-described clustering method embodiments and achieve the same technical effects. To avoid repetition, these will not be described again here. The processor is the processor in the electronic device or electronic system described in the above embodiments. The readable storage medium includes a computer-readable storage medium, such as a computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0217] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above-described clustering method embodiments and achieve the same technical effect. To avoid repetition, it will not be described again here.

[0218] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0219] This application provides a computer program product that is stored in a storage medium and executed by at least one processor to implement the various processes of the clustering method embodiments described above, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0220] In the embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between apparatuses or units through some interfaces, and may be electrical, mechanical, or other forms.

[0221] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0222] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0223] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application embodiment, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0224] The above descriptions are merely embodiments of this application and do not limit the patent scope of this application. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art, under the guidance of this application, may make equivalent structural or procedural transformations based on the description and drawings of the embodiments of this application, or directly or indirectly apply them to other related technical fields, without departing from the spirit and scope of protection of the claims. All such transformations are similarly included within the patent protection scope of the embodiments of this application.

Claims

1. A clustering method, characterized in that, include: Acquire noisy target data; Extreme value filtering is performed on the target data to remove samples that significantly deviate from the data distribution, forming an effective sample set; The effective sample set is standardized to eliminate feature scale differences, and then multi-scale noise filtering is performed to reduce scattered noise interference, resulting in denoised data. The comprehensive density score of each sample is calculated based on the density fusion model. The density fusion model accelerates nearest neighbor search by constructing an index structure, calculates local density by fusing spatial distance and feature similarity, and obtains the comprehensive density score by combining global distribution features for correction. By combining adaptive threshold screening to select cluster candidate centers and removing redundant candidate centers, an initial set of cluster centers is obtained. Valid samples are assigned to the clusters corresponding to the nearest initial cluster centers. The cluster centers are iteratively updated, and samples with low scores but meeting the spatial proximity condition are reassigned to generate the final clustering result.

2. The method according to claim 1, characterized in that, The extreme value filtering method determines the extreme value range by calculating the first quartile and the third quartile of each feature dimension of the target data, and marks samples outside the extreme value range as samples to be removed.

3. The method according to claim 2, characterized in that, The extreme value filtering also includes performing a secondary verification on the samples within the extreme value range, removing samples whose feature values ​​deviate from the mean of the dimension by more than a preset standard deviation multiple, thus forming the effective sample set.

4. The method according to claim 1, characterized in that, The multi-scale noise filtering includes time-domain filtering using moving average filtering and frequency-domain filtering using wavelet decomposition. The results of time-domain filtering and frequency-domain filtering are fused to obtain the noise-reduced data.

5. The method according to claim 1, characterized in that, The standardization process uses a preset standardization method to calculate the standardized value. Samples whose absolute value exceeds a preset threshold after standardization are re-marked as samples to be removed. The mean and standard deviation of the effective samples are updated and then a second standardization is performed.

6. The method according to claim 5, characterized in that, The preset standardization method is Z-score standardization.

7. The method according to claim 1, characterized in that, When constructing the index structure, the features are sorted by variance of the feature dimension, and the feature with the largest variance is selected as the segmentation dimension. The index structure is a K-dimensional tree.

8. The method according to claim 1, characterized in that, The number of neighbors is dynamically adjusted during nearest neighbor search, increasing the number of neighbors in densely populated areas and decreasing the number of neighbors in sparsely populated areas.

9. The method according to claim 1, characterized in that, When calculating local density by fusing spatial distance and feature similarity, the contribution ratio of the two is adjusted by a weighting coefficient. The spatial distance is Euclidean distance, and the feature similarity is cosine similarity.

10. The method according to claim 9, characterized in that, The calculation of the local density adopts a nonlinear fusion method, which fuses the distance difference corresponding to Euclidean distance with the directional difference corresponding to cosine similarity.

11. The method according to claim 1, characterized in that, The global distribution features include the global maximum local density and the global dispersion coefficient within a preset range. The global density weight is calculated using the global maximum local density and the global dispersion coefficient. The comprehensive density score is the product of the local density and the global density weight.

12. The method according to claim 1, characterized in that, The adaptive threshold is calculated based on the overall variance of the target data, the historical maximum variance, and the noise ratio. The final threshold is determined after iterative calibration.

13. The method according to claim 12, characterized in that, The stopping condition for the iterative calibration is that the noise recognition accuracy reaches a preset standard or a preset number of iterations. The iterative calibration process gradually adjusts the threshold by scaling up.

14. The method according to claim 1, characterized in that, Redundancy removal of candidate centers is achieved by iteratively comparing the distances between candidate centers, and removing samples whose distance from the retained candidate centers is less than a preset multiple of the average distance of candidate centers.

15. The method according to claim 1, characterized in that, When iteratively updating cluster centers, samples with a comprehensive density score within a cluster lower than the adaptive threshold are first excluded, and then the mean of the remaining samples is calculated as the new cluster centers.

16. The method according to claim 15, characterized in that, The stopping condition for the iterative update of the cluster centers is that the change in the center is less than the preset convergence threshold or the preset upper limit of the number of iterations is reached. When the sample size exceeds the preset scale, the upper limit of the number of iterations is used.

17. The method according to claim 1, characterized in that, The secondary allocation of samples involves redistributing samples whose comprehensive density score is below a threshold but whose distance from the center of a cluster is less than a preset multiple of the cluster radius. The allocation is based on the spatial proximity of the sample to the cluster center and the distribution characteristics of the cluster.

18. The method according to claim 1, characterized in that, Density calculation is optimized using GPU parallelism. A three-dimensional mesh thread model is designed based on the CUDA architecture, and each kernel function independently calculates the Euclidean distance and cosine similarity of sample pairs.

19. The method according to claim 18, characterized in that, The number of threads in each thread block of the three-dimensional mesh thread model is configured according to the GPU hardware characteristics, and the dimension of the three-dimensional mesh is dynamically adjusted according to the number of samples and GPU hardware resources.

20. The method according to claim 18, characterized in that, The GPU parallel optimization also includes caching neighboring sample features through GPU shared memory. The shared memory adopts a block loading strategy to load neighboring sample features into the shared memory to improve access efficiency.

21. The method according to claim 1, characterized in that, When the target data is image data, pixel or texture features are extracted first, and principal component analysis is used to reduce the dimensionality while retaining the principal components with a preset variance percentage.

22. The method according to claim 1, characterized in that, When the target data is a 3D point cloud, the normal vector angle parameter is introduced in the local density calculation and the corresponding weight coefficient is configured, and the index structure is constructed in layers according to the acquisition distance.

23. A clustering device, characterized in that, The clustering device includes: The first acquisition module is used to acquire target data containing noise. The first filtering module is used to perform extreme value filtering on the target data to remove samples that deviate significantly from the data distribution, thereby forming an effective sample set; The second filtering module is used to standardize the effective sample set to eliminate feature scale differences, and then perform multi-scale noise filtering to reduce scattered noise interference, so as to obtain denoised data. The first fusion module is used to calculate the comprehensive density score of each sample in the denoised data based on the density fusion model. The density fusion model accelerates the nearest neighbor search by constructing an index structure, calculates the local density by fusing spatial distance and feature similarity, and obtains the comprehensive density score by combining global distribution features for correction. The first elimination module is used to filter cluster candidate centers by combining adaptive thresholds, and obtain an initial cluster center set after eliminating redundant candidate centers; The first iteration module is used to assign valid samples to the clusters corresponding to the nearest initial cluster centers, iteratively update the cluster centers, and perform secondary assignment on samples with low scores but meeting the spatial proximity condition to generate the final clustering result.

24. A computer device, characterized in that, It includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the method according to any one of claims 1 to 22.

25. A computer-readable storage medium, characterized in that, It stores a computer program thereon, which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 22.

26. A computer program product, characterized in that, The computer program includes a program or instructions that, when executed by a processor, implement the steps of the model fusion method as described in any one of claims 1 to 22.

Images