Abnormal data identification method, device, equipment and storage medium

Abstract

The present application relates to the field of data identification technology, and discloses a method, apparatus, device, and storage medium for identifying abnormal data. The method comprises: extracting data distribution characteristics within each time window in a high-dimensional time series data stream, generating low-dimensional data after dimensionality reduction based on the data distribution characteristics and the adjusted random projection matrix of the high-dimensional time series data stream, configuring local sensitive hash resolution parameters according to the local volatility of the low-dimensional data, constructing a multi-resolution hash table, and identifying abnormal data points based on the data density in the bucket of the multi-resolution hash table. The present application adjusts the dimensionality reduction process by extracting data distribution characteristics, reduces computational complexity while retaining key information, constructs a multi-resolution hash table based on the local volatility of the dimensionality reduction data, realizes adaptive analysis of different data regions, identifies abnormal data points based on the data density in the bucket, and can effectively deal with dimensionality problems and distribution change problems, while ensuring detection accuracy and improving processing efficiency.

Description

Technical Field

[0001] The present application relates to the field of data identification technology, and in particular to a method, apparatus, device and storage medium for identifying abnormal data. Background Art

[0002] In smart campus big data management, real-time monitoring of semiconductor manufacturing equipment faces the challenge of processing high-dimensional time-series data. For example, wafer fab etching machines generate hundreds of sensor data streams in various dimensions, including temperature, air pressure, and RF power. These data include both periodic process fluctuations and transient anomalies caused by equipment failures. Existing anomaly detection technologies suffer from two major drawbacks: first, traditional dimensionality reduction methods are poorly adaptable to dynamic data, resulting in the loss of key fault signatures; second, fixed-threshold detection algorithms are unable to cope with distribution drift caused by process switching, equipment aging, and other factors. These drawbacks result in low accuracy in identifying anomaly data. Summary of the Invention

[0003] The main purpose of this application is to provide a method, device, equipment and storage medium for identifying abnormal data, aiming to solve the technical problem of low anomaly identification accuracy caused by high data dimension and dynamic distribution changes in the existing technology.

[0004] To achieve the above objectives, the present application proposes a method for identifying abnormal data, which includes:

[0005] Extract data distribution features within each time window in high-dimensional time series data stream;

[0006] Generate low-dimensional data after dimensionality reduction based on the data distribution characteristics and the adjusted random projection matrix of the high-dimensional time series data stream, wherein the adjusted random projection matrix is a matrix obtained by jointly optimizing the data distribution characteristics and the initial random projection matrix of the high-dimensional time series data stream;

[0007] Configure the local sensitive hashing resolution parameters according to the local volatility of low-dimensional data and construct a multi-resolution hash table;

[0008] Identify abnormal data points based on the data density in the bucket of the multi-resolution hash table.

[0009] In one embodiment, the step of identifying abnormal data points based on the data density in the bucket of the multi-resolution hash table includes:

[0010] Determine the data density within each hash bucket of the multi-resolution hash table;

[0011] Adjusting the initial density determination threshold according to the resolution parameter of the multi-resolution hash table to obtain the target density determination threshold;

[0012] Marking abnormal hash buckets by comparing the data density in the bucket with the target density determination threshold;

[0013] A secondary verification is performed on the data points in the abnormal hash bucket to obtain a verification result, and the abnormal data point is determined by combining the verification result and the density distribution characteristics of adjacent hash buckets.

[0014] In one embodiment, the step of adjusting the initial density determination threshold according to the resolution parameter of the multi-resolution hash table to obtain the target density determination threshold includes:

[0015] Determining a base density threshold according to a resolution parameter of a multi-resolution hash table;

[0016] Obtaining aggregate distribution features of multiple time windows in a high-dimensional time series data stream, and determining a density correction coefficient based on the aggregate distribution features;

[0017] The basic density threshold is multiplied by the density correction coefficient to obtain a target density determination threshold.

[0018] In one embodiment, the step of obtaining aggregate distribution features of multiple time windows in a high-dimensional time series data stream and determining a density correction coefficient based on the aggregate distribution features includes:

[0019] Obtaining aggregate distribution features of multiple time windows in a high-dimensional time series data stream, wherein the aggregate distribution features include the statistical characteristic matrix and principal component evolution trajectory of each dimensional data;

[0020] A spatiotemporal association graph model is constructed based on the aggregated distribution features. The spatiotemporal association graph model uses the data distribution features of each time window as nodes and represents the distribution similarity between windows through edge weights. The spatiotemporal association graph model is used to identify the stable evolution stage and the sudden transition stage of the high-dimensional time series data stream, and output the distribution evolution stage identifier. The density correction coefficient is determined according to the distribution evolution stage identifier.

[0021] In one embodiment, the step of extracting data distribution features within each time window in the high-dimensional time series data stream includes:

[0022] Perform multi-scale window segmentation processing on the input high-dimensional time series data stream to obtain window analysis results, which include at least one of a short-term window with a fixed length, an event window automatically divided according to data mutation points, and a long-term window containing historical data;

[0023] By determining the statistics of each dimension of data within each short-period window, short-period statistical features are generated;

[0024] Generate data event features by performing waveform analysis on each event window;

[0025] Smoothing and trend decomposition of data in long-period windows to generate long-term trend characteristics;

[0026] Based on the short-term statistical features, the data event features, and the long-term trend features, the data distribution features in each time window in the high-dimensional time series data stream are obtained.

[0027] In one embodiment, the step of generating the reduced-dimensional low-dimensional data based on the data distribution characteristics and the adjusted random projection matrix of the high-dimensional time series data stream includes:

[0028] Generate an initial random projection matrix according to the high-dimensional time series data stream, and analyze the variance contribution of each dimensional data in the principal component direction from the data distribution characteristics to generate a feature importance vector;

[0029] Determining a sparsification weight of an initial random projection matrix based on the feature importance vector and a column vector correlation of the random projection matrix;

[0030] Performing structured sparse processing on the initial random projection matrix based on the sparsification weights to obtain an adjusted random projection matrix;

[0031] The high-dimensional time series data stream and the adjusted random projection matrix are multiplied to generate low-dimensional data after dimensionality reduction.

[0032] In one embodiment, the step of configuring the locality-sensitive hashing resolution parameter according to the local volatility of the low-dimensional data and constructing the multi-resolution hash table includes:

[0033] Analyze the local volatility of each data point in the low-dimensional data within a preset spatiotemporal neighborhood to obtain the fluctuation characteristic value;

[0034] Based on a preset partition threshold and the fluctuation characteristic value, obtaining data space partitions by partitioning the preset spatiotemporal neighborhood;

[0035] Configuring a corresponding locality-sensitive hashing resolution parameter for the data space partition, wherein the preset partition threshold is a value determined by clustering historical data;

[0036] A multi-resolution hash table is constructed according to the data space partition and the locality-sensitive hashing resolution parameter.

[0037] In addition, to achieve the above-mentioned purpose, the present application also proposes an abnormal data identification device, which includes:

[0038] Feature extraction module, used to extract data distribution features within each time window in high-dimensional time series data stream;

[0039] a data dimensionality reduction module, configured to generate reduced low-dimensional data based on the data distribution characteristics and an adjusted random projection matrix of the high-dimensional time series data stream, wherein the adjusted random projection matrix is a matrix obtained by jointly optimizing the data distribution characteristics and the initial random projection matrix of the high-dimensional time series data stream;

[0040] The parameter adjustment module is used to configure the local sensitive hashing resolution parameters according to the local volatility of low-dimensional data and construct a multi-resolution hash table;

[0041] The anomaly identification module is used to identify abnormal data points based on the data density in the bucket of the multi-resolution hash table.

[0042] In addition, to achieve the above-mentioned purpose, the present application also proposes an abnormal data identification device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the abnormal data identification method as described above.

[0043] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and stores a computer program on the storage medium. When the computer program is executed by the processor, the steps of the abnormal data identification method described above are implemented.

[0044] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps of the abnormal data identification method described above are implemented.

[0045] The technical solution proposed in this application extracts the data distribution characteristics within each time window in a high-dimensional time series data stream, generates low-dimensional data after dimensionality reduction based on the data distribution characteristics and the adjusted random projection matrix of the high-dimensional time series data stream, configures the local sensitive hash resolution parameters according to the local volatility of the low-dimensional data, constructs a multi-resolution hash table, and identifies abnormal data points based on the data density within the bucket of the multi-resolution hash table. This application adjusts the dimensionality reduction process by extracting data distribution characteristics, reducing computational complexity while retaining key information, constructs a multi-resolution hash table based on the local volatility of the dimensionality reduction data, realizes adaptive analysis of different data regions, and identifies abnormal data points based on the data density within the bucket. It can effectively address dimensionality problems and distribution change problems, and improves processing efficiency while ensuring detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0047] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0048] Figure 1 A flowchart of the first embodiment of the abnormal data identification method of this application is provided;

[0049] Figure 2 A flowchart of the second embodiment of the abnormal data identification method of this application is provided;

[0050] Figure 3 A flowchart of the third embodiment of the abnormal data identification method of this application is provided;

[0051] Figure 4 This is a schematic diagram of the module structure of the abnormal data identification device according to an embodiment of the present application;

[0052] Figure 5 Schematic diagram of the device structure of the hardware operating environment involved in the abnormal data identification method in the embodiment of the present application.

[0053] The purpose, features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0054] It should be understood that the specific embodiments described herein are merely used to explain the technical solutions of the present application and are not intended to limit the present application.

[0055] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0056] Existing anomaly detection technology has two major flaws: first, traditional dimensionality reduction methods have poor adaptability to dynamic data, resulting in the loss of key fault characteristics; second, fixed-threshold detection algorithms cannot cope with distribution drift caused by process switching, equipment aging, etc. These flaws lead to low accuracy in anomaly data identification.

[0057] Therefore, in order to overcome the above-mentioned defects, this application provides a solution, which adjusts the dimensionality reduction process by extracting data distribution characteristics, reduces computational complexity while retaining key information, constructs a multi-resolution hash table based on the local volatility of the dimensionality reduction data, realizes adaptive analysis of different data areas, and identifies abnormal data points according to the data density in the bucket, which can effectively deal with dimensionality problems and distribution change problems, and improves processing efficiency while ensuring detection accuracy.

[0058] It should be noted that the execution entity of each embodiment of this application can be a computing service system with data processing, network communication, and program execution functions, such as an electronic system or abnormal data identification system capable of implementing the above functions. The following uses the abnormal data identification system as an example (hereinafter referred to as the "system") to illustrate the following embodiments.

[0059] Based on this, the present invention provides a method for identifying abnormal data. Figure 1 , Figure 1 This is a flow chart of the first embodiment of the abnormal data identification method of the present application.

[0060] In this embodiment, the abnormal data identification method includes steps S10 to S40:

[0061] Step S10: extracting data distribution features in each time window in the high-dimensional time series data stream.

[0062] In the field of Industrial Internet of Things and intelligent monitoring, anomaly detection in high-dimensional time-series data streams faces three core challenges: First, high-dimensional data from multi-source sensors (such as 300+ dimensional process parameters in semiconductor equipment monitoring) suffers from significant dimensionality disasters, and traditional dimensionality reduction methods will cause the loss of key fault characteristics; second, data distribution has time-varying characteristics, and factors such as process switching and equipment aging cause statistical characteristics to continuously drift; third, anomaly patterns exhibit multi-scale characteristics, ranging from sudden anomalies at the millisecond level to gradual degradation lasting for months.

[0063] This application's abnormal data identification method innovatively integrates improved Random Projection (RP) and Locality Sensitive Hashing (LSH) technologies to construct an adaptive dynamic detection system. Compared to traditional technologies, this application dynamically adjusts the sparsification weights of the random projection matrix based on data distribution characteristics and intelligently assigns dimension importance based on principal component contributions. This improves the retention rate of key features compared to fixed sparsity patterns and effectively addresses the problem of key feature loss caused by the curse of dimensionality. An adaptive multi-resolution hashing mechanism based on the local volatility of dimensionality-reduced data automatically configures differentiated resolution parameters by quantifying the spatiotemporal neighborhood volatility index, replacing the fixed-resolution design of traditional LSH. This improves the F1 value (the harmonic mean of precision and recall) of mutation anomaly detection and addresses the need to detect multi-scale features of abnormal patterns. A spatiotemporal correlation graph model is introduced to achieve dynamic evolution of the density correction coefficient. By analyzing the changing patterns of data distribution similarity between windows, this approach overcomes the problem of decreased detection accuracy caused by the time-varying nature of data distribution.

[0064] It is understandable that this step first extracts features that reflect the data distribution characteristics from the high-dimensional time series data stream. High-dimensional time series data streams typically contain dynamically changing data in multiple dimensions (such as sensor monitoring, financial transactions, etc.). Directly processing high-dimensional data is computationally complex and difficult to capture effective information. Before extracting features, the raw data can be preprocessed. For example, the time series data can be segmented using a sliding window, or event windows can be dynamically divided using a mutation point detection algorithm (such as the Cumulative Sum (CUSUM) sequential analysis method or the Bayesian change point detection method) to accommodate the non-stationary nature of the data stream. High-dimensional time series data streams refer to continuous time series data composed of multiple monitoring dimensions (such as temperature, pressure, vibration, etc.). Time windows are a technical means of dividing an infinite data stream into finite analysis units.

[0065] For each time window, data distribution features can be extracted, including statistics (mean, variance, skewness), frequency domain features (such as the main frequency component extracted through Fourier transform or wavelet analysis), or time series model parameters (such as the coefficients of the Autoregressive Integrated Moving Average Model (ARIMA)). Furthermore, multi-scale analysis can be introduced, combining short-term statistical features (such as extreme values within the sliding window), waveform features of the event window (such as peak duration), and long-term trend features (such as HP (Hodrick-Prescott) filter decomposition) to comprehensively characterize the spatiotemporal characteristics of the data distribution. This step addresses the problem of non-robust feature representation in high-dimensional time series data due to the curse of dimensionality and noise interference.

[0066] Step S20, based on the data distribution characteristics and the adjusted random projection matrix of the high-dimensional time series data stream, generates low-dimensional data after dimensionality reduction, wherein the adjusted random projection matrix is a matrix obtained by jointly optimizing the data distribution characteristics and the initial random projection matrix of the high-dimensional time series data stream.

[0067] Traditional random projections map high-dimensional data to a low-dimensional space using a random matrix, but they fail to consider the distribution characteristics of the data itself, potentially leading to the loss of key feature information. The innovation of this step lies in optimizing the initial random projection matrix based on the data distribution characteristics.

[0068] In a specific implementation, an initial random projection matrix (either an initial random Gaussian matrix or a sparse matrix) is first generated, and then structural adjustments are made to the initial random projection matrix based on the data distribution characteristics. For example, the column vectors of the initial matrix can be weighted using feature importance vectors (calculated via variance contribution) to increase the projection weights of highly contributing dimensions, or redundant dimensions can be removed through sparsification (such as L1 regularization). The adjusted random projection matrix preserves the primary distribution patterns of the data, thereby better reflecting the discriminative features of the original data in the reduced low-dimensional data. This addresses the issue of decreased sensitivity to anomalies that occurs in traditional dimensionality reduction methods due to neglect of data distribution, while also improving computational efficiency through matrix optimization.

[0069] Step S30: configuring local sensitive hashing resolution parameters according to the local volatility of the low-dimensional data to construct a multi-resolution hash table.

[0070] Locality-sensitive hashing typically uses fixed-resolution parameters, making it difficult to adapt to the dynamic fluctuations of time series data. This step dynamically configures multi-resolution hash parameters by analyzing the local volatility of the reduced data (for example, calculating the standard deviation or entropy value within the neighborhood of a data point). Multi-resolution hash tables can be constructed using a family of parallel hash functions, each corresponding to a different resolution parameter, thereby accommodating data distributions of varying granularity within the same hash structure. This step addresses the issue of missed or false anomalies in fixed-resolution hashing due to uneven data density, while also improving the hash table's adaptability to dynamic data through multi-resolution design.

[0071] Step S40: identifying abnormal data points based on the data density in the bucket of the multi-resolution hash table.

[0072] It should be understood that this step achieves precise positioning of outliers through density comparison. First, the density of each hash bucket is counted (such as based on neighbor counting or kernel density estimation), and the density judgment threshold is dynamically adjusted based on the resolution parameter of the hash table. For example, the threshold of the high-resolution bucket is lower to capture sparse anomalies. Subsequently, the abnormal bucket is marked by comparing the bucket density with the threshold, and the data points in the bucket are verified twice, and then a list of outliers with comprehensive verification results is output. The density layered judgment through multi-resolution hashing in this step solves the problem of missed detection or false detection caused by the global density threshold in traditional methods, and is particularly suitable for the detection of local anomalies in high-dimensional time series data.

[0073] This embodiment extracts the data distribution characteristics within each time window in a high-dimensional time series data stream, generates low-dimensional data after dimensionality reduction based on the data distribution characteristics and the adjusted random projection matrix of the high-dimensional time series data stream, configures the local sensitive hash resolution parameters according to the local volatility of the low-dimensional data, constructs a multi-resolution hash table, and identifies abnormal data points based on the data density within the bucket of the multi-resolution hash table. This application adjusts the dimensionality reduction process by extracting data distribution characteristics, reducing computational complexity while retaining key information, constructs a multi-resolution hash table based on the local volatility of the dimensionality reduction data, implements adaptive analysis of different data regions, and identifies abnormal data points based on the data density within the bucket. It can effectively address dimensionality problems and distribution change problems, and improves processing efficiency while ensuring detection accuracy.

[0074] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above embodiment 1 can be referred to the above introduction and will not be described in detail later. Figure 2 , the step S40 may include steps S401 to S404:

[0075] Step S401: Determine the data density in each hash bucket in the multi-resolution hash table.

[0076] It's understandable that the core of this step is to quantify the degree of data aggregation within each hash bucket, providing a density benchmark for anomaly detection. Specifically, this can be achieved by using density estimation algorithms based on nearest neighbor statistics (such as kernel density estimation (KDE) or simple density calculations based on counts), combined with normalization based on the spatial volume of the hash buckets to mitigate bias caused by differences in bucket size at different resolutions. For dynamic time series data, a time decay factor (such as an exponentially weighted moving average) can be introduced to adjust the influence of historical data on the current density, thereby more accurately reflecting real-time changes in data distribution.

[0077] Step S402: adjusting the initial density determination threshold according to the resolution parameter of the multi-resolution hash table to obtain the target density determination threshold.

[0078] It's important to note that the initial density threshold can be trained using historical normal data (e.g., based on the 3σ principle or percentile statistics), while the adjustment process requires consideration of hash bucket resolution parameters (e.g., bucket width or hash function parameters). In practice, a resolution-threshold mapping model (e.g., a log-linear relationship or a machine learning-based regression model) can be established to automatically lower the threshold in high-resolution areas to capture subtle anomalies, while raising it in low-resolution areas to reduce false positives. A sliding window mechanism can also be introduced to dynamically update this mapping based on recent data stream characteristics.

[0079] As an implementation method, the above-mentioned step S402 in this embodiment may include: determining a basic density threshold according to the resolution parameter of the multi-resolution hash table; obtaining the aggregate distribution characteristics of multiple time windows in the high-dimensional time series data stream, and determining a density correction coefficient based on the aggregate distribution characteristics; multiplying the basic density threshold by the density correction coefficient to obtain a target density determination threshold.

[0080] Understandably, to determine the target density determination threshold, a quantitative relationship between hash resolution and density benchmark must first be established. Specifically, a resolution-parameter-driven threshold generation algorithm can be employed, where the base density threshold is directly linked to the geometric properties of the hash bucket. For high-resolution regions (smaller bucket volumes), a lower base threshold can be derived using a sparse data model based on a Poisson process. For low-resolution regions (larger bucket volumes), a higher threshold is determined by fitting a normal data distribution using a Gaussian mixture model. A resolution-threshold response curve (such as an exponential decay function) can be introduced to achieve adaptive parameter calibration by fitting the optimal detection thresholds at different resolution levels in historical data. This approach overcomes the limitations of traditional fixed thresholds and addresses the density incomparability caused by varying bucket volumes in multi-resolution environments.

[0081] Furthermore, contextual adaptation of the threshold is achieved through spatiotemporal feature fusion. During the feature extraction phase, a sliding window mechanism is used to aggregate the three-dimensional feature tensor (including statistical moments, principal component loading matrices, and temporal autocorrelation coefficients) of recent time windows. Feature evolution patterns are then modeled using a spatiotemporal attention network. Density correction coefficients can be generated by combining two mechanisms: 1) hard correction based on feature drift detection, which uses Kullback-Leibler Divergence (KLD) to calculate a sharp adjustment coefficient when a sudden change in the distribution is detected; and 2) soft correction based on neural differential equations, which fine-tunes the coefficients by modeling the gradual changes in features through a continuous-time recurrent neural network.

[0082] Finally, a composite calculation is used to precisely calibrate the detection threshold. This involves multiplying the base density threshold by the density correction factor to obtain the target density threshold. During the multiplication fusion process, logarithmic space operations can be introduced to maintain the stability of thresholds of varying magnitudes, and dynamic bounds can be added (for example, using a sigmoid function to constrain the correction factor to a preset interval of [0.5, 2.0]) to prevent over-adjustment. For time series data with both periodicity and trending, a seasonal factor derived from Fourier basis function decomposition can be added for periodic modulation.

[0083] As an implementation method, the steps of obtaining the aggregate distribution characteristics of multiple time windows in a high-dimensional time series data stream and determining the density correction coefficient based on the aggregate distribution characteristics in this embodiment may include: obtaining the aggregate distribution characteristics of multiple time windows in the high-dimensional time series data stream, the aggregate distribution characteristics including the statistical characteristic matrix and principal component evolution trajectory of each dimensional data; constructing a spatiotemporal association graph model based on the aggregate distribution characteristics, the spatiotemporal association graph model is a model that uses the data distribution characteristics of each time window as nodes and represents the distribution similarity between windows through edge weights; identifying the stable evolution stage and the sudden transition stage of the high-dimensional time series data stream through the spatiotemporal association graph model, and outputting a distribution evolution stage identifier; and determining the density correction coefficient based on the distribution evolution stage identifier.

[0084] It is understandable that the features of continuous time windows need to be aligned and fused first: for the statistical characteristic matrix, tensor splicing can be used to retain high-order statistics such as the mean and variance of each window, and moving average filtering can be used to eliminate instantaneous fluctuations; for the principal component evolution trajectory, it is necessary to align the principal component analysis (PCA) components of different windows through dynamic time warping (DTW), and then extract their time series features such as trajectory curvature and directional change rate.

[0085] When constructing a spatiotemporal correlation graph model based on aggregated distribution features, the data distribution features of each time window are converted into graph nodes, and the distribution similarity between windows is calculated as the edge weight. This process can be further combined with a temporal attention mechanism and a graph neural network to enable the model to adaptively capture the dependencies between different time windows. Specifically, a graph attention neural network can be used to calculate the dynamic similarity between nodes, and combined with time series embedding (such as Transformer encoding) to enhance the expressive power of temporal features. In addition, edge weight calculation can be optimized to multimodal similarity fusion, for example, combining statistical distance and shape similarity to more comprehensively characterize the spatiotemporal evolution of data distribution.

[0086] Next, the data stream evolution pattern is analyzed using a spatiotemporal correlation graph model, identifying phases of stable data evolution and transitional phases of sudden changes. The corresponding distribution evolution phase identifiers are then output. Finally, the density correction coefficient is dynamically adjusted based on these phase identifiers, with a higher coefficient for stable phases to reduce detection sensitivity and a lower coefficient for sudden changes to improve anomaly detection.

[0087] Step S403 : marking abnormal hash buckets by comparing the data density in the bucket with the target density determination threshold.

[0088] It's understandable that during the density-threshold comparison phase, a multi-level decision strategy can be employed: first, a hard threshold comparison is performed to mark buckets with obvious anomalies. Then, a soft decision based on statistical tests (such as the Student's t test or the Mann-Whitney U test) is performed on boundary buckets (those with density close to the threshold). For time series scenarios, time series consistency verification can be added (for example, requiring N consecutive windows to be marked before an anomaly is considered). The marking results can be used to form an anomaly probability heat map, where the anomaly probability of each bucket is determined by the degree to which its density deviates from the threshold. This dual filtering mechanism, combining statistical tests and time series verification, effectively distinguishes true anomalies from transient noise.

[0089] Step S404 : performing secondary verification on the data points in the abnormal hash bucket to obtain a verification result, and determining the abnormal data point by combining the verification result and the density distribution characteristics of adjacent hash buckets.

[0090] This step accurately locates outliers and eliminates false positives. Secondary verification employs two parallel techniques: 1) model-based verification, for example, using a pretrained autoencoder to calculate reconstruction error; and 2) density-based local verification, for example, comparing the density ratio of the target point to its nearest neighbor in adjacent buckets. The secondary verification results are then combined with the density gradient features of adjacent buckets (e.g., using graph neural networks to model inter-bucket density propagation), with particular attention paid to data points with high validation scores that lie at the boundary of density mutations. This effectively addresses the problem of boundary point misclassification caused by complex local density variations in high-dimensional data.

[0091] This embodiment ensures detection sensitivity by automatically adjusting the judgment threshold according to the resolution parameter of the hash table. The false alarm rate is reduced by secondary verification of the abnormal hash bucket and combining the density distribution characteristics of adjacent buckets. In addition, the basic density threshold based on the resolution parameter is combined with the correction coefficient of the aggregate distribution characteristics to achieve dynamic optimization of the abnormal judgment threshold. The distribution similarity between windows is quantified through the spatiotemporal correlation graph model, and the stable evolution stage and the sudden transition stage are identified. The density correction coefficient is dynamically adjusted, enabling the system to predict data change trends in the front stage and adjust the detection strategy in advance, significantly improving the response speed to sudden anomalies.

[0092] Based on the first embodiment of the present application, in the third embodiment of the present application, the same or similar contents as those in the first embodiment can be referred to the above introduction and will not be described in detail later. Figure 3 , the step S10 may include steps S101 to S105:

[0093] Step S101, perform multi-scale window segmentation processing on the input high-dimensional time series data stream to obtain window analysis results, which include at least one of a short-period window with a fixed length, an event window automatically divided according to a data mutation point, and a long-period window containing historical data.

[0094] As can be understood, this step employs a three-level window partitioning strategy to achieve multi-scale feature capture. Specifically, for short-term windows of fixed duration, equally spaced sliding windows are used to ensure basic analysis granularity. Event windows are dynamically generated through mutation point detection based on the CUSUM algorithm to ensure that key data change events are fully captured. Long-term windows are constructed using an exponentially weighted historical data backtracking mechanism to focus on capturing macro-trends. Furthermore, a window overlap compensation algorithm can be introduced to automatically adjust the partitioning boundaries when the event window overlaps with the fixed window, ensuring data integrity at each scale. For example, in industrial vibration monitoring, this method can simultaneously capture transient abnormal bearing vibrations (event window), minute-by-minute operating condition fluctuations (short-term window), and equipment aging trends (long-term window).

[0095] For example, taking the monitoring of wafer fab etching equipment as an example, the system intelligently selects the window combination strategy based on actual needs: in the conventional process monitoring mode, a short-period window with an interval of 1 second is used to capture the instantaneous fluctuation of plasma, an event window triggered by mutations is used to fully record the abnormal discharge process, and a long-period window of 200 process cycles is used to track electrode loss; in the equipment health assessment mode, only the long-period window is used to analyze the aging trend; and in the process debugging stage, the short-period window and the event window are combined to capture real-time anomalies.

[0096] Step S102 : generating short-term statistical features by determining the statistics of data in each dimension within each short-term window.

[0097] For each short-period window, multidimensional statistics are calculated, including those in the time domain (mean, variance, and kurtosis) and the frequency domain (FFT dominant frequency amplitude and spectral entropy). Specifically, to understand the inter-dimensional correlation characteristics of high-dimensional data, traditional statistics are supplemented by calculating the mutual information matrix for pairs of dimensions within the window. For example, in chemical process monitoring, not only is the mean and variance of each sensor reading recorded, but the changes in mutual information for key dimension pairs such as temperature and pressure are also calculated, forming a 20-dimensional short-period feature vector.

[0098] Step S103: Generate data event features by performing waveform analysis on each event window.

[0099] The feature extraction of the event window focuses on the waveform morphological characteristics: first, the data in the window is normalized, and then morphological indicators such as the number of waveform extreme points, zero-crossing rate, and approximate entropy are extracted. The data event characteristics at different scales are obtained through continuous wavelet transform.

[0100] Step S104: smoothing and trend decomposition are performed on the data in the long-period window to generate long-term trend features.

[0101] STL (Seasonal-Trend decomposition using Loess), a technique used for time series decomposition, is used to decompose long-term window data, separating the trend term, period term, and residual term. The innovation lies in constructing derivative features of the trend term, quantifying the rate and acceleration of trend change by calculating the first- and second-order derivatives at each point on the trend line. For example, in server cluster temperature monitoring, not only is the 24-hour average temperature trend extracted, but more importantly, the acceleration of trend change is calculated. This allows early warning signals of an abnormal increase in trend curvature to be detected three days before equipment cooling system failure.

[0102] Step S105 , based on the short-term statistical features, the data event features, and the long-term trend features, obtain the data distribution features in each time window in the high-dimensional time series data stream.

[0103] Multi-scale feature fusion is achieved through feature concatenation and attention weighting, and short-term statistical features (such as 20 dimensions), event features and long-term trend features are spliced into data distribution features in each time window in the high-dimensional time series data stream. The final output distribution features can include original statistics, cross-scale correlation indicators (such as the covariance of short-term variance and long-term trend slope) and attention weight values.

[0104] As an implementation method, step S20 may include: generating an initial random projection matrix based on the high-dimensional time series data stream, and analyzing the variance contribution of each dimensional data in the principal component direction from the data distribution characteristics to generate a feature importance vector; determining the sparsification weight of the initial random projection matrix based on the feature importance vector and the column vector correlation of the random projection matrix; performing structured sparse processing on the initial random projection matrix based on the sparsification weight to obtain an adjusted random projection matrix; performing a product operation on the high-dimensional time series data stream and the adjusted random projection matrix to generate low-dimensional data after dimensionality reduction.

[0105] It is understood that the initial random projection matrix is generated using a Gaussian random distribution (dimension d×k, where d is the original dimension and k is the target dimension), which meets the theoretical guarantee of the Johnson-Lindenstrauss Lemma. Furthermore, principal component analysis results are extracted from the data distribution characteristics, and the sum of squared loads of each original dimension on the top-k principal components (i.e., the top k most important principal components) is calculated as the variance contribution to form the feature importance vector. Furthermore, incremental PCA can be used to process streaming data, dynamically updating the principal component directions. For nonlinear feature relationships, kernel PCA or latent space feature importance extracted by autoencoders can be introduced. The correlation between the feature importance vector and the column vectors of the random projection matrix can be calculated by calculating the cosine similarity between the column vectors of the initial matrix and the feature importance vector, thereby generating the sparsification weights of the initial random projection matrix. Among them, the complete projection relationship is retained for high-importance dimensions (such as cosine similarity > 0.7), probabilistic sparseness is performed on medium-importance dimensions (such as cosine similarity between 0.3-0.7), and low-importance dimensions (such as cosine similarity < 0.3) are forced to zero.

[0106] The structured sparsity processing and dimensionality reduction calculation process uses an improved approximate Gram-Schmidt orthogonalization process. While maintaining the matrix's approximate orthogonality, the matrix elements are zeroed according to the sparsification weights, preserving the cross-projection relationships between important dimensions (such as the joint projection of temperature and pressure sensors). This results in an adjusted random projection matrix. The high-dimensional time series data stream is then multiplied by the adjusted random projection matrix to generate the reduced low-dimensional data.

[0107] As an implementation method, step S30 may include: analyzing the local volatility of each data point in the low-dimensional data within a preset spatiotemporal neighborhood to obtain a fluctuation characteristic value; obtaining a data space partition by partitioning the preset spatiotemporal neighborhood based on a preset partition threshold and the fluctuation characteristic value; configuring corresponding local sensitive hash resolution parameters for the data space partition, wherein the preset partition threshold is a value determined by clustering historical data; and constructing a multi-resolution hash table based on the data space partition and the local sensitive hash resolution parameter.

[0108] It's understandable that the core of adaptive multi-resolution hashing by analyzing the local fluctuation characteristics of data lies in establishing an intelligent mapping relationship between the dynamic characteristics of the data and the hash resolution. Specifically, for each data point in the low-dimensional data, a local fluctuation characteristic value is calculated within its spatiotemporal neighborhood (e.g., the set of points within the three preceding and following time points and the Euclidean radius r). This value is a weighted combination of standard deviation, approximate entropy, and trend slope, and can quantify the stability of the data in that area. Based on the fluctuation threshold obtained by historical data clustering (such as the density-based spatial clustering of applications with noise (DBSCAN) algorithm), the data space is divided into three types of partitions (high volatility zone, transition zone, and stable zone): the first partition corresponds to areas with frequent data mutations (such as periods of industrial equipment failure), and the third partition corresponds to normal operation.

[0109] The locality-sensitive hashing resolution parameters are configured for each partition: the first partition (high volatility area) uses a fine resolution (small hash bucket width) to capture subtle anomalies, the third partition (stable area) uses a coarse resolution (large bucket width) to improve computational efficiency, and the second partition (transition area) uses an intermediate value. The resulting multi-resolution hash table uses a hierarchical storage structure, and spatial indexing allows for rapid location of different resolution regions.

[0110] For ease of understanding, the following examples are given, but they are not intended to limit the abnormal data identification method of this application. In the smart park monitoring scenario of semiconductor manufacturing equipment, the abnormal data identification method of this application can effectively solve the problem of equipment status monitoring in the chip production process. Taking the wafer etching machine as an example, when the equipment is running, it will generate sensor data streams containing hundreds of dimensions such as temperature, air pressure, and RF power. These data contain both periodic process parameter fluctuations and sudden transient mutations caused by equipment failure or process abnormalities.

[0111] The system first performs multi-scale window segmentation on the original data stream: a short-period window of 5 seconds is used to capture the instantaneous fluctuations of the plasma state, and the process abnormality event window is dynamically divided through Bayesian change point detection. At the same time, a long window containing 200 production cycles is set to analyze the aging trend of equipment.

[0112] During the feature extraction phase, a short-period window calculates the time-frequency characteristics of each sensor reading (such as the spectral entropy of RF power). An event window analyzes the peak duration and rising slope of abnormal discharge waveforms. A long-period window extracts the device performance degradation curve through STL decomposition. These features are then reduced in dimension using a random projection matrix optimized based on principal component contributions. The 50-dimensional low-dimensional data still retains most of the critical information. By analyzing the local fluctuations of the plasma state within the etching chamber, the system divides the data space into a stable process region (resolution parameter 0.1), a transition region (resolution parameter 0.05), and an abnormal discharge region (resolution parameter 0.01), and constructs a corresponding multi-resolution hash table.

[0113] In actual testing, when an etcher experienced electrode aging, the system first detected the slow rise in electrode impedance within a long-term trend, lowering the trigger density correction factor to 0.8 to improve detection sensitivity. Subsequently, abnormal RF power fluctuations were detected within a short-term window. A multi-resolution hash table marked the abnormal hash bucket at a fine resolution level (0.01). After verifying the autoencoder reconstruction error and analyzing the plasma density gradient of adjacent buckets, the fault was ultimately located as poor electrode contact in reaction chamber 3. Compared to traditional fixed-threshold detection, this method, in field testing at a TSMC smart park, provided earlier warning of equipment failures and reduced false alarm rates.

[0114] This embodiment adopts the method of multi-scale window segmentation and multi-dimensional feature fusion to fully capture the distribution characteristics of high-dimensional time series data. The sparsification weight adjustment of the random projection matrix is driven by the data distribution characteristics, thereby realizing adaptive optimization of the dimensionality reduction process. The sparsity pattern is dynamically adjusted according to the contribution of each dimension in the direction of the principal component, so that the low-dimensional data after dimensionality reduction can retain key feature information to the greatest extent while significantly reducing the computational complexity. The multi-resolution hash table construction method based on local volatility balances detection accuracy and computational efficiency. By quantifying the local fluctuation characteristics of each data point, the data space is divided into different regions and differentiated resolution parameters are configured, so that high-fluctuation areas obtain fine detection capabilities and low-fluctuation areas maintain efficient screening, thereby improving the real-time performance and accuracy of the system as a whole.

[0115] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the abnormal data identification method of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0116] This application also provides an abnormal data identification device, please refer to Figure 4 , the abnormal data identification device includes:

[0117] Feature extraction module 10, used to extract data distribution features in each time window in the high-dimensional time series data stream;

[0118] A data dimensionality reduction module 20 is configured to generate reduced low-dimensional data based on the data distribution characteristics and the adjusted random projection matrix of the high-dimensional time series data stream, wherein the adjusted random projection matrix is a matrix obtained by jointly optimizing the data distribution characteristics and the initial random projection matrix of the high-dimensional time series data stream;

[0119] A parameter adjustment module 30 is used to configure local sensitive hashing resolution parameters according to the local volatility of low-dimensional data and construct a multi-resolution hash table;

[0120] The anomaly identification module 40 is used to identify abnormal data points based on the data density in the bucket of the multi-resolution hash table.

[0121] The abnormal data identification device provided in this application, which utilizes the abnormal data identification method described in the aforementioned embodiments, can address the technical problem of low anomaly identification accuracy in the prior art due to high data dimensionality and dynamically changing data distribution. Compared to the prior art, the beneficial effects of the abnormal data identification device provided in this application are the same as those of the abnormal data identification method described in the aforementioned embodiments. Other technical features of the abnormal data identification device are the same as those disclosed in the aforementioned embodiments and are not further elaborated here.

[0122] The present application provides an abnormal data identification device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the abnormal data identification method in the above-mentioned embodiment 1.

[0123] Reference below Figure 5 , which shows a schematic diagram of the structure of an abnormal data identification device suitable for implementing the embodiments of the present application. The abnormal data identification device in the embodiments of the present application may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Devices), PMPs (Portable Media Players), in-vehicle terminals (such as in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 5 The abnormal data identification device shown is merely an example and should not limit the functions and scope of use of the embodiments of the present application.

[0124] like Figure 5As shown, the abnormal data identification device may include a processing device 1001 (e.g., a central processing unit, graphics processing unit, etc.), which can perform various appropriate actions and processes based on programs stored in a read-only memory 1002 or programs loaded from a storage device 1003 into a random access memory 1004. The random access memory 1004 also stores various programs and data required for the operation of the abnormal data identification device. The processing device 1001, the read-only memory 1002, and the random access memory 1004 are interconnected via a bus 1005. An input / output interface 1006 is also connected to the bus. Typically, the following systems can be connected to the input / output interface 1006: an input device 1007 including, for example, a touchscreen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; a storage device 1003 including, for example, a magnetic tape or hard disk; and a communication device 1009. The communication device 1009 can allow the abnormal data identification device to communicate wirelessly or wired with other devices to exchange data. Although the figure shows an abnormal data identification device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or provided instead.

[0125] In particular, according to the embodiments disclosed in the present application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed in the present application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program comprising program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via a communication device, or installed from a storage device 1003, or installed from a read-only memory 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are performed.

[0126] The abnormal data identification device provided in this application, which utilizes the abnormal data identification method described in the above-mentioned embodiments, can address the technical problem of low anomaly identification accuracy in the prior art due to high data dimensionality and dynamically changing data distribution. Compared to the prior art, the beneficial effects of the abnormal data identification device provided in this application are the same as those of the abnormal data identification method described in the above-mentioned embodiments. Other technical features of this abnormal data identification device are the same as those disclosed in the above-mentioned embodiments and are not further elaborated here.

[0127] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0128] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

[0129] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, a computer program) stored thereon, wherein the computer-readable program instructions are used to execute the abnormal data identification method in the above-mentioned embodiment.

[0130] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0131] The computer-readable storage medium may be included in the abnormal data identification device; or may exist independently without being assembled into the abnormal data identification device.

[0132] The above-mentioned computer-readable storage medium carries one or more programs. When the above-mentioned one or more programs are executed by the abnormal data identification device, the abnormal data identification device is enabled to: extract the data distribution characteristics in each time window in the high-dimensional time series data stream, generate low-dimensional data after dimensionality reduction based on the data distribution characteristics and the adjusted random projection matrix of the high-dimensional time series data stream, configure the local sensitive hash resolution parameters according to the local volatility of the low-dimensional data, construct a multi-resolution hash table, and identify abnormal data points according to the data density in the bucket of the multi-resolution hash table.

[0133] Computer program code for performing the operations of the present application may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, and C++, as well as conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, entirely on a remote computer or server, or on an ARM (Advanced RISC Machines) development board. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0134] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, program segment or a part of code, and the module, program segment or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the specified function or operation, or can be implemented by a combination of dedicated hardware and computer instructions.

[0135] The modules described in the embodiments of the present application may be implemented in software or hardware, wherein the name of a module does not necessarily limit the unit itself.

[0136] The computer-readable storage medium provided in this application stores computer-readable program instructions (i.e., a computer program) for executing the above-described abnormal data identification method. This computer-readable storage medium can address the technical problem of low anomaly identification accuracy caused by high data dimensionality and dynamically changing data distribution in the prior art. Compared to the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the abnormal data identification method provided in the above-described embodiments, and are not further elaborated here.

[0137] The present application also provides a computer program product, comprising a computer program, which implements the steps of the above-mentioned abnormal data identification method when executed by a processor.

[0138] The computer program product provided in this application can solve the technical problem of low anomaly identification accuracy caused by high data dimensionality and dynamic data distribution in the prior art. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the abnormal data identification method provided in the above embodiment, and will not be elaborated here.

[0139] The above descriptions are only some embodiments of the present application and do not limit the patent scope of the present application. All equivalent structural transformations made by using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect application in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A method for identifying abnormal data, characterized in that: The method comprises the following steps: Extracting data distribution features within each time window in a high-dimensional time series data stream, wherein the high-dimensional time series data stream is a multi-dimensional process parameter obtained by multi-source sensors; Generate low-dimensional data after dimensionality reduction based on the data distribution characteristics and the adjusted random projection matrix of the high-dimensional time series data stream, wherein the adjusted random projection matrix is a matrix obtained by jointly optimizing the data distribution characteristics and the initial random projection matrix of the high-dimensional time series data stream; Configure the local sensitive hashing resolution parameters according to the local volatility of low-dimensional data and construct a multi-resolution hash table; Identify abnormal data points of semiconductor manufacturing equipment based on the data density within the bucket of a multi-resolution hash table; The step of identifying abnormal data points of semiconductor manufacturing equipment based on the data density in the bucket of the multi-resolution hash table includes: Determine the data density within each hash bucket of the multi-resolution hash table; Adjusting the initial density determination threshold according to the resolution parameter of the multi-resolution hash table to obtain the target density determination threshold; Marking abnormal hash buckets by comparing the data density in the bucket with the target density determination threshold; A secondary verification is performed on the data points in the abnormal hash bucket to obtain a verification result, and the abnormal data points of the semiconductor manufacturing equipment are determined in combination with the verification result and the density distribution characteristics of adjacent hash buckets.

2. The abnormal data identification method according to claim 1, characterized in that: The step of adjusting the initial density determination threshold according to the resolution parameter of the multi-resolution hash table to obtain the target density determination threshold includes: Determining a base density threshold according to a resolution parameter of a multi-resolution hash table; Obtaining aggregate distribution features of multiple time windows in a high-dimensional time series data stream, and determining a density correction coefficient based on the aggregate distribution features; The basic density threshold is multiplied by the density correction coefficient to obtain a target density determination threshold.

3. The abnormal data identification method according to claim 2, characterized in that: The step of obtaining aggregate distribution features of multiple time windows in the high-dimensional time series data stream and determining a density correction coefficient based on the aggregate distribution features includes: Obtaining aggregate distribution features of multiple time windows in a high-dimensional time series data stream, wherein the aggregate distribution features include the statistical characteristic matrix and principal component evolution trajectory of each dimensional data; A spatiotemporal association graph model is constructed based on the aggregated distribution features. The spatiotemporal association graph model uses the data distribution features of each time window as nodes and represents the distribution similarity between windows through edge weights. The spatiotemporal association graph model is used to identify the stable evolution stage and the sudden transition stage of the high-dimensional time series data stream, and output the distribution evolution stage identifier. The density correction coefficient is determined according to the distribution evolution stage identifier.

4. The abnormal data identification method according to any one of claims 1 to 3, characterized in that: The step of extracting data distribution features within each time window in the high-dimensional time series data stream includes: Perform multi-scale window segmentation processing on the input high-dimensional time series data stream to obtain window analysis results, which include at least one of a short-term window with a fixed length, an event window automatically divided according to data mutation points, and a long-term window containing historical data; By determining the statistics of each dimension of data within each short-period window, short-period statistical features are generated; Generate data event features by performing waveform analysis on each event window; Smoothing and trend decomposition of data in long-period windows to generate long-term trend characteristics; Based on the short-term statistical features, the data event features, and the long-term trend features, the data distribution features in each time window in the high-dimensional time series data stream are obtained.

5. The abnormal data identification method according to any one of claims 1 to 3, characterized in that: The step of generating the low-dimensional data after dimensionality reduction based on the data distribution characteristics and the adjusted random projection matrix of the high-dimensional time series data stream includes: Generate an initial random projection matrix according to the high-dimensional time series data stream, and analyze the variance contribution of each dimensional data in the principal component direction from the data distribution characteristics to generate a feature importance vector; Determining a sparsification weight of an initial random projection matrix based on the feature importance vector and a column vector correlation of the random projection matrix; Performing structured sparse processing on the initial random projection matrix based on the sparsification weights to obtain an adjusted random projection matrix; The high-dimensional time series data stream and the adjusted random projection matrix are multiplied to generate low-dimensional data after dimensionality reduction.

6. The abnormal data identification method according to any one of claims 1 to 3, characterized in that: The step of configuring the local sensitive hashing resolution parameter according to the local volatility of the low-dimensional data and constructing a multi-resolution hash table includes: Analyze the local volatility of each data point in the low-dimensional data within a preset spatiotemporal neighborhood to obtain the fluctuation characteristic value; Based on a preset partition threshold and the fluctuation characteristic value, obtaining data space partitions by partitioning the preset spatiotemporal neighborhood; Configuring a corresponding locality-sensitive hashing resolution parameter for the data space partition, wherein the preset partition threshold is a value determined by clustering historical data; A multi-resolution hash table is constructed according to the data space partition and the locality-sensitive hashing resolution parameter.

7. An abnormal data identification device, characterized in that: The abnormal data identification device includes: A feature extraction module is used to extract data distribution features within each time window in a high-dimensional time series data stream, wherein the high-dimensional time series data stream is a multi-dimensional process parameter obtained by a semiconductor device; a data dimensionality reduction module, configured to generate reduced low-dimensional data based on the data distribution characteristics and an adjusted random projection matrix of the high-dimensional time series data stream, wherein the adjusted random projection matrix is a matrix obtained by jointly optimizing the data distribution characteristics and the initial random projection matrix of the high-dimensional time series data stream; The parameter adjustment module is used to configure the local sensitive hashing resolution parameters according to the local volatility of low-dimensional data and construct a multi-resolution hash table; An anomaly identification module, used to identify abnormal data points of semiconductor manufacturing equipment based on the data density in the buckets of the multi-resolution hash table; The anomaly identification module is further configured to determine the data density within each hash bucket in the multi-resolution hash table; adjust an initial density determination threshold according to a resolution parameter of the multi-resolution hash table to obtain a target density determination threshold; mark an abnormal hash bucket by comparing the data density within the bucket with the target density determination threshold; perform a secondary verification on the data points within the abnormal hash bucket to obtain a verification result, and determine abnormal data points of semiconductor manufacturing equipment in combination with the verification result and density distribution characteristics of adjacent hash buckets.

8. An abnormal data identification device, characterized in that: The abnormal data identification device includes: a memory, a processor, and an abnormal data identification program stored in the memory and executable on the processor. When the abnormal data identification program is executed by the processor, the abnormal data identification method according to any one of claims 1 to 6 is implemented.

9. A storage medium, characterized in that: The storage medium stores an abnormal data identification program, which, when executed by a processor, implements the abnormal data identification method according to any one of claims 1 to 6.

Images