A method and system for information processing based on multi-dimensional features

Abstract

The application relates to the technical field of data processing, and discloses an information processing method and system based on multidimensional features. The method comprises the following steps: collecting system logs, CPU and memory usage rate sequences and alarm event data, and extracting an initial feature set; removing low-variance redundant features based on feature correlation analysis to form an optimized feature set; performing PCA dimension reduction on the optimized set to construct a compression space, and obtaining an independent subspace through ICA iterative optimization after whitening transformation; matching the independent subspace with a historical fault case library, combining evolution features to determine a strongly correlated subspace in terms of Euclidean distance; extracting a causal propagation chain and embedding a description template to generate a fault semantic expression; constructing a fault propagation weighted topology graph based on the strongly correlated subspace, fusing the semantic expression to perform reverse BFS, and generating a structured monitoring report. The method can improve the accuracy of server anomaly detection.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to an information processing method and system based on multi-dimensional features. Background Technology

[0002] Against the backdrop of accelerated enterprise digital transformation, the amount of heterogeneous data generated by servers, including logs, performance metrics, alarms, and sensor readings, is exploding. This data serves as the core basis for ensuring stable system operation, and its value extraction heavily relies on efficient big data analytics technologies. However, current big data analytics practices face bottlenecks when handling complex scenarios. Significant differences in semantic expression between different data sources, coupled with redundant feature dimensions and nonlinear interactions, lead to low accuracy in identifying root causes of failures. Therefore, there is an urgent need to overcome the limitations of existing big data analytics frameworks and develop an innovative technology system capable of adaptively separating redundant interference, accurately aligning multimodal semantics, and generating highly reliable root cause reports.

[0003] In one existing technique, multi-source data such as server logs, performance metrics, and alarms are first collected. Logs are then segmented and vectorized using TF-IDF. Statistical measures such as mean and variance are extracted from metrics, and alarm events are encoded as discrete sequences. All features are concatenated into a high-dimensional fusion vector, which is then input into a random forest model for fault classification training. Finally, the fault type is identified based on the model's output. This method directly concatenates heterogeneous features, failing to address semantic differences and redundant information, leading to the curse of dimensionality and noise interference.

[0004] Therefore, existing technologies cannot improve the accuracy of server anomaly detection. Summary of the Invention

[0005] This invention provides an information processing method and system based on multi-dimensional features to improve the accuracy of server anomaly detection.

[0006] Firstly, in order to solve the above-mentioned technical problems, the present invention provides an information processing method based on multi-dimensional features, comprising: Acquire log text data, processor utilization sequence, memory utilization sequence, and alarm event data; Based on log text data, processor usage sequence, memory usage sequence, and alarm event data, data features are extracted to obtain an initial feature set; Based on the initial feature set, redundant feature pairs are identified through feature correlation analysis, and features with small variances in the redundant feature pairs are removed to obtain an optimized feature set. Principal component analysis is performed on the optimized feature set to obtain a principal component feature set, and an orthogonal space is constructed based on the principal component feature set to obtain a compressed feature space; The compressed feature space is subjected to whitening transformation to obtain whitening features, and rapid independent component analysis is performed on the whitening features for iterative optimization to obtain independent subspaces; The fault patterns are matched with the pre-stored historical fault case library based on the independent subspaces, and the Euclidean distance threshold is determined based on the fault patterns and the evolution characteristics of the independent subspaces to obtain strongly correlated subspaces. The causal propagation chain is extracted from the strongly correlated subspace to obtain a propagation path set, and a description template is embedded based on the propagation path set to obtain a fault semantic expression; Fault propagation paths are constructed based on the strongly correlated subspace, a weighted topology graph is generated, and a reverse breadth-first search is performed based on the weighted topology graph and the fault semantic expression to generate a monitoring report.

[0007] Secondly, the present invention provides an information processing system based on multi-dimensional features, comprising: The data acquisition module is used to acquire log text data, processor utilization sequence, memory utilization sequence, and alarm event data; The feature extraction module is used to extract data features based on log text data, processor usage sequence, memory usage sequence and alarm event data to obtain an initial feature set; The feature optimization module is used to determine redundant feature pairs based on the initial feature set through feature correlation analysis, and to remove features with small feature variance from the redundant feature pairs to obtain an optimized feature set. The compressed feature module is used to perform principal component analysis on the optimized feature set to obtain a principal component feature set, and to construct a compressed feature space based on the principal component feature set using an orthogonal space. An independent module is used to perform whitening transformation on the compressed feature space to obtain whitening features, and to perform fast independent component analysis and iterative optimization based on the whitening features to obtain an independent subspace; The strongly correlated module is used to match fault patterns with the pre-stored historical fault case library based on the independent subspace, and to determine the Euclidean distance threshold based on the fault patterns and the evolution characteristics of the independent subspace to obtain the strongly correlated subspace. The fault semantic module is used to extract the causal propagation chain from the strongly correlated subspace to obtain a propagation path set, and to embed a description template based on the propagation path set to obtain a fault semantic expression. The output module is used to construct a fault propagation path based on the strongly correlated subspace, generate a weighted topology graph, and perform a reverse breadth-first search based on the weighted topology graph and the fault semantic expression to generate a monitoring report.

[0008] Compared with the prior art, the present invention has the following beneficial effects: (1) This invention eliminates low-variance redundant features through feature correlation analysis to form an optimized feature set, effectively solving the problems of semantic differences and feature redundancy in multi-source heterogeneous data, avoiding the curse of dimensionality and noise interference, and providing high-quality input for subsequent feature extraction and fault analysis.

[0009] (2) This invention constructs a compressed space by performing PCA dimensionality reduction on the optimized feature set, and obtains an independent subspace by ICA iterative optimization after whitening transformation. This effectively decouples multi-source features and separates an independent semantic subspace that is strongly related to the fault, so that the key fault modes can be clearly separated, thereby improving the accuracy and interpretability of fault detection.

[0010] (3) This invention matches independent subspaces with historical fault case libraries, combines evolutionary features to determine strongly correlated subspaces by Euclidean distance, extracts causal propagation chains and embeds description templates to generate fault semantic expressions, constructs a fault propagation weighted topology graph based on strongly correlated subspaces, integrates semantic expressions to perform reverse BFS, generates structured monitoring reports, realizes accurate positioning of fault root causes from raw data and generates interpretable anomaly monitoring reports, and improves the accuracy of server anomaly detection. Attached Figure Description

[0011] Figure 1 This is a schematic flowchart of the information processing method based on multi-dimensional features provided in the first embodiment of the present invention; Figure 2 This is a schematic diagram of the structure of an information processing system based on multi-dimensional features provided in the second embodiment of the present invention. Detailed Implementation

[0012] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0013] Reference Figure 1 The first embodiment of the present invention provides an information processing method based on multi-dimensional features, comprising the following steps: S11, obtain log text data, processor utilization sequence, memory utilization sequence and alarm event data; S12, based on log text data, processor utilization sequence, memory utilization sequence and alarm event data, perform data feature extraction to obtain an initial feature set; S13, Based on the initial feature set, redundant feature pairs are determined through feature correlation analysis, and features with small feature variances in the redundant feature pairs are removed to obtain an optimized feature set; S14, perform principal component analysis on the optimized feature set to obtain a principal component feature set, and construct an orthogonal space based on the principal component feature set to obtain a compressed feature space; S15, perform whitening transformation on the compressed feature space to obtain whitening features, and perform fast independent component analysis iterative optimization based on the whitening features to obtain independent subspaces; S16, Match fault patterns with the pre-stored historical fault case library according to the independent subspace, and determine the Euclidean distance threshold according to the fault patterns and the evolution characteristics of the independent subspace to obtain a strongly correlated subspace; S17, extract the causal propagation chain from the strongly correlated subspace to obtain a propagation path set, and embed a description template based on the propagation path set to obtain a fault semantic expression; S18. Construct a fault propagation path based on the strongly correlated subspace, generate a weighted topology graph, and perform a reverse breadth-first search based on the weighted topology graph and the fault semantic expression to generate a monitoring report.

[0014] In step S11, log text data, processor usage sequence, memory usage sequence, and alarm event data are obtained.

[0015] Specifically, step S11 acquires log text data, processor usage sequence, memory usage sequence, and alarm event data. This step provides raw data input for the entire information processing flow, ensuring that subsequent feature extraction and fault analysis are based on complete, timely, and multi-source heterogeneous data.

[0016] The log text data comes from the server system log files, including system operation logs, application logs, and security logs. Each log entry contains a timestamp, log level, source module, and log content field. The text content is extracted periodically in a 5-minute time window. During extraction, the timestamp is retained and the text is cleaned to remove irrelevant characters.

[0017] The processor utilization sequence is generated by collecting CPU utilization values ​​every 5 minutes via the system API, forming a continuous time series. Each time point in the sequence corresponds to an integer between 0 and 100 representing the CPU utilization percentage. The memory utilization sequence is obtained in the same way as the processor utilization sequence, collecting memory utilization values ​​every 5 minutes via the system API, forming a time series aligned with the processor utilization sequence. Each time point corresponds to an integer between 0 and 100 representing the memory utilization percentage. Alarm event data comes from the alarm system, extracting alarm events every 5 minutes via the alarm system API. Each alarm includes a timestamp, alarm level, alarm type, and alarm description fields. Alarm types include system anomalies, network failures, storage warnings, etc., and the frequency of each type of alarm is statistically analyzed by time window. The 5-minute time window size was determined by analyzing historical monitoring data. Based on 95% of the historical data collection frequency distribution, the 5-minute time window is set to balance data timeliness and computational efficiency.

[0018] This step ensures the alignment of multi-source data in the time dimension, providing a unified time benchmark for subsequent feature extraction, avoiding feature matching errors caused by time asynchrony, enabling the system to perform fault analysis based on the latest and most complete data, and laying a data foundation for improving the accuracy of server anomaly detection.

[0019] In step S12, based on log text data, processor usage sequence, memory usage sequence, and alarm event data, data feature extraction is performed to obtain an initial feature set, including: Based on the log text data, context semantic encoding is performed using a bidirectional encoder to generate a high-dimensional dense vector and bind it with a time window identifier to obtain text features; The performance statistical characteristics are obtained by taking the arithmetic mean and standard deviation of the corresponding values ​​of the processor utilization sequence and the memory utilization sequence in each time window; Based on the processor usage sequence and the memory usage sequence, the linear slope of the numerical change within each time window is calculated using a linear regression fitting method to obtain the performance trend characteristics; The alarm type frequency is obtained by statistically analyzing the cumulative occurrence count of each alarm type within each time window of the alarm event data. The number of times different alarm types co-occur within each time window is counted to obtain the alarm co-occurrence count. A co-occurrence relationship graph is constructed with alarm type as nodes and alarm co-occurrence count as edge weights. The edge weights are extracted as the association strength to obtain the co-occurrence feature. The text features, performance statistics features, performance trend features, alarm type frequency, and co-occurrence features are combined into a fixed-length feature vector to obtain an initial feature set.

[0020] Specifically, step S12 extracts features based on log text data, processor usage sequences, memory usage sequences, and alarm event data. The log text data originates from server system log files, including system operation logs, application logs, and security logs. Each log entry contains a timestamp, log level, source module, and log content fields. A pre-built bidirectional encoding model is used to process the log text data. This model is trained on server log text data from the past year, with a data source of 1 million entries from system operation logs, application logs, and security logs (actual data volume is based on historical monitoring periods). The vocabulary size is determined through historical word frequency statistics, selecting words with a cumulative frequency of 95%, resulting in a vocabulary size of 10,000. The model structure includes an input layer, an embedding layer (embedding dimension 768), 12 layers of bidirectional self-attention encoding layers (768 neurons per layer), and an output layer. Training uses the Adam optimizer with a learning rate of 0.00005, a batch size of 32, and 10 training epochs. After inputting log text data, the model processes it through word segmentation and embedding layers, generates high-dimensional dense vectors through 12 layers of bidirectional self-attention encoding layers, binds them with time window identifiers, and outputs text features.

[0021] For a processor utilization sequence, calculate the average processor utilization within each time window, which is the sum of all values ​​in the sequence divided by the sequence length. Calculate the standard deviation of processor utilization, which is the sum of the squares of the differences between each value in the sequence and the average processor utilization, divided by the square root of the sequence length minus one. Similarly, process the memory utilization sequence, calculating the average memory utilization and its standard deviation. The average processor utilization equals the sum of the values ​​in the sequence divided by the sequence length; the average memory utilization is calculated in the same way. The standard deviation of processor utilization is calculated as the square root of the sum of the squares of the differences between each value in the sequence and the average processor utilization, divided by the sequence length minus one; the calculation process for the standard deviation of memory utilization is the same.

[0022] The slope of the processor utilization trend is calculated based on the processor utilization sequence using a linear regression fitting method. The slope calculation is based on the sum of the products of the sample size and utilization rate at each time point, the sum of the time points, the sum of the utilization rates, and the sum of squares at each time point. Specifically, it is calculated by multiplying the sample size by the sum of the products of the time points and utilization rates, subtracting the sum of the time points multiplied by the sum of the utilization rates, and then dividing by the sample size multiplied by the sum of squares at each time point, minus the square of the sum of the time points. The slope of the memory utilization trend is calculated in the same way, using the memory utilization sequence and corresponding time points for the same calculation.

[0023] Alarm event data is used to statistically analyze the frequency of alarm types within each time window. This is achieved by traversing the alarm event list and counting the occurrences of each alarm type (e.g., system anomaly, network failure, storage warning), generating an alarm type frequency vector. When calculating the co-occurrence count of alarms, the alarm event data is traversed, and the number of times each pair of alarm types occurs simultaneously within the window is calculated. A co-occurrence graph is constructed, where nodes represent alarm types and edge weights represent co-occurrence counts. All edge weights are extracted as association strengths to obtain the alarm co-occurrence characteristics.

[0024] The text features, average processor utilization, standard deviation of processor utilization, average memory utilization, standard deviation of memory utilization, slope of processor utilization trend, slope of memory utilization trend, alarm type frequency, and alarm co-occurrence features are combined into a fixed-length feature vector to form an initial feature set. This initial feature set provides multi-dimensional feature support for subsequent fault diagnosis, ensuring that anomaly detection is based on complete and aligned multi-source data, thereby improving the accuracy and efficiency of system fault identification.

[0025] In step S13, redundant feature pairs are determined based on the initial feature set through feature correlation analysis, and features with smaller feature variances in the redundant feature pairs are removed to obtain an optimized feature set, including: Based on the initial feature set, the degree of linear correlation between features is quantified by calculating the Pearson correlation coefficient, and a correlation coefficient matrix is ​​generated. Based on the correlation coefficient matrix, feature index pairs corresponding to off-diagonal elements whose absolute values ​​are greater than a preset redundancy threshold are identified as redundant feature pairs. Based on the initial feature set, calculate the numerical variance of each feature dimension across all samples to obtain the feature variance vector; The features with smaller feature variance vectors in the redundant feature pairs are marked as redundant features, and all redundant features are removed from the initial feature set to generate an optimized feature set.

[0026] Specifically, step S13 performs feature correlation analysis based on the initial feature set to determine redundant features. The initial feature set is derived from the output of step S12 and includes text features, average processor utilization, standard deviation of processor utilization, average memory utilization, standard deviation of memory utilization, slope of processor utilization trend, slope of memory utilization trend, alarm type frequency, and alarm co-occurrence features. The redundancy threshold is determined through historical data statistics. The specific implementation steps are as follows: 1000 samples are extracted from the initial feature set data of the past year, the absolute value of the Pearson correlation coefficient between each pair of features is calculated, the distribution of these absolute values ​​is statistically analyzed, and the value corresponding to the 95th percentile is selected as the redundancy threshold. For example, the 95th percentile value of historical data is 0.7, so the redundancy threshold is set to 0.7.

[0027] When calculating the Pearson correlation coefficient, for each pair of feature sequences in the initial feature set, the covariance is first calculated. The covariance is equal to the sum of the products of the differences between corresponding values ​​in the two sequences and their respective means, divided by the sample size minus one. Then, the standard deviation of each sequence is calculated. The standard deviation is equal to the square root of the sum of the squares of the differences between each value in the sequence and the mean, divided by the sample size minus one. The correlation coefficient is then calculated by dividing the covariance by the product of the standard deviations of the two sequences, generating the correlation coefficient matrix.

[0028] Iterate through the off-diagonal elements of the correlation coefficient matrix, identifying feature index pairs corresponding to elements whose absolute values ​​are greater than the redundancy threshold of 0.7 as redundant feature pairs. For example, if the absolute value of the correlation coefficient between average processor utilization and average memory utilization exceeds 0.7, they are marked as redundant feature pairs.

[0029] Calculate the variance of each feature across all samples. The variance is equal to the sum of the squares of the differences between all values ​​in the feature sequence and the mean of that feature, divided by the number of samples minus one, resulting in the feature variance vector. Iterate through all redundant feature pairs. For each pair, compare their variance values ​​and mark the feature with the smaller variance as a redundant feature.

[0030] All features marked as redundant are removed from the initial feature set to generate an optimized feature set. This optimized feature set retains key features, removes highly correlated redundant features, reduces feature dimensionality, improves the computational efficiency and prediction accuracy of subsequent fault diagnosis models, and reduces the impact of noise interference on system anomaly detection.

[0031] In step S14, principal component analysis is performed on the optimized feature set to obtain a principal component feature set, and an orthogonal space is constructed based on the principal component feature set to obtain a compressed feature space, including: Based on the optimized feature set, the features are standardized using the Z-Score method to obtain the standard feature set; Based on the standard feature set, calculate the covariance values ​​between features and construct a symmetric matrix to obtain the covariance matrix; Based on the covariance matrix, all eigenvectors and corresponding eigenvalues ​​of the matrix are solved by eigenvalue decomposition to obtain the eigenvector set and eigenvalue set. Based on the eigenvalue set and the eigenvector set, principal vectors are extracted by principal component analysis to obtain the principal component vector set. The principal component vector set is then projected into a linear transformation matrix to obtain the projection matrix. Based on the standard feature set and the projection matrix, the standard feature set is projected onto the principal component direction through matrix multiplication to obtain the principal component feature set; By using the orthogonal space construction method, the principal component feature set is used as a basis to construct a low-dimensional representation space, thus obtaining a compressed feature space.

[0032] Specifically, step S14 performs principal component analysis based on the optimized feature set to construct a compressed feature space. The optimized feature set is derived from the output of step S13 and includes text features, average processor utilization, standard deviation of processor utilization, average memory utilization, standard deviation of memory utilization, slope of processor utilization trend, slope of memory utilization trend, alarm type frequency, and alarm co-occurrence features. First, features are standardized using the Z-Score method. The mean and standard deviation of each feature dimension in the optimized feature set are calculated. For each sample, the mean of the corresponding dimension is subtracted from the feature value, and then divided by the standard deviation of the corresponding dimension to obtain the standardized value. All feature dimensions are processed to form the standard feature set.

[0033] When calculating the covariance matrix, for each pair of feature dimensions in the optimized feature set, iterate through all samples, calculate the difference between the value of the first feature dimension and its mean, multiply this by the difference between the value of the second feature dimension and its mean, sum these products for all samples, and then divide the sum by the total number of samples minus one to obtain the covariance value. Fill all the covariance values ​​into a symmetric matrix to form the covariance matrix. Perform eigenvalue decomposition on the covariance matrix. This decomposition process uses numerical methods to obtain all eigenvalues ​​and eigenvectors. The eigenvalues ​​represent the variance of the data along the corresponding eigenvector direction.

[0034] The eigenvalues ​​are sorted from largest to smallest, and the cumulative contribution rate of each eigenvalue is calculated. The cumulative contribution rate equals the sum of the current eigenvalue and all previous eigenvalues ​​divided by the sum of all eigenvalues. The threshold for the number of principal components is determined through historical data statistics. 1000 samples are extracted from the optimized feature set data of the past year, the cumulative contribution rate distribution is calculated, and the value corresponding to the 95th percentile is selected as the threshold. For example, the historical 95th percentile value is 0.95, so the cumulative contribution rate threshold is set to 95%.

[0035] The principal component vector set is extracted, consisting of the top k feature vectors with a cumulative contribution rate exceeding 95% after sorting. These principal component vectors are then used as columns to construct a projection matrix. The standard feature set is projected onto the principal component directions using matrix multiplication; that is, each sample vector of the standard feature set is multiplied by the projection matrix to obtain the principal component feature set. Finally, a low-dimensional representation space is constructed based on the principal component feature set, forming a compressed feature space. This compressed feature space reduces feature dimensionality, preserves the main directions of data variation, improves the computational efficiency and prediction accuracy of subsequent fault diagnosis models, reduces memory usage and computational overhead, and provides efficient data representation for real-time anomaly detection.

[0036] In step S15, the compressed feature space is subjected to a whitening transformation to obtain whitening features, and rapid independent component analysis is performed iteratively based on the whitening features to obtain independent subspaces, including: Based on the covariance matrix, a whitening transformation is performed using a whitening algorithm to obtain a whitening matrix. Then, a linear transformation is performed on the compressed feature space based on the whitening matrix to obtain whitened features. Based on the whitening characteristics, the weight vector is iteratively optimized using a fast independent component analysis algorithm. With the goal of maximizing negative entropy, the weight vector is iteratively updated until the change between adjacent iterations is less than a preset convergence threshold, thus obtaining a converged weight vector. The convergence weight vectors are arranged in row order to construct the unmixing matrix. The whitening feature is then multiplied by the unmixing matrix through a linear projection transformation to output an independent subspace.

[0037] Specifically, step S15 performs whitening transformation and fast independent component analysis iterative optimization on the compressed feature space. The compressed feature space is derived from the output of step S14 and contains the principal component feature set. The whitening transformation is based on the covariance matrix, which is derived from the covariance matrix calculated in S14. The whitening matrix is ​​constructed by calculating the reciprocal of the square root of each eigenvalue from the covariance matrix, multiplying this value by the corresponding eigenvector to form the column vectors of the whitening matrix, and combining all column vectors to form the whitening matrix. A linear transformation is then performed on the compressed feature space, that is, each sample vector in the compressed feature space is multiplied by the whitening matrix to obtain the whitened features.

[0038] The Fast Independent Component Analysis (ICA) algorithm iteratively optimizes the weight vector with the objective of maximizing negative entropy. The initial weight vector is randomly generated. During iteration, a nonlinear transformation of the whitening feature is calculated on the current weight vector. By statistically analyzing the distribution characteristics of the whitening feature, the weights are adjusted to increase negative entropy. The weight update rule is as follows: based on the product of the whitening feature and the current weight vector, the expected value of the nonlinear function is calculated, and this is subtracted from the expected value of the weight vector itself to obtain the updated weight. This updated weight is then added to the current weight vector to obtain the new weight. The convergence threshold is determined through historical data statistics. 1000 samples are extracted from the compressed feature space data of the past year, and the ICA algorithm is run multiple times. The weight changes in each iteration are recorded, and the distribution of these changes is statistically analyzed. The value corresponding to the 95th percentile is selected as the threshold; for example, the historical 95th percentile value is 0.001, therefore the convergence threshold is set to 0.001.

[0039] The weight vector is iteratively updated until the weight change between two consecutive iterations is less than the convergence threshold of 0.001, resulting in a converged weight vector. The converged weight vector is arranged in row order to construct an unmixing matrix. The whitening features are multiplied by the unmixing matrix through a linear projection transformation, that is, each sample vector of the whitening feature is multiplied by the unmixing matrix, outputting an independent subspace.

[0040] This independent subspace enhances the independence of features, eliminates redundant correlations, provides clearer feature representations for subsequent fault diagnosis, improves the model's classification accuracy and real-time processing efficiency, and reduces the risk of false alarms and false negatives.

[0041] In step S16, fault patterns are matched between the independent subspace and the pre-stored historical fault case library, and Euclidean distance threshold determination is performed based on the fault patterns and the evolution characteristics of the independent subspace to obtain strongly correlated subspaces, including: Based on the independent subspace, the arithmetic mean, standard deviation, and average rate of change of the difference between adjacent time points of each dimension are calculated within the time window to obtain the evolutionary characteristics; Based on the independent subspace, fault modes are matched from the pre-stored historical fault case library, and combined with the evolution characteristics, the time axis is flexibly aligned using a dynamic time warping algorithm to generate aligned sequence pairs. Based on the aligned sequence pairs, the Euclidean distance between corresponding points of the two sequences in all feature dimensions is calculated to obtain the distance value, and the independent subspaces whose distance values ​​are not greater than a preset distance threshold are labeled as strongly correlated subspaces.

[0042] Specifically, step S16 matches fault patterns against the independent subspace and the historical fault case library, and performs Euclidean distance determination. The independent subspace is derived from the output of step S15 and includes the linear transformation results of the principal component feature set. The historical fault case library is constructed based on server fault records from the past three years, including fault mode labels, corresponding evolutionary features, and time series data, sourced from system monitoring logs and fault work orders. First, the evolutionary features of the independent subspace are calculated. For each dimension, all samples are traversed, and the arithmetic mean is calculated as the sum of all values ​​in that dimension divided by the total number of samples. The standard deviation is calculated as the square root of the sum of the squares of the differences between each value in that dimension and the mean, divided by the total number of samples minus one. The average rate of change of the differences between adjacent time points is calculated. The time series is traversed, and for each time point, the difference between the current value and the previous time point is calculated. All differences are summed and divided by the number of time points minus one to obtain the average rate of change. After processing all dimensions, an evolutionary feature vector is formed.

[0043] Based on the evolutionary characteristics of independent subspaces, fault modes are matched from a historical fault case database. The matching process compares the similarity between the evolutionary feature vector and the evolutionary features of each fault mode in the historical case database. Similarity is calculated based on the ratio of the dot product to the magnitude of the feature vectors, and the fault mode with the highest similarity is selected as the matching result. Combining the matched fault modes and evolutionary features, a dynamic time warping algorithm is used for flexible time axis alignment. This algorithm traverses the independent subspace sequences and historical fault sequences, calculates the cumulative distance for each pair, and uses dynamic programming to find the alignment path that minimizes the total distance, generating time-axis aligned sequence pairs.

[0044] The Euclidean distance between aligned sequence pairs is calculated. For each feature dimension, the square root of the sum of squared differences between corresponding points is taken to obtain the distance value between sequences. The distance threshold is determined through historical data statistics. Specifically, 1000 aligned sequence pairs are extracted from the historical fault case database, the Euclidean distance of all sequence pairs is calculated, the distance distribution is statistically analyzed, and the value corresponding to the 95th percentile is selected as the threshold (for example, the historical 95th percentile value is 2.5, so the distance threshold is set to 2.5). The distance values ​​are compared with the threshold, and independent subspaces that are less than or equal to the threshold are labeled as strongly correlated subspaces. This strongly correlated subspace improves the accuracy of fault diagnosis by quantifying the similarity between anomalies and historical faults, reducing the false alarm rate, providing a reliable basis for real-time fault early warning, and ensuring that the system can efficiently identify key fault modes with low latency.

[0045] In step S17, causal propagation chains are extracted from the strongly correlated subspace to obtain a propagation path set, and a description template is embedded based on the propagation path set to obtain a fault semantic expression, including: The unmixing matrix is ​​indexed according to the index of the strongly correlated subspace in the independent subspace to obtain the strongly correlated weights, and the original data dimension indexes corresponding to the elements whose strongly correlated weights are greater than the preset strongly correlated weight threshold are retained to obtain the original index set. Based on the original index set, the preset metadata configuration table is traversed, and each index is mapped to the corresponding topology identifier and signal code to obtain the identifier-code pair; Based on the identifier-encoding pair, a pre-built semantic tag library is queried, and the identifier-encoding pair is converted into Chinese semantic phrases to obtain a semantic phrase set; Based on the set of semantic phrases, a breadth-first search algorithm is used to perform a hierarchical traversal along the causal relationship edges in the preset industrial mechanism knowledge graph, starting with semantic phrases as the starting nodes, to extract the propagation chain formed by the associated nodes and edges, and obtain the propagation path set. Using the set of semantic phrases as the backbone and the causal relationships in the propagation path set as logical connectors embedded in a preset description template, a fault semantic expression containing fault phenomena and potential impacts is generated.

[0046] Specifically, step S17 extracts causal propagation chains and generates fault semantic representations based on the strongly correlated subspace. The strongly correlated subspace is derived from the output of step S16, and the unmixing matrix is ​​derived from the construction result of step S15. The strongly correlated weight threshold is determined through historical data statistics. The specific implementation steps are as follows: 1000 samples are extracted from the strongly correlated weight data of the past year, the sorted sequence of all weight values ​​is calculated, and the value corresponding to the 95th percentile is selected as the threshold (for example, the historical 95th percentile value is 0.3), so the strongly correlated weight threshold is set to 0.3. According to the index of the strongly correlated subspace in the independent subspace, the corresponding row of the unmixing matrix is ​​indexed to obtain the strongly correlated weight vector. The strongly correlated weight vector is traversed, and the original data dimension index corresponding to the elements with weights greater than 0.3 is retained to form the original index set.

[0047] The metadata configuration table is built based on historical system configuration data, sourced from server hardware topology and signal definition documents, and includes device identifiers, signal codes, and topology relationship mappings. Each index in the original index set is traversed, and the metadata configuration table is queried to map each index to a corresponding topology identifier (e.g., CPU module, memory controller) and signal code (e.g., CPU utilization, memory usage), generating identifier-code pairs. The semantic tag library is built based on historical fault reports and standard terminology, sourced from descriptive text in fault work orders from the past three years, establishing a mapping relationship between identifier codes and Chinese semantic phrases. The semantic tag library is queried to convert identifier-code pairs into Chinese semantic phrases (e.g., "CPU utilization too high" "abnormal memory usage"), forming a semantic phrase set.

[0048] It should be noted that, to ensure that all feature dimensions can be accurately mapped and interpreted, a complete metadata configuration table needs to be established during the system initialization phase. This table assigns a unique index number to each feature dimension generated in the feature extraction phase (S12) and records its corresponding data source, such as CPU utilization sequence, statistical type, such as mean, slope, alarm type, or text feature identifier. The semantic tag library predefines corresponding natural language description phrases for each data source-statistical type pair or alarm type. By looking up the table, the abstract index number can be chained into a specific and understandable description of the fault phenomenon.

[0049] The industrial mechanism knowledge graph is built based on system design documents and historical failure cases, containing nodes (devices or signals) and causal relationship edges (such as "high CPU usage leads to memory leak"). A breadth-first search algorithm is used, starting with each semantic phrase in the semantic phrase set as a node, and traversing hierarchically along the causal relationship edges. During traversal, the access path for each node is recorded, and when a new node is encountered, it is added to a queue. This process is repeated until all associated nodes have been visited, and the propagation chains formed by associated nodes and edges are extracted to form a propagation path set.

[0050] Specifically, the construction of the industrial mechanism knowledge graph can be based on the target server system's architecture design documents, operation and maintenance manuals, and historical fault analysis reports. Hardware components (such as CPU, memory modules, and disks), software services (such as database processes and web services), and performance indicators (such as CPU utilization, memory usage, and I / O wait time) mentioned in the documents are extracted as graph nodes. Secondly, based on the physical connection relationships, data flow dependencies, and causal chains summarized in historical fault reports (e.g., "disk I / O saturation" may lead to "database response delay," which in turn causes "application service timeout"), directed "causal relationship" edges are established between graph nodes. This graph can be stored and maintained in the form of a graph database (such as Neo4j) and can be continuously iterated and updated as operation and maintenance experience accumulates.

[0051] Using a set of semantic phrases as the backbone and causal relationships in the propagation path set as logical connectors (such as "leads to" or "triggers"), a pre-defined description template is embedded. The description template structure is "Fault Phenomenon: [Semantic Phenomenon 1], [Logical Connector], [Semantic Phenomenon 2]; Potential Impact: [Semantic Phenomenon 3], [Logical Connector], [Semantic Phenomenon 4]". This generates a fault semantic expression containing both the fault phenomenon and potential impact. This fault semantic expression improves the interpretability of fault diagnosis, enabling the system to automatically generate fault descriptions that conform to the language habits of maintenance personnel, reducing manual analysis time, improving the accuracy and response efficiency of fault location, and providing a clear basis for automated maintenance decision-making.

[0052] In step S18, a fault propagation path is constructed based on the strongly correlated subspace to generate a weighted topology graph. Then, based on the weighted topology graph and the fault semantic expression, a reverse breadth-first search is performed to generate a monitoring report, including: The corresponding initial feature set is extracted based on the strongly correlated subspace, and the feature components are compared with the preset feature component threshold one by one based on the initial feature set. The feature components that exceed the feature component threshold are regarded as abnormal signals, and the absolute deviation between the abnormal signal and the feature component threshold is calculated to obtain the deviation vector. Based on the abnormal signal, the deviation vector, and the propagation path set, a fault propagation path matching the abnormal pattern represented by the deviation vector is retrieved using a breadth-first search algorithm. The deviation vector is then assigned to the starting node of the fault propagation path as an initial weight. The weights are then passed along the causal edges according to a preset attenuation coefficient to obtain a weighted topology graph. Based on the weighted topology graph, the reverse breadth-first search algorithm is used to traverse backwards from the end node, accumulate the path weights, and locate the node with the largest accumulated weight as the root cause node. The semantic phrase set associated with the root cause node, the initial feature set, and the fault semantic expression are embedded into a preset report template to generate a monitoring report.

[0053] Specifically, step S18 constructs the fault propagation path and generates monitoring reports based on the strongly correlated subspace. The strongly correlated subspace originates from the output of step S16, and the initial feature set originates from the output of step S12. The feature component threshold is determined through historical data statistics. The specific implementation steps are as follows: 1000 samples are extracted from the initial feature set data of the past year, the numerical distribution of each feature component is calculated, and the value corresponding to its 95th percentile is used as the threshold (for example, the historical 95th percentile value is 10.5), so the feature component threshold is set to 10.5. Based on the initial feature set, each feature component is traversed, and the feature component value is compared with the feature component threshold. If the feature component value is greater than the threshold, it is marked as an abnormal signal. The absolute deviation between the abnormal signal and the threshold is calculated, that is, the absolute deviation is equal to the absolute value of the difference between the feature component value and the threshold, forming a deviation vector.

[0054] Based on the abnormal signal, deviation vector, and propagation path set, a breadth-first search algorithm is used to retrieve matching fault propagation paths. The propagation path set originates from step S17 and contains causal edges. For each path in the propagation path set, the deviation vector is assigned to the starting node of the path as an initial weight. Weights are passed along the causal edges, with the new weight equal to the current weight multiplied by a decay coefficient. The decay coefficient is determined through historical data statistics; 1000 samples are extracted from the historical weight passing data, and the value corresponding to the 95th percentile is calculated (e.g., the historical 95th percentile value is 0.8), therefore the decay coefficient is set to 0.8. After traversing all paths, a weighted topology graph is obtained, where the node weights are the passed-down values.

[0055] It should be noted that the search for the fault propagation path that matches the abnormal pattern represented by the deviation vector refers to traversing the set of propagation paths and checking whether the starting node of the path, i.e. the source of the abnormal signal, is included in the set of abnormal features indicated by the deviation vector. If the feature dimension corresponding to the starting node of a propagation path is marked as abnormal in the deviation vector, i.e., its deviation value is greater than 0, then the path is considered to match the current abnormal pattern and should be included in the construction of the subsequent weighted topology graph.

[0056] Based on the weighted topology graph, a reverse breadth-first search algorithm is used to traverse the graph from the end node (the node with no outgoing edges). During traversal, the node weights are accumulated for each path, and the total weight of the path is calculated. After traversing all paths, the node with the largest accumulated weight is identified as the root cause node. The set of semantic phrases associated with the root cause node comes from step S17, and the initial feature set is the output of S12.

[0057] The semantic phrase set, initial feature set, and fault semantic expression associated with the root cause node are embedded into a pre-defined report template. The report template structure is "Root Cause: [Root Cause Semantics]; Anomaly Features: [Initial Feature Set Summary]; Fault Description: [Fault Semantic Expression]". A monitoring report containing the root cause, anomaly features, and a detailed description is generated. This report improves the operability of fault diagnosis, enabling operations and maintenance personnel to quickly locate the root cause and take measures, reducing system downtime and improving operational efficiency and decision-making accuracy.

[0058] Reference Figure 2 The second embodiment of the present invention provides an information processing system based on multi-dimensional features, comprising: The data acquisition module is used to acquire log text data, processor utilization sequence, memory utilization sequence, and alarm event data; The feature extraction module is used to extract data features based on log text data, processor usage sequence, memory usage sequence and alarm event data to obtain an initial feature set; The feature optimization module is used to determine redundant feature pairs based on the initial feature set through feature correlation analysis, and to remove features with small feature variance from the redundant feature pairs to obtain an optimized feature set. The compressed feature module is used to perform principal component analysis on the optimized feature set to obtain a principal component feature set, and to construct a compressed feature space based on the principal component feature set using an orthogonal space. An independent module is used to perform whitening transformation on the compressed feature space to obtain whitening features, and to perform fast independent component analysis and iterative optimization based on the whitening features to obtain an independent subspace; The strongly correlated module is used to match fault patterns with the pre-stored historical fault case library based on the independent subspace, and to determine the Euclidean distance threshold based on the fault patterns and the evolution characteristics of the independent subspace to obtain the strongly correlated subspace. The fault semantic module is used to extract the causal propagation chain from the strongly correlated subspace to obtain a propagation path set, and to embed a description template based on the propagation path set to obtain a fault semantic expression. The output module is used to construct a fault propagation path based on the strongly correlated subspace, generate a weighted topology graph, and perform a reverse breadth-first search based on the weighted topology graph and the fault semantic expression to generate a monitoring report.

[0059] It should be noted that the information processing system based on multi-dimensional features provided in this embodiment of the invention is used to execute all the process steps of the information processing method based on multi-dimensional features in the above embodiment. The working principles and beneficial effects of the two are one-to-one, so they will not be described again.

[0060] It should be noted that the system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, in the accompanying drawings of the system embodiments provided by this invention, the connection relationships between modules indicate that they have communication connections, which can be specifically implemented as one or more communication buses or signal lines. Those skilled in the art can understand and implement this without any creative effort.

[0061] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. In particular, it should be noted that any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention for those skilled in the art.

Claims

1. An information processing method based on multi-dimensional features, characterized in that, include: Acquire log text data, processor utilization sequence, memory utilization sequence, and alarm event data; Based on log text data, processor usage sequence, memory usage sequence, and alarm event data, data features are extracted to obtain an initial feature set; Based on the initial feature set, redundant feature pairs are identified through feature correlation analysis, and features with small variances in the redundant feature pairs are removed to obtain an optimized feature set. Principal component analysis is performed on the optimized feature set to obtain a principal component feature set, and an orthogonal space is constructed based on the principal component feature set to obtain a compressed feature space; The compressed feature space is subjected to whitening transformation to obtain whitening features, and rapid independent component analysis is performed on the whitening features for iterative optimization to obtain independent subspaces; The fault patterns are matched with the pre-stored historical fault case library based on the independent subspaces, and the Euclidean distance threshold is determined based on the fault patterns and the evolution characteristics of the independent subspaces to obtain strongly correlated subspaces. The causal propagation chain is extracted from the strongly correlated subspace to obtain a propagation path set, and a description template is embedded based on the propagation path set to obtain a fault semantic expression; Fault propagation paths are constructed based on the strongly correlated subspace, a weighted topology graph is generated, and a reverse breadth-first search is performed based on the weighted topology graph and the fault semantic expression to generate a monitoring report.

2. The information processing method based on multi-dimensional features according to claim 1, characterized in that, The process involves extracting data features from log text data, processor usage sequences, memory usage sequences, and alarm event data to obtain an initial feature set, including: Based on the log text data, context semantic encoding is performed using a bidirectional encoder to generate a high-dimensional dense vector and bind it with a time window identifier to obtain text features; The performance statistical characteristics are obtained by taking the arithmetic mean and standard deviation of the corresponding values ​​of the processor utilization sequence and the memory utilization sequence in each time window; Based on the processor usage sequence and the memory usage sequence, the linear slope of the numerical change within each time window is calculated using a linear regression fitting method to obtain the performance trend characteristics; The alarm type frequency is obtained by statistically analyzing the cumulative occurrence count of each alarm type within each time window of the alarm event data. The number of times different alarm types co-occur within each time window is counted to obtain the alarm co-occurrence count. A co-occurrence relationship graph is constructed with alarm type as nodes and alarm co-occurrence count as edge weights. The edge weights are extracted as the association strength to obtain the co-occurrence feature. The text features, performance statistics features, performance trend features, alarm type frequency, and co-occurrence features are combined into a fixed-length feature vector to obtain an initial feature set.

3. The information processing method based on multi-dimensional features according to claim 1, characterized in that, The step of determining redundant feature pairs based on the initial feature set through feature correlation analysis, and removing features with small variance from the redundant feature pairs to obtain an optimized feature set includes: Based on the initial feature set, the degree of linear correlation between features is quantified by calculating the Pearson correlation coefficient, and a correlation coefficient matrix is ​​generated. Based on the correlation coefficient matrix, feature index pairs corresponding to off-diagonal elements whose absolute values ​​are greater than a preset redundancy threshold are identified as redundant feature pairs. Based on the initial feature set, calculate the numerical variance of each feature dimension across all samples to obtain the feature variance vector; The features with smaller feature variance vectors in the redundant feature pairs are marked as redundant features, and all redundant features are removed from the initial feature set to generate an optimized feature set.

4. The information processing method based on multi-dimensional features according to claim 1, characterized in that, The step of performing principal component analysis on the optimized feature set to obtain a principal component feature set, and constructing a compressed feature space based on the principal component feature set using an orthogonal space, includes: Based on the optimized feature set, the features are standardized using the Z-Score method to obtain the standard feature set; Based on the standard feature set, calculate the covariance values ​​between features and construct a symmetric matrix to obtain the covariance matrix; Based on the covariance matrix, all eigenvectors and corresponding eigenvalues ​​of the matrix are solved by eigenvalue decomposition to obtain the eigenvector set and eigenvalue set. Based on the eigenvalue set and the eigenvector set, principal vectors are extracted by principal component analysis to obtain the principal component vector set. The principal component vector set is then projected into a linear transformation matrix to obtain the projection matrix. Based on the standard feature set and the projection matrix, the standard feature set is projected onto the principal component direction through matrix multiplication to obtain the principal component feature set; By using the orthogonal space construction method, the principal component feature set is used as a basis to construct a low-dimensional representation space, thus obtaining a compressed feature space.

5. The information processing method based on multi-dimensional features according to claim 4, characterized in that, The process involves performing a whitening transformation on the compressed feature space to obtain whitened features, and then performing rapid independent component analysis iterative optimization based on the whitened features to obtain independent subspaces, including: Based on the covariance matrix, a whitening transformation is performed using a whitening algorithm to obtain a whitening matrix. Then, a linear transformation is performed on the compressed feature space based on the whitening matrix to obtain whitened features. Based on the whitening characteristics, the weight vector is iteratively optimized using a fast independent component analysis algorithm. With the goal of maximizing negative entropy, the weight vector is iteratively updated until the change between adjacent iterations is less than a preset convergence threshold, thus obtaining a converged weight vector. The convergence weight vectors are arranged in row order to construct the unmixing matrix. The whitening feature is then multiplied by the unmixing matrix through a linear projection transformation to output an independent subspace.

6. The information processing method based on multi-dimensional features according to claim 1, characterized in that, The step of matching fault patterns between the independent subspaces and a pre-stored historical fault case library, and determining a strongly correlated subspace based on the Euclidean distance threshold of the fault patterns and the evolution characteristics of the independent subspaces, includes: Based on the independent subspace, the arithmetic mean, standard deviation, and average rate of change of the difference between adjacent time points of each dimension are calculated within the time window to obtain the evolutionary characteristics; Based on the independent subspace, fault modes are matched from the pre-stored historical fault case library, and combined with the evolution characteristics, the time axis is flexibly aligned using a dynamic time warping algorithm to generate aligned sequence pairs. Based on the aligned sequence pairs, the Euclidean distance between corresponding points of the two sequences in all feature dimensions is calculated to obtain the distance value, and the independent subspaces whose distance values ​​are not greater than a preset distance threshold are labeled as strongly correlated subspaces.

7. The information processing method based on multi-dimensional features according to claim 5, characterized in that, The step involves extracting causal propagation chains from the strongly correlated subspace to obtain a propagation path set, and embedding a description template based on the propagation path set to obtain a fault semantic expression, including: The unmixing matrix is ​​indexed according to the index of the strongly correlated subspace in the independent subspace to obtain the strongly correlated weights, and the original data dimension indexes corresponding to the elements whose strongly correlated weights are greater than the preset strongly correlated weight threshold are retained to obtain the original index set. Based on the original index set, the preset metadata configuration table is traversed, and each index is mapped to the corresponding topology identifier and signal code to obtain the identifier-code pair; Based on the identifier-encoding pair, a pre-built semantic tag library is queried, and the identifier-encoding pair is converted into Chinese semantic phrases to obtain a semantic phrase set; Based on the set of semantic phrases, a breadth-first search algorithm is used to perform a hierarchical traversal along the causal relationship edges in the preset industrial mechanism knowledge graph, starting with semantic phrases as the starting nodes, to extract the propagation chain formed by the associated nodes and edges, and obtain the propagation path set. Using the set of semantic phrases as the backbone and the causal relationships in the propagation path set as logical connectors embedded in a preset description template, a fault semantic expression containing fault phenomena and potential impacts is generated.

8. The information processing method based on multi-dimensional features according to claim 1, characterized in that, The step of constructing fault propagation paths based on the strongly correlated subspaces and generating a weighted topology graph includes: The corresponding initial feature set is extracted based on the strongly correlated subspace, and the feature components are compared with the preset feature component threshold one by one based on the initial feature set. The feature components that exceed the feature component threshold are regarded as abnormal signals, and the absolute deviation between the abnormal signal and the feature component threshold is calculated to obtain the deviation vector. Based on the abnormal signal, the deviation vector, and the propagation path set, a fault propagation path matching the abnormal pattern represented by the deviation vector is retrieved using a breadth-first search algorithm. The deviation vector is then assigned to the starting node of the fault propagation path as an initial weight, and the weights are passed along the causal edges according to a preset attenuation coefficient to obtain a weighted topology graph.

9. The information processing method based on multi-dimensional features according to claim 7, characterized in that, The step of performing a reverse breadth-first search based on the weighted topology graph and the fault semantic representation to generate a monitoring report includes: Based on the weighted topology graph, the reverse breadth-first search algorithm is used to traverse backwards from the end node, accumulate the path weights, and locate the node with the largest accumulated weight as the root cause node. The semantic phrase set associated with the root cause node, the initial feature set, and the fault semantic expression are embedded into a preset report template to generate a monitoring report.

10. An information processing system based on multi-dimensional features, characterized in that, include: The data acquisition module is used to acquire log text data, processor utilization sequence, memory utilization sequence, and alarm event data; The feature extraction module is used to extract data features based on log text data, processor usage sequence, memory usage sequence and alarm event data to obtain an initial feature set; The feature optimization module is used to determine redundant feature pairs based on the initial feature set through feature correlation analysis, and to remove features with small feature variance from the redundant feature pairs to obtain an optimized feature set. The compressed feature module is used to perform principal component analysis on the optimized feature set to obtain a principal component feature set, and to construct a compressed feature space based on the principal component feature set using an orthogonal space. An independent module is used to perform whitening transformation on the compressed feature space to obtain whitening features, and to perform fast independent component analysis and iterative optimization based on the whitening features to obtain an independent subspace; The strongly correlated module is used to match fault patterns with the pre-stored historical fault case library based on the independent subspace, and to determine the Euclidean distance threshold based on the fault patterns and the evolution characteristics of the independent subspace to obtain the strongly correlated subspace. The fault semantic module is used to extract the causal propagation chain from the strongly correlated subspace to obtain a propagation path set, and to embed a description template based on the propagation path set to obtain a fault semantic expression. The output module is used to construct a fault propagation path based on the strongly correlated subspace, generate a weighted topology graph, and perform a reverse breadth-first search based on the weighted topology graph and the fault semantic expression to generate a monitoring report.

Images