An adaptive system data patrol and exception repair method for a wisdom center

Abstract

The application discloses a kind of self-adapting system data inspection and exception repair method for intelligent computing center, it is related to intelligent computing system operation and maintenance and data processing technical field, including the system running data of the acquisition of several computing nodes of intelligent computing center, and based on resource consumption proportion, system running data is dynamically weighted standardization processing, obtains weight standardization data matrix;According to weight standardization data matrix, node correlation matrix is constructed, and system state expected value is generated through bidirectional recurrent neural network model;Based on the deviation distribution of system state expected value and current weight standardization data, adaptive interval estimation algorithm is used to determine exception identification boundary;For the data beyond exception identification boundary, data reconstruction is carried out using multi-source collaborative compensation mechanism, and through inverse weight standardization processing, the system running data after repair is output.The application realizes the whole-process intelligent inspection and accurate exception repair of intelligent computing center system running data.

Description

Technical Field

[0001] This invention relates to the field of intelligent computing system operation and maintenance and data processing technology, and in particular to an adaptive system data inspection and anomaly repair method for intelligent computing centers. Background Technology

[0002] Intelligent computing centers typically consist of thousands or even tens of thousands of computing nodes, forming a complex distributed computing architecture. Their system operation data exhibits characteristics such as high dimensionality, multi-source heterogeneity, and dynamic changes. To ensure the stable operation and computing efficiency of intelligent computing centers, real-time monitoring, anomaly detection, and fault early warning of system operation data have become critical technical requirements. Traditional data inspection methods mainly rely on preset threshold judgments and rule matching, identifying system anomalies by monitoring basic indicators such as CPU utilization, memory usage, and network traffic. However, these methods often suffer from high false alarm rates and poor adaptability when facing the complex and ever-changing operating environment of intelligent computing centers, making it difficult to meet the requirements of modern intelligent computing centers for accurate anomaly detection.

[0003] Existing system data anomaly detection technologies suffer from the following shortcomings: First, traditional static threshold detection methods struggle to adapt to the dynamically changing resource consumption patterns of intelligent computing centers, leading to increased false positive or false negative rates. Second, anomaly prediction based on a single model fails to adequately consider the correlation between nodes, resulting in a lack of global consistency in the generation of expected state values. Furthermore, existing anomaly repair methods often rely on a single data source or simple weighted averaging, failing to effectively utilize the sparsity of inter-node correlations, leading to insufficient robustness of repair results. Particularly noteworthy is the limited dynamic processing capability of current technologies for weighted standardized data; the lack of a dynamic weighting mechanism based on resource consumption ratios during standardization further reduces the sensitivity of anomaly identification. In the repair phase, existing methods lack a constraint adjustment mechanism for expected values, potentially introducing the accumulation of compensation biases. To address these issues, an adaptive system data inspection and anomaly repair method for intelligent computing centers is proposed. Summary of the Invention

[0004] In view of the problems of poor adaptability, high false alarm rate and lack of intelligent data reconstruction capability in existing intelligent computing center system monitoring technology, this invention is proposed.

[0005] Therefore, the problem to be solved by this invention is how to achieve dynamic adaptive inspection and high-precision anomaly repair of data in intelligent computing center systems.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0007] In a first aspect, embodiments of the present invention provide a method for adaptive system data inspection and anomaly repair in an intelligent computing center, comprising,

[0008] System operation data from several computing nodes in the intelligent computing center are collected, and the system operation data is dynamically weighted and standardized based on the resource consumption ratio to obtain a weighted standardized data matrix.

[0009] A node correlation matrix is ​​constructed based on the weighted standardized data matrix, and the expected value of the system state is generated through a bidirectional recursive neural network model.

[0010] Based on the deviation distribution between the expected system state value and the current weighted standardized data, an adaptive interval estimation algorithm is used to determine the anomaly identification boundary.

[0011] For data that exceeds the anomaly identification boundary, a multi-source collaborative compensation mechanism is used to reconstruct the data, and the repaired system operation data is output through inverse weight standardization processing.

[0012] As a preferred embodiment of the adaptive system data inspection and anomaly repair method for intelligent computing centers described in this invention, the method involves: for data exceeding the anomaly identification boundary, a multi-source collaborative compensation mechanism is used for data reconstruction, and the repaired system operation data is output through inverse weight standardization processing, including:

[0013] The current time data of the weighted standardized data matrix T is compared with the anomaly identification boundary parameter set J to obtain the multi-source compensation node set N, wherein the anomaly identification boundary parameter set J includes an upper boundary threshold and a lower boundary threshold.

[0014] For the corresponding index data of each node in the multi-source compensation node set N, weight coefficients are assigned according to the correlation values ​​in the sparse node correlation matrix M' to construct a compensation weight vector C;

[0015] The preliminary compensation value vector B is obtained by using the compensation weight vector C to perform a weighted average calculation on the corresponding index data in the multi-source compensation node set N;

[0016] Based on the expected value at the corresponding position in the system state expected value vector P, the initial compensation value vector B is constrained and adjusted to obtain the compensation value vector A;

[0017] Replace the data at the abnormal positions in the weighted standardized data matrix T with the corresponding values ​​in the compensation value vector A to obtain the repaired weighted standardized data matrix T”;

[0018] The repaired weighted normalized data matrix T” is subjected to inverse weighted normalization to obtain the repaired system operation data matrix Z.

[0019] As a preferred embodiment of the adaptive system data inspection and anomaly repair method for intelligent computing centers described in this invention, the method involves judging the current-time data of the weighted standardized data matrix T against the anomaly identification boundary parameter set J, including:

[0020] When the actual weighted standardized data is greater than the upper boundary threshold of the anomaly identification boundary parameter set J, or when the actual weighted standardized data is less than the lower boundary threshold of the anomaly identification boundary parameter set J, the location of the abnormal data is marked, an anomaly identification matrix K is generated, and a multi-source collaborative compensation mechanism is used for data reconstruction.

[0021] When the actual weighted standardized data is less than or equal to the upper boundary threshold of the anomaly identification boundary parameter set J and greater than or equal to the lower boundary threshold of the anomaly identification boundary parameter set J, it is determined to be normal data.

[0022] As a preferred embodiment of the adaptive system data inspection and anomaly repair method for intelligent computing centers described in this invention, the method of employing a multi-source collaborative compensation mechanism for data reconstruction includes:

[0023] Based on the anomaly identification matrix K, the node index and indicator index of the anomaly data are extracted, and the anomaly data location index set Q is constructed.

[0024] Based on the sparse node correlation matrix M', select several normal nodes with the highest correlation to the location of abnormal data to form a multi-source compensation node set N.

[0025] As a preferred embodiment of the adaptive system data inspection and anomaly repair method for intelligent computing centers described in this invention, the anomaly identification boundary is determined using an adaptive interval estimation algorithm based on the deviation distribution of the expected system state value and the current weighted standardized data, including:

[0026] The deviation vector D is obtained by subtracting the current actual observation value from the system state expected value vector P in the weighted standardized data matrix T element by element.

[0027] The deviation vector D is grouped and arranged according to node type and resource index type to construct the deviation distribution matrix E. The cumulative distribution function value of each element is calculated to obtain the cumulative probability distribution matrix F.

[0028] Based on the cumulative probability distribution matrix F, the dynamic confidence interval parameter set G is calculated using the quantile regression method;

[0029] Based on the time series change trend of the deviation values ​​in the deviation distribution matrix E, calculate the adaptive adjustment factor H and obtain the upper boundary threshold vector U and the lower boundary threshold vector L.

[0030] Based on the system state expected value vector P and the upper boundary threshold vector U and the lower boundary threshold vector L, an anomaly identification boundary interval set I is constructed.

[0031] The set of anomaly identification boundary intervals I is smoothed to obtain the set of anomaly identification boundary parameters J.

[0032] As a preferred embodiment of the adaptive system data inspection and anomaly repair method for intelligent computing centers described in this invention, wherein: the method for generating the system state expected value vector P is as follows:

[0033] Based on the weighted standardized data matrix T, the Pearson correlation coefficient of each computing node is calculated to obtain the correlation coefficient between nodes;

[0034] The correlation coefficients between nodes are arranged in order of node number to construct a node correlation matrix M, wherein the node correlation matrix M is a symmetric matrix;

[0035] The node correlation matrix M is subjected to threshold filtering to obtain a sparse node correlation matrix M', and a bidirectional recurrent neural network model is constructed.

[0036] The bidirectional recurrent neural network model is used to perform forward calculations on the current time data of the weighted standardized data matrix T to generate the expected value vector P of the system state.

[0037] As a preferred embodiment of the adaptive system data inspection and anomaly repair method for intelligent computing centers described in this invention, the method includes: performing threshold filtering on the node correlation matrix M, including:

[0038] If the correlation value of the node correlation matrix M is greater than the preset correlation threshold θ, then the correlation value is retained in the corresponding position of the sparse matrix to maintain the original correlation strength between nodes; if the correlation value of the node correlation matrix M is less than or equal to the preset correlation threshold θ, then the correlation value is set to zero in the corresponding position of the sparse matrix to obtain the sparse node correlation matrix M'.

[0039] As a preferred embodiment of the adaptive system data inspection and anomaly repair method for intelligent computing centers described in this invention, wherein: the method for obtaining the weighted standardized data matrix T is as follows:

[0040] The system operation data is obtained by data acquisition agent programs deployed on each computing node of the intelligent computing center;

[0041] The system operation data is arranged chronologically according to the collection timestamps to construct the original data matrix R;

[0042] Calculate the total resource consumption of the computing nodes in the original data matrix R within the current time window to form a resource consumption vector V;

[0043] Based on the consumption ratio of each resource in the resource consumption vector V, the data in the original data matrix R is weighted and normalized using the resource consumption vector V to obtain the weighted standardized data matrix T.

[0044] In a second aspect, embodiments of the present invention provide a computer device, including a memory and a processor, wherein the memory stores a computer program, wherein: when the computer program instructions are executed by the processor, they implement the steps of the adaptive system data inspection and anomaly repair method for intelligent computing centers as described in the first aspect of the present invention.

[0045] Thirdly, embodiments of the present invention provide a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program instructions are executed by a processor, they implement the steps of the adaptive system data inspection and anomaly repair method for intelligent computing centers as described in the first aspect of the present invention.

[0046] Compared with existing technologies, the beneficial effects of this invention are as follows: This invention achieves unified quantitative representation and normalization of heterogeneous system operation data of different types and scales by collecting system operation data from several computing nodes in an intelligent computing center and performing dynamic weight standardization based on resource consumption ratios. This solves the technical problem of direct comparison and comprehensive analysis of diverse heterogeneous resource data in intelligent computing centers. By constructing a node correlation matrix based on Pearson correlation coefficients and using a bidirectional recurrent neural network model to generate expected system state values, this invention achieves deep learning modeling of the spatiotemporal dependencies between complex nodes in intelligent computing centers and intelligent prediction of system operation trends. This solves the technical limitation of traditional static threshold methods in accurately capturing the dynamic correlation characteristics and temporal evolution patterns between nodes. The deviation distribution between the expected system state value and the current weighted standardized data is determined by an adaptive interval estimation algorithm to identify anomalies. This enables dynamic adaptive adjustment and quantile regression optimization of anomaly identification thresholds under different periods, load conditions, and business scenarios, solving the technical defects of high false alarm rate and high false negative rate of fixed threshold methods in complex and variable operating environments. By adopting a multi-source collaborative compensation mechanism to intelligently reconstruct data exceeding the anomaly identification boundary and outputting repaired system operation data through inverse weighted standardization, accurate repair of anomaly data based on the correlation information of multiple normal nodes and complete restoration of the original data format are achieved. This solves the technical problems of insufficient repair accuracy from a single data source, mismatched repair data formats, and difficulty in ensuring data consistency. Attached Figure Description

[0047] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein:

[0048] Figure 1 This is a flowchart of an adaptive system data inspection and anomaly repair method for intelligent computing centers. Detailed Implementation

[0049] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0050] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0051] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0052] As mentioned in the background section, existing system data anomaly detection technologies suffer from the following shortcomings: First, traditional static threshold detection methods struggle to adapt to the dynamically changing resource consumption patterns of intelligent computing centers, leading to increased false positive or false negative rates. Second, anomaly prediction based on a single model fails to adequately consider the inter-node correlations, resulting in a lack of global consistency in the generation of expected state values. Furthermore, existing anomaly repair methods often rely on a single data source or simple weighted averaging, failing to effectively utilize the sparsity of inter-node correlations, leading to insufficient robustness of the repair results. Particularly noteworthy is the limited dynamic processing capability of current technologies for weighted standardized data; the lack of a dynamic weighting mechanism based on resource consumption ratios during standardization further reduces the sensitivity of anomaly identification. In the repair phase, existing methods lack a constraint adjustment mechanism for expected values, potentially introducing the accumulation of compensation biases. To address these issues, an adaptive system data inspection and anomaly repair method for intelligent computing centers is proposed.

[0053] Figure 1 This is a flowchart illustrating an adaptive system data inspection and anomaly repair method for an intelligent computing center according to an embodiment of the present invention. Figure 1As shown, an adaptive system data inspection and anomaly repair method for intelligent computing centers includes:

[0054] S1: Collect system operation data from several computing nodes in the intelligent computing center, and perform dynamic weight standardization processing on the system operation data based on the resource consumption ratio to obtain a weighted standardized data matrix;

[0055] S2: Construct a node correlation matrix based on the weighted standardized data matrix, and generate the expected value of the system state through a bidirectional recurrent neural network model;

[0056] S3: Based on the deviation distribution of the expected system state value and the current weighted standardized data, an adaptive interval estimation algorithm is used to determine the anomaly identification boundary;

[0057] S4: For data that exceeds the anomaly identification boundary, a multi-source collaborative compensation mechanism is used to reconstruct the data, and the repaired system operation data is output through inverse weight standardization.

[0058] In this embodiment of the application, step S1 includes:

[0059] System operation data includes CPU utilization, memory usage, network transmission rate, and storage I / O response time;

[0060] S1.1: Obtain system operation data through data acquisition agent programs deployed on various computing nodes in the intelligent computing center;

[0061] It should be noted that the system operation data includes CPU utilization parameters, memory usage parameters, disk I / O throughput parameters, network bandwidth utilization parameters, GPU computing load parameters, and storage access frequency parameters.

[0062] For example, there are 3 computing nodes (Node1, Node2, and Node3) whose system operating status needs to be monitored; the collected data includes CPU utilization, memory usage, network transmission rate, and storage I / O response time, collected every 5 seconds to form the raw data matrix R; for example, at a certain moment, Node1 has a CPU utilization of 65%, a memory usage of 48%, a network transmission rate of 120Mbps, and a storage I / O response time of 15ms.

[0063] S1.2: Arrange the system operation data in chronological order according to the collection timestamps to construct the original data matrix R;

[0064] It should be noted that the rows of the original data matrix R represent different points in time, and the columns represent different types of system resource indicators.

[0065] S1.3: Calculate the total resource consumption of the nodes in the original data matrix R within the current time window, forming the resource consumption vector V;

[0066] Preferably, the specific formula for the resource consumption vector V is as follows:

[0067]

[0068] Among them, V j Let α be the resource consumption vector value of the j-th computing node, n be the total number of system resource types, and α be the vector value of the resource consumption vector of the j-th computing node. i Let β be the weight coefficient for the i-th resource. i Let be the time decay factor for the i-th resource, Δt be the difference between the current time and the reference time, tanh(*) be the hyperbolic tangent function, and R be the time decay factor for the i-th resource. i,j Let σ be the collected data value of the i-th resource at the j-th node. i Let γ be the standard deviation parameter of the i-th resource, m be the number of associated nodes involved in the calculation, and γ be the standard deviation parameter of the i-th resource. k This represents the influence weight of the kth associated node.

[0069] Furthermore, the range of this formula is [0,2]; when V j When the value is in the range [0, 0.3], it indicates that the resource consumption of the j-th node is in a low-load state, and the system is running stably; when V j When the value is in the range (0.3, 0.8), it indicates that the node resource consumption is within the normal operating range; when V j When the value is in the range (0.8, 1.5), it indicates that the node's resource consumption is high, and resource allocation needs to be monitored; when V... j When the value is in the range (1.5, 2), it indicates that the node resource consumption is abnormal and there is a risk of performance bottleneck.

[0070] It should be noted that the total resource consumption includes CPU resource consumption, memory resource consumption, storage resource consumption, and network resource consumption.

[0071] S1.4: Based on the consumption ratio of each resource in the resource consumption vector V, the data in the original data matrix R is weighted and normalized using the resource consumption vector V to obtain the weighted standardized data matrix T.

[0072] Preferably, the data in the original data matrix R is subjected to weighted normalization, including: calculating the consumption proportion weight coefficient of each resource type in the resource consumption vector V; subtracting the historical mean of the corresponding indicator from each data element in the original data matrix R to obtain a mean-free data matrix; dividing each element in the mean-free data matrix by the historical standard deviation of the corresponding indicator to obtain a standardized intermediate matrix S, eliminating the dimensional differences between different indicators; multiplying each element in the standardized intermediate matrix by the weight coefficient to obtain a weighted standardized matrix; and normalizing the weighted standardized matrix by dividing it by the norm or maximum value of each row of data to ensure that the data distribution is within a reasonable numerical range, thus obtaining a weighted standardized data matrix T.

[0073] For example, the resource consumption vector V of each node is calculated; for instance, the resource consumption vector value of Node1 is calculated to be 0.75, which falls in the interval (0.3, 0.8], indicating that its resource consumption is within the normal range; while the calculated value of Node3 is 1.6, which falls in the interval (1.5, 2], indicating that it may have a performance bottleneck risk; the system performs weighted normalization on the original data matrix R according to the resource consumption ratio to generate a weighted standardized data matrix T, eliminating the difference in the dimensions of different indicators.

[0074] In this embodiment of the application, step S2 includes:

[0075] S2.1: Based on the weighted standardized data matrix T, calculate the Pearson correlation coefficient of each computing node to obtain the correlation coefficient between nodes;

[0076] S2.2: Arrange the correlation coefficients between nodes in order of node number to construct the node correlation matrix M, where the node correlation matrix M is a symmetric matrix;

[0077] It should be noted that the Pearson correlation coefficient represents the correlation strength between the system operating states of different nodes; the matrix elements represent the numerical values ​​of the correlation strength between nodes.

[0078] S2.3: Perform threshold filtering on the node correlation matrix M to obtain the sparsed node correlation matrix M', and construct the bidirectional recurrent neural network model';

[0079] Preferably, the node correlation matrix M is subjected to threshold filtering, including: if the correlation value of the node correlation matrix M is greater than the preset correlation threshold θ, then the correlation value is retained in the corresponding position of the sparse matrix to maintain the original correlation strength between nodes; if the correlation value in the node correlation matrix M is less than or equal to the preset correlation threshold θ, then the correlation value is set to zero in the corresponding position of the sparse matrix to obtain the sparse node correlation matrix M'.

[0080] It should be noted that the preset association threshold θ is determined based on the statistical characteristics of the association degree distribution between nodes. The mean and standard deviation of the association degree values ​​are calculated by analyzing the statistical distribution of all association degree values ​​in the node association degree matrix M. The bidirectional recurrent neural network model includes an input layer, a hidden layer and an output layer. The input layer structure includes two branches: a forward input sequence and a backward input sequence. Each branch accepts time-series data from the weighted standardized data matrix T.

[0081] Furthermore, the method for establishing a bidirectional recurrent neural network model includes: dividing the weighted standardized data matrix T into a training dataset and a validation dataset according to the time series order; the training dataset occupies the first part of the historical time window; the validation dataset occupies the second part of the historical time window; during the forward propagation of the bidirectional recurrent neural network model, the sparse node correlation matrix M' is used as the attention weight matrix to adjust the contribution of different node features to the prediction result; the forward hidden layer and the backward hidden layer process the forward and backward information of the time series data, respectively; through the backpropagation algorithm of the bidirectional recurrent neural network model, the connection weight parameters and bias parameters between network layers are updated using the training dataset until the prediction error of the model on the validation dataset converges to the preset error range.

[0082] S2.4: Use a bidirectional recurrent neural network model to perform forward calculations on the current time data of the weighted standardized data matrix T to generate the expected value vector P of the system state.

[0083] It should be noted that the system state expected value vector P includes the expected operating state parameters of each computing node at the next moment.

[0084] For example, the system calculates the Pearson correlation coefficient between nodes based on matrix T, finding that the correlation between Node1 and Node2 is 0.85 (strong correlation), while the correlation between Node1 and Node3 is 0.2 (weak correlation). After filtering by a preset threshold θ = 0.5, a sparse correlation matrix M' is generated, in which the correlation between Node1 and Node3 is set to zero. The bidirectional recurrent neural network model uses M' as attention weights to predict the expected value vector P of the system state at the next time step. For example, the expected CPU utilization of Node1 is 68%, which is close to the actual observed value of 65%.

[0085] In this embodiment of the application, step S3 includes:

[0086] S3.1: Subtract the current actual observation value of the weighted standardized data matrix T from the expected value vector P of the system state element by element to obtain the deviation vector D;

[0087] It should be noted that each element of the deviation vector D represents the prediction deviation of the corresponding node and indicator.

[0088] S3.2: Group the deviation vector D according to node type and resource index type, construct the deviation distribution matrix E, calculate the cumulative distribution function value of each element, and obtain the cumulative probability distribution matrix F;

[0089] It should be noted that the rows of the deviation distribution matrix E represent different computing nodes, and the columns represent the deviation distribution characteristics of different resource indicators; the cumulative probability distribution matrix F reflects the probability distribution characteristics of the resource indicator deviations of each node.

[0090] S3.3: Based on the cumulative probability distribution matrix F, the quantile regression method is used to calculate the set of dynamic confidence interval parameters G;

[0091] It should be noted that the set of dynamic confidence interval parameters G includes upper quantile boundary parameters and lower quantile boundary parameters.

[0092] S3.4: Based on the time series change trend of the deviation values ​​in the deviation distribution matrix E, calculate the adaptive adjustment factor H and obtain the upper boundary threshold vector U and the lower boundary threshold vector L;

[0093] Preferably, the adaptive adjustment factor H is used to dynamically correct the boundary parameters in the dynamic confidence interval parameter set G; the upper quantile boundary parameters of the dynamic confidence interval parameter set G are multiplied by the adaptive adjustment factor H to obtain the upper boundary threshold vector U; at the same time, the lower quantile boundary parameters of the dynamic confidence interval parameter set G are multiplied by the adaptive adjustment factor H to obtain the lower boundary threshold vector L.

[0094] S3.5: Based on the expected value vector P of the system state, the upper boundary threshold vector U, and the lower boundary threshold vector L, construct the set of anomaly identification boundary intervals I;

[0095] Preferably, the construction process of the anomaly identification boundary interval set I involves adding or subtracting each element in the system state expected value vector P with the corresponding elements in the upper boundary threshold vector U and the lower boundary threshold vector L; for the i-th resource indicator of the j-th computing node of the intelligent computing center, the upper boundary value of its normal operation interval is determined by P. i,j +U i,j The lower boundary value is obtained through calculation via P. i,j -U i,j The calculation yields the anomaly identification boundary interval [P] for this node and this indicator. i,j -U i,j ,P i,j +U i,j ].

[0096] It should be noted that the anomaly identification boundary interval set I includes the normal operation range of each node and each indicator. Its structure is presented in the form of a multi-dimensional matrix, with the row dimension corresponding to different computing nodes and the column dimension corresponding to different resource indicator types. Each matrix element contains the upper and lower boundary value pairs of the corresponding node indicator.

[0097] S3.6: Smooth the set of anomaly identification boundary intervals I to eliminate abrupt fluctuations in boundary parameters and obtain the set of anomaly identification boundary parameters J.

[0098] For example, the system compares the deviation between the actual observed value and the expected value to generate a deviation vector D. For instance, the memory usage deviation of Node2 is large, exceeding the 90th percentile of the historical distribution. Therefore, after the dynamic confidence interval is adjusted, its upper boundary threshold is set to +12%. The anomaly identification boundary set I finally determines that the normal range of Node2's memory usage is [50%, 74%].

[0099] In this embodiment of the application, step S4 includes:

[0100] S4.1: Compare the current data of the weighted standardized data matrix T with the anomaly identification boundary parameter set J to obtain the multi-source compensation node set N, where the anomaly identification boundary parameter set J includes the upper boundary threshold and the lower boundary threshold;

[0101] Preferably, when the actual weighted standardized data is greater than the upper boundary threshold of the anomaly identification boundary parameter set J or less than the lower boundary threshold of the anomaly identification boundary parameter set J, the location of the abnormal data is marked, an anomaly identification matrix K is generated, and a multi-source collaborative compensation mechanism is used for data reconstruction; when the actual weighted standardized data is less than or equal to the upper boundary threshold of the anomaly identification boundary parameter set J and greater than or equal to the lower boundary threshold of the anomaly identification boundary parameter set J, it is determined to be normal data.

[0102] Furthermore, a multi-source collaborative compensation mechanism is adopted for data reconstruction, including: extracting the node index and indicator index of abnormal data based on the anomaly identification matrix K, and constructing an abnormal data location index set Q; and selecting several normal nodes with the highest correlation with the abnormal data location based on the sparse node correlation matrix M', forming a multi-source compensation node set N.

[0103] It should be noted that the non-zero elements in the anomaly identification matrix K indicate that there is abnormal data at the corresponding position; the abnormal data position index set Q records the coordinate information of all data that needs to be repaired.

[0104] S4.2: For the corresponding index data of each node in the multi-source compensation node set N, assign weight coefficients according to the correlation values ​​in the sparse node correlation matrix M', and construct the compensation weight vector C;

[0105] It should be noted that the compensation weight vector C determines the contribution ratio of different compensation sources.

[0106] S4.3: Use the compensation weight vector C to perform a weighted average calculation on the corresponding index data in the multi-source compensation node set N to obtain the preliminary compensation value vector B;

[0107] S4.4: Based on the expected value at the corresponding position in the system state expected value vector P, the initial compensation value vector B is constrained and adjusted to obtain the compensation value vector A;

[0108] S4.5: Replace the data at the outlier positions in the weighted normalized data matrix T with the corresponding values ​​in the compensation value vector A to obtain the repaired weighted normalized data matrix T”;

[0109] S4.6: Based on the expected value at the corresponding position in the system state expected value vector P, the initial compensation value vector B is constrained and adjusted to obtain the final compensation value vector A;

[0110] Ensure that the compensated data meets the physical constraints of the system operation.

[0111] S4.7: Perform inverse weight standardization on the repaired weighted standardized data matrix T” to obtain the repaired system operation data matrix Z.

[0112] Preferably, the inverse weight standardization process includes dividing each element of the repaired weighted standardized data matrix T” by the corresponding weight coefficient in the dynamic weight coefficient set W, then multiplying by the historical standard deviation and adding the historical mean.

[0113] For example, if the current storage I / O response time of Node3 is 25ms, which exceeds its anomaly identification boundary [10ms, 20ms], then multi-source collaborative compensation is triggered; Node1 and Node2 are selected as compensation nodes according to the correlation matrix M', and their storage I / O data (18ms and 16ms) are weighted and averaged according to weights (0.6 and 0.4) to generate a compensation value of 19ms; inverse weight normalization processing restores the compensation value to the actual repair value, and outputs the repaired system operation data matrix Z.

[0114] In summary, this invention achieves unified quantitative representation and normalization of heterogeneous system operation data of different types and scales by collecting system operation data from several computing nodes in an intelligent computing center and performing dynamic weighting and standardization based on resource consumption ratios. This solves the technical challenge of directly comparing and comprehensively analyzing diverse heterogeneous resource data in intelligent computing centers. Furthermore, by constructing a node correlation matrix based on the Pearson correlation coefficient and using a bidirectional recurrent neural network model to generate expected system state values, this invention achieves deep learning modeling of the complex spatiotemporal dependencies between nodes in an intelligent computing center and intelligent prediction of system operation trends. This overcomes the technical limitation of traditional static threshold methods in accurately capturing the dynamic correlation characteristics and temporal evolution patterns between nodes. Finally, by using expected system state values… The deviation distribution from the current weighted standardized data is determined by an adaptive interval estimation algorithm to identify anomalies. This enables dynamic adaptive adjustment and quantile regression optimization of anomaly identification thresholds under different periods, load conditions, and business scenarios, solving the technical defects of high false alarm and false negative rates of fixed threshold methods in complex and variable operating environments. By adopting a multi-source collaborative compensation mechanism to intelligently reconstruct data exceeding the anomaly identification boundary and outputting repaired system operation data through inverse weighted standardization, accurate repair of anomaly data based on the correlation information of multiple normal nodes and complete restoration of the original data format are achieved. This solves the technical problems of insufficient repair accuracy from a single data source, mismatched repair data formats, and difficulty in ensuring data consistency.

[0115] This embodiment also provides a computer device applicable to the adaptive system data inspection and anomaly repair method for intelligent computing centers, including a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the adaptive system data inspection and anomaly repair method for intelligent computing centers as proposed in the above embodiment.

[0116] The computer device can be a terminal, comprising a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0117] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for adaptive system data inspection and anomaly repair in intelligent computing centers, characterized in that: include, System operation data from several computing nodes in the intelligent computing center are collected, and the system operation data is dynamically weighted and standardized based on the resource consumption ratio to obtain a weighted standardized data matrix. A node correlation matrix is ​​constructed based on the weighted standardized data matrix, and the expected value of the system state is generated through a bidirectional recursive neural network model. Based on the deviation distribution between the expected system state value and the current weighted standardized data, an adaptive interval estimation algorithm is used to determine the anomaly identification boundary. For data that exceeds the anomaly identification boundary, a multi-source collaborative compensation mechanism is used to reconstruct the data, and the repaired system operation data is output through inverse weight standardization processing. The current time data of the weighted standardized data matrix T is compared with the anomaly identification boundary parameter set J to obtain the multi-source compensation node set N, wherein the anomaly identification boundary parameter set J includes an upper boundary threshold and a lower boundary threshold. For the corresponding index data of each node in the multi-source compensation node set N, weight coefficients are assigned according to the correlation values ​​in the sparse node correlation matrix M' to construct a compensation weight vector C; The preliminary compensation value vector B is obtained by using the compensation weight vector C to perform a weighted average calculation on the corresponding index data in the multi-source compensation node set N; Based on the expected value at the corresponding position in the system state expected value vector P, the initial compensation value vector B is constrained and adjusted to obtain the compensation value vector A; Replace the data at the abnormal positions in the weighted normalized data matrix T with the corresponding values ​​in the compensation value vector A to obtain the repaired weighted normalized data matrix T''. The repaired weighted normalized data matrix T'' is subjected to inverse weighted normalization to obtain the repaired system operation data matrix Z; The deviation vector D is obtained by subtracting the current actual observation value from the system state expected value vector P in the weighted standardized data matrix T element by element. The deviation vector D is grouped and arranged according to node type and resource index type to construct the deviation distribution matrix E. The cumulative distribution function value of each element is calculated to obtain the cumulative probability distribution matrix F. Based on the cumulative probability distribution matrix F, the dynamic confidence interval parameter set G is calculated using the quantile regression method; Based on the time series change trend of the deviation values ​​in the deviation distribution matrix E, calculate the adaptive adjustment factor H and obtain the upper boundary threshold vector U and the lower boundary threshold vector L. Based on the system state expected value vector P and the upper boundary threshold vector U and the lower boundary threshold vector L, an anomaly identification boundary interval set I is constructed. The anomaly identification boundary interval set I is smoothed to obtain the anomaly identification boundary parameter set J; The method for generating the system state expected value vector P is as follows: Based on the weighted standardized data matrix T, the Pearson correlation coefficient of each computing node is calculated to obtain the correlation coefficient between nodes; The correlation coefficients between nodes are arranged in order of node number to construct a node correlation matrix M, wherein the node correlation matrix M is a symmetric matrix; The node correlation matrix M is subjected to threshold filtering to obtain a sparse node correlation matrix M', and a bidirectional recurrent neural network model is constructed. The bidirectional recurrent neural network model is used to perform forward calculation on the current time data of the weighted standardized data matrix T to generate the expected value vector P of the system state; The method for obtaining the weighted standardized data matrix T is as follows: The system operation data is obtained by data acquisition agent programs deployed on each computing node of the intelligent computing center; The system operation data is arranged chronologically according to the collection timestamps to construct the original data matrix R; Calculate the total resource consumption of the computing nodes in the original data matrix R within the current time window to form a resource consumption vector V; Based on the consumption ratio of each resource in the resource consumption vector V, the data in the original data matrix R is weighted and normalized using the resource consumption vector V to obtain the weighted standardized data matrix T.

2. The adaptive system data inspection and anomaly repair method for intelligent computing centers as described in claim 1, characterized in that: The current time data of the weighted standardized data matrix T is compared with the anomaly identification boundary parameter set J, including: When the actual weighted standardized data is greater than the upper boundary threshold of the anomaly identification boundary parameter set J, or when the actual weighted standardized data is less than the lower boundary threshold of the anomaly identification boundary parameter set J, the location of the abnormal data is marked, an anomaly identification matrix K is generated, and a multi-source collaborative compensation mechanism is used for data reconstruction. When the actual weighted standardized data is less than or equal to the upper boundary threshold of the anomaly identification boundary parameter set J and greater than or equal to the lower boundary threshold of the anomaly identification boundary parameter set J, it is determined to be normal data.

3. The adaptive system data inspection and anomaly repair method for intelligent computing centers as described in claim 2, characterized in that: The data reconstruction using a multi-source collaborative compensation mechanism includes: Based on the anomaly identification matrix K, the node index and indicator index of the anomaly data are extracted, and the anomaly data location index set Q is constructed. Based on the sparse node correlation matrix M', select several normal nodes with the highest correlation to the location of abnormal data to form a multi-source compensation node set N.

4. The adaptive system data inspection and anomaly repair method for intelligent computing centers as described in claim 1, characterized in that: The threshold filtering process for the node correlation matrix M includes: If the correlation value of the node correlation matrix M is greater than the preset correlation threshold θ, then the correlation value is retained in the corresponding position of the sparse matrix to maintain the original correlation strength between nodes; if the correlation value of the node correlation matrix M is less than or equal to the preset correlation threshold θ, then the correlation value is set to zero in the corresponding position of the sparse matrix to obtain the sparse node correlation matrix M'.

5. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, it implements the steps of the adaptive system data inspection and anomaly repair method for intelligent computing centers as described in any one of claims 1 to 4.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the steps of the adaptive system data inspection and anomaly repair method for intelligent computing centers as described in any one of claims 1 to 4.

Images