Operation and maintenance information processing method and system for information system

Abstract

The invention relates to the technical field of intelligent operation and maintenance data processing, in particular to an operation and maintenance information processing method and system for an information system. An operation and maintenance information processing method for an information system comprises the following steps: S1, acquiring time sequence index data, data point identification data and abnormal data point data, and constructing a sequence data set according to the time sequence index data and the data point identification data; and calculating the linear correlation strength of the sequence data set, and determining a first total correlation and a second total correlation according to the linear correlation strength. According to the invention, a multi-dimensional fault assessment system is constructed through intelligent acquisition and correlation analysis, and accurate monitoring of the operation and maintenance state is realized; based on an influence quantification and type identification dynamic optimization strategy, a historical library and real-time resource adaptive matching are combined, backspacing and compression are intelligently adjusted, a closed-loop decision mechanism is formed, and system reliability and edge adaptability are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent operation and maintenance data processing, and particularly relates to an operation and maintenance information processing method and system for an information system. BACKGROUND

[0002] With the continuous improvement of the scale and complexity of information systems, intelligent operation and maintenance information processing methods have gradually become the key support for ensuring the stable operation of systems. The existing technology significantly improves operation and maintenance efficiency and system reliability through real-time monitoring, anomaly detection and automatic response mechanisms, and has made positive progress in automatic deployment and fault recovery, providing an effective means for large-scale cluster management.

[0003] However, the existing methods still have obvious deficiencies in dealing with multi-constraint scenarios of edge devices. Although traditional methods focus on processing efficiency and time cost, they are mostly based on preset rules and fixed strategies, and lack the ability to analyze the correlation of data, resource status and fault types in coordination, making it difficult to achieve accurate fault decision-making and resource adaptive scheduling under complex conditions such as network fluctuations, storage fullness and abnormal concurrency, thereby affecting the overall robustness and response real-time performance of the system.

[0004] Therefore, it is urgent to study an intelligent operation and maintenance information processing method for multi-constraint environments to overcome the limitations of existing strategies in adaptability, correlation maintenance and resource optimization, and to improve the operation and maintenance performance and reliability of information systems in complex scenarios. SUMMARY

[0005] In order to overcome the poor dynamic adaptive ability, the present application provides an operation and maintenance information processing method and system for an information system.

[0006] The technical implementation scheme of the present application is: an operation and maintenance information processing method for an information system, comprising the following steps: S1: acquiring time series index data, data point identification data and abnormal data point data, constructing a sequence data set according to the time series index data and the data point identification data; calculating the linear correlation strength of the sequence data set, and determining a first total correlation and a second total correlation according to the linear correlation strength; S2: taking the absolute difference between the first total correlation and the second total correlation as a correlation change amount, and determining a processing method for the abnormal data point data according to the correlation change amount; acquiring the characteristic attributes of the abnormal data point data according to the processing method of the abnormal data point data, and extracting the corresponding fault type according to the characteristic attributes, and determining an instantaneous fault impact range evaluation benchmark according to the fault type; S3: determining an impact range evaluation benchmark of the cumulative fault according to the fault type, determining a theoretical rollback range according to the impact range evaluation benchmark corresponding to the abnormal data point data, determining a maximum allowed rollback range according to a ratio of the remaining storage capacity to the average storage consumption of a single data point, and determining a compressed data packet according to the theoretical rollback range; S4: calculating storage space consumption of the compressed data packet, and determining a compressed rollback operation and a system restart operation according to the storage space consumption.

[0007] Preferably, the time series indicator data, data point identification data and abnormal data point data are obtained, and a sequence data set is constructed according to the time series indicator data and the data point identification data, including: The data point identification data refers to a timestamp or a serial number that uniquely identifies each data point; The sequence data set refers to a collection of time series indicator data points arranged in chronological order; The abnormal data point data refers to a collection of outlier data points suspected to be generated by faults or errors, which are pre-identified by an anomaly detection algorithm or rules; The time series indicator data refers to numerical operation and maintenance performance data collected at consecutive time points.

[0008] Preferably, the linear correlation strength of the sequence data set is calculated, and the first total correlation and the second total correlation are determined according to the linear correlation strength, including: The linear correlation strength of the sequence data set is calculated using the Pearson correlation coefficient and defined as the first correlation; If the first correlation is less than a preset linear threshold, the nonlinear correlation strength of the sequence data set is calculated using the normalized mutual information and defined as the second correlation; If the first correlation is greater than or equal to the preset linear threshold, the first correlation is taken as the total correlation, and if the first correlation is less than the preset linear threshold, the second correlation is taken as the total correlation; The total correlation of the sequence data set before excluding the abnormal data point data is calculated and defined as the first total correlation, and the total correlation of the sequence data set after excluding the abnormal data point data is calculated and defined as the second total correlation.

[0009] Preferably, the absolute difference between the first total correlation and the second total correlation is taken as the correlation change amount, and the processing method of the abnormal data point data is determined according to the correlation change amount, including: If the correlation change amount is less than a preset change amount threshold, it is determined that the abnormal data point data has no impact on the overall correlation, and the abnormal data point data is selected to be skipped; If the change in correlation is greater than or equal to a preset change threshold, it is determined that the abnormal data point has an impact on the overall correlation, and the process is selected to proceed to the subsequent fault handling process.

[0010] Preferably, the step of obtaining the feature attributes of the abnormal data points according to the processing method of the abnormal data points, extracting the corresponding fault type according to the feature attributes, and determining the impact range assessment benchmark of the instantaneous fault according to the fault type includes: The fault types include transient faults and cumulative faults; If the fault type is a transient fault, the maximum impact range of the transient fault is obtained from the historical fault database as the impact range assessment benchmark. The transient fault refers to the type of fault characterized by consecutive abnormal data points affected after the abnormal data point.

[0011] Preferably, the step of determining the impact range assessment benchmark of the cumulative fault based on the fault type includes: If the fault type is a cumulative fault, the assessment strategy is dynamically determined based on the current remaining capacity, the normalized remaining capacity is calculated, and the result is obtained using the formula... Calculate the percentile of historical impact range; in, Percentile of historical impact range For the aggressive strategy percentile threshold, For conservative strategy percentile threshold, The attenuation coefficient is... To normalize the remaining capacity; The impact range value corresponding to the cumulative fault at the historical impact range percentile is obtained from the historical fault database based on the historical impact range percentile as the impact range assessment benchmark. The cumulative fault refers to the fault type represented by a series of consecutive abnormal data points from the start of the abnormal data point until the system recovers. A time-series sequence set is constructed by sorting the sequence dataset in chronological order according to the timestamp attribute. Based on the time series set, calculate the remaining capacity of the current sequence dataset, where the remaining capacity refers to the number of data points from the current abnormal data point to the end of the sequence; If the remaining capacity is compared with the determined impact range assessment benchmark, and the remaining capacity is greater than or equal to the impact range assessment benchmark, then the current data batch will continue to be uploaded. If the remaining capacity is less than the impact range assessment benchmark, the upload will be stopped and the subsequent fault handling process will begin.

[0012] Preferably, the step of determining the theoretical rollback range based on the impact range assessment benchmark corresponding to the abnormal data point, determining the maximum allowable rollback range based on the ratio of remaining storage capacity to average storage consumption per data point, and determining the compressed data packet based on the theoretical rollback range includes: If the maximum allowable rollback range is greater than or equal to the theoretical rollback range, then a full rollback operation is performed, and the rollback step size is set to the theoretical rollback range; If the maximum allowable rollback range is less than the theoretical rollback range, then the data compression process is initiated to calculate the correlation matrix of data points within the theoretical rollback range; The data segments in the correlation matrix that meet the preset similarity threshold are compressed and encoded. Data points in the correlation matrix that do not meet the preset similarity threshold are processed to retain their original format. The compressed encoding process is combined with the data after the original format preservation process to form a compressed data packet.

[0013] Preferably, the step of calculating the storage space consumption of the compressed data packet and determining the compression rollback operation and system restart operation based on the storage space consumption includes: If the storage space consumed by the compressed data packet exceeds the remaining storage capacity, the system restart process is triggered: reconstruct the system state snapshot based on the key data points in the compressed data packet, store the system state snapshot in persistent storage, perform the system restart operation, and restore the service state based on the system state snapshot after the restart is completed; If the storage space consumption of the compressed data packet is less than or equal to the remaining storage capacity, the real-time response prediction time is obtained according to the real-time response time prediction formula. If the real-time response prediction time is less than or equal to the average system restart time, then the compression rollback operation is selected. If the predicted real-time response time is greater than the average system restart time, then the system restart operation is selected.

[0014] Preferably, if the storage space consumption of the compressed data packet is less than or equal to the remaining storage capacity, then the real-time response prediction time is obtained according to the real-time response time prediction formula, including: the real-time response time prediction formula is as follows, ; ; in, For real-time response time, This is the theoretical backoff range. For resource adaptation coefficient, For sensitivity parameters, To handle the efficiency constant, For the remaining storage capacity, This represents the average storage consumption per data point.

[0015] Preferably, an operation and maintenance information processing system for an information system includes: The time series correlation analysis module is used to acquire time series indicator data, data point identification data, and outlier data point data, construct a time series dataset, calculate the linear and nonlinear correlation strengths, and generate the first total correlation and the second total correlation. The anomaly impact assessment module is used to calculate the correlation change and determine the impact of abnormal data points, extract the fault type and classify it into instantaneous faults or cumulative faults, and determine the specific impact range assessment benchmark based on the historical fault database or dynamic strategy. The rollback range decision module is used to determine the theoretical rollback range based on the impact range assessment benchmark, calculate the maximum allowable rollback range in combination with the remaining storage capacity, perform correlation analysis and compression encoding on the data within the theoretical rollback range, and generate the final compressed data packet; The operation decision execution module is used to calculate the storage space consumption of compressed data packets and predict the real-time response time. By comparing the remaining storage capacity with the average system restart time, it decides whether to execute a compression rollback operation or a system restart operation.

[0016] Beneficial Effects: This invention constructs a multi-dimensional fault assessment system through intelligent collection and correlation analysis of time-series indicator data, data point identification data, and abnormal data point data, enabling precise monitoring and dynamic response to the operational status of information systems. Based on the quantitative analysis of correlational changes, the invention quantifies the impact of faults and dynamically optimizes processing strategies through fault type identification and impact range assessment mechanisms, significantly improving the accuracy of fault diagnosis and resource utilization efficiency. By leveraging the adaptive matching of historical fault databases and real-time resource status, the invention intelligently adjusts the rollback scope and compression strategies, ensuring a high degree of coordination between operational processing and system resource status. Through intelligent decision-making mechanisms for compression rollback and system restart, a closed-loop optimized operational processing system is formed, ultimately improving the reliability of information systems while enhancing the adaptability of edge devices in multi-constraint environments, and optimizing operational efficiency and resource scheduling effectiveness. Attached Figure Description

[0017] Figure 1 This is a flowchart of the operation and maintenance information processing method for information systems according to the present invention; Figure 2 This is a structural diagram of the operation and maintenance information processing system for information systems according to the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] Example 1: A method for processing operation and maintenance information for information systems, such as... Figure 1 As shown, it includes the following steps: S1-1: Obtain time series indicator data, data point identification data, and abnormal data point data; construct a sequence dataset based on the time series indicator data and the data point identification data. The data point identifier data refers to the timestamp or sequence number that uniquely identifies each data point. The sequence dataset refers to a collection of time series indicator data points arranged in chronological order. The anomalous data points refer to the set of outlier data points that are pre-identified by anomaly detection algorithms or rules and are suspected to be caused by faults or errors. The time series index data refers to numerical operation and maintenance performance data collected at continuous time points.

[0020] It should be noted that step S1-1 of this invention aims to solve the problems of delayed fault response and misjudgment caused by discrete data and lack of temporal correlation in existing operation and maintenance systems. This step provides structured input for subsequent intelligent fault decision-making by systematically collecting and integrating multi-source operation and maintenance data.

[0021] Specifically, acquiring time-series metric data refers to collecting performance values ​​at continuous time points in real time from various monitoring agents of the information system, such as CPU utilization, memory usage, or network throughput collected per second. Data point identification data uses high-precision timestamps or globally auto-incrementing sequence numbers to ensure that each data point has unique traceability. Abnormal data points are identified using a pre-defined isolated forest algorithm. This algorithm automatically marks outliers exceeding a threshold range by calculating the degree of deviation of data points from the overall distribution; for example, a CPU utilization exceeding 95% for three consecutive time points is considered abnormal.

[0022] Based on time-series index data and data point identifier data, a sequence dataset is constructed in chronological order. For example, 500 CPU utilization data points collected within 5 minutes are sorted by millisecond-level timestamps to form an ordered dataset. The core function of constructing a sequence dataset is to establish the temporal correlation between data points, enabling the system to analyze the data evolution patterns before and after a fault occurs. This overcomes the limitations of traditional methods that rely solely on isolated data points to determine faults, laying the foundation for subsequent correlation analysis and accurate fault location.

[0023] S1-2: Calculate the linear association strength of the sequence dataset, and determine the first total association and the second total association based on the linear association strength; The linear association strength of the sequence dataset is calculated using the Pearson correlation coefficient and defined as the first association. If the first correlation is less than a preset linear threshold, the nonlinear correlation strength of the sequence dataset is calculated using normalized mutual information and defined as the second correlation. If the first correlation is greater than or equal to a preset linear threshold, the first correlation is taken as the total correlation; if the first correlation is less than the preset linear threshold, the second correlation is taken as the total correlation. The total correlation of the sequence dataset before removing the abnormal data points is calculated and defined as the first total correlation, and the total correlation of the sequence dataset after removing the abnormal data points is defined as the second total correlation.

[0024] It should be noted that steps S1-2 of this invention address the problem of high computational complexity and difficulty in real-time response of traditional fault detection methods in resource-constrained edge device environments by proposing a hierarchical correlation analysis mechanism. This step first uses the computationally efficient Pearson correlation coefficient to evaluate the linear correlation strength of the sequence dataset, as shown in the formula: ,in, The calculated Pearson correlation coefficient is a dimensionless statistic with a range of [-1, 1], used to precisely quantify the degree of linear correlation between two variables. Represents a time sequence number. This represents the operation and maintenance indicator value corresponding to the time sequence number. and The mean, This represents the number of sequence data points. For example, when the calculation result is higher than the preset threshold of 0.7, linear correlation is directly used as the total correlation; when the first correlation is lower than the preset threshold, it indicates the existence of a nonlinear pattern, but considering the limited computing power of edge devices, only normalized mutual information is used to supplement the analysis of key data segments.

[0025] By calculating the overall correlation before and after removing outlier data points (the first overall correlation reflects the complete system state, and the second overall correlation characterizes the state after the outlier removal), the impact of the outlier on the overall data correlation is accurately quantified. If the difference between the two is less than a threshold, it indicates that the outlier is an isolated event and can be skipped without affecting data integrity; if the difference is significant, it needs to proceed to the subsequent fault type analysis process. This design effectively solves the contradiction between ensuring analytical accuracy and controlling computational complexity under resource constraints for edge devices, providing a reliable basis for subsequent intelligent decision-making.

[0026] It should be noted that the preset linear threshold is usually set based on the statistical significance level or empirical values ​​for specific application scenarios, and is used to determine how well the linear model describes the data relationship. For example, in the field of operation and maintenance, a value of 0.7 (representing strong correlation) or higher is often used to adopt linear correlation, while a value below this is required to initiate nonlinear analysis.

[0027] S2-1: The absolute difference between the first total correlation and the second total correlation is taken as the correlation change amount, and the processing method of the abnormal data point data is determined according to the correlation change amount; If the change in correlation is less than a preset threshold, it is determined that the abnormal data point has no impact on the overall correlation, and the abnormal data point is skipped. If the change in correlation is greater than or equal to a preset change threshold, it is determined that the abnormal data point has an impact on the overall correlation, and the process is selected to proceed to the subsequent fault handling process.

[0028] It should be noted that step S2-1 of this invention addresses the deficiency in existing operation and maintenance systems that lack quantitative evaluation standards when processing abnormal data, making it difficult to accurately determine the degree of fault impact. By introducing the concept of correlation change, a scientific fault decision-making mechanism is established. This step calculates the absolute difference between the first total correlation and the second total correlation, aiming to quantify the degree of damage caused by abnormal data points to the overall data correlation structure. The absolute difference eliminates the influence of positive and negative deviations canceling each other out, ensuring the accuracy of the evaluation results.

[0029] The correlation change value characterizes the change in the correlation strength between anomaly data points and the overall dataset. When this value is less than a preset threshold (usually set in the range of 0.15-0.25 based on historical data statistical analysis), it indicates that the anomaly is an isolated event with weak correlation to preceding and following data, and skipping its processing will not compromise data integrity. When the correlation change value is greater than or equal to the threshold, it indicates a strong correlation between the anomaly and the dataset, which could trigger a chain reaction of failures, necessitating further analysis in the subsequent failure handling process. This decision-making mechanism based on quantitative indicators effectively solves the randomness problem of traditional methods relying on experience-based judgment, providing a reliable data processing basis for edge devices in resource-constrained environments.

[0030] It should be noted that the preset change threshold is determined by performing extensive simulations and statistical analysis on the distribution of "correlated changes" in historical normal data sequences. Typically, the high percentile value of this distribution (such as the 95th or 99th percentile) is selected as the threshold to ensure that the fault handling process is only triggered when the correlated change significantly exceeds the normal fluctuation range.

[0031] S2-2: Obtain the characteristic attributes of the abnormal data points according to the abnormal data point data processing method, extract the corresponding fault type according to the characteristic attributes, and determine the impact range assessment benchmark of the instantaneous fault according to the fault type; The fault types include transient faults and cumulative faults; If the fault type is a transient fault, the maximum impact range of the transient fault is obtained from the historical fault database as the impact range assessment benchmark. The transient fault refers to the type of fault characterized by consecutive abnormal data points affected after the abnormal data point.

[0032] It should be noted that when the change in correlation is greater than or equal to a preset threshold, the abnormal data point that significantly affects the overall data correlation cannot be skipped directly. At this point, the in-depth analysis stage is entered, extracting the multidimensional feature attributes of the abnormal data point (such as numerical deviation, duration, and change gradient), and accurately identifying the fault type (instantaneous fault or cumulative fault) based on these feature attributes, providing a basis for subsequently formulating precise fault handling strategies.

[0033] Step S2-2 of this invention addresses the problem of coarse fault classification and lack of accurate evaluation benchmarks in existing operation and maintenance systems by establishing a fine-grained fault feature analysis mechanism to improve fault handling accuracy. This step extracts multi-dimensional feature attributes, including numerical deviation, duration, and rate of change, from abnormal data points and uses a machine learning-based classification algorithm to distinguish between transient and cumulative faults. Transient faults refer to sudden, short-duration anomalies, such as transient network congestion; cumulative faults refer to progressive, continuously deteriorating system problems, such as the gradual depletion of storage space.

[0034] When a transient fault is identified, the maximum impact range of that type of fault is obtained from the historical fault database (a pre-built database storing the characteristics and quantified impact range data of various historical fault cases. Its core function is to provide an evaluation benchmark based on historical experience for newly detected faults, thereby achieving accurate prediction of fault impact and adaptive optimization of handling strategies) as the evaluation benchmark. This benchmark represents the maximum cascading effect range caused by this type of fault. For example, the historical maximum impact range of a certain type of network transient fault is the subsequent five consecutive abnormal data points, which is used as the criterion for determining the impact boundary of the current fault. This evaluation method based on historical maximum values ​​ensures that the system retains sufficient safety margin and avoids underestimating the impact of faults. The impact of a transient fault is specifically manifested as the number of subsequent contaminated consecutive data points starting from the abnormal data point. This quantitative indicator provides accurate data support for subsequent decisions and effectively solves the problem that traditional methods are too conservative or aggressive in assessing the impact of faults.

[0035] S3-1: Determine the assessment benchmark for the impact range of cumulative faults based on the fault type described; If the fault type is a cumulative fault, the assessment strategy is dynamically determined based on the current remaining capacity, the normalized remaining capacity is calculated, and the result is obtained using the formula... Calculate the percentile of historical impact range; It should be noted that the formula is based on As a benchmark value, ( It defines the maximum dynamic range of the strategy from "aggressive" to "conservative" and quantifies the maximum "resource margin" cost that the system can pay in pursuit of ultimate reliability. It is a resource that varies with the remaining resources An increasing but exponentially decaying adjustment factor. Its function is: when resources are scarce ( When the value is small, the factor approaches 1, making the strategy... Approaching conservative The aim is to maximize the success rate by utilizing sufficient safety margins, prioritizing system reliability; when resources are sufficient ( When the value is large, the factor approaches 0, making the strategy... Approaching radical The aim is to achieve higher resource utilization efficiency and processing speed while accepting controllable marginal risks. The entire formula uses an exponential decay function to normalize the real-time resource state. Smoothly and adaptively mapped to a state between (Radical) and Optimal strategy value between (conservative) This achieves an optimal trade-off between resource constraints and fault handling integrity.

[0036] in, Percentile of historical impact range For the aggressive strategy percentile threshold, For conservative strategy percentile threshold, The attenuation coefficient is... To normalize the remaining capacity; It should be noted that the percentile threshold for the aggressive strategy ( The threshold for conservative strategies is typically set based on the system's sensitivity to resource constraints. By analyzing historical failure data, a low percentile (such as P10) is selected to prioritize system availability during resource-constrained periods. The setting of the aggressive strategy focuses on the completeness of fault repair. By selecting a high percentile (such as P90 or P95) of the historical fault impact range, it ensures that there is sufficient safety margin to fully cover the potential fault impact. The aggressive strategy percentile threshold ( The percentile value represents the lower bound of acceptable risk for the system under resource constraints. Choosing this strategy implies prioritizing resource efficiency over the completeness of fault handling. The conservative strategy percentile threshold (…) ) is a high percentile value, representing the safety margin reserved by the system to ensure maximum reliability. Choosing this strategy means prioritizing ensuring that faults are fully accommodated rather than saving resources.

[0037] The impact range value corresponding to the cumulative fault at the historical impact range percentile is obtained from the historical fault database based on the historical impact range percentile as the impact range assessment benchmark. The cumulative fault refers to the fault type represented by a series of consecutive abnormal data points from the start of the abnormal data point until the system recovers. A time-series sequence set is constructed by sorting the sequence dataset in chronological order according to the timestamp attribute. Based on the time series set, calculate the remaining capacity of the current sequence dataset, where the remaining capacity refers to the number of data points from the current abnormal data point to the end of the sequence; If the remaining capacity is compared with the determined impact range assessment benchmark, and the remaining capacity is greater than or equal to the impact range assessment benchmark, then the current data batch will continue to be uploaded. If the remaining capacity is less than the impact range assessment benchmark, the upload will be stopped and the subsequent fault handling process will begin.

[0038] It should be noted that step S3-1 of this invention addresses a specific scenario in the continuous data processing of edge devices: when the system is processing the current batch of data and is about to complete, an abnormal fault is detected, while the next batch of data is waiting to be uploaded. At this point, it is crucial to accurately assess the relationship between the fault's impact and remaining resources. Traditional methods employ fixed assessment strategies, which cannot adapt to this dynamic resource environment, often leading to improper processing of the current batch or affecting subsequent data processing.

[0039] This step addresses this issue through a dynamic evaluation mechanism: when accumulated faults are detected, the normalized remaining capacity is first calculated. This reflects the current scarcity of available storage resources. (The formula is used to express this information.) Calculate the historical impact percentile, including the exponential decay term. Intelligent switching of assessment strategies has been implemented—when remaining capacity is strained. (Small value), the calculation result quickly approaches the conservative strategy Employing the maximum safety margin ensures that faults can be fully handled, prioritizing system operational safety under resource constraints; when remaining capacity is sufficient ( (Large value), the calculation result approaches that of an aggressive strategy. While ensuring basic reliability, a small safety margin is adopted, and resources are released first to ensure the data throughput and processing efficiency of the system to continue to run.

[0040] This dynamic assessment is based on the premise that if the estimated impact of a fault exceeds the remaining capacity, a conservative strategy will exhaust resources and cause system crashes; conversely, if the impact is much smaller than the remaining capacity, an aggressive strategy will ignore important fault characteristics. By comparing the remaining capacity with the dynamically acquired impact assessment benchmark in real time, the system can make precise decisions—continuing to upload the current batch or aborting processing to reserve resources for subsequent data. This maximizes resource utilization while ensuring system continuity, effectively solving the resource conflict problem for edge devices in multi-batch data processing. The core of this design lies in preventative resource management—devices must make decisions based on real-time resources (not resources to be released in the future) because new data streams continuously flow in and may contain new anomalies, putting resources in a state of dynamic competition. By increasing P as R decreases (i.e., the more scarce the resources, the larger the P value), a more conservative impact assessment benchmark (such as a higher percentile) is adopted. This makes it easier to meet the condition 'remaining capacity < impact assessment benchmark,' thus causing the system to abort uploading the current data batch. After the upload is stopped, the subsequent fault handling process (including rollback range decision and operation decision) will be initiated, which will eventually trigger an earlier system restart or compression rollback operation to prioritize the continuous operation of the system and avoid resource exhaustion that could lead to failure in the middle of the process.

[0041] It should be noted that the "normalized residual capacity" mentioned in this invention... It is not an absolute quantity, but a standardized dimensionless ratio, calculated as follows: ,in, This represents the number of data points from the current abnormal data point to the end of the sequence (i.e., the remaining data capacity). This represents the total number of data points in the entire sequence dataset. This definition ensures... The range of values ​​is strictly within the interval [0,1]. =1 indicates that the sequence data has just started and there is sufficient remaining capacity; A value of 0 indicates that the sequence data has been processed and the remaining capacity has been exhausted. This normalization process eliminates evaluation bias caused by differences in the absolute length of the data sequence, resulting in a lower decay coefficient. This becomes a universal sensitivity adjustment parameter that is unaffected by system size, greatly enhancing the adaptability and portability of the evaluation strategy.

[0042] S3-2: Determine the theoretical rollback range based on the impact range assessment benchmark corresponding to the abnormal data point data, determine the maximum allowable rollback range based on the ratio of remaining storage capacity to average storage consumption per data point, and determine the compressed data packet based on the theoretical rollback range; It should be noted that the impact range assessment benchmark is a predicted quantity extracted from the historical failure database and optimized by the strategy. It is used to characterize the number of abnormal data points that are expected to be affected by a specific type of failure, and is the core basis for subsequent resource allocation and recovery strategy decisions.

[0043] If the maximum allowable rollback range is greater than or equal to the theoretical rollback range, then a full rollback operation is performed, and the rollback step size is set to the theoretical rollback range; If the maximum allowable rollback range is less than the theoretical rollback range, then the data compression process is initiated to calculate the correlation matrix of data points within the theoretical rollback range; The data segments in the correlation matrix that meet the preset similarity threshold are compressed and encoded. Data points in the correlation matrix that do not meet the preset similarity threshold are processed to retain their original format. The compressed encoding process is combined with the data after the original format preservation process to form a compressed data packet.

[0044] It should be noted that step S3-2 of this invention builds upon the fault impact range assessment benchmark determined in the previous step, optimizing the rollback strategy for edge devices processing historical data backlogs in resource-constrained environments. Traditional methods use fixed rollback step sizes, which cannot adapt to dynamically changing storage states, often leading to rollback operations exhausting resources or failing to fully cover the fault impact range.

[0045] This step first determines the theoretical rollback range based on the impact assessment benchmark corresponding to the abnormal data point. This range represents the amount of data that needs to be reprocessed to ensure system consistency. Simultaneously, it calculates the maximum allowable rollback range based on the ratio of remaining storage capacity to the average storage consumption per data point. This ratio reflects the maximum number of rollback steps that currently available resources can support. When the maximum allowable rollback range is greater than or equal to the theoretical rollback range, it indicates sufficient resources. A complete rollback operation is then performed, with the rollback step size set to the theoretical rollback range to ensure complete coverage of the fault-affected area.

[0046] When the maximum allowable rollback range is less than the theoretical rollback range, a data compression process is initiated. The correlation matrix of data points within the theoretical rollback range is constructed by calculating the Pearson correlation coefficient to quantify the similarity between data segments. Data segments with similarity exceeding a preset threshold (typically set to 0.8-0.9) are compressed using differential encoding, storing only the changes. Key data points with similarity below the threshold retain their original format to ensure data integrity. Finally, the compressed encoded data is combined with the retained original data to form a compressed data packet. For example, 10 similar data segments are compressed into a combination of one baseline data point and nine difference values, maximizing data information retention under limited resources. This dynamic rollback and compression mechanism effectively solves the contradiction between ensuring data integrity and controlling storage consumption under resource constraints for edge devices.

[0047] It should be noted that the construction process of the correlation matrix is ​​as follows: Determine the data range: From the sorted time series set, extract the data segment corresponding to the theoretical backtracking range. Let this data segment contain N data points, with a value set of V=[v1,v2,...,vN], where v1 is the earliest point and vN is the latest point. Define the matrix dimension: Construct an NxN symmetric matrix M, where N is the number of data points within the theoretical backtracking range. Calculate matrix elements: For each element M[i][j] in matrix M (where i,j are indices from 1 to N), its value is the Pearson correlation coefficient between data points vi and vj. Since it calculates the correlation between the values ​​of the same indicator at different times, the Pearson correlation coefficient calculation formula is simplified to measure the degree of linear correlation. The element M[i][i] on the main diagonal of matrix M represents the correlation between the data point and itself, and its value is always 1. Matrix output: Finally, output this NxN correlation coefficient matrix M as the basis for subsequent data compression.

[0048] It's important to note that the preset similarity threshold depends on the trade-off between the target fidelity and compression ratio of the data compression. Typically, through experimental evaluation, a correlation coefficient value is selected that achieves a considerable compression ratio while ensuring no loss of key data features; for example, a value between 0.8 and 0.9 indicates that compression is only applied to highly similar or repetitive data segments. Here, the "preset similarity threshold" refers to the absolute value of the Pearson correlation coefficient. That is, when |r| ≥ the threshold, the data segments are considered highly similar and compression is performed. This ensures that both positive and negative correlations, two strongly regular patterns, are captured simultaneously, thereby improving compression efficiency.

[0049] S4: Calculate the storage space consumption of the compressed data packet, and determine the compression rollback operation and system restart operation based on the storage space consumption.

[0050] If the storage space consumed by the compressed data packet exceeds the remaining storage capacity, the system restart process is triggered: reconstruct the system state snapshot based on the key data points in the compressed data packet, store the system state snapshot in persistent storage, perform the system restart operation, and restore the service state based on the system state snapshot after the restart is completed; It should be noted that remaining capacity refers to the number of remaining data points (unit: points) in the current data sequence to be processed, from the anomaly point to the end of the sequence. This is a logical quantity used to assess whether the impact of a fault exceeds the subsequent data processing capacity. Remaining storage capacity refers to the remaining available space on the device's physical storage medium (unit: KB, MB, or GB, etc.). This is a physical quantity used to determine whether there are sufficient hardware resources to perform operations such as rollback, compression, or storage snapshots. Both of these, from the perspectives of logical workload and physical resources, jointly ensure reliable decision-making by the system under constraints. Furthermore, the theoretical rollback range involved in the solution ( Both the maximum allowable rollback range and the unit for the maximum allowable rollback range are the number of data points (number of data points), and the average storage consumption per data point (number of data points). The unit is storage space / unit (e.g., KB / unit), and the real-time response time is... The units for system restart average time are time units (such as ms or s). All calculations must ensure that the units are consistent and the dimensions are uniform to guarantee the correctness of the decision-making algorithm and its comparability.

[0051] If the storage space consumption of the compressed data packet is less than or equal to the remaining storage capacity, the real-time response prediction time is obtained according to the real-time response time prediction formula. The formula for predicting real-time response time is as follows: ; ; in, For real-time response time, This is the theoretical backoff range. For resource adaptation coefficient, For sensitivity parameters, To handle the efficiency constant, For the remaining storage capacity, Average storage consumption per data point; It should be noted that the real-time response time prediction formula introduced in this invention solves the problem of lack of quantitative basis in the final decision-making stage. This formula is a piecewise function: when the resource adaptation coefficient... When the dimensionless coefficient is greater than 1 (resources are sufficient), a linear model is used. = Calculation is efficient, when When the value is greater than 1, it indicates that there are remaining storage resources ( ) far exceeds the basic resources required for theoretical rollback operations ( Resource contention no longer exists. In this state, the bottleneck for processing performance is no longer I / O wait or underlying resource conflicts such as memory swapping, but rather stabilizes at the basic rate of CPU computation and data transfer. Therefore, real-time response time... Withdrawal range It exhibits a simple linear proportional relationship. (Coefficient) This represents the average time taken to process a single data point in this device environment. The total time taken for the entire rollback process is simply the unit time multiplied by the number of data points, without the need to introduce complex nonlinear factors for correction. Using a linear model offers both accuracy and simplicity when resources are sufficient. It provides sufficiently accurate predictions (because the processing rate tends to stabilize) while avoiding the unnecessary computational overhead of exponential models, meeting the needs of efficient decision-making in edge devices; when… When ≤1 (resource scarcity), an exponential model is used. = It accurately reflects the nonlinear decay of processing efficiency under resource constraints, when When ≤1, it indicates that the remaining storage resources ( This is insufficient to adequately cover the resources required for the theoretical rollback operation. When the system enters a resource contention state, linear models cannot accurately depict the non-linear deterioration in processing efficiency caused by underlying contention mechanisms such as decreased cache hit rate, I / O wait, and garbage collection under resource stress. Therefore, an exponential model must be introduced. To simulate this accelerated performance degradation. Real-time response time. With resource adaptation coefficient It decreases but grows exponentially. Specifically, the changes are: When the value decreases from 1 (resources become tighter), the exponent term (1- ) increases, making The multiplier effect amplifies dramatically, leading to a significant increase in predicted response time. This change truly reflects the objective law that the marginal cost of processing each data point rises sharply under resource constraints. This exponential model provides a more accurate quantitative basis for system decision-making in resource-scarce scenarios. It ensures that the system can anticipate the significant delays caused by resource competition, thus favoring a more thorough recovery strategy of restarting rather than getting bogged down in slow rollback operations, ultimately guaranteeing the system's real-time objective. The system compares the calculated... Compared to the average system restart time, scientific decision-making and execution of compressed rollback ( ≤ Average system restart time) or system restart ( The average system restart time allows for an optimal balance between processing efficiency and resource utilization.

[0052] If the real-time response prediction time is less than or equal to the average system restart time, then the compression rollback operation is selected. If the predicted real-time response time is greater than the average system restart time, then the system restart operation is selected.

[0053] It should be noted that step S4 of this invention addresses the final decision-making problem faced by edge devices during fault handling. Through an intelligent resource assessment and response time prediction mechanism, it solves the inefficiency problem caused by fixed strategies in traditional methods. This step first calculates the storage space consumption of the compressed data packet, i.e., the actual storage size occupied by the compressed data, and compares it with the current remaining storage capacity.

[0054] When the compressed data packet storage space consumption exceeds the remaining storage capacity, it indicates that the system cannot support the resources required for the rollback operation, triggering the system restart process. The system extracts key data points (i.e., core indicator data affecting the system state) from the compressed data packet, reconstructs a system state snapshot (a system image containing key information such as service status and operating parameters), stores the system state snapshot in persistent storage (such as solid-state drives or flash non-volatile storage media), then performs a system restart and restores the service state based on the snapshot after completion, ensuring that the system quickly returns to a stable state.

[0055] When the compressed data packet storage space consumption is less than or equal to the remaining storage capacity, a real-time response time prediction formula is used to evaluate processing efficiency. This formula uses a resource adaptation coefficient. The predicted value is dynamically adjusted based on the ratio of remaining storage capacity to the resources theoretically required for rollback. When ≤1, an exponential model is used to accurately predict response time under resource constraints; When the time is greater than 1, a linear model is used to simplify the calculation. By comparing the real-time response prediction time with the average system restart time, the optimal handling scheme is selected: if the prediction time is short, a compression rollback operation is performed to prioritize service continuity; if the prediction time is long, a system restart is selected to avoid prolonged service interruption. This decision-making mechanism based on quantitative evaluation effectively solves the problem of low efficiency of traditional methods in resource-constrained environments, achieving an optimal balance between fault handling efficiency and resource utilization.

[0056] The following is an example of a formula for predicting real-time response time, assuming the average system restart time... =200ms, processing efficiency constant =2ms / sample, sensitivity parameter =1.5.

[0057] Scenario 1: Sufficient resources, assuming theoretical rollback range =50 (data points), remaining storage capacity =1000KB, average single-point consumption =10KB / item. Therefore, the resource adaptation coefficient... = =2. Because... >1. Use a linear model: = =2 50 = 100ms. Comparison: (100ms)< (200ms), therefore the system chooses to perform a compression rollback operation because its expected time is less than that of a restart operation. In this example, the storage unit (KB) is for illustrative purposes only; the formula itself applies to any consistent unit system, i.e. It is a dimensionless coefficient.

[0058] Scenario 2: Resource shortage situation, assuming =100, =800KB =10KB / unit. Then = =0.8. Because ≤1, use the exponential model: = = = ≈ =270ms. Comparison: (270ms)> (200ms) Therefore, the system chooses to perform a system restart operation because the expected rollback processing time has exceeded the restart time.

[0059] Through the above example, the prediction formula can dynamically and quantitatively recommend the optimal recovery strategy based on real-time resource conditions, thereby achieving optimal efficiency.

[0060] Example 2: Based on Example 1, a system for processing operation and maintenance information for information systems, such as... Figure 2 As shown, it includes: The time series correlation analysis module is used to acquire time series indicator data, data point identification data, and outlier data point data, construct a time series dataset, calculate the linear and nonlinear correlation strengths, and generate the first total correlation and the second total correlation. The anomaly impact assessment module is used to calculate the correlation change and determine the impact of abnormal data points, extract the fault type and classify it into instantaneous faults or cumulative faults, and determine the specific impact range assessment benchmark based on the historical fault database or dynamic strategy. The rollback range decision module is used to determine the theoretical rollback range based on the impact range assessment benchmark, calculate the maximum allowable rollback range in combination with the remaining storage capacity, perform correlation analysis and compression encoding on the data within the theoretical rollback range, and generate the final compressed data packet; The operation decision execution module is used to calculate the storage space consumption of compressed data packets and predict the real-time response time. By comparing the remaining storage capacity with the average system restart time, it decides whether to execute a compression rollback operation or a system restart operation.

[0061] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An operation and maintenance information processing method for an information system, characterized by, The method comprises the following steps: S1: acquiring time series index data, data point identification data and abnormal data point data, constructing a sequence data set according to the time series index data and the data point identification data; calculating the linear correlation strength of the sequence data set, and determining a first total correlation and a second total correlation according to the linear correlation strength; S2: taking the absolute difference value between the first total correlation and the second total correlation as a correlation change amount, and determining a processing method of the abnormal data point data according to the correlation change amount; acquiring a characteristic attribute of the abnormal data point data according to the processing method of the abnormal data point data, and extracting a corresponding fault type according to the characteristic attribute, and determining an influence range evaluation benchmark of an instantaneous fault according to the fault type; S3: determining an influence range evaluation benchmark of a cumulative fault according to the fault type; determining a theoretical rollback range according to the influence range evaluation benchmark corresponding to the abnormal data point data, determining a maximum allowed rollback range according to the ratio of the remaining storage capacity to the average storage consumption of a single data point, and determining a compressed data packet according to the theoretical rollback range; S4: calculating the storage space consumption of the compressed data packet, and determining a compression rollback operation and a system restart operation according to the storage space consumption.

2. The operation and maintenance information processing method for an information system according to claim 1, characterized by, The acquisition of the time series index data, the data point identification data and the abnormal data point data, and the construction of the sequence data set according to the time series index data and the data point identification data comprises: The data point identification data refers to a timestamp or a serial number that uniquely identifies each data point; The sequence data set refers to a collection of time series index data points arranged in chronological order; The abnormal data point data refers to a collection of outlier data points suspected to be generated by faults or errors, which are pre-identified by an anomaly detection algorithm or a rule; The time series index data refers to numerical operation and maintenance performance data collected at consecutive time points.

3. The operation and maintenance information processing method for an information system according to Claim 1, wherein The calculation of the linear correlation strength of the sequence data set and the determination of the first total correlation and the second total correlation according to the linear correlation strength comprises: The linear correlation strength of the sequence data set is calculated using the Pearson correlation coefficient and defined as the first correlation; If the first correlation is less than a preset linear threshold, the nonlinear correlation strength of the sequence data set is calculated using normalized mutual information and defined as the second correlation; If the first correlation is greater than or equal to the preset linear threshold, the first correlation is taken as the total correlation, and if the first correlation is less than the preset linear threshold, the second correlation is taken as the total correlation; The total correlation of the sequence data set before excluding the abnormal data point data is calculated and defined as the first total correlation, and the total correlation of the sequence data set after excluding the abnormal data point data is calculated and defined as the second total correlation.

4. The operation and maintenance information processing method for an information system according to Claim 1, wherein The taking of the absolute difference value between the first total correlation and the second total correlation as the correlation change amount and the determination of the processing method of the abnormal data point data according to the correlation change amount comprises: If the correlation change amount is less than a preset change amount threshold, it is determined that the abnormal data point data has no impact on the overall correlation, and the abnormal data point data is selected to be skipped; If the correlation change amount is greater than or equal to the preset change amount threshold, it is determined that the abnormal data point data has an impact on the overall correlation, and a subsequent fault processing process is selected.

5. The operation and maintenance information processing method for an information system according to Claim 1, wherein The processing method according to the abnormal data point data obtains a characteristic attribute of the abnormal data point data, extracts a corresponding fault type according to the characteristic attribute, and determines an influence range evaluation benchmark of a transient fault according to the fault type, including: The fault type includes a transient fault and a cumulative fault; If the fault type is a transient fault, the maximum influence range of the transient fault is obtained from a historical fault library as an influence range evaluation benchmark; The transient fault refers to a fault type represented by a continuous abnormal data point affected after the abnormal data point data.

6. The operation and maintenance information processing method for an information system according to Claim 1, wherein The influence range evaluation benchmark of the cumulative fault is determined according to the fault type, including: If the fault type is cumulative fault, the evaluation strategy is dynamically determined according to the current residual capacity, the normalized residual capacity is calculated, and the formula The historical influence range percentile is calculated; wherein, is a historical impact range percentile, is an aggressive strategy percentile threshold, is a conservative strategy percentile threshold, is a decay coefficient, is a normalized remaining capacity; According to the historical influence range percentile, an influence range value corresponding to the cumulative fault at the historical influence range percentile is obtained from a historical fault library as an influence range evaluation benchmark; The cumulative fault refers to a fault type represented by a continuous abnormal data point from the abnormal data point data to the recovery of the system; According to the timestamp attribute of the sequence data set, a time sequence set is constructed in chronological order; Based on the time sequence set, the remaining capacity of the current sequence data set is calculated, and the remaining capacity refers to the number of data points from the current abnormal data point to the end of the sequence; According to the comparison between the remaining capacity and the determined influence range evaluation benchmark, if the remaining capacity is greater than or equal to the influence range evaluation benchmark, the current data batch is selected to continue to be uploaded; If the remaining capacity is less than the influence range evaluation benchmark, the uploading is stopped and a subsequent fault processing process is entered.

7. The operation and maintenance information processing method for an information system according to Claim 1, wherein According to the influence range evaluation benchmark corresponding to the abnormal data point data, a theoretical rollback range is determined, a maximum allowed rollback range is determined according to the ratio of the remaining storage capacity to the average storage consumption of a single data point, and a compressed data packet is determined according to the theoretical rollback range, including: If the maximum allowed rollback range is greater than or equal to the theoretical rollback range, a complete rollback operation is performed, and the rollback step is set to the theoretical rollback range; If the maximum allowed rollback range is less than the theoretical rollback range, a data compression processing process is started, and a correlation matrix of the data points in the theoretical rollback range is calculated; The data segments in the correlation matrix that meet a preset similarity threshold are compressed and encoded; The data points in the correlation matrix that do not meet the preset similarity threshold are processed in the original format; The compressed and encoded data and the data processed in the original format are combined into a compressed data packet.

8. The operation and maintenance information processing method for an information system according to Claim 1, wherein The storage space consumption of the compressed data packet is calculated, and a compressed rollback operation and a system restart operation are determined according to the storage space consumption, including: If the storage space consumption of the compressed data packet is greater than the remaining storage capacity, a system restart process is triggered: reconstructing a system state snapshot according to key data points in the compressed data packet, storing the system state snapshot into persistent storage, performing a system restart operation, and restoring the service state based on the system state snapshot after the restart is completed; If the storage space consumption of the compressed data packet is less than or equal to the remaining storage capacity, a real-time response prediction time is obtained according to a real-time response time prediction formula; If the real-time response prediction time is less than or equal to the system restart average time, a compressed rollback operation is selected to be performed; If the real-time response prediction time is greater than the system restart average time, a system restart operation is selected.

9. The method for processing operation and maintenance information of an information system as described in claim 1, characterized in that, If the storage space consumption of the compressed data packet is less than or equal to the remaining storage capacity, a real-time response prediction time is obtained according to a real-time response time prediction formula, including that the real-time response time prediction formula is as follows, ； ； wherein, is a real-time response time, is a theoretical back-off range, is a resource adaptation coefficient, is a sensitivity parameter, is a processing efficiency constant, is a remaining storage capacity, is a single data point average storage consumption.

10. An operation and maintenance information processing system for an information system, for implementing the operation and maintenance information processing method according to any one of claims 1-9, characterized in that, including: a time series correlation analysis module, configured to obtain time series index data, data point identification data, and abnormal data point data, construct a time series dataset and calculate linear and nonlinear correlation strengths, and generate first total correlation and second total correlation; an abnormal influence evaluation module, configured to calculate a correlation change amount and determine an influence degree of the abnormal data point data, extract a fault type and distinguish between transient faults and cumulative faults, and determine a specific influence range evaluation benchmark according to a historical fault library or a dynamic strategy; a rollback range decision module, configured to determine a theoretical rollback range according to the influence range evaluation benchmark, calculate a maximum allowable rollback range in combination with the remaining storage capacity, perform correlation analysis and compression encoding on data in the theoretical rollback range, and generate a final compressed data packet; an operation decision execution module, configured to calculate the storage space consumption of the compressed data packet and predict a real-time response time, and decide to perform a compressed rollback operation or a system restart operation by comparing the remaining storage capacity and the system restart average time.

Images