Operation and maintenance exception automatic detection system based on machine automatic learning

Abstract

The application belongs to the technical field of computer data processing and relates to an operation and maintenance exception automatic detection system based on machine automatic learning, which comprises: a cost function construction module, which generates a dynamic cost function based on operation and maintenance data flow and business configuration; an initialization and evaluation module, which initializes a pipeline population, combines function calculation branch total cost scalar values; a population screening diagnosis module, which screens the population based on scalar values, extracts operator features of an elimination set to generate a performance degradation feature vector; a drift root cause positioning module, which extracts high-risk operator features according to the vector, scans data flow to generate a drift root cause label; a reconstruction instruction generation module, which matches candidate operators and adjusts selection probability to generate a directional reconstruction instruction; and a closed-loop detection execution module, which injects the instruction into the population to breed a new generation set and outputs a diagnosis report package. The application solves the problems of poor adaptability of an exception detection model to data distribution drift and lack of automatic diagnosis and directional self-healing mechanism.

Description

Technical Field

[0001] This invention belongs to the technical field of computer data processing and relates to an automatic detection system for operation and maintenance anomalies based on machine learning. Background Technology

[0002] In modern information technology infrastructure and large-scale distributed application systems, intelligent operation and maintenance (O&M) to ensure business continuity has become a crucial technical support. These systems continuously generate multi-dimensional O&M data, such as server performance metrics and network traffic logs. However, in actual operation, the underlying statistical distribution of data streams often experiences unexpected distribution drift due to factors such as version iterations or hardware aging. This can easily lead to performance degradation of anomaly detection models and generate a large number of false positives and false negatives, thereby affecting O&M decision-making and fault handling efficiency.

[0003] To address the anomaly detection needs in the aforementioned scenarios, commonly used techniques include fixed-threshold alarms and static machine learning model-based methods. Fixed-threshold methods manually preset static alarm limits for key indicators and trigger alarms when these limits are exceeded. Static machine learning methods utilize historical data to train models such as isolated forests or autoencoders to identify anomalies. Once trained and deployed, these models typically maintain fixed internal parameters and structure, using this fixed state to continuously infer and judge real-time data streams.

[0004] These traditional approaches have certain technical limitations when dealing with complex and ever-changing real-world online environments. Because the fixed threshold methods lack dynamic adjustment mechanisms, they struggle to achieve a good balance between sensitivity and specificity when handling indicators with complex periodicity or dynamic baselines. Furthermore, the static machine learning models with fixed parameters and structures after deployment are less adaptable to data distribution shifts; their detection performance is prone to unpredictable degradation when the statistical characteristics of online data change. In addition, existing strategies often rely on manual discovery of failures followed by offline data labeling and retraining, resulting in long response cycles and a lack of diagnostic analysis of the root causes of failures, making it difficult to effectively guide model structure optimization. Summary of the Invention

[0005] To address the aforementioned problems, this invention provides an automatic detection system for operational anomalies based on machine learning.

[0006] An automated operation and maintenance anomaly detection system based on machine learning includes: The cost function construction module is used to obtain the original operation and maintenance data stream and business impact monitoring configuration, and match the cost weights of false alarms and false alarms to generate a dynamic cost function; The initialization and evaluation module is used to initialize the anomaly detection pipeline population and input the benchmark test set, and calculate the total cost scalar value of each candidate pipeline branch in combination with the dynamic cost function. The population screening and diagnosis module is used to screen the population based on the total cost scalar value, extract the current optimal pipeline and the set of pipelines to be eliminated, and generate a performance degradation feature vector based on the operator node characteristics of the elimination set. The drift root cause localization module is used to extract high-risk operator features based on the performance degradation feature vector, scan the original operation and maintenance data stream along the time domain distribution direction it points to, extract abnormal missing attributes, and generate drift root cause labels. The reconstruction instruction generation module is used to match candidate operator categories based on abnormal missing attributes, adjust the selection probability of operator categories by combining performance degradation feature vectors, and generate targeted reconstruction instructions. The closed-loop detection execution module is used to inject directional reconstruction instructions into the next-generation pipeline set for population reproduction, integrate the current best pipeline, the next-generation pipeline set, and safety alarm information converted from drift root cause labels, and output a diagnostic report package.

[0007] In a further embodiment of the present invention, the cost function construction module is used to perform the following operations: Extract the identification identifiers of the monitored objects from the original operation and maintenance data stream, query the business impact monitoring configuration, and obtain the specific business impact level corresponding to the monitored object; Based on the specific business impact level, false alarm cost weights and false alarm cost weights are assigned to false alarm and false alarm cost weights for the monitored objects, respectively. Obtain the number of false positive samples and false negative samples generated by the upstream inspection and classification process; Calculate the product of the number of false positive samples and the cost weight of false alarms, and the product of the number of false negative samples and the cost weight of false negatives. Sum the product results of all monitored objects to generate a dynamic cost function.

[0008] In a further embodiment of the present invention, the initialization and evaluation module is used to perform the following operations: Arrange feature processing operators and anomaly detection models to generate an anomaly detection pipeline population containing multiple candidate pipeline branches; Slice a recent historical benchmark set containing known fault labels and synchronously input it into the anomaly detection pipeline population to perform parallel inference and output a predicted label set; Compare the predicted label set with the real labels of recent historical benchmark sets, and statistically analyze the elements of the verification confusion matrix for various monitoring indicators; The elements of the verification confusion matrix are input into the dynamic cost function for weighted calculation, and the total cost scalar value corresponding to each candidate pipeline branch is output.

[0009] In a further aspect of the present invention, when the population screening and diagnosis module screens the population based on the total cost scalar value, it performs the following operations: Sort all candidate pipeline branches in the population according to the ascending order of total cost scalar values ​​from low to high; Select the first candidate pipeline branch in the ranking sequence as the current optimal pipeline; Select candidate pipeline branches at the end of the ranking sequence with a preset proportion and a number not less than a preset low target threshold, and combine them to construct an elimination pipeline set.

[0010] In a further aspect of this invention, the population screening and diagnosis module performs the following operations when generating performance degradation feature vectors: Analyze the eliminated operator nodes contained in the elimination pipeline set; The first occurrence frequency of eliminated operator nodes in the elimination pipeline set and the second occurrence frequency in the anomaly detection pipeline population are statistically analyzed. Calculate the ratio of the first occurrence frequency to the second occurrence frequency to obtain the failure correlation parameters between the eliminated operator node and the detected blocking state; The failure-related parameters of all operator nodes are combined to generate a performance degradation feature vector.

[0011] In a further aspect of the present invention, the drift root cause localization module performs the following operations when extracting high-risk operator features: Analyze the performance degradation feature vector and sort the component values ​​of each dimension of the performance degradation feature vector in descending order; Extract the operators corresponding to the dimensions with the highest preset number of component values ​​as high-risk operator features; High-risk operator features are configured as dedicated diagnostic filters to intercept and build a processing cache that stores the original operation and maintenance data stream according to a preset time window.

[0012] In a further aspect of the present invention, the drift root cause localization module performs the following operations when generating drift root cause tags: Calculate the stability parameter of the original operation and maintenance data stream within the current time window along the time domain distribution direction pointed to by the characteristics of the high-risk operators; Calculate the normalized mean difference of the original operation and maintenance data stream between adjacent time windows, and use it as the jump parameter; The stability parameter is compared with the preset first diagnostic threshold and the jump parameter is compared with the preset second diagnostic threshold. If either parameter exceeds its corresponding diagnostic threshold, the original operation and maintenance data stream is determined to be in a sudden change state. Based on the mutation state, determine the reasons for the loss of statistical characteristics of the original operation and maintenance data stream, extract the corresponding abnormal missing attributes, and generate drift root cause labels containing abnormal missing attributes.

[0013] In a further aspect of this invention, the reconstruction instruction generation module performs the following operations when matching candidate operator categories: Retrieve the preset rule mapping matching matrix; Read the abnormal missing attributes exposed in the drift root cause tags; Using the abnormal missing attribute as the query keyword, a search is performed in the rule mapping matching matrix to obtain candidate operator categories with attribute compensation characteristics.

[0014] In a further aspect of the present invention, the reconstruction instruction generation module performs the following operations when generating directional reconstruction instructions: The expected selection probability threshold of the candidate operator category is increased based on a preset positive adjustment constant. Based on the performance degradation feature vector, the failure operator category is located, and the probability of basic mutation of the failure operator category is reduced based on the preset negative adjustment constant; The probabilities of all operator categories are normalized using a normalized exponential function to generate operator probability adjustment parameters. The drift root cause label and the operator probability adjustment parameter are connected in series to form a directional reconstruction instruction.

[0015] In a further embodiment of the present invention, the closed-loop detection execution module is used to perform the following operations: Remove the operator probability adjustment parameters from the front end of the directional reconstruction instruction and replace the built-in mutation control matrix of the anomaly detection pipeline population; Based on the updated mutation control matrix, crossover and directed mutation operations are triggered to generate a new generation of pipeline sets; Remove the drift root cause label attached to the targeted reconstruction instruction, perform network topology mapping by connecting to the pre-configured business asset network topology dictionary, and export security alarm information; The system merges and packages the current best pipeline, the next-generation pipeline set, and safety alarm information, renders and transmits them to the front-end monitoring dashboard system, and outputs a diagnostic report package.

[0016] In summary, the present invention has the following beneficial technical effects: 1. By acquiring the business impact monitoring configuration and matching the cost weights of false positives and false negatives, a dynamic cost function is generated. This function is then used to calculate the total cost scalar value of each candidate pipeline branch within the population. This technical solution changes the traditional approach of relying solely on conventional classification accuracy to evaluate the model, establishing an objective correlation between the model's technical performance and its actual business impact. Based on this, the quantified cost scalar value serves as the basis for subsequent model population evolution selection, helping to consider risk control from a business perspective during evaluation. This improves the signal-to-noise ratio of the alarm system, ensuring a high degree of correlation between the system's output alarm information and the actual operational status of the business.

[0017] 2. A hierarchical screening process based on the total cost scalar value is used to construct a set of elimination pipelines, and performance degradation feature vectors are generated based on the characteristics of the eliminated operator nodes within this set. This mechanism transforms high-cost models in the screening process into diagnostic information sources. By statistically analyzing the operator combination characteristics within the eliminated models, a reverse derivation path from model performance degradation to specific structural defects is established. This process enables automated attribution analysis of model maladaptation problems caused by data distribution drift, providing clear data guidance for identifying the specific links in the decline of detection capabilities and subsequent targeted model optimization.

[0018] 3. By extracting high-risk operator features using performance degradation feature vectors and generating drift root cause labels in the original operation and maintenance data stream, targeted reconstruction instructions are generated to adjust the operator selection probability in subsequent population propagation. This scheme verifies the state of relevant statistical features on the original data stream, which can more accurately reflect the true root cause of data distribution drift. Simultaneously, based on diagnostic conclusions, it dynamically intervenes in genetic operations, appropriately increasing the application weight of candidate operators and suppressing the mutation probability of failed operators, transforming the conventional divergent model evolution into a guided targeted reconstruction process. This mechanism constructs an automated closed loop from failure diagnosis to adaptive model updates, improving the adaptive adjustment efficiency and recovery capability of the anomaly detection system in the face of variable data environments. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. The drawings are used to provide a further understanding of the present invention.

[0020] Figure 1 This is a schematic diagram of the framework in the embodiments of this application.

[0021] Figure 2 This is a flowchart illustrating an embodiment of this application.

[0022] Figure 3 This is a schematic diagram of the ascending order of the total cost of the anomaly detection pipeline population in an embodiment of the present invention.

[0023] Figure 4 This is a distribution map of performance degradation feature vector operators and location map of high-risk operators in an embodiment of the present invention.

[0024] Figure 5 This is a schematic diagram illustrating the location of abrupt changes in the time-domain statistical characteristics of the original operation and maintenance data stream in an embodiment of the present invention.

[0025] Figure 6 This is a comparison chart showing the adjustment of operator mutation probability under the directional reconstruction instruction in this embodiment of the invention. Detailed Implementation

[0026] The following is in conjunction with the appendix Figures 1-6 A preferred description of the present invention is provided below.

[0027] See attached document Figures 1-6 This invention proposes an automatic detection system for operation and maintenance anomalies based on machine learning, comprising the following modules: The cost function construction module is used to obtain the original operation and maintenance data stream and business impact monitoring configuration, and match the cost weights of false alarms and false alarms to generate a dynamic cost function; The initialization and evaluation module is used to initialize the anomaly detection pipeline population and input the benchmark test set, and calculate the total cost scalar value of each candidate pipeline branch in combination with the dynamic cost function. The population screening and diagnosis module is used to screen the population based on the total cost scalar value, extract the current optimal pipeline and the set of pipelines to be eliminated, and generate a performance degradation feature vector based on the operator node characteristics of the elimination set. The drift root cause localization module is used to extract high-risk operator features based on the performance degradation feature vector, scan the original operation and maintenance data stream along the time domain distribution direction it points to, extract abnormal missing attributes, and generate drift root cause labels. The reconstruction instruction generation module is used to match candidate operator categories based on abnormal missing attributes, adjust the selection probability of operator categories by combining performance degradation feature vectors, and generate targeted reconstruction instructions. The closed-loop detection execution module is used to inject directional reconstruction instructions into the next-generation pipeline set for population reproduction, integrate the current best pipeline, the next-generation pipeline set, and safety alarm information converted from drift root cause labels, and output a diagnostic report package.

[0028] In one embodiment of the present invention, the cost function construction module is configured to perform the following steps: Extract the identification identifiers of the monitored objects from the original operation and maintenance data stream, query the business impact monitoring configuration, and obtain the specific business impact level corresponding to the monitored object; Based on the specific business impact level, false alarm cost weights and false alarm cost weights are assigned to false alarm and false alarm cost weights for the monitored objects, respectively. Obtain the number of false positive samples and false negative samples generated by the upstream inspection and classification process; Calculate the product of the number of false positive samples and the cost weight of false alarms, and the product of the number of false negative samples and the cost weight of false negatives. Sum the product results of all monitored objects to generate a dynamic cost function.

[0029] Specifically, the process of acquiring raw operational data streams and business impact monitoring configurations, and constructing a dynamic cost function to assess business losses, is executed by a cost function construction module deployed on a central server or cloud platform. During execution, the cost function construction module receives continuous raw operational data streams collected and reported by front-end probes in real time through standard message queue interfaces, such as subscribing to Kafka topics or RabbitMQ queues.

[0030] It should be noted that raw operational data streams refer to time-series data continuously collected from various assets of the information technology system, including but not limited to server CPU utilization, memory usage, disk I / O rate, network device bandwidth throughput, packet loss rate, and application service interface response time and error log entry count. Each data record, whether it is an indicator type or a log type, is encapsulated as a structured data object, such as a JSON format message, which must contain an identifier field to uniquely identify the monitored object.

[0031] The identification identifier is a globally unique name assigned to each monitored physical or logical asset, such as the hostname "server-prod-db-01" or the microservice name "order-processing-service".

[0032] After obtaining the identification identifier, the cost function construction module uses this identifier as the primary key to initiate a query request to an external configuration management system. This system can be a configuration management database or LDAP service. It is understood that the configuration management system is a centralized system for storing assets and their business attributes, typically an enterprise-level configuration management database, which records the business applications carried by each asset and their importance. The purpose of the query operation is to obtain the specific business impact level associated with the identification identifier. In this embodiment, the specific business impact level is a manually tiered classification of the business based on factors such as economic losses, customer impact, and brand reputation damage caused by business interruption, such as setting it to three levels: L1 - Core Transaction, L2 - Important Support, and L3 - General Application.

[0033] The cost function construction module configures numerical judgment cost weights for the two potential misjudgment events of the monitored object—false positives and false negatives—based on the specific business impact level returned by the query. The judgment cost weights quantify the business impact of different types of detection errors. Their setting is based on business risk assessment; for core business operations, the false negative cost weight is [not specified in the original text]. Typically much higher than the cost of false positives. ,like Set to 50.0 to 100.0. The value is set to between 0.5 and 1.5. For general business operations, the weight difference is relatively small. This configuration process is completed by querying the weight mapping table of the low target threshold. This table maps each business impact level to a specific false positive cost weight and false negative cost weight pair. The construction module stores the association between the identification identifier and each decision cost weight as a cost mapping table. This mapping table uses an in-memory hash table or a physical database table to support data retrieval. The cost mapping table is a data structure that maps a specific business impact level to its corresponding decision cost weight pair, such as a database table containing three columns: impact level, false positive weight, and false negative weight.

[0034] The cost function construction module iterates through the set of confusion matrix elements output by the upstream testing and classification process. This set summarizes the anomaly detection results for each monitored object within a specific evaluation period. It should be noted that the confusion matrix elements refer to key indicators extracted from the classification model performance evaluation results; in this step, they specifically refer to the number of false positive samples. Number of false negative samples During the traversal, the number of false positive samples and false negative samples associated with each monitored object are extracted from the confusion matrix elements. Using the identification identifier of the currently traversed monitored object, a matching query is performed in the previously generated cost mapping table to retrieve the corresponding judgment cost weights, namely the false positive cost weight and the false negative cost weight. The cost function construction module accumulates the costs of all monitored objects by calculating the sum of the products of all false positive samples and false positive cost weights, plus the sum of the products of all false negative samples and false negative cost weights, to obtain the weighted total of business losses. This sum is the specific output of the dynamic cost function in the current evaluation period. It quantifies the overall loss of the current detection model at the business level in the form of a scalar value and serves as the benchmark for system alarm evaluation.

[0035] The weighted sum of business losses is used to quantify the degree of business risk caused by system detection performance degradation. Its calculation is essentially the sum of the weighted costs of each monitored object. The specific process of constructing a dynamic cost function and calculating the weighted sum of business losses is expressed by the following formula: ; In the formula, The total cost calculated by the dynamic cost function is a scalar value that quantifies the cost of business losses and serves as the final indicator for evaluating model performance. This represents the total number of monitored objects. Represents the currently calculated number of One monitored object. Representative and the The false alarm cost weight associated with each monitored object. This weight is a unit cost coefficient that quantifies the impact of a single false alarm. The representative, within the current evaluation period, is addressing the [specific issue / representative's] [specific action / representative's] [action ... The number of false positive samples detected for each monitored object, i.e., the number of false alarms. Representative and the The weight of the cost of missed detection associated with each monitored object. This weight is a unit cost coefficient that quantifies the impact of a single false positive. The representative, within the current evaluation period, is addressing the [specific issue / representative's] [specific action / representative's] [action ... The number of false negative samples detected for each monitored object, i.e., the number of missed reports.

[0036] For example, the intelligent operations and maintenance system monitors two business assets: a core database service identified by the identifier "asset-db-core" and a front-end portal website identified by the identifier "asset-web-portal". The system receives raw operations and maintenance data streams containing these two identifiers. By querying the configuration management system, it is determined that the specific business impact level of "asset-db-core" is L1-core transaction, while the specific business impact level of "asset-web-portal" is L3-general application. Based on a preset cost mapping table, the system assigns a false alarm cost weight to the L1-core transaction level. and underreporting cost weight The decision cost weight assigned to L3-general application level is the false alarm cost weight. and underreporting cost weight At the end of an evaluation cycle, the confusion matrix elements output by the upstream testing and classification process showed the number of false positive samples generated for the "asset-db-core" test. Number of false negative samples The number of false positive samples generated by the detection of "asset-web-portal" was [number missing]. Number of false negative samples Next, the cost function construction module performs calculations, substituting these values ​​into the formula of the dynamic cost function to obtain the total cost. ,have to Finally, the system generates and outputs a dynamic cost function representing the current system alarm evaluation benchmark, which is 113.0. This scalar value will be used in subsequent steps to measure the merits of different candidate detection models.

[0037] In one embodiment of the present invention, the initialization and evaluation module is configured to perform the following steps: Arrange feature processing operators and anomaly detection models to generate an anomaly detection pipeline population containing multiple candidate pipeline branches; Slice a recent historical benchmark set containing known fault labels and synchronously input it into the anomaly detection pipeline population to perform parallel inference and output a predicted label set; Compare the predicted label set with the real labels of recent historical benchmark sets, and statistically analyze the elements of the verification confusion matrix for various monitoring indicators; The elements of the verification confusion matrix are input into the dynamic cost function for weighted calculation, and the total cost scalar value corresponding to each candidate pipeline branch is output.

[0038] Specifically, the process of initializing the anomaly detection pipeline population of the carrying model and calculating the fitness parameters of each candidate pipeline branch within the population using a dynamic cost function is executed by the initialization and evaluation module deployed on a central server or cloud platform. During execution, the initialization and evaluation module generates a basic population structure composed of multi-level anomaly detection pipelines by orchestrating and combining a pre-defined operator library and model library.

[0039] The basic population structure refers to a collection of multiple parallel anomaly detection pipelines with different structures, and its scale is... This is a configurable hyperparameter, typically set between 30 and 100 depending on computational resources. For example, zero-mean normalization operators and principal component analysis operators can be extracted from an operator library and concatenated, and topologies such as isolated forests or autoencoders can be selected from a model library for combination. It should be noted that feature processing operators are algorithmic modules used for data preprocessing and feature extraction, such as min-maximum normalization operators, standard deviation normalization operators, fast Fourier transform operators, wavelet transform operators, etc.; among them, the anomaly detection model topology refers to the algorithmic model structure used to ultimately determine whether data points are anomalies, such as statistically based multivariate Gaussian distribution models, distance-based K-nearest neighbor models, isolated forest models based on ensemble learning, or deep learning-based autoencoders and variational autoencoder networks. The selected features and models are encapsulated as independent pipeline entities. It can be understood that independent computational units refer to each pipeline being implemented in engineering as an independently deployable and schedulable entity, such as microservice instances or containerized applications, ensuring the feasibility of parallel inference within the population.

[0040] Repeat this process until the number generated is [number missing]. Individual pipeline units collectively constitute the basic population structure. A recent historical benchmark set containing known fault labels is extracted and sliced ​​using a fixed time window approach, such as generating a data slice every 5 minutes. In this embodiment, the recent historical benchmark set refers to a data set selected and labeled by experts based on recent historical business records, used as a benchmark for offline or asynchronous evaluation of model accuracy. The same data slice is then synchronously input into all units of the basic population structure via broadcast. In each anomaly detection pipeline, parallel inference computation processes are triggered. After processing the received data slices, each pipeline outputs a set of predicted labels for the corresponding time period, which consists of a series of judgment results marked as normal or abnormal.

[0041] The predicted label set is the output sequence of a single pipeline after processing a data slice. Each element in the sequence corresponds to a binary classification label for a data point, such as 0 representing normal and 1 representing abnormal. The true labels of this recent historical benchmark set constitute the benchmark data for evaluation. By comparing the difference between the predicted label set and the true label of each pipeline point by point, i.e., the overlap error, a separate verification confusion matrix element is calculated for each monitored object, and the sample count values ​​of four indicators—true positive, false positive, true negative, and false negative—are calculated. The verification confusion matrix element is the false positive result calculated by comparing the predicted label set and the true labels of the recent historical benchmark set. False negative The specific value.

[0042] For each candidate pipeline branch within the basic population structure, its corresponding check confusion matrix element is passed to the aforementioned constructed dynamic cost function for weighted calculation. This calculation process yields a total cost scalar value for each pipeline, which serves as the core parameter for measuring the pipeline's operational fitness and is output for subsequent screening. The total cost scalar value is used as a measure of the fitness of each individual pipeline; the lower the value, the better the pipeline's overall performance in balancing operational impact and detection accuracy. The process of calculating the total cost scalar value using the dynamic cost function to evaluate the performance of each candidate pipeline is expressed by the following formula: ; In the formula, Representative regarding the first The total cost scalar value calculated for each candidate pipeline. This represents the index number of the candidate pipeline in the basic population structure, and its value range is... arrive ,in Population size. This represents the total number of monitored objects; this value is inherited from the previous step. Represents the currently calculated number of One monitored object. Representative and the The false alarm cost weight associated with each monitored object is provided by the cost mapping table and has been determined in the previous steps. Representing the The candidate pipeline, within the current evaluation cycle, is for the [number]th [project / line]. The number of false positive samples detected per monitored object. Representative and the The cost weight associated with each monitored object is also provided by the cost mapping table. Representing the The candidate pipeline, within the current evaluation cycle, is for the [number]th [project / line]. The number of false negative samples detected per monitored object.

[0043] For example, following the steps described above, the system needs to evaluate the performance of candidate pipelines for two monitored assets: a core database service identified by the identifier "asset-db-core" and a front-end portal website identified by the identifier "asset-web-portal". The system initializes a basic population structure containing two pipelines, pipeline A and pipeline B. Pipeline A uses a structure of standard deviation normalization operator + isolated forest model, while pipeline B uses a structure of fast Fourier transform operator + long short-term memory network autoencoder. The system synchronously inputs a slice of recent historical benchmark data into these two pipelines. Simultaneously, the system obtains the ground truth labels corresponding to this data slice, showing that "asset-db-core" has 1 ground truth anomaly and "asset-web-portal" has 2 ground truth anomalies. After parallel inference, pipeline A predicts that it detects 3 anomalies for "asset-db-core", including 1 true positive and 2 false positives. Therefore... , Eleven anomalies were detected in the "asset-web-portal" database, including one true positive, ten false positives, and one missed true anomaly. , Pipeline B's prediction result was that one outlier was detected for "asset-db-core," but this was a false positive, missing a real anomaly. Therefore... , Six anomalies were detected in the "asset-web-portal" system, of which two were true positives and four were false positives. There were no missed detections. , The system then invokes the dynamic cost function and uses the decision cost weights determined in the previous steps, i.e. , , , Calculate the total cost scalar value for each of the two pipelines. For pipeline A, its total cost scalar value is... For pipeline B, its total cost scalar value is... Finally, the system outputs the total cost scalar value corresponding to each candidate pipeline branch, that is, the fitness parameter of pipeline A is 17.0 and the fitness parameter of pipeline B is 83.0.

[0044] In one embodiment of the present invention, the population screening and diagnosis module is used to perform the following steps: Sort all candidate pipeline branches in the population according to the ascending order of total cost scalar values ​​from low to high; Select the first candidate pipeline branch in the ranking sequence as the current optimal pipeline; Select candidate pipeline branches at the end of the ranking sequence with a preset proportion and a number not less than a preset low target threshold, and combine them to construct an elimination pipeline set. When generating the performance degradation feature vector, the discarded operator nodes contained in the discard pipeline set are analyzed; the first occurrence frequency of the discarded operator nodes in the discard pipeline set and the second occurrence frequency in the anomaly detection pipeline population are counted; the ratio of the first occurrence frequency to the second occurrence frequency is calculated to obtain the failure association parameters between the discarded operator nodes and the detection stall state; the failure association parameters of all operator nodes are combined to generate the performance degradation feature vector.

[0045] Specifically, the process of performing hierarchical screening and stripping of the basic population structure based on the total cost scalar value, and outputting the optimal model link and performance degradation feature vector for the current period, is executed by the population screening and diagnosis module deployed on a central server or cloud platform. This module receives the total cost scalar value calculated for each candidate pipeline within the basic population structure. According to a preset ascending order sorting rule based on the total cost scalar value from low to high, a global sorting and tabulation process is performed on all candidate pipelines within the basic population structure to generate an ordered pipeline ranking sequence.

[0046] The ascending order rule is the sole criterion guiding the ranking; that is, the lower the total cost of a pipeline, the higher its ranking. The pipeline branch at the top of the ranking sequence is extracted as the current optimal pipeline. It should be noted that the current optimal pipeline is the best-performing pipeline among all candidates within the current evaluation period, with the smallest total cost scalar value. The complete configuration of this optimal pipeline, including its feature processing operator sequence and model topology, is serialized and sent to the online inference engine to replace the old model and perform online anomaly detection on the real-time data stream of the target business assets.

[0047] Simultaneously, based on a preset ratio, such as selecting 20% ​​to 30% of the pipeline branches at the bottom of the ranking sequence, with a total number not less than a preset low target threshold, these lagging pipeline branches with the highest total cost are selected. It should be understood that the preset ratio is a configurable hyperparameter used to determine the proportion of the bottom-ranked pipelines selected to construct the elimination set, typically set between 0.2 and 0.4. To ensure the statistical significance of subsequent extraction of degradation features, this screening operation is limited by a minimum number of failure samples. This parameter setting represents a trade-off between obtaining sufficiently rich failure sample information and controlling the computational cost of subsequent analysis.

[0048] Extract the structural definitions of the aforementioned lagging pipeline branches to construct a set of obsolete pipelines that records failure cases. This obsolete pipeline set is a data structure storing the complete topology definitions of all pipelines deemed to have the worst performance. Analyze the obsolete operator nodes of each pipeline in the obsolete pipeline set. In this embodiment, the obsolete operator node refers to the specific functional unit constituting the pipeline topology, which can be any feature processing operator or anomaly detection model. Calculate the first occurrence frequency of each operator node within this set.

[0049] By comparing the first occurrence frequency of a certain type of operator in the elimination pipeline set with its second occurrence frequency in the entire anomaly detection pipeline population, the failure correlation parameter between the operator and the detection performance degradation state, i.e., the detection stall state, can be calculated. It can be understood that the detection stall state refers to the state where the pipeline cannot effectively identify anomalies under the current data mode, resulting in a significant increase in its total cost. The failure correlation parameter is a quantitative indicator used to measure the correlation strength between a specific operator node and the detection stall state; the higher the parameter value, the more likely that the operator is one of the causes of the current detection performance degradation.

[0050] By combining the failure correlation parameters of all operator nodes, a performance degradation feature vector is ultimately generated that reflects which processing mode tends to fail under the current data distribution. It should be noted that the performance degradation feature vector is a vector constructed based on all operators in the operator library. Each component of the vector is the failure correlation parameter of the corresponding operator, and together they depict the pattern of performance degradation in the current system. The calculation process for each dimension component of the performance degradation feature vector, i.e., the failure correlation parameter, is expressed by the following formula: ; In the formula, The th element in the performance degradation feature vector The value of the nth component, that is, for the nth component The failure correlation parameter of each operator node. This parameter is a dimensionless relative frequency ratio. Represents the first in the operator library Operator nodes, such as the Fast Fourier Transform operator. Representing the first in the basic population structure One candidate pipeline. This is an indicator function; its value is 1 when the condition is true, and 0 otherwise. Represents the judgment operator Does it exist in the assembly line? In the structure. The set of pipelines to be eliminated is a subset of the set of all candidate pipelines. This represents the number of pipelines included in the set of pipelines to be phased out. This represents the total size of the basic population structure, i.e., the total number of candidate pipelines. For a preset minimum smoothing constant, such as This is used to prevent a computational anomaly where the denominator is zero when an operator is eliminated from the global population. The numerator of this formula calculates the operator's... Frequency of occurrence in the set of obsolete pipelines. A value greater than 1 indicates that the operator accounts for too high a proportion in the elimination set, which is positively correlated with performance degradation.

[0051] For example, such as Figure 3 As shown, the system has calculated the total cost of pipeline A to be 17.0 and pipeline B to be 83.0. The basic population structure contains a total of 10 pipelines. Besides A and B, the total cost scalar values ​​for the other 8 pipelines are: C = 25.0, D = 31.0, E = 45.0, F = 50.0, G = 68.0, H = 75.0, I = 80.0, and J = 92.0. Combined with... Figure 3 As can be seen, the population screening and diagnosis module first sorts these 10 pipelines in ascending order of total cost, resulting in the sequence: A (17.0), C (25.0), D (31.0), E (45.0), F (50.0), G (68.0), H (75.0), I (80.0), B (83.0), J (92.0). The pipeline A, ranked first, is extracted as the current optimal pipeline, and its standard deviation normalization operator + isolated forest model structure is deployed online. Simultaneously, with a preset proportion of 30%, the three pipelines at the bottom of the ranking, namely I, B, and J, are selected to form the elimination pipeline set.

[0052] refer to Figure 4 The performance degradation feature vector operator distribution is shown, and the system calculates the failure correlation parameters of each operator. Taking the Fast Fourier Transform operator as an example, its frequency of occurrence in the elimination set is 2 / 3, and its frequency of occurrence in the entire population is 3 / 10. Therefore, neglecting the influence of the minimum smoothing constant ϵ, its failure correlation parameter... Taking the Long Short-Term Memory (LSTM) network model as an example, its frequency of occurrence in the elimination set is 1, and its frequency of occurrence in the entire population is 0.5. Therefore, its failure association parameter... Taking the standard deviation normalization operator as an example, if its frequency of occurrence in the elimination set is 0, then its failure correlation parameter... After calculating all operators using this method, the system generates and outputs the following: Figure 4 The performance degradation feature vector shown by the feature localization of medium- and high-risk operators can take the following form: This vector clearly indicates that, within the current data cycle, the processing mode based on Fast Fourier Transform and Long Short-Term Memory networks is highly correlated with high costs and is the main mode leading to performance degradation.

[0053] In one embodiment of the present invention, the drift root cause localization module is configured to perform the following steps: Analyze the performance degradation feature vector and sort the component values ​​of each dimension of the performance degradation feature vector in descending order; Extract the operators corresponding to the dimensions with the highest preset number of component values ​​as high-risk operator features; Configure high-risk operator features as dedicated diagnostic filters to intercept and build a processing cache that stores the original operation and maintenance data stream according to a preset time window; When generating drift root cause labels, the stationarity parameter of the original operation and maintenance data stream within the current time window is calculated along the time-domain distribution direction pointed to by the high-risk operator features; the normalized mean difference between adjacent time windows of the original operation and maintenance data stream is calculated as the jump parameter; the stationarity parameter is compared with the preset first diagnostic threshold, and the jump parameter is compared with the preset second diagnostic threshold. If either parameter exceeds its corresponding diagnostic threshold, the original operation and maintenance data stream is determined to be in a mutation state; the cause of the loss of statistical features in the original operation and maintenance data stream is determined based on the mutation state, the corresponding abnormal missing attributes are extracted, and drift root cause labels containing abnormal missing attributes are generated.

[0054] Specifically, the process of activating the drift root cause locator, which is attached to the external evolutionary architecture, and projecting the performance degradation feature vector onto the original operational data stream to perform targeted feature diagnostic analysis is executed by a drift root cause localization module that is independent of the main process. It should be noted that the drift root cause localization module is a diagnostic service that is logically or physically separated from the main population evolution process, ensuring the independence of the diagnostic task and avoiding performance impact on the main process.

[0055] During execution, the main control unit initiates the drift root cause localization module via command and transmits the performance degradation feature vector. The drift root cause localization module receives and parses the performance degradation feature vector, sorting its component values ​​in descending order to extract a preset number of dimensions with the highest component values. The operators corresponding to these dimensions are identified as the structural component dimensions with the highest degradation risk. The drift root cause localization module configures high-risk operator features as dedicated diagnostic filters. These filters act as analytical guides, determining the direction of subsequent diagnostics. High-risk operator features refer to the operator types in the performance degradation feature vector with the highest numerical correlation to performance degradation, such as Fast Fourier Transform or mean filling. The dedicated diagnostic filter serves as a guiding mechanism, shifting the diagnostic scope from a blind, comprehensive scan of the data to focusing on specific statistical attributes indicated by the high-risk operator features.

[0056] Simultaneously, connecting to the underlying data bus, the system synchronously intercepts and temporarily stores the original operation and maintenance data stream consistent with the data stream processed by the basic population structure, constructing a cyclic processing buffer based on a time window or fixed capacity. In this embodiment, the processing buffer is a first-in-first-out memory data structure used to temporarily store the original operation and maintenance data stream for a recent period for statistical calculations. The drift root cause localization module scans the data in the processing buffer along the key time-domain distribution direction pointed to by the high-risk operator features. For example, if the high-risk operator feature is a Fast Fourier Transform operator used to extract periodic features, the diagnostic direction focuses on whether the periodicity in the original data has been structurally lost; if it is a statistical calculation operator related to the mean or variance, the diagnostic direction focuses on whether the stationarity of the data has abruptly changed. On the determined diagnostic direction, the stationarity parameter and jump parameter on a specific vector subset are calculated.

[0057] The stationarity parameter is an indicator used to measure whether the statistical characteristics of time series data remain constant over time. In this embodiment, the probability value of the augmented Dickey-Fowler test is used, and its judgment threshold is usually set to 0.05. If it is lower than this preset threshold, the data is judged to be stationary. The jump parameter is an indicator used to measure whether there are structural abrupt changes in time series data. In this embodiment, the difference of the normalized mean of adjacent windows is used, and its judgment threshold is usually set to 3.0. This value is based on the three sigma criterion in statistical process control theory and empirical settings. If it exceeds this threshold, it indicates that a small probability of a significant change has occurred.

[0058] The stationarity parameter, calculated in real time, is compared with a preset first diagnostic threshold, and the jump parameter is compared with a preset second diagnostic threshold to determine whether a sudden change has occurred. If a sudden change is confirmed in either parameter, the cause of statistical feature loss is determined based on the type of change. A sudden change refers to a change in the value of the stationarity parameter from less than 0.05 to greater than 0.05, or a change in the value of the jump parameter from less than 3.0 to greater than 3.0. Abnormal missing attributes are extracted, and corresponding drift root cause labels are generated. These drift root cause labels are formatted strings, such as "periodic feature loss" or "step change in the data mean," used to clearly describe the nature of the data distribution change.

[0059] The calculation process for the stability parameter and the jump parameter can be expressed by the following formula: The stationarity parameter can be the probability value of the augmented Dickey-Fowler test. To quantify: ; The jump parameter can be the normalized mean difference between adjacent windows. To quantify: ; In the formula, This parameter represents the stationarity of the data within the current time window; the closer its value is to 0, the more stable the data. The augmented Dickie-Fowler statistical test function represents the standard. Represents the current time window A vector subset of the original operation and maintenance data stream within. This represents the jump parameter, which quantifies the magnitude of change in the data mean. and These represent the current time window. The sample mean and sample variance of the internal data subset. and Representing the previous adjacent time window The sample mean and sample variance of the internal data subset. To maintain consistency with the dimensions of the sample variance, a minimal smoothing constant can be chosen in practical engineering as a minimal proportion of the historical global variance of the corresponding indicator. This is used to prevent absolute stationarity of the data, i.e., the variance is... The calculation anomaly of zero denominator is triggered to ensure the consistency of the dimensions and the numerical stability of the formula.

[0060] For example, the drift root cause localization module receives a performance degradation feature vector. Analyzing this vector reveals the failure correlation parameters corresponding to the Fast Fourier Transform operator. The value was the highest, therefore it was identified as a high-risk operator feature. Based on this, the diagnostic focus shifted to periodic analysis of the raw operational data stream, and caching of this data stream was initiated. For example... Figure 5The diagram illustrates the location of abrupt changes in the time-domain statistical characteristics of the original operation and maintenance data stream. The module analyzes an indicator that exhibits a clear 24-hour periodicity under normal conditions. Combined with... Figure 5 Mid-time window As can be seen from the curve, the sample mean is calculated from a subset of data. ,variance The corresponding stability parameter The value is below the preset value of 0.05, meeting the stability requirements. (Continue to refer to...) Figure 5 Mid-time window The curve, due to a malfunction in the upstream data acquisition system, lost its periodicity and presented as random noise. At this point, the recalculated sample mean abruptly changed... ,variance The corresponding stability parameter It exceeds 0.05. Substituting this into the formula, we obtain the jump parameter for the quantified data jump. The value increased sharply and exceeded the preset diagnostic threshold of 3.0. Due to a sudden change in the periodic features associated with the Fast Fourier Transform operator, the cause of the statistical feature loss in the original operational data stream was determined to be the loss of periodic features. Therefore, the final drift root cause label is generated and output as periodic feature loss. This label will be passed to subsequent steps to guide the targeted reconstruction of the population.

[0061] In one embodiment of the present invention, the refactoring instruction generation module is configured to perform the following steps: Retrieve the preset rule mapping matching matrix; Read the abnormal missing attributes exposed in the drift root cause tags; Using the abnormal missing attribute as the query keyword, a search is performed in the rule mapping matching matrix to obtain the candidate operator category with attribute compensation characteristics; When generating directional reconstruction instructions, the expected selection probability threshold of candidate operator categories is increased based on a preset positive adjustment constant; the failure operator category is located based on the performance degradation feature vector, and the basic mutation probability of the failure operator category is reduced based on a preset negative adjustment constant; the probabilities of all adjusted operator categories are normalized using a normalized exponential function to generate operator probability adjustment parameters; the drift root cause label and the operator probability adjustment parameters are concatenated to form directional reconstruction instructions.

[0062] Specifically, the process of extracting and integrating the missing attributes contained in the drift root cause tags, and combining them to generate targeted reconstruction instructions for guiding population evolution, is executed by a reconstruction instruction generation module integrated within or closely cooperating with the drift root cause localization module. During execution, this unit retrieves a pre-built rule mapping matching matrix stored in memory. This rule mapping matching matrix is ​​a key-value pair data structure pre-installed in system memory, where the keys are possible drift root cause tags, and the values ​​are lists containing multiple operator category names, establishing a mapping relationship from problem diagnosis to solution. This unit reads the drift root cause tag content output by the drift root cause localization module and extracts abnormal missing attributes, such as periodicity or stationarity. Using this missing attribute as a query keyword, it searches the rule mapping matching matrix. The matrix is ​​designed to map the absence of a data attribute to one or more candidate operator categories with corresponding attribute compensation characteristics.

[0063] After a successful query, the unit obtains a set of recommended candidate operator categories. Candidate operator categories refer to feature processing operators or models recommended based on rule-mapped matching matrices, considered capable of effectively handling the current data characteristics, and are components that should be prioritized in the next generation of the pipeline population. The refactoring instruction generation module calculates the expected selection probability threshold of these recommended candidate operator categories in the next round of natural evolution, essentially increasing their probability of being selected in the selection, crossover, and mutation operations of the genetic algorithm. The expected selection probability threshold refers to a higher base selection probability target set for the candidate operator category, such as from the original... Upgraded to .

[0064] Simultaneously, this unit uses the performance degradation feature vector generated by the population screening and diagnosis module to locate the category of failed operators pointed to within it, and specifically reduces the probability of their basic mutation. In this embodiment, the category of failed operators refers to the category of operators with the highest correlation to performance degradation in the performance degradation feature vector; the probability of basic mutation is a fundamental parameter for controlling population diversity in genetic algorithms, and reducing this probability means discouraging the system from continuing to try such proven ineffective operators.

[0065] After probability adjustment, the newly generated set of parameters containing the adjusted probabilities of all operator nodes is called the operator probability adjustment parameters. In essence, the operator probability adjustment parameters are data objects, such as dictionaries or hash maps, that encapsulate the latest selection probabilities of all operator categories, and will serve as direct input for subsequent genetic mutation operations. The reconstruction instruction generation module concatenates and encapsulates the text-formatted drift root cause labels with the structured operator probability adjustment parameters, combining them to form targeted reconstruction instructions that can regulate the logic of the population reproduction process.

[0066] The directed reconfiguration command is a composite data packet containing both high-level semantic information, i.e., drift root cause tags, and low-level, executable control parameters, i.e., operator probability adjustment parameters. The system establishes an independent communication link between the drift root cause localization module and the closed-loop detection execution module. This link can utilize a remote procedure call channel. It should be noted that this low-level independent communication link is a dedicated channel established to ensure that control commands can bypass regular business data flows and achieve high-priority, low-coupling transmission; in engineering, it can be implemented using methods such as gRPC or dedicated message queue topics. The system then issues the directed reconfiguration command to the evolution controller via this link.

[0067] The basic selection probability of the operator node is adjusted, and its update process can be represented by the following formula: ; In the formula, Representing the The new selection probability after adjusting each operator node is a component of the operator probability adjustment parameter. This represents the total number of operator categories in the basic operator component library. Representing the The logarithmic probabilities or basic weights of each operator node before adjustment can typically be initialized from a uniform distribution or a historical empirical distribution. It is a probability adjustment factor, and its value is determined according to the following rules: ; in, Representing the An operator, It is a set of candidate operator categories. It is a set of failure operator categories. It is a positive adjustment constant. It is a negative adjustment constant. The entire formula ensures that the sum of all adjusted probabilities remains 1 through a normalized exponential function.

[0068] It should be noted that the preset rule mapping matching matrix is ​​a key mapping dictionary constructed based on common data distribution characteristics in the operations and maintenance field. To ensure the effectiveness of self-healing reconstruction, this matrix in this embodiment specifically includes, but is not limited to, the following four sets of core mapping rules: Periodic feature loss mapping: When the abnormal missing attribute is a periodic feature loss, the candidate operators for mapping include: Fast Fourier Transform operator, Wavelet Transform operator, and Autocorrelation coefficient calculation operator; Mean step / drift mapping: When the missing attribute is a step change in the mean of the data, the candidate operators for the mapping include: first-order difference operator, moving average operator, and exponential smoothing operator. Variance / volatility mutation mapping: When the anomalous missing attribute is a sudden increase in variance or heteroscedasticity, the candidate operator categories for mapping include: logarithmic transformation operator, Z-Score normalization operator, and Box-Cox transformation operator; High-frequency noise increase mapping: When the abnormal missing attribute is a decrease in signal-to-noise ratio, the candidate operator categories for mapping include: Kalman filter operator, low-pass filter operator, and median filter operator.

[0069] By pre-setting a complete mapping matrix covering frequency domain, time domain trends, and fluctuation characteristics, the system can provide attribute compensation operators with clear physical meaning and mathematical support for most common operational data distribution drift phenomena.

[0070] For example, the reconstruction instruction generation module receives a drift root cause label as periodic feature loss. The unit retrieves a rule mapping matching matrix, which contains a mapping rule of {"periodic feature loss": ["mean operator", "variance operator", "difference operator"]}. Therefore, the system determines the candidate operator categories to be these three statistical operators. Simultaneously, the system references the performance degradation feature vector generated in the previous steps, which indicates that the Fast Fourier Transform operator is the failure operator category. The operator library contains 10 operators, each with an initial base selection probability of 0.1 and corresponding base weights. All are constants. The system is set to positively adjust constants. negative adjustment constant For the mean operator, its adjustment factor For the Fast Fourier Transform operator, its adjustment factor For other unspecified operators, the adjustment factor is 0. (Combined) Figure 6 The comparison chart of operator mutation probability adjustments shows that, after calculation using the formula, the new selection probabilities of the three operators—mean, variance, and difference—are... The probability of a single operator will be significantly increased to approximately 26.1%, while the probability of a new choice for the Fast Fourier Transform operator will be significantly reduced to approximately 0.5%. The probabilities of other operators will also be adjusted accordingly to make their sum equal to 1. Finally, the system packages the generated operator probability adjustment parameters, such as a JSON object {"mean operator": 0.261, "variance operator": 0.261, "difference operator": 0.261, ..., "Fast Fourier Transform operator": 0.005, ...}, together with the periodic feature loss of the drift root cause label, into a directional reconstruction instruction. Finally, this instruction is encrypted and sent to the loop closure detection execution module via the gRPC channel, ready to take effect in the next round of evolution.

[0071] In one embodiment of the present invention, the closed-loop detection execution module is configured to perform the following steps: Remove the operator probability adjustment parameters from the front end of the directional reconstruction instruction and replace the built-in mutation control matrix of the anomaly detection pipeline population; Based on the updated mutation control matrix, crossover and directed mutation operations are triggered to generate a new generation of pipeline sets; Remove the drift root cause label attached to the targeted reconstruction instruction, perform network topology mapping by connecting to the pre-configured business asset network topology dictionary, and export security alarm information; The system merges and packages the current best pipeline, the next-generation pipeline set, and safety alarm information, renders and transmits them to the front-end monitoring dashboard system, and outputs a diagnostic report package.

[0072] Specifically, the process of extracting directional reconstruction instruction parameters to forcibly intervene in the pipeline population evolution process and outputting a diagnostic report package with traceability guidance to complete the closed-loop detection business is collaboratively executed by the closed-loop detection execution module deployed in the system core. It should be noted that the closed-loop detection execution module is the core engine of the automated machine learning framework, responsible for managing the lifecycle of the pipeline population, including genetic operations such as selection, crossover, and mutation.

[0073] During execution, the closed-loop detection execution module receives and parses the directional reconstruction command through an underlying independent communication link. The module extracts the operator probability adjustment parameter from the command message front-end; this parameter is a set containing the probability of all operators being selected in the new round. The module then replaces the mutation control matrix built into the basic population structure with this probability parameter. This replacement operation forcibly alters the behavioral preferences of subsequent genetic operators.

[0074] The mutation control matrix is ​​a set of parameters and rules used in genetic algorithms to define and control mutation and crossover operations. Essentially, it's the core data structure controlling the randomness and directionality of population evolution. In this embodiment, its core component is the selection probability distribution of each operator. The closed-loop detection execution module, based on the updated mutation control matrix, controls the basic population structure to trigger evolutionary operations, namely crossover and directional mutation. Crossover is a standard reproduction operation in genetic algorithms, generating new offspring pipelines by exchanging parts of the structures of two parent pipelines. Directional mutation refers to the selective mutation of operators, rather than random replacement or addition, guided by operator probability adjustments.

[0075] During the propagation operation, the system prioritizes candidate operators with increased probability and avoids using ineffective operators. This generates a new generation of pipeline sets adapted to the current data characteristics. It should be understood that the new generation of pipeline sets is a new population generated after one round of directed evolution, and theoretically, its average fitness, i.e., average total cost, will be better than that of the previous generation.

[0076] Meanwhile, the closed-loop detection execution module extracts another part from the targeted reconstruction instructions: the drift root cause label. Using the asset identification identifier contained in this label, it interfaces with a pre-built business asset network topology dictionary that records the mapping relationship between business assets and physical network devices to perform network topology mapping.

[0077] In this embodiment, the business asset network topology dictionary is a database or configuration file that maintains the relationship between the logical identifiers of IT assets and their physical deployment locations, network addresses, and business domains. It serves as a crucial bridge from technical alerts to business impact analysis. Network topology mapping refers to mapping diagnostic tags to language descriptions that are relevant to the business scenario and understandable and executable by operations and maintenance personnel. This mapping process associates abstract diagnostic tags, such as periodic feature loss, with specific business systems and potentially problematic physical devices or application modules, thereby deriving a readable security alert message with clear guidance on tracing the source of the anomaly.

[0078] Security alerts include speculative guidance on the cause of the anomaly, such as: "The core database service metric 'asset-db-core' is periodically missing. We recommend checking the execution status of the nighttime ETL batch task 'job-daily-sync'."

[0079] The closed-loop detection execution module merges the current optimal pipeline, the next-generation pipeline set, and safety alarm information into a diagnostic report package. This diagnostic report package is a comprehensive output object that encapsulates all key information about this self-healing closed-loop event, including the optimal model, new population status, and diagnostic conclusions, providing a complete record for operational auditing and continuous improvement. The system pushes the diagnostic report package to the monitoring dashboard system via the application programming interface. The monitoring dashboard displays the automatic diagnostic and root cause localization results, completing the detection closed loop.

[0080] For example, the closed-loop detection execution module receives a directional reconstruction instruction containing periodic feature loss of drift root cause labels and operator probability adjustment parameters. The closed-loop detection execution module immediately updates its internal mutation control matrix using new probability parameters, such as {"mean operator": 0.261, ..., "fast Fourier transform operator": 0.005}. The closed-loop detection execution module initiates a new round of evolutionary processes. When selecting a parent for crossover, the system is more likely to choose pipelines that already include the mean operator or difference operator. When performing mutation operations, if a pipeline needs to introduce a new feature processing operator, the system has a 26.1% probability of choosing the mean operator and only a 0.5% probability of choosing the fast Fourier transform operator. Through this series of controlled evolutionary operations, a completely new generation of pipelines is ultimately derived, most of which tend to use statistical operators rather than frequency domain analysis operators. Simultaneously, the closed-loop detection execution module detects the loss of periodic characteristics of the drift root cause label and identifies the associated asset as "asset-db-core". By querying the business asset network topology dictionary, it is found that the periodicity of the "asset-db-core" indicator is generated by a nighttime data aggregation task named "job-daily-report-aggregation". Therefore, the diagnostic label is converted into a security alert message: "Alert Level: High; Affected Asset: Core Database Service; Diagnostic Conclusion: Periodic characteristics of the indicator are lost; Recommended Investigation: Immediately check whether the nighttime data aggregation task 'job-daily-report-aggregation' was successfully executed in the past 24 hours". The metadata of the currently used best pipeline, the next-generation pipeline set more adapted to non-periodic data, and the above detailed security alert message are integrated together to form a diagnostic report package in JSON format, which is sent to the front-end dashboard of the monitoring center. The operations team can directly troubleshoot the problem based on this report.

[0081] Each of the modules can be implemented in whole or in part through software, hardware, or a combination thereof. It supports hardware embedded in or independent of the processor in the computer device, and also supports software stored in the memory of the computer device, so that the processor can call and execute the operations corresponding to each of the above modules.

[0082] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. An automatic detection system for operational anomalies based on machine learning, characterized in that, include: The cost function construction module is used to obtain the original operation and maintenance data stream and business impact monitoring configuration, and match the cost weights of false alarms and false alarms to generate a dynamic cost function; The initialization and evaluation module is used to initialize the anomaly detection pipeline population and input the benchmark test set, and calculate the total cost scalar value of each candidate pipeline branch in combination with the dynamic cost function. The population screening and diagnosis module is used to screen the population based on the total cost scalar value, extract the current optimal pipeline and the set of pipelines to be eliminated, and generate a performance degradation feature vector based on the operator node characteristics of the elimination set. The drift root cause localization module is used to extract high-risk operator features based on the performance degradation feature vector, scan the original operation and maintenance data stream along the time domain distribution direction it points to, extract abnormal missing attributes, and generate drift root cause labels. The reconstruction instruction generation module is used to match candidate operator categories based on abnormal missing attributes, adjust the selection probability of operator categories by combining performance degradation feature vectors, and generate targeted reconstruction instructions. The closed-loop detection execution module is used to inject directional reconstruction instructions into the next-generation pipeline set for population reproduction, integrate the current best pipeline, the next-generation pipeline set, and safety alarm information converted from drift root cause labels, and output a diagnostic report package.

2. The automatic detection system for operation and maintenance anomalies based on machine learning according to claim 1, characterized in that, The cost function building block is used to perform the following operations: Extract the identification identifiers of the monitored objects from the original operation and maintenance data stream, query the business impact monitoring configuration, and obtain the specific business impact level corresponding to the monitored object; Based on the specific business impact level, false alarm cost weights and false alarm cost weights are assigned to false alarm and false alarm cost weights for the monitored objects, respectively. Obtain the number of false positive samples and false negative samples generated by the upstream inspection and classification process; Calculate the product of the number of false positive samples and the cost weight of false alarms, and the product of the number of false negative samples and the cost weight of false negatives. Sum the product results of all monitored objects to generate a dynamic cost function.

3. The automatic detection system for operation and maintenance anomalies based on machine learning according to claim 1, characterized in that, The initialization and evaluation module is used to perform the following operations: Arrange feature processing operators and anomaly detection models to generate an anomaly detection pipeline population containing multiple candidate pipeline branches; Slice a recent historical benchmark set containing known fault labels and synchronously input it into the anomaly detection pipeline population to perform parallel inference and output a predicted label set; Compare the predicted label set with the real labels of recent historical benchmark sets, and statistically analyze the elements of the verification confusion matrix for various monitoring indicators; The elements of the verification confusion matrix are input into the dynamic cost function for weighted calculation, and the total cost scalar value corresponding to each candidate pipeline branch is output.

4. The automatic detection system for operation and maintenance anomalies based on machine learning according to claim 1, characterized in that, When the population screening and diagnostic module screens the population based on the total cost scalar value, it performs the following operations: Sort all candidate pipeline branches in the population according to the ascending order of total cost scalar values ​​from low to high; Select the first candidate pipeline branch in the ranking sequence as the current optimal pipeline; Select candidate pipeline branches at the end of the ranking sequence with a preset proportion and a number not less than a preset low target threshold, and combine them to construct an elimination pipeline set.

5. The automatic detection system for operation and maintenance anomalies based on machine learning according to claim 1, characterized in that, The population screening and diagnostic module performs the following operations when generating performance degradation feature vectors: Analyze the eliminated operator nodes contained in the elimination pipeline set; The first occurrence frequency of eliminated operator nodes in the elimination pipeline set and the second occurrence frequency in the anomaly detection pipeline population are statistically analyzed. Calculate the ratio of the first occurrence frequency to the second occurrence frequency to obtain the failure correlation parameters between the eliminated operator node and the detected blocking state; The failure-related parameters of all operator nodes are combined to generate a performance degradation feature vector.

6. The automatic detection system for operation and maintenance anomalies based on machine learning according to claim 1, characterized in that, The drift root cause localization module performs the following operations when extracting high-risk operator features: Analyze the performance degradation feature vector and sort the component values ​​of each dimension of the performance degradation feature vector in descending order; Extract the operators corresponding to the dimensions with the highest preset number of component values ​​as high-risk operator features; High-risk operator features are configured as dedicated diagnostic filters to intercept and build a processing cache that stores the original operation and maintenance data stream according to a preset time window.

7. The automatic detection system for operation and maintenance anomalies based on machine learning according to claim 1, characterized in that, The drift root cause localization module performs the following operations when generating drift root cause tags: Calculate the stability parameter of the original operation and maintenance data stream within the current time window along the time domain distribution direction pointed to by the characteristics of the high-risk operators; Calculate the normalized mean difference of the original operation and maintenance data stream between adjacent time windows, and use it as the jump parameter; The stability parameter is compared with the preset first diagnostic threshold and the jump parameter is compared with the preset second diagnostic threshold. If either parameter exceeds its corresponding diagnostic threshold, the original operation and maintenance data stream is determined to be in a sudden change state. Based on the mutation state, determine the reasons for the loss of statistical characteristics of the original operation and maintenance data stream, extract the corresponding abnormal missing attributes, and generate drift root cause labels containing abnormal missing attributes.

8. The automatic detection system for operation and maintenance anomalies based on machine learning according to claim 1, characterized in that, The reconstruction instruction generation module performs the following operations when matching candidate operator categories: Retrieve the preset rule mapping matching matrix; Read the abnormal missing attributes exposed in the drift root cause tags; Using the abnormal missing attribute as the query keyword, a search is performed in the rule mapping matching matrix to obtain candidate operator categories with attribute compensation characteristics.

9. The automatic detection system for operation and maintenance anomalies based on machine learning according to claim 1, characterized in that, When generating targeted refactoring instructions, the refactoring instruction generation module performs the following operations: The expected selection probability threshold of the candidate operator category is increased based on a preset positive adjustment constant. Based on the performance degradation feature vector, the failure operator category is located, and the probability of basic mutation of the failure operator category is reduced based on the preset negative adjustment constant; The probabilities of all operator categories are normalized using a normalized exponential function to generate operator probability adjustment parameters. The drift root cause label and the operator probability adjustment parameter are connected in series to form a directional reconstruction instruction.

10. The automatic detection system for operation and maintenance anomalies based on machine learning according to claim 1, characterized in that, The closed-loop detection execution module is used to perform the following operations: Remove the operator probability adjustment parameters from the front end of the directional reconstruction instruction and replace the built-in mutation control matrix of the anomaly detection pipeline population; Based on the updated mutation control matrix, crossover and directed mutation operations are triggered to generate a new generation of pipeline sets; Remove the drift root cause label attached to the targeted reconstruction instruction, perform network topology mapping by connecting to the pre-configured business asset network topology dictionary, and export security alarm information; The system merges and packages the current best pipeline, the next-generation pipeline set, and safety alarm information, renders and transmits them to the front-end monitoring dashboard system, and outputs a diagnostic report package.

Images