A real-time process monitoring method based on data analysis

Abstract

The application relates to a real-time process monitoring method based on data analysis, comprising the following steps: the method comprises the following steps: S1, based on the component running scene type, real-time resource occupancy and data dependency, a three-dimensional component priority evaluation model is constructed, and a collection strategy is dynamically matched; S2, according to the data source, a cleaning rule is adapted, a feature extraction dimension is adjusted in combination with process dynamic characteristics, and an improved time decomposition algorithm is used to separate data trends, fluctuations and abnormal residuals; S3, according to the scene label, an online learning algorithm is used to construct a special baseline submodel, and a scene switching mechanism is established; S4, in combination with the abnormal feature of component interaction, an abnormal grade is determined through a hybrid detection model; S5, based on abnormal processing and user feedback, an error correction model is constructed to optimize parameters; and S6, a report containing an abnormal propagation path is generated, and a hierarchical collaborative response of associated components is triggered. The application aims to solve the problems of single collection dimension, no scene adaptability of preprocessing and incomplete abnormal detection of existing monitoring technology.

Description

Technical Field

[0001] This invention belongs to the field of data processing and monitoring technology, and relates to a real-time process monitoring method based on data analysis. Background Technology

[0002] Real-time process monitoring is a core technology for ensuring the stable operation of industrial production, internet services, and equipment maintenance. It continuously collects and analyzes information such as component operating status, data interaction, and resource consumption during the process, enabling early identification of anomalies, risk warnings, and rapid response, directly impacting process efficiency, resource utilization, and business continuity. With the advancement of Industry 4.0 and digital transformation, real-time processes are increasingly characterized by dynamic switching across multiple scenarios, such as startup and commissioning, full-load operation, and shutdown maintenance in industrial production; peak and off-peak traffic scenarios in internet services; close component interrelationships, such as data dependencies between equipment on production lines and interactions between modules in service architectures; and diverse data types, such as time-series data collected by sensors, system-generated text logs, and business-level interactive data. These characteristics place higher demands on the accuracy, adaptability, and coordination of monitoring technologies. They must not only cover the operating status of individual components but also relate the interactions between components; not only adapt to fixed scenarios but also dynamically respond to changes in scenarios; and not only detect anomalies but also form a closed-loop mechanism of detection-processing-optimization.

[0003] However, in current production and practical use of real-time process monitoring, existing technologies still have multi-dimensional problems, making it difficult to meet the needs of dynamic and complex processes: Data collection is singular; most solutions determine collection strategies based solely on component historical performance or fault records, without considering the importance of the real-time process scenario, such as the different monitoring accuracy requirements of core business scenarios and edge scenarios, fluctuations in real-time resource usage, and data dependencies between components. This easily leads to insufficient collection and missed anomalies in core scenarios, and redundant collection and wasted resources in edge scenarios. Data preprocessing lacks scenario adaptability; different types of data, such as sensor data, log data, and interaction data, are treated with a uniform cleaning and time-series decomposition strategy, without considering the dynamic characteristics of process changes, such as equipment start-up and shutdown, and sudden traffic adjustment logic. This results in low data accuracy after preprocessing, unable to support subsequent accurate detection. The baseline model is static and lacks scenario differentiation; a single fixed baseline is built based on historical normal data, which cannot adapt to the dynamic switching of real-time process scenarios. When the scenario changes, the baseline and the actual data become inconsistent. Significant deviations in actual process behavior lead to a sharp increase in false alarm and false negative rates. Anomaly detection only covers a single dimension and algorithm, judging anomalies solely by comparing real-time data with a single baseline, without considering anomalies in data interaction between components, such as transmission delay deviations and data packet loss rates. It cannot identify cross-component collaborative faults, resulting in insufficient detection comprehensiveness. Adaptive adjustment lacks a closed loop, adjusting model parameters only based on basic system feedback or simple user feedback, without linking it to anomaly handling effects, such as anomaly resolution rate, fault recovery time, and business impact, such as the decline in business indicators. The adjustment direction lacks specificity, and model performance continuously degrades with process changes. Reporting and response lack contextualization and collaboration. Monitoring reports only list abnormal events without anomaly propagation paths or contextualized optimization suggestions. Anomaly responses only target single abnormal components, failing to trigger collaborative control of related components through component dependencies. This leads to difficulties in fault tracing, rapid anomaly propagation, and poor processing efficiency, ultimately manifesting as low monitoring accuracy, serious resource waste, high false alarm and false negative rates, and poor fault handling efficiency.

[0004] After reviewing relevant materials, several technical solutions have been proposed to address this issue. A representative example is the Chinese invention patent CN116955092A, "Multimedia System Monitoring Method and System Based on Data Analysis." The core principle of this patent is as follows: data acquisition is optimized by dynamically adjusting the frequency and depth of data collection; the collected data undergoes cleaning, feature extraction, and time-series analysis preprocessing; a single baseline model is constructed based on normal operating data; anomaly detection is achieved by comparing real-time data with the baseline model using a support vector machine algorithm; model parameters are adjusted based on system and user feedback; and finally, a monitoring report is generated and an automatic response is triggered. From an application perspective, this solution overcomes the limitations of traditional multimedia system monitoring, which relies on manual inspection or fixed-frequency log review. It improves the anomaly detection rate and response timeliness through data analysis technology, making it applicable to specific scenarios such as monitoring audio, video, and text data stream components in multimedia systems. However, when applied to general real-time process monitoring, it has significant drawbacks: First, it lacks scenario adaptability. Designed for multimedia systems, it cannot adapt to the dynamic switching of multiple scenarios in real-time processes, such as the differences between industrial production and internet services. The baseline model is a single static model, and the monitoring accuracy drops significantly when the scenario changes. Second, the data acquisition and detection dimensions are limited. Data acquisition is based solely on the criticality of component performance and historical fault assessment, without covering real-time resource usage fluctuations and component dependencies. Anomaly detection only compares real-time data with the baseline using a single support vector machine algorithm, without considering anomalies in inter-component interactions, and cannot handle complex cross-component collaborative faults in real-time processes. Third, the optimization and response mechanisms are imperfect. Adaptive adjustment is not associated with anomaly handling effects and business impacts, lacks closed-loop logic, and anomaly response is only for a single component, without associated component collaborative control, which cannot meet the needs of real-time processes for rapid fault tracing and propagation control. Summary of the Invention

[0005] This invention provides a real-time process monitoring method based on data analysis, which solves the problems of existing technologies in real-time process monitoring, such as single data acquisition dimension, lack of scenario adaptability in preprocessing, static baseline model without scenario differentiation, single-dimensional and single-algorithm anomaly detection, lack of closed-loop adaptive adjustment, and lack of scenario-based and collaborative reporting and response. These problems result in low monitoring accuracy, resource waste, high rate of anomaly missed detection and false alarm, and poor fault handling efficiency.

[0006] To solve the above problems, the technical solution adopted by the invention is as follows:

[0007] A real-time process monitoring method based on data analysis includes the following steps: S1: Dynamic multi-source data acquisition; based on the component operation scenario type, real-time resource occupancy rate, and data dependency between components in the real-time process, a three-dimensional component priority evaluation model is constructed, and a data acquisition strategy is dynamically matched; the three-dimensional component priority evaluation model calculates a priority score by integrating scenario weight, resource weight, and dependency weight, using the following formula: , The importance of the component's runtime scenario, R represents the scene weight, and R represents the real-time resource utilization rate. Where D is the resource weight, and D is the data dependency between components. For dependent weights, and S2: Scenario-based data preprocessing: Adapt cleaning rules to the source type of real-time process data, adjust feature extraction dimensions based on the dynamic changes of the process, and use an improved time-series decomposition algorithm to separate data trends, fluctuations, and abnormal residuals; S3: Multi-scenario baseline model construction: Based on different operating scenario labels of the real-time process, use online learning algorithms to build scenario-specific baseline sub-models, update sub-model parameters with newly added normal data within the scenario in real time, and establish a scenario switching trigger mechanism; S4: Correlated anomaly detection: Input the preprocessed real-time data into the corresponding scenario baseline sub-model, combine the abnormal features of data interaction between components, and determine the anomaly level through a hybrid detection model that integrates classification algorithms and anomaly scoring algorithms; S5: Feedback-driven adaptive optimization: Based on anomaly handling effect feedback and user scenario-based feedback, build a multi-dimensional error correction model to dynamically optimize data acquisition parameters, preprocessing rules, and baseline model thresholds; S6: Scenario-based report generation and collaborative response: Generate scenario-based monitoring reports containing anomaly propagation paths based on anomaly level and real-time process scenario characteristics, and trigger hierarchical collaborative response measures for related components based on component dependency graphs.

[0008] The principle and advantages of this solution are as follows:

[0009] The core principle of this solution is to construct a real-time process monitoring framework around scenario-based adaptation and multi-dimensional collaboration, achieving a dual improvement in monitoring accuracy and efficiency through dynamic optimization across the entire chain. In the data acquisition phase, a three-dimensional priority evaluation model is built based on component operating scenario types, real-time resource occupancy, and data dependencies between components. Priority scores are calculated by quantifying the weight of scenario importance, resource fluctuations, and dependency strength, thereby dynamically matching the acquisition frequency and dimensions to ensure priority allocation of acquisition resources for core scenarios and high-risk components. In the data preprocessing phase, specific cleaning rules are adapted for different source types such as sensor data, log data, and interaction data. The focus of feature extraction is adjusted based on dynamic changes such as process startup, stability, and shutdown. Simultaneously, an improved time-series decomposition algorithm is introduced to separate data trends, fluctuations, and abnormal residuals, eliminating interference from unstable processes on data accuracy. In the baseline modeling phase, a scenario-specific baseline sub-model is constructed based on the real-time process's operating scenario labels, and online learning is used to... The algorithm integrates newly added normal data in real time to update model parameters and designs a scene switching trigger mechanism to achieve smooth transition and dynamic adaptation of baselines under different scenarios. In the anomaly detection stage, preprocessed data is input into the corresponding scenario baseline sub-model, while interactive anomaly features such as transmission delay deviation between components and data packet loss rate are extracted. The anomaly level is quantified through a hybrid model of classification algorithm and anomaly scoring algorithm, covering component-specific and cross-component collaborative anomalies. In the adaptive optimization stage, the anomaly handling effect, resolution rate, recovery time and user scenario-based feedback are integrated to build a multi-dimensional error correction model, dynamically optimizing the collection parameters, preprocessing rules and baseline thresholds to form a closed loop of detection-processing-feedback-optimization. Finally, through scenario-based report generation and collaborative response supported by component dependency graph, accurate source tracing and rapid control of anomalies are achieved.

[0010] Compared to existing technologies, which employ a single baseline and algorithm for detection and cannot identify cross-component collaborative anomalies, this solution eliminates baseline deviations caused by scene switching through multi-scene-specific baselines. By combining component interaction features with a hybrid detection model, it significantly improves the anomaly detection rate compared to existing technologies. Upgraded to The above measures reduce the rate of missed detections for cross-component collaborative anomalies. For example, in industrial production line monitoring, this solution can accurately identify collaborative faults where transmission delays in device A cause abnormal processing in device B, a type of fault that traditional solutions cannot cover. Existing technologies suffer from resource redundancy in edge scenarios or insufficient data collection in core scenarios due to their single data acquisition dimension. This solution dynamically matches acquisition resources using a three-dimensional priority model, reducing the amount of data collected in edge scenarios. Improved data acquisition accuracy in core scenarios Taking internet service monitoring as an example, the frequency of edge service data collection during off-peak hours is reduced to [missing information]. During peak periods, the core service collection frequency has increased to [number missing]. This approach aims to reduce server resource consumption while ensuring effective monitoring; however, existing static baseline technologies suffer from a high false alarm rate during scene switching. Furthermore, lacking closed-loop optimization logic, this solution utilizes a scene switching trigger mechanism and online learning to control the false alarm rate during scene switching within a certain range. Within this timeframe, and based on a multi-dimensional error correction model, parameters are automatically optimized every 30 minutes. For example, when switching from start-up and commissioning to full-load operation in industrial production, the baseline model can smoothly transition within 0.8 seconds, and the detection threshold is continuously optimized as operational data accumulates. Existing technologies report no abnormal propagation path and respond only to a single component, resulting in a fault recovery time of up to 45 seconds. This solution, through scenario-based reporting of abnormal source tracing paths and collaborative responses using component dependency graphs, reduces the response time to within 0.6 seconds, significantly lowering the fault recovery time. For example, when a core service fails, the system can quickly sever non-critical links between the failed component and upstream data providers and downstream data receivers through the dependency graph, while simultaneously activating backup components to prevent the failure from spreading to the entire service architecture and significantly reduce the risk of business interruption.

[0011] Furthermore, the dynamic matching data acquisition strategy in S1 specifically includes: S11: Dividing the acquisition level according to the priority score P, This is a level 1 data collection. This is a level 2 data collection. It is a three-level collection. Level 4 acquisition; S12: Level 1 acquisition uses the acquisition frequency The data collected includes component running status, resource consumption details, internal interaction logs, and error stack information; secondary data collection uses a specific collection frequency. The collected data includes component running status, resource consumption summaries, and key interaction logs; the three-level collection adopts a collection frequency... The data collected includes the basic operational status of components and average resource consumption; the fourth-level data collection uses a specific collection frequency. The collected data only includes the online status data of components; S13: Real-time monitoring of component priority score changes, when the fluctuation of P exceeds... When the time is right, it automatically switches to the corresponding acquisition level and adjusts the response time accordingly. .

[0012] Furthermore, the scenario-based data preprocessing in S2 specifically includes: S21: Sensor data is filled with missing values ​​using interpolation, through improvement... Rules for identifying outliers; log data uses keyword matching to complete missing fields and extracts semantic features using natural language processing; interactive data uses association-based completion to complete missing items and identifies abrupt changes using temporal difference; S22: In the improved temporal decomposition algorithm, the scene adaptation coefficient... ,pass The fluctuation decomposition results are corrected, enhancing fluctuation sensitivity in core scenarios and reducing fluctuation interference in edge scenarios; S23: During feature extraction, the proportion of high-frequency feature extraction is increased to 70% in core scenarios, and the proportion of low-frequency feature extraction is increased to [missing information] in edge scenarios. .

[0013] Furthermore, the multi-scenario baseline model construction in S3 specifically includes: S31: The scenario labels are divided based on the core business indicators of the real-time process. Each scenario label corresponds to a set of clear business indicator threshold ranges. According to the business characteristics and data features of different scenarios, the parameter setting rules for the initial baseline sub-model are as follows: The initial model learning rate for core scenarios is set to 0.02, the number of decision trees is set to 80, and the maximum depth is set to 15; the initial model learning rate for edge scenarios is set to 0.05, the number of decision trees is set to 50, and the maximum depth is set to 8; the initial model learning rate for transitional scenarios is set to 0.03, and the decision tree depth is set to 15. The number of trees is set to 60, and the maximum depth is set to 10. During the construction of the initial baseline sub-model, scene feature vectors are introduced, and the K-means clustering algorithm is used to train scene classification on historical normal data. S32: The online learning algorithm adopts incremental random forest. The sub-model parameters are updated once every 1000 new normal data are received. The historical optimal parameters are retained during the update process. If the model accuracy drops by more than 3% after the update, it will automatically roll back to the historical optimal parameters. S33: The scene switching trigger mechanism is implemented by real-time monitoring of core business indicators. When the indicator exceeds the current scene threshold range for a prolonged period of time, the scene will be switched. When the scene is switched automatically, the corresponding scene baseline sub-model is called. During the switching process, a smooth transition algorithm is used to avoid detecting faults.

[0014] Furthermore, the correlation-based anomaly detection in S4 specifically includes: S41: Anomaly characteristics of data interaction between components are achieved by constructing an interaction health index, the index calculation formula being: , This is the difference between the actual transmission delay and the historical average delay. Let L be the historical average latency and L be the data loss rate. Marked as an interaction anomaly; S42: In the hybrid detection model, the classification algorithm uses a lightweight neural network to output the probability of normality. The anomaly scoring algorithm uses an isolated forest to output anomaly scores. The final abnormality level determination formula is: ; This is a serious abnormality. This is a general abnormality. It is a minor abnormality. Normal; S43: Minor anomalies trigger automatic review process, general anomalies trigger operation and maintenance personnel warning, and serious anomalies directly trigger emergency response.

[0015] Furthermore, the feedback-driven adaptive optimization in S5 specifically includes: S51: Calculating the comprehensive error using a multi-dimensional error correction model. , For model prediction error, , These are the model's predicted values. This is the actual value. This is due to anomaly handling error. =, For user feedback on errors, S52: When At that time, the adaptive momentum gradient descent method was used to optimize the model parameters, and the learning rate was... Simultaneously, the data collection frequency was adjusted, with the first-level collection frequency increased by 20% in severe abnormal scenarios and the fourth-level collection frequency decreased by 15% in minor abnormal scenarios; S53: User scenario-based feedback was collected through scenario-specific questionnaires, including the degree of impact of the anomaly on the business and the timeliness of the handling. The feedback results were used to adjust feature weights, with the weight of high-impact features marked by users increased by 30%.

[0016] Furthermore, the scenario-based report generation in S6 specifically includes: S61: The scenario-based monitoring report includes a scenario overview, anomaly details, anomaly propagation path, and scenario-based optimization suggestions; S62: The report generation frequency is dynamically adjusted according to the anomaly level, with severe anomalies generated in real time, general anomalies generated every 2 minutes, and minor anomalies generated every 10 minutes; S63: Reports in core scenarios add business impact simulation analysis, and reports in edge scenarios add anomaly trend prediction.

[0017] Furthermore, the collaborative response in S6 specifically includes: S61: Establishing a mapping table of anomaly level, response measures, and related components. The response measures for severe anomalies are: suspending the business processing of the abnormal component, starting the backup component to take over the business, cutting off the interaction link between the abnormal component and the core component, and notifying the technical lead; the response measures for general anomalies are: limiting the resource usage of the abnormal component, starting the component's self-diagnosis program, and notifying the operations and maintenance personnel; the response measures for minor anomalies are: recording the anomaly log and starting anomaly trend monitoring; S62: The collaborative response of related components is implemented based on the component dependency graph, identifying the direct upstream and downstream components of the abnormal component. The upstream component suspends sending non-critical data to the abnormal component, and the downstream component starts a data caching mechanism; S63: After the response measures are executed, the operating status of the abnormal component and related components is monitored in real time. When the anomaly is resolved, normal operation is restored in the order of downstream component cache data synchronization → abnormal component business recovery → upstream component data transmission recovery. During the recovery process, core business indicators are continuously monitored to ensure that there are no secondary anomalies.

[0018] Furthermore, the incremental random forest optimization in S32 specifically includes: S321: The number of decision trees in the incremental random forest is set to 50-100, and the maximum depth of each decision tree is dynamically adjusted according to the amount of scene data; S322: When training with new data, a weight update strategy is adopted, the weight of historical data decays over time, and the weight of new data is set to 1.0; S323: The incremental random forest is pruned and optimized every 7 days, removing decision trees with an accuracy of less than 85% and supplementing with newly trained decision trees to maintain the overall accuracy of the model ≥95%. After the initial baseline sub-models for different scenarios are constructed, the effectiveness is verified through a scenario adaptability verification mechanism. If the model's recognition rate of historical abnormal data in the scenario is less than 95%, the model parameters are adjusted based on the scenario feature vector, and retraining is performed until the recognition rate requirement is met.

[0019] Furthermore, the optimization of the hybrid detection model in S42 specifically includes: S421: calibrating the weights of the hybrid detection model monthly based on historical anomaly data, using a grid search method to traverse the weight combinations. ,choose The highest-scoring weighted combination, among which represents the weight coefficients of the lightweight neural network (CNN), characterizing the contribution of the CNN algorithm in anomaly level determination, with a value range of [0.3, 0.7]. These are the weight coefficients of the Time Series Feature Extraction (LSTM) module, representing the LSTM module's ability to extract anomalous features from time series data. Their values ​​range from [0.1, 0.3]. S422: The weight coefficients of the Isolation Forest (IF) algorithm represent its importance in anomaly scoring, with values ​​ranging from [0.1, 0.4]. S422: When a new scene is added in the real-time process, transfer learning is used to initialize the hybrid detection model for that scene. The model parameters from similar scenes are combined with the new scene data for training. The training time is... Within hours, it can quickly realize anomaly detection in new scenarios; S423: Set anomaly false alarm rate threshold. When the false alarm rate of a certain scenario exceeds the threshold for 3 consecutive days, the weight in the anomaly level judgment formula will be automatically adjusted, and manual review of false alarm cases will be triggered to correct the model judgment logic. Attached Figure Description

[0020] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0021] Example 1

[0022] As attached Figure 1As shown, a real-time process monitoring method based on data analysis includes the following steps: S1: Dynamic multi-source data acquisition; based on the component operation scenario type, real-time resource occupancy rate, and data dependency between components in the real-time process, a three-dimensional component priority evaluation model is constructed, and a data acquisition strategy is dynamically matched; the three-dimensional component priority evaluation model calculates a priority score by integrating scenario weight, resource weight, and dependency weight, using the following formula: , The importance of the component's runtime scenario, R represents the scene weight, and R represents the real-time resource utilization rate. Where D is the resource weight, and D is the data dependency between components. For dependent weights, and Weights of the 3D component priority evaluation model ( , , The business importance coefficient is derived based on the target scenario, where the industrial production scenario is the default. =0.4、 =0.3、 =0.3, default value for internet service scenarios =0.3、 =0.4、 =0.3, the specific value can be adjusted through scenario adaptation experiments; data dependency D between components,

[0023] The maximum interaction frequency and maximum transmission volume are the peak thresholds for the normal operation of components in this scenario. S2: Scenario-based data preprocessing: Adapt cleaning rules according to the source type of real-time process data, adjust feature extraction dimensions based on the dynamic changes of the process, and use an improved time-series decomposition algorithm to separate data trends, fluctuations, and abnormal residuals; S3: Multi-scenario baseline model construction: Based on different operating scenario labels of the real-time process, use an online learning algorithm to build scenario-specific baseline sub-models, update sub-model parameters with newly added normal data in the scenario in real time, and establish a scenario switching trigger mechanism; S4: Correlated anomaly detection: Input the preprocessed real-time data into the corresponding scenario baseline sub-model, combine the abnormal characteristics of data interaction between components, and determine the anomaly level through a hybrid detection model that integrates classification algorithms and anomaly scoring algorithms; S5: Feedback-driven adaptive optimization: Based on anomaly handling effect feedback and user scenario-based feedback, build a multi-dimensional error correction model to dynamically optimize data acquisition parameters, preprocessing rules, and baseline model thresholds; S6: Scenario-based report generation and collaborative response: Generate scenario-based monitoring reports containing anomaly propagation paths based on anomaly level and real-time process scenario characteristics, and trigger hierarchical collaborative response measures for related components based on component dependency graphs.

[0024] The core principle of this solution is to construct a real-time process monitoring framework around scenario-based adaptation and multi-dimensional collaboration, achieving a dual improvement in monitoring accuracy and efficiency through dynamic optimization across the entire chain. In the data acquisition phase, a three-dimensional priority evaluation model is built based on component operating scenario types, real-time resource occupancy, and data dependencies between components. Priority scores are calculated by quantifying the weight of scenario importance, resource fluctuations, and dependency strength, thereby dynamically matching the acquisition frequency and dimensions to ensure priority allocation of acquisition resources for core scenarios and high-risk components. In the data preprocessing phase, specific cleaning rules are adapted for different source types such as sensor data, log data, and interaction data. The focus of feature extraction is adjusted based on dynamic changes such as process startup, stability, and shutdown. Simultaneously, an improved time-series decomposition algorithm is introduced to separate data trends, fluctuations, and abnormal residuals, eliminating interference from unstable processes on data accuracy. In the baseline modeling phase, a scenario-specific baseline sub-model is constructed based on the real-time process's operating scenario labels, and online learning is used to... The algorithm integrates newly added normal data in real time to update model parameters and designs a scene switching trigger mechanism to achieve smooth transition and dynamic adaptation of baselines under different scenarios. In the anomaly detection stage, preprocessed data is input into the corresponding scenario baseline sub-model, while interactive anomaly features such as transmission delay deviation between components and data packet loss rate are extracted. The anomaly level is quantified through a hybrid model of classification algorithm and anomaly scoring algorithm, covering component-specific and cross-component collaborative anomalies. In the adaptive optimization stage, the anomaly handling effect, resolution rate, recovery time and user scenario-based feedback are integrated to build a multi-dimensional error correction model, dynamically optimizing the collection parameters, preprocessing rules and baseline thresholds to form a closed loop of detection-processing-feedback-optimization. Finally, through scenario-based report generation and collaborative response supported by component dependency graph, accurate source tracing and rapid control of anomalies are achieved.

[0025] Existing technologies employ a single baseline and single algorithm for detection, failing to identify cross-component collaborative anomalies. This solution eliminates baseline deviations caused by scene switching through multi-scene-specific baselines, and combines component interaction features with a hybrid detection model, significantly improving the anomaly detection rate compared to existing technologies. Upgraded to The above measures reduce the rate of missed detections for cross-component collaborative anomalies. For example, in industrial production line monitoring, this solution can accurately identify collaborative faults where transmission delays in device A cause abnormal processing in device B, a type of fault that traditional solutions cannot cover. Existing technologies suffer from resource redundancy in edge scenarios or insufficient data collection in core scenarios due to their single data acquisition dimension. This solution dynamically matches acquisition resources using a three-dimensional priority model, reducing the amount of data collected in edge scenarios. Improved data acquisition accuracy in core scenarios Taking internet service monitoring as an example, the frequency of edge service data collection during off-peak hours is reduced to [missing information]. During peak periods, the core service collection frequency has increased to [number missing]. This approach aims to reduce server resource consumption while ensuring effective monitoring; however, existing static baseline technologies suffer from a high false alarm rate during scene switching. Furthermore, lacking closed-loop optimization logic, this solution utilizes a scene switching trigger mechanism and online learning to control the false alarm rate during scene switching within a certain range. Within this range, and based on a multi-dimensional error correction model, parameters are automatically optimized every 30 minutes. For example, when industrial production switches from start-up and commissioning to full-load operation, the baseline model can... A smooth transition is completed within seconds, and the detection threshold is continuously optimized as operational data accumulates. Existing technologies report no abnormal propagation path, respond only to a single component, and have a fault recovery time of up to 45 seconds. This solution, through scenario-based reporting of abnormal source tracing paths and collaborative responses using component dependency graphs, reduces the response time to [time missing]. Within seconds, the fault recovery time is reduced. For example, when a core service fails, the system can quickly sever non-critical links between the failed component and upstream data providers and downstream data receivers through the dependency graph, while simultaneously activating backup components to prevent the failure from spreading to the entire service architecture and significantly reduce the risk of business interruption.

[0026] The dynamic matching data acquisition strategy in S1 specifically includes: S11: Dividing the acquisition level according to the priority score P. This is a level 1 data collection. This is a level 2 data collection. It is a three-level collection. Level 4 acquisition; S12: Level 1 acquisition uses the acquisition frequency The data collected includes component running status, resource consumption details, internal interaction logs, and error stack information; secondary data collection uses a specific collection frequency. The collected data includes component running status, resource consumption summaries, and key interaction logs; the three-level collection adopts a collection frequency... The data collected includes the basic operational status of components and average resource consumption; the fourth-level data collection uses a specific collection frequency. The collected data only includes the online status data of components; S13: Real-time monitoring of component priority score changes, when the fluctuation of P exceeds... When the time is right, it automatically switches to the corresponding acquisition level and adjusts the response time accordingly. By combining quantitative scoring with hierarchical data collection, the problem of "insufficient data collection and missed anomalies in core scenarios, and redundant data collection and waste of resources in edge scenarios" in the traditional fixed data collection mode is avoided. For example, when industrial production is running at full load, core equipment is collected at level one to capture key data at high frequency, while edge components are collected at level four to reduce resource consumption during shutdown and maintenance. On the other hand, real-time fluctuation monitoring and second-level switching response can quickly adapt to the dynamic changes in component priorities. For example, when the traffic of Internet services changes from off-peak to peak, core services can be upgraded from level two to level one within 1 second, ensuring that data collection is always accurately matched with the real-time risk level and scenario requirements of components, which not only ensures monitoring accuracy but also achieves efficient utilization of system resources.

[0027] The scenario-based data preprocessing in S2 specifically includes: S21: Sensor data is filled with missing values ​​using interpolation, through improvements... Rules for identifying outliers; log data uses keyword matching to complete missing fields and extracts semantic features using natural language processing; interactive data uses association-based completion to complete missing items and identifies abrupt changes using temporal difference; S22: In the improved temporal decomposition algorithm, the scene adaptation coefficient... ,pass The fluctuation decomposition results are corrected, enhancing fluctuation sensitivity in core scenarios and reducing fluctuation interference in peripheral scenarios; S23: During feature extraction, the proportion of high-frequency feature extraction is increased in core scenarios. In edge scenarios, increase the proportion of low-frequency feature extraction to ,improve The rule adapts to non-normal data using dynamic standard deviation multiples, and is adopted in core scenarios. Enhanced sensitivity, adopted in edge scenarios To reduce false alarms, the standard deviation can be calibrated based on historical anomaly data from the scene. Dedicated preprocessing rules are adapted for three core data types: sensor data, log data, and interaction data. Sensor data uses interpolation to fill in missing values ​​and improves upon existing methods. The algorithm identifies anomalies by using keyword matching to complete fields in log data and extracting semantic features through natural language processing. Interactive data is filled with missing items using correlation analysis and abrupt anomalies are captured using temporal difference analysis to ensure the accuracy of preprocessing data from different sources. A scene adaptation coefficient is introduced into the improved temporal decomposition algorithm. ,pass The dynamic correction of fluctuation decomposition results enhances the sensitivity to fluctuations in core scenarios and reduces fluctuation interference in edge scenarios, avoiding the impact of scene-specific differences on time-series data decomposition. S23 adjusts the feature extraction dimensions according to scene importance, increasing the proportion of high-frequency feature extraction to 70% in core scenarios to capture subtle data changes, and increasing the proportion of low-frequency feature extraction to [missing information] in edge scenarios. To focus on key trends. Among them, improvements The rule, through its design of adapting to nonnormal data using dynamic standard deviation multiples, has significant advantages: traditional The rules are fixed. The threshold is prone to false alarms or missed alarms in real-time process data that is not normally distributed, while the improved core scenario adopts... Narrowing the threshold range allows for more sensitive identification of minute anomalies, such as subtle temperature fluctuations in core industrial equipment. This is particularly useful in edge scenarios. Expanding the threshold range effectively filters out non-critical fluctuation interference, such as instantaneous data deviations caused by network jitter in edge components. Simultaneously, the standard deviation factor supports calibration based on historical anomaly data for different scenarios, further aligning with the actual data characteristics of various scenarios. For example, in medical equipment monitoring scenarios, the threshold can be fine-tuned based on historical fault data. It solves the traditional The rules have limitations in adapting to different scenarios, but the accuracy of anomaly identification is improved through dynamic calibration. This provides high-quality data support for subsequent multi-scenario baseline modeling and correlation-based anomaly detection, avoiding monitoring errors caused by preprocessing data deviations.

[0028] The construction of the multi-scenario baseline model in S3 specifically includes: S31: The scenario labels are divided based on the core business indicators of the real-time process. Each scenario label corresponds to a set of clear business indicator threshold ranges. According to the business characteristics and data features of different scenarios, the parameter setting rules of the initial baseline sub-model are as follows: The initial model learning rate of the core scenario is set to 0.02, the number of decision trees is set to 80, and the maximum depth is set to 15; the initial model learning rate of the edge scenario is set to 0.05, the number of decision trees is set to 50, and the maximum depth is set to 8; the initial model learning rate of the transition scenario is set to 0.03, the number of decision trees is set to 60, and the maximum depth is set to 10. In the process of constructing the initial baseline sub-model, scenario feature vectors are introduced, and the K-means clustering algorithm is used to classify and train the scenario using historical normal data; S32: The online learning algorithm adopts incremental random forest. The sub-model parameters are updated once every 1000 new normal data are received. The historical optimal parameters are retained during the update process. If the model accuracy drops by more than 10% after the update, the parameter will be updated. Automatically rollback to historical optimal parameters; S33: The scene switching trigger mechanism is implemented through real-time monitoring of core business indicators. When the indicator exceeds the current scene threshold range and the duration is... When the time is right, scene switching is automatically triggered, and the corresponding scene baseline sub-model is called. A smooth transition algorithm is used during the switching process to avoid detection gaps. The four-level acquisition level is divided by priority score P, which can clearly distinguish the monitoring priority of different components. Level 1 acquisition is for core high-risk components. At the highest frequency Collect the most comprehensive information, including error stacks and interaction logs, to ensure that anomalies in core processes can be accurately traced; Level 4 data collection targets low-risk edge components. At the lowest frequency Only basic online status is collected, significantly reducing resource consumption; however, when the fluctuation range of P exceeds... If a component enters a high-risk state due to a sudden load, The rapid switching response within seconds allows for timely upgrades to the collection level, preventing missed anomalies due to outdated collection strategies. Conversely, it also enables timely downgrading of collection when component risk decreases, reducing unnecessary resource consumption. For example, in internet service monitoring, during peak periods, when the P-value of the core transaction component rises to 0.9, it automatically switches to Level 1 collection, capturing resource consumption details and interaction logs every second to ensure real-time identification of transaction anomalies. During off-peak periods, when the P-value drops to 0.3, it automatically switches to Level 4 collection, checking online status only every 20 minutes, thereby reducing server CPU utilization. This ensures both the accuracy of monitoring core scenarios and the efficient use of resources.

[0029] The correlation-based anomaly detection in S4 specifically includes: S41: Anomaly characteristics of data interaction between components are realized by constructing an interaction health index, the index calculation formula is: , This is the difference between the actual transmission delay and the historical average delay. Let L be the historical average latency and L be the data loss rate. Marked as an interaction anomaly; S42: In the hybrid detection model, the classification algorithm uses a lightweight neural network to output the probability of normality. The anomaly scoring algorithm uses an isolated forest to output anomaly scores. The final abnormality level determination formula is: ; This is a serious abnormality. This is a general abnormality. It is a minor abnormality. Normal; S43: Minor anomalies trigger an automatic review process, general anomalies trigger an alert from operations and maintenance personnel, and severe anomalies directly trigger an emergency response. By accurately quantifying the health status of interactions between components and integrating the advantages of multiple algorithms, the system achieves a scientific determination of anomaly levels. Simultaneously, a tiered response mechanism ensures the targeted and efficient handling of anomalies, avoiding both missed or misjudged anomalies in single components and preventing resource waste and risk spread caused by excessive or insufficient anomaly responses. Specifically, this is achieved through an interaction health index. Abstract interaction characteristics such as inter-component transmission latency deviation and data loss rate are transformed into quantifiable health values. The system accurately marks interaction anomalies, effectively identifying cross-component collaboration failures such as data blockage in component B caused by transmission delays in component A. Existing single-dimensional detection technologies often overlook such anomalies. The hybrid detection model integrates the normal probability output of a lightweight neural network. Anomaly scores with isolated forests ,pass The formula implements anomaly level stratification, which significantly reduces the false positive and false negative rates compared to existing single-algorithm judgment technologies. For example, when a component experiences a slight data deviation due to instantaneous load fluctuations, the model can accurately determine it as a slight anomaly. Instead of directly classifying it as a serious fault; coupled with a tiered mechanism of automatic review of minor anomalies, early warning of general anomalies, and emergency response to serious anomalies, this approach can avoid resource consumption caused by excessive responses triggered by minor anomalies, eliminate false alarms through automatic review without the need for operations and maintenance personnel intervention, and ensure that serious anomalies ( The abnormal link can be dealt with quickly, such as by directly initiating an emergency process to cut off the abnormal link. For example, when the health of a core device in industrial production is H=0.5 and G=0.85, the system can directly trigger an emergency response, suspend the abnormal device and start the backup device to prevent the fault from spreading to the entire production line. However, if the existing technology only responds to the abnormality of a single device, it will often miss the best time to deal with it.

[0030] The feedback-driven adaptive optimization in S5 specifically includes: S51: Calculating the comprehensive error using a multi-dimensional error correction model. , For model prediction error, , These are the model's predicted values. This is the actual value. This is due to anomaly handling error. , For user feedback on errors, S52: When At that time, the adaptive momentum gradient descent method was used to optimize the model parameters, and the learning rate was... Simultaneously, the data acquisition frequency was adjusted: the first-level acquisition frequency was increased by 20% in severe anomaly scenarios, while the fourth-level acquisition frequency was decreased in minor anomaly scenarios. S53: User scenario-based feedback is collected through scenario-specific questionnaires, including scores on the impact of anomalies on business and the timeliness of handling. Feedback results are used to adjust feature weights, with user-marked high-impact features seeing a 30% increase in weight. This constructs a complete closed loop of detection-processing-feedback-optimization, overcoming the limitations of existing technologies that only adjust parameters based on simple feedback. By comprehensively considering model prediction accuracy, anomaly handling effectiveness, and user business perception, dynamic and precise optimization of the entire monitoring chain is achieved. This ensures both long-term model adaptation to real-time process changes and a deep alignment between optimization strategies and business needs, avoiding performance degradation caused by untargeted adjustments. Specifically, the multi-dimensional error correction model... The formula will determine the model prediction error. Error handling Reflecting the effectiveness of the treatment and user feedback errors Reflecting the impact on business. By integrating the errors into a comprehensive error, compared to existing technologies that only focus on a single prediction error, a more comprehensive evaluation of the overall performance of the monitoring system can be achieved. For example, although a certain model may have a high prediction accuracy, it may not be able to fully assess the overall performance of the monitoring system. Small, but with a low success rate in exception handling. Even with a large overall error E, optimization will still be triggered to avoid the problem of accurate detection but poor processing; when In this case, the adaptive momentum gradient descent method is used to optimize the model parameters, and the acquisition frequency is adjusted accordingly. In severe abnormal scenarios, the acquisition frequency is increased by one level. To obtain finer-grained data, the frequency of level four data collection is reduced in slightly abnormal scenarios. To save resources and achieve coordinated optimization of model parameters and data collection strategies, avoiding inefficient optimization caused by the disconnect between existing technical parameters and data collection; in addition, by collecting user feedback on the degree of impact of anomalies and the timeliness of handling through scenario-specific questionnaires, and increasing the weight of high-impact features by 30%, the optimization strategy can be more aligned with core business needs. For example, in e-commerce transaction scenarios, users reported that anomalies in the payment component had the greatest impact on the business, so the system increased the weight of payment component-related features, which can more accurately identify anomalies in the component in the future. Compared with existing technologies that do not have a business orientation for optimization, this greatly improves the monitoring's ability to protect core business.

[0031] The scenario-based report generation in S6 specifically includes: S61: Scenario-based monitoring reports include scenario overview, anomaly details, anomaly propagation paths, and scenario-based optimization suggestions; S62: The report generation frequency is dynamically adjusted according to the anomaly level, with severe anomalies generated in real time, general anomalies generated every 2 minutes, and minor anomalies generated every 10 minutes; S63: Reports in core scenarios add business impact simulation analysis, and reports in edge scenarios add anomaly trend prediction. By deeply binding with the characteristics of real-time process scenarios and the urgency of anomalies, reports with greater decision-making value and practicality are generated. This not only helps operations and maintenance personnel quickly and accurately grasp the core information of anomalies, but also provides targeted support for risk control and optimization decisions in different scenarios, avoiding information redundancy or missing key information caused by indiscriminate reporting. The report includes a scenario overview, anomaly details, anomaly propagation path, and scenario-based optimization suggestions. Compared to existing technologies that only list anomaly events, this report allows operations and maintenance personnel to intuitively understand the context of the anomaly, such as whether it's a core production scenario or an edge support scenario, and the scope of the anomaly's impact on related components. It traces the source and spread direction through the propagation path and directly obtains optimization solutions adapted to the current scenario, such as prioritizing resource expansion in core scenarios and focusing on parameter fine-tuning in edge scenarios, significantly shortening fault tracing and decision-making time. The report generation frequency dynamically adjusts according to the anomaly level: severe anomalies are generated in real time, general anomalies every 2 minutes, and minor anomalies every 10 minutes, ensuring accurate and timely reporting. It ensures real-time synchronization of information on serious anomalies, avoiding missed opportunities for handling due to reporting delays, while also preventing information overload caused by frequent reporting of minor anomalies, thus reducing the workload of operations and maintenance personnel. At the same time, core scenario reports are supplemented with business impact simulation analysis, and edge scenario reports are supplemented with anomaly trend prediction, such as predicting the probability of anomalies occurring in edge components in the next hour. This can provide differentiated support for management decisions in different scenarios. For example, when anomalies occur in core e-commerce transaction scenarios, operations and maintenance personnel can quickly determine whether to activate backup links through business impact simulation, while edge logistics information scenarios can make preventive maintenance in advance through trend prediction, improving the practical value and scenario adaptability of the reports.

[0032] The collaborative response in S6 specifically includes: S61: Establishing a mapping table of anomaly level, response measures, and related components. Response measures for severe anomalies include: suspending the abnormal component's business processing, initiating a backup component to take over the business, cutting off the interaction link between the abnormal component and the core component, and notifying the technical lead. Response measures for general anomalies include: limiting the abnormal component's resource usage, initiating the component's self-diagnosis program, and notifying maintenance personnel. Response measures for minor anomalies include: recording anomaly logs and initiating anomaly trend monitoring. S62: Collaborative response of related components is implemented based on a component dependency graph, identifying the direct upstream and downstream components of the abnormal component. The upstream component suspends sending non-critical data to the abnormal component, and the downstream component initiates a data caching mechanism. S63: After the response measures are executed, the operating status of the abnormal component and related components is monitored in real time. When the anomaly is resolved, normal operation is restored in the order of downstream component cache data synchronization → abnormal component business recovery → upstream component data transmission recovery. During the recovery process, core business indicators are continuously monitored to ensure no secondary anomalies. The mapping table of anomaly level, response measures, and related components ensures that anomalies of different urgency levels have clear handling plans. In the case of severe anomalies, the abnormal component's business processing is suspended. Business operations, activating backup components, and severing core interaction links can isolate risks in the shortest possible time, preventing the failure from spreading to the entire system. In contrast, existing technologies often lead to the spread of anomalies due to the failure to sever related links when responding to a single component. Based on the collaborative response of component dependency graphs, the upstream and downstream related parties of the abnormal component are accurately identified. The upstream suspends the transmission of non-critical data, and the downstream activates caching. This ensures that critical business data is not lost and avoids new problems caused by abnormal data input or output interruptions in related components. For example, when a data transmission component malfunctions, the upstream database suspends the transmission of non-core query data, and the downstream application activates caching. This does not affect the flow of core business data and prevents the downstream application from crashing due to data supply interruption. The orderly recovery process after the response, including downstream cache synchronization, abnormal component recovery, upstream data transmission recovery, and continuous monitoring of core indicators, ensures that the recovery process is gradual and free of secondary risks. Compared with the disordered recovery of existing technologies, which is prone to data inconsistency or component linkage failures, this significantly improves the stability of system recovery. For example, when recovering from an abnormality in an industrial production line, the downstream equipment cache data is synchronized first, and then the abnormal equipment is restored. This can avoid production deviations caused by data gaps, and continuous monitoring of core indicators further ensures that the recovery effect meets business requirements.

[0033] The incremental random forest optimization in S32 specifically includes: S321: The number of decision trees in the incremental random forest is set to 50-100, and the maximum depth of each decision tree is dynamically adjusted according to the amount of scene data; S322: When training with new data, a weight update strategy is adopted, the weight of historical data decays over time, and the weight of new data is set to 1.0; S323: The incremental random forest is pruned and optimized every 7 days, removing trees with accuracy lower than 1.0. The decision tree is supplemented with a newly trained decision tree to maintain the overall accuracy of the model. The number of decision trees is set to The tree depth is dynamically adjusted based on the amount of data in the scenario, striking a balance between model complexity and computational efficiency. In edge scenarios with limited data, such as equipment standby, the decision tree depth is reduced to lower computational resource consumption. In core scenarios with large data volumes, such as full-load industrial production, the depth is increased to improve model fitting accuracy. Compared to traditional random forests with fixed parameters, this approach is more adaptable to the differences in real-time scenarios. When training with new data, a strategy of historical data weight decay and new data weight set to 1.0 is employed, allowing the model to focus more on learning the features and patterns of the latest process data. For example, when the operating mode of components changes slowly with equipment aging during real-time processing, the model can quickly adapt to the new operating patterns through weight adjustments, avoiding the adaptation lag caused by the over-reliance on old data in traditional models. Regular pruning every 7 days optimizes and maintains overall accuracy. This allows for the timely removal of ineffective or inefficient decision trees from the model, preventing redundant structures from slowing down computation. Simultaneously, by supplementing with new trees, the model maintains high recognition accuracy. For example, in internet service scenarios, as user access patterns iterate, the accuracy of old decision trees in recognizing new anomaly patterns decreases. The new trees added after pruning can quickly capture new features, ensuring the model's detection rate for novel anomalies remains consistently high. In comparison, the accuracy of traditional random forests declines year by year due to a lack of long-term updates, which significantly improves the model's long-term adaptability and detection stability.

[0034] The optimization of the hybrid detection model in S42 specifically includes: S421: calibrating the weights of the hybrid detection model monthly based on historical anomaly data, using a grid search method to traverse the weight combinations. ,choose The highest-scoring weighted combination, among which represents the weight coefficients of the lightweight neural network (CNN), characterizing the contribution of the CNN algorithm in anomaly level determination, with a value range of [0.3, 0.7]. The weight coefficients of the Time Series Feature Extraction (LSTM) module represent the LSTM module's ability to extract abnormal features from time series data, and their values ​​range from [0.1, 0.3]. S422: The weight coefficients of the Isolation Forest (IF) algorithm represent its importance in anomaly scoring, with values ​​ranging from [0.1, 0.4]. S422: When a new scene is added in the real-time process, transfer learning is used to initialize the hybrid detection model for that scene. The model parameters from similar scenes are combined with the new scene data for training. The training time is... Within hours, it quickly achieves anomaly detection in new scenarios; S423: Sets anomaly false alarm rate threshold. When the false alarm rate of a certain scenario exceeds the threshold for 3 consecutive days, it automatically adjusts the weights in the anomaly level judgment formula and triggers manual review of false alarm cases to correct the model's judgment logic. Monthly, it calibrates the weights using a grid search method based on historical anomaly data, ensuring that the advantages of the classification algorithm (CNN-LSTM) and the anomaly scoring algorithm are always adapted to the changes in anomaly characteristics in the real-time process. For example, when the number of component interaction anomalies increases during a certain period, the sensitivity to interaction anomalies can be enhanced by increasing the weights of the isolated forest, which significantly improves the accuracy of anomaly detection at different times compared to a fixed-weight model; When a new scenario is added to the real-time process, such as adding a new production line to monitor in industrial production, it uses transfer learning to train by combining parameters from similar scenarios with new scenario data, and the training time is... Within hours, the system can quickly initialize new scenario models, avoiding the inefficiency of traditional models that require hours or even days to train from scratch, ensuring rapid deployment of new scenario monitoring. It also sets a false positive rate threshold; when the false positive rate for a scenario exceeds the limit for three consecutive days, the system automatically adjusts the weights and triggers manual review. This not only reduces false positives through dynamic weight correction but also corrects the model's judgment logic through manual review. For example, in temporary scenarios during e-commerce promotions, if the false positive rate increases due to sudden traffic surges, the system can quickly adjust the weights and optimize the judgment rules through manual review, causing the false positive rate to fall back below the threshold within one day. Compared to existing technologies with high and difficult-to-correct false positive rates, this significantly improves the model's practical value and operational efficiency.

[0035] Example 2

[0036] This embodiment takes the real-time monitoring scenario of an industrial production line as an example, covering three core scenarios: startup and debugging, full-load operation, and shutdown maintenance, to provide a detailed description of the real-time process monitoring method based on data analysis of the present invention.

[0037] I. Dynamic Multi-Source Data Acquisition

[0038] 1.1 Construction of a 3D Component Priority Evaluation Model

[0039] First, identify the key components in the industrial production line, including core processing equipment such as CNC lathes, data transmission modules, sensor groups: temperature and pressure sensors, and backup equipment switching units, totaling four categories of components.

[0040] Scenario Importance S-quantification: Assigning values ​​based on the impact of the component's scenario on production, including core processing equipment under full-load operation scenarios. Core processing equipment in startup and debugging scenarios All components during downtime maintenance scenarios Data transmission module in full-load operation scenarios Other scenarios Scenario of full-load operation of sensor group and backup equipment switching unit Other scenarios .

[0041] Weighting: Set scenario weights based on the characteristics of industrial production lines. Real-time resource utilization weight =0.3, weight of data dependency between components ,satisfy .

[0042] Real-time resource utilization (R) acquisition: The CPU utilization, memory utilization, and bandwidth utilization of components are collected in real time through the industrial control system, and the average of the three is taken as the R value. For example, when the core processing equipment is running at full load... During power-on debugging .

[0043] Data dependency (D) calculation between components: Based on the evaluation of component interaction frequency and data transmission volume, the interaction frequency between the core processing equipment and the data transmission module is as follows: The transmission volume is , The sensor array transmits data to the data transmission module at a frequency of 5 times / second and a transmission volume of 50MB / minute, with D=0.7; the backup equipment switching unit only interacts with the core equipment in case of a fault. .

[0044] Priority score (P) calculation: based on the formula Calculations, such as those for core processing equipment operating at full load. Core processing equipment in startup and debugging scenarios .

[0045] 1.2 Data Acquisition Strategy Matching

[0046] Data collection level classification: 4 levels are defined based on the P-value. This is a level 1 data collection. This is a level 2 data collection. It is a three-level collection. Level 4 data acquisition. Core processing equipment operating at full load. For primary data acquisition, the data transmission module... This is for secondary data acquisition; core processing equipment for startup and debugging. For three-level data acquisition, sensor group For Level 3 data acquisition; all components in the downtime maintenance scenario. This is a level four data collection.

[0047] Data acquisition parameter settings:

[0048] Level 1 Acquisition: Frequency The data collected includes the operating status of core processing equipment, spindle speed, processing accuracy, resource consumption details, CPU utilization every 0.5 seconds, memory usage refreshed in real time, internal interaction logs, interaction records with the data transmission module, error stack information, and the complete code path when the device reports an error.

[0049] Secondary acquisition: frequency It collects the operating status of the data transmission module, including transmission rate, connection stability, resource consumption summary, CPU and memory average every 5 seconds, key interaction logs, and key interaction records with core devices and sensor groups.

[0050] Level 3 Acquisition: Frequency Collect basic operating status data of core processing equipment, such as power on / off status, error reporting, average resource consumption, and average CPU and memory usage per minute; and basic data from sensor groups, such as average temperature and pressure.

[0051] Level 4 Acquisition: Frequency Only the online status of all components is collected.

[0052] Dynamic switching of data collection levels: Real-time monitoring of P-value fluctuations; when the fluctuation exceeds... Switching levels in time, response time For example, in a full-load operation scenario, if a core processing equipment experiences a sudden failure, causing the R value to rise to 0.95, the P value will become... The fluctuation range is 0.03 (not exceeding ±0.2), maintaining level one acquisition; if the data transmission module suddenly disconnects, the D value drops to 0.1, and the P value becomes... The fluctuation range was 0.23 ± 0.2, and it switched to level 3 acquisition within 1 second.

[0053] II. Scenario-based data preprocessing

[0054] 2.1 Data Cleaning Rule Adaptation

[0055] Sensor data processing: For temperature and pressure sensors, missing values ​​are filled using linear interpolation. For example, if temperature data is missing for a given second, linearly fitted values ​​from the preceding and following 10 seconds are used. Outliers are identified using an improved 3σ rule. The mean μ and standard deviation σ of historical sensor data are calculated, and values ​​exceeding these deviations are excluded. Data that falls within a range and lasts for ≥2 seconds is marked as anomaly, such as average temperature. , , If it lasts for more than 2 seconds, it is considered abnormal.

[0056] Log data processing: System logs use keyword matching to complete missing fields. For example, if a "transmission failure" log is missing the "failure reason" field, it is filled by matching common reasons for "transmission failure" in historical logs of the same type, such as "insufficient bandwidth" or "connection timeout". Semantic features are extracted using the BERT model, converting the log text into vector form. For example, "abnormal spindle speed" is converted into... For subsequent analysis.

[0057] Interactive data processing: Inter-component interaction data uses an associative completion method. For example, if the "transmission duration" field is missing in the interaction record between the data transmission module and the core device, the transmission duration is calculated as transmission data volume / transmission rate to complete the data. Temporal difference analysis is used to identify abrupt changes by calculating the difference between two adjacent data sets, such as a change in transmission rate from... Mutation Difference If it lasts for more than 3 seconds, it is considered abnormal.

[0058] 2.2 Application of the Improved Time Series Decomposition Algorithm

[0059] Calculation of scene adaptation coefficient α: based on the formula Core scenarios, running at full load. , Edge scenarios, downtime for maintenance. , .

[0060] Timing decomposition execution: Based on the STL algorithm, α-corrected fluctuation decomposition results are introduced. When decomposing the timing data of the spindle speed of the core machining equipment, Enhanced fluctuation sensitivity, capable of capturing minute speed fluctuations within 0.5 seconds; when decomposing online status time-series data of components under shutdown and maintenance scenarios, Reduce fluctuation interference and ignore momentary offline false alarms caused by network jitter. After decomposition, separate trend items, such as the slow decreasing trend of spindle speed with equipment running time, the periodic fluctuation of speed during processing, and abnormal residuals, such as sudden drop in speed caused by sudden failure.

[0061] 2.3 Adjustment of Feature Extraction Dimensions

[0062] In core scenarios such as full-load operation: the proportion of high-frequency feature extraction This includes the instantaneous rate of change of the spindle speed of the core equipment, calculated every 0.1 seconds; the real-time bandwidth usage of the data transmission module, sampled every 0.5 seconds; high-frequency fluctuations in sensor data, such as the pressure change every 0.2 seconds; and the proportion of low-frequency features. Examples include the 5-minute average of device resource consumption and the 10-minute average of interaction success rate.

[0063] In edge scenarios such as downtime and maintenance: the proportion of low-frequency feature extraction Examples include the 10-minute stability of the component's online status and the 30-minute average of resource consumption; the proportion of high-frequency features. Only key abnormal triggering features are retained, such as the instantaneous signal of a device suddenly going offline.

[0064] III. Construction of Multi-Scenario Baseline Models

[0065] 3.1 Segmentation of Running Scene Tags

[0066] Based on core business indicators of industrial production lines, scenario tags and threshold ranges are defined for production efficiency, product qualification rate, and equipment failure rate:

[0067] Startup and debugging scenario: production efficiency Product qualification rate Equipment failure rate .

[0068] Full load operation scenario: production efficiency Product qualification rate Equipment failure rate .

[0069] Downtime maintenance scenario: production efficiency Product qualification rate Equipment failure rate .

[0070] 3.2 Incremental Random Forest Model Construction and Update

[0071] Model initialization: The number of decision trees is set to 80, and the maximum depth of each decision tree is adjusted according to the amount of scenario data. The full-load operation is based on the amount of scenario data. Maximum depth set to 15; boot-up debugging scenario data volume Maximum depth is set to 10; data volume during downtime maintenance scenarios. The maximum depth is set to 8.

[0072] Parameter update mechanism: Sub-model parameters are updated every 1000 new normal data entries received. The updated parameters retain the historical best values, such as the model accuracy after the previous update. The parameters are set. If the model accuracy drops by more than 3% after the update, it will automatically roll back to the historical optimal parameters. For example, in a full-load operation scenario, 100,000 new normal data points are added daily, triggering 100 parameter updates. After each update, the accuracy is verified using a test set containing 5,000 normal data points and 1,000 abnormal data points to ensure that it is not lower than the previous best. .

[0073] Model optimization: Pruning is performed every 7 days, removing decision trees with accuracy below 85% and supplementing with newly trained decision trees to maintain the overall accuracy of the model. .

[0074] 3.3 Scene Switching Trigger Mechanism

[0075] Real-time monitoring of core business metrics; when a metric exceeds the current scenario threshold range for an extended period of time... Scene switching is triggered every second, employing a smooth transition algorithm, such as weighted averaging. For the first 5 seconds, the current scene baseline weight is 0.8, and the new scene baseline weight is 0.2. In the following 5 seconds, the weight is gradually adjusted to 0.2 for the current scene and 0.8 for the new scene, avoiding detection gaps. For example, when a production line switches from start-up testing to full-load operation, production efficiency increases from 30% to... It lasts for 10 seconds, triggering a switch. Within 10 seconds, the baseline model transitions from the startup debugging sub-model to the full-load running sub-model, with no false alarms or missed anomalies detected during the transition.

[0076] IV. Correlational Anomaly Detection

[0077] 4.1 Calculation of Interaction Health Index (H)

[0078] Parameter acquisition: This represents the difference between the actual transmission delay and the historical average delay, where the historical average delay of the data transmission module and the core device is [missing information]. The actual delay at a certain moment is 1.2 seconds. Seconds; L is the data loss rate, where 20 out of 1000 data entries are lost during the same transmission. .

[0079] Index calculation: based on the formula

[0080] Calculated This is marked as an interaction exception.

[0081] 4.2 Anomaly Level Determination in Hybrid Detection Model

[0082] Model Output: The MobileNet architecture is used to output the normal probability of the preprocessed core device data. =0.3; Isolation Forest algorithm outputs anomaly score .

[0083] Anomaly level calculation: based on the formula The calculated value is G = 0.7 × (1 - 0.3) + 0.3 × 0.9 = 0.49 + 0.27 = 0.76, which falls within the range of 0.6 ≤ G < 0.8, and is therefore classified as a general anomaly.

[0084] 4.3 Tiered Response Trigger

[0085] Minor abnormalities ( ): Trigger the automatic review process. The system re-collects data 3 times. If it is still judged as abnormal after 2 or more times, it is upgraded to general abnormal. If it is normal after 2 or more times, it is marked as "suspected abnormal" and recorded in the log.

[0086] General abnormalities ( ): Triggers an alert for operations and maintenance personnel, notifying them via SMS and system pop-ups, including information such as the abnormal component, the type of abnormality, and the health index of the interaction, requiring a response within 3 minutes.

[0087] Serious abnormality ( ): Directly trigger emergency response, suspend the abnormal component's business, start the backup component, and notify the technical lead, who must intervene within 1 minute.

[0088] V. Feedback-driven adaptive optimization

[0089] 5.1 Construction of Multi-Dimensional Error Correction Model

[0090] Error calculation:

[0091] Model prediction error Select 100 sets of data, model prediction values Compared with actual value The absolute difference of the average 。

[0092] Error handling There were a total of 50 anomalies during a certain period, and 42 of them were successfully resolved. .

[0093] User feedback error Total feedback issues: 30; User satisfaction: 21. .

[0094] Overall error calculation: based on the formula Calculated Parameter optimization will not be triggered for the time being; if Initiate the optimization process.

[0095] 5.2 Parameter Optimization Execution

[0096] Model parameter optimization: Adaptive momentum gradient descent is used, with a learning rate of [missing information]. The decision tree splitting threshold for incremental random forest and the number of sample bins for isolated forest were adjusted, and the process was iterated 100 times until the model accuracy improved. .

[0097] Acquisition frequency adjustment: In severe abnormal scenarios, the primary acquisition frequency is increased. ( Become In cases of minor anomalies, the sampling frequency of Level 4 is reduced by 1. (from Become ).

[0098] 5.3 User Feedback Application

[0099] Feedback was collected through scenario-specific questionnaires. The full-load operation scenario questionnaire included "the degree of impact of the anomaly on production efficiency (1-5 points)" and "the timeliness of handling (1-5 points)". If the user marked "abnormal spindle speed" as a high-impact feature (score of 5 points), the weight of this feature was increased by 30% from 0.2 to 0.26 to enhance the model's sensitivity to the anomaly.

[0100] VI. Contextualized Report Generation and Collaborative Response

[0101] 6.1 Generation of Contextualized Monitoring Reports

[0102] Report content:

[0103] Scenario Overview: Full-load operation scenario, production efficiency Product qualification rate The equipment failure rate is 0.8%.

[0104] Error details: Data transmission module is experiencing a general error. Interactive health The anomaly occurred at 14:30 and lasted for 8 seconds.

[0105] Abnormal propagation path: Data transmission module malfunction → Data reception delay of core processing equipment → Spindle speed fluctuation → Product processing accuracy deviation (deviation value 0.02mm).

[0106] Scenario-based optimization suggestions: Check the bandwidth of the data transmission module and it is recommended to expand it to 100Mbps (currently 80Mbps) and optimize the transmission protocol.

[0107] Generation frequency: For general anomalies, a report is generated every 2 minutes and pushed to the operation and maintenance personnel's terminal in real time; for serious anomalies, a report is generated in real time and the anomaly progress is updated every second; for minor anomalies, a report is generated every 10 minutes.

[0108] Scenario-specific supplements: The core scenario full-load operation report now includes business impact simulation analysis; for example, if the anomaly persists for 10 minutes, it will lead to a decrease in production efficiency. 20 more defective products were added; the edge scenario report now includes predictions of abnormal trends, such as the probability of devices going offline in the next hour. .

[0109] 6.2 Collaborative Response Execution

[0110] Response measure matching: Based on the anomaly level - response measure - associated component mapping table, the response measure for general anomalies is: limit the resource usage of the data transmission module, and set the CPU utilization limit from [previous level]. Down to The module initiates a self-diagnostic program to detect transmission protocols, bandwidth usage, and notify maintenance personnel.

[0111] Related component collaboration: Based on the component dependency graph, the direct upstream component sensor group and the direct downstream component core processing equipment of the data transmission module are identified. The upstream sensor group suspends sending non-critical data such as historical temperature data to it, and only transmits real-time critical data, such as the current pressure value. The downstream core processing equipment starts a data caching mechanism to cache the processing parameter data within 10 minutes to avoid processing interruption due to data supply failure.

[0112] Recovery process: After the anomaly is resolved, the data transmission module delay is reduced to 0.5 seconds, the loss rate is reduced to 0.5%, and the process proceeds in the following order: downstream core device cache data synchronization → data transmission module service recovery → upstream sensor group data transmission recovery. During the synchronization process, key indicators such as production efficiency and product qualification rate are continuously monitored to ensure that there are no secondary anomalies. The entire process recovery is completed within 5 minutes.

[0113] The above are merely embodiments of the present invention. Commonly known structures and characteristics of the solutions are not described in detail here. Those skilled in the art are aware of all common technical knowledge in the field prior to the application date or priority date, are aware of all prior art in that field, and have the ability to apply conventional experimental methods prior to that date. Those skilled in the art can, under the guidance of this application, improve and implement this solution in combination with their own capabilities. Some typical known structures or methods should not be obstacles for those skilled in the art to implement this application. It should be noted that those skilled in the art can make several modifications and improvements without departing from the structure of the present invention. These should also be considered within the scope of protection of the present invention, and will not affect the effectiveness of the implementation of the present invention or the practicality of the patent. The scope of protection claimed in this application should be determined by the content of its claims, and the description of specific embodiments in the specification can be used to interpret the content of the claims.

Claims

1. A method for real-time process monitoring based on data analysis, characterized in that, Comprise the following steps: S1: dynamic multi-source data acquisition, based on real-time process component running scene type, real-time resource occupancy and component data dependency, construct three-dimensional component priority evaluation model, dynamic matching data acquisition strategy; the three-dimensional component priority evaluation model calculates the priority score by fusing scene weight, resource weight and dependency weight, the formula is: , is the component running scene importance, is the scene weight, is the real-time resource occupancy, is the resource weight, is the component data dependency, is the dependency weight, and ; S2: scene data preprocessing, according to the source type of real-time process data Adapt cleaning rules, combined with the dynamic characteristics of the process Adjust the feature extraction dimension, using the improved time decomposition algorithm Separate the trend, volatility and abnormal residual of the data; S3: multi-scene baseline model construction, based on the different running scene labels of real-time process, using online learning algorithm to construct scene-specific baseline sub-model, real-time fusion of scene within the new normal data update sub-model parameters, while establishing scene switching trigger mechanism; S4: correlation anomaly detection, input the preprocessed real-time data into the corresponding scene baseline sub-model, combined with the abnormal characteristics of data interaction between components, through the mixed detection model of fusion classification algorithm and abnormal scoring algorithm to determine the abnormal level; S5: feedback driven adaptive optimization, based on abnormal processing effect feedback and user scene feedback, build multi-dimensional error correction model, dynamically optimize data acquisition parameters, preprocessing rules and baseline model threshold; The feedback driven adaptive optimization in S5 specifically comprises: S51: The multi-dimensional error correction model calculates the comprehensive error , is the model prediction error, , is the model prediction value, is the actual value, is the abnormal processing error, , is the user feedback error, ; S52: When the adaptive momentum gradient descent method is used to optimize the model parameters, the learning rate ; at the same time, the data acquisition frequency is adjusted, the first level acquisition frequency is increased by 20% under the serious abnormal scene, and the fourth level acquisition frequency is reduced by 15% under the slight abnormal scene; S53: user scene feedback is collected through scene-specific questionnaire, including the impact of abnormality on business, processing timeliness score, feedback results are used to adjust feature weight, user marked high impact feature weight is improved by 30%; S6: scene report generation and collaborative response, according to the abnormal level and real-time process scene characteristics, generate scene monitoring report containing abnormal propagation path, trigger hierarchical collaborative response measures of associated components based on component dependency graph.

2. The real-time process monitoring method based on data analysis according to claim 1, characterized in that, The dynamic matching data acquisition strategy in S1 specifically comprises: S11: classifying the collection level according to the priority score P, for first-class collection, for second-class collection, for third-class collection, for fourth-class collection; S12: the first level collection adopts a collection frequency , and the collection content includes component running state, resource consumption details, internal interaction log and error stack information; the second level collection adopts a collection frequency , and the collection content includes component running state, resource consumption summary and key interaction log; the third level collection adopts a collection frequency , and the collection content includes component basic running state and resource consumption average; the fourth level collection adopts a collection frequency , and the collection content only contains component online state data; S13: Real-time monitoring of component priority score changes, when P fluctuation amplitude exceeds , automatically switch to the corresponding acquisition level, switching response time seconds.

3. The real-time process monitoring method based on data analysis of claim 1, wherein, The scene data preprocessing in S2 specifically comprises: S21: The sensor data is filled with missing values by interpolation method, and the outliers are identified by improved rule; the log data is filled with missing fields by keyword matching method, and the semantic features are extracted by natural language processing technology; the interactive data is filled with missing items by association filling method, and the mutation anomaly is identified by time series difference method; S22: in the improved timing decomposition algorithm, scene adaptation coefficient , by correcting the fluctuation decomposition result, enhancing the fluctuation sensitivity under the core scene, and reducing the fluctuation interference under the edge scene; S23: During feature extraction, the proportion of high-frequency feature extraction under the core scene is increased to , and the proportion of low-frequency feature extraction under the edge scene is increased to .

4. The real-time process monitoring method based on data analysis of claim 1, wherein, The multi-scene baseline model construction in S3 specifically comprises: S31: running scene label is based on core business indicators of real-time process, each scene label corresponds to a group of explicit business indicator threshold range, according to the business characteristics and data characteristics of different scenes, the parameter setting rules of initial baseline sub-model are as follows: the initial model learning rate of core scene is set to 0.02, the number of decision trees is set to 80, and the maximum depth is set to 15; The initial model learning rate of edge scene is set to 0.05, the number of decision trees is set to 50, and the maximum depth is set to 8; The initial model learning rate of transition scene is set to 0.03, the number of decision trees is set to 60, and the maximum depth is set to 10, in the construction process of initial baseline sub-model, scene feature vector is introduced, and K-means clustering algorithm is used to train scene classification of historical normal data; S32: online learning algorithm uses incremental random forest, each receives 1000 new normal data to trigger sub-model parameter update, retains the historical optimal parameter in the update process, if the model accuracy decreases by more than 3% after updating, automatically roll back to the historical optimal parameter; S33: The scene switching trigger mechanism is realized by monitoring core business indicators in real time. When the indicators exceed the current scene threshold range and the duration is greater than the threshold, the scene switching is automatically triggered, the corresponding scene baseline sub-model is called, and a smooth transition algorithm is used in the switching process to avoid detection faults.

5. The real-time process monitoring method based on data analysis of claim 1, wherein, The correlation anomaly detection in S4 specifically comprises: S41: The inter-component data interaction abnormality feature is realized by constructing an interaction health index, and the calculation formula of the index is: , is the difference between the actual transmission delay and the historical average delay, is the historical average delay, L is the data loss rate, is marked as interaction abnormality; S42: In the mixed detection model, the classification algorithm uses a lightweight neural network to output normal probability , the anomaly scoring algorithm uses Isolation Forest to output anomaly score , and the final anomaly level determination formula is: ; is a serious anomaly, is a general anomaly, is a slight anomaly, is normal; S43: slight abnormality triggers automatic review process, general abnormality triggers operation and maintenance personnel warning, serious abnormality directly triggers emergency response.

6. The real-time process monitoring method based on data analysis of claim 1, wherein, The scene report generation in S6 specifically comprises: S61: scene monitoring report contains scene overview, abnormal details, abnormal propagation path and scene optimization suggestions; S62: The report generation frequency is dynamically adjusted according to the abnormality level, a serious abnormality is generated in real time, a general abnormality is generated every 2 minutes, and a slight abnormality is generated every 10 minutes; S63: The report under the core scene increases the business impact simulation analysis, and the report under the edge scene increases the abnormality trend prediction.

7. The real-time process monitoring method based on data analysis of claim 1, wherein, The S6 cooperative response specifically includes: S61: Establish a mapping relationship table of abnormality level-response measure-association component, the response measures for a serious abnormality are: suspending abnormal component business processing, starting backup component to take over business, cutting off the interaction link between the abnormal component and the core component, and notifying the technical person in charge; the response measures for a general abnormality are: limiting abnormal component resource occupation, starting component self-diagnosis program, and notifying operation and maintenance personnel; the response measures for a slight abnormality are: recording abnormal logs and starting abnormal trend monitoring; S62: The association component cooperative response is realized based on the component dependency graph, the direct upstream component and the direct downstream component of the abnormal component are identified, the upstream component suspends sending non-critical data to the abnormal component, and the downstream component starts the data caching mechanism; S63: After the response measures are executed, the running state of the abnormal component and the associated component is monitored in real time, when the abnormality is resolved, the normal operation is restored in the order of downstream component cache data synchronization, abnormal component business recovery, and upstream component data transmission recovery, the core business indicators are continuously monitored during the recovery process to ensure no secondary abnormality.

8. The real-time process monitoring method based on data analysis according to claim 4, characterized in that, The S32 incremental random forest optimization specifically includes: S321: The number of decision trees of the incremental random forest is set to 50-100, and the maximum depth of each decision tree is dynamically adjusted according to the scene data volume; S322: When training new data, a weight update strategy is adopted, the historical data weight decays over time, and the new data weight is set to 1.0; S323: Pruning optimization of the incremental random forest is performed periodically every 7 days, removing decision trees with accuracy lower than 0.7, supplementing new trained decision trees, maintaining the overall accuracy of the model. .​ 9. The real-time process monitoring method based on data analysis of claim 5, wherein, The S42 mixed detection model optimization specifically includes: S421: Calibrate the weights of the hybrid detection model based on historical abnormal data every month, use grid search method to traverse the weight combination, , select the highest score weight combination, where is the weight coefficient of the lightweight neural network CNN, representing the contribution of the CNN algorithm in the abnormal level judgment, the value range is [0.3, 0.7]; is the weight coefficient of the time series feature extraction module LSTM, representing the extraction ability weight of the LSTM module to the time series data abnormal feature, the value range is [0.1, 0.3], is the weight coefficient of the isolated forest IF algorithm, representing the importance of the isolated forest algorithm in the abnormal score, the value range is [0.1, 0.4]; S422: When a real-time process adds a scene, initialize the hybrid detection model of the scene using transfer learning, train the model parameters of similar scenes combined with new scene data, and the training time is a few hours, quickly realize anomaly detection in new scenes; S423: Set an abnormal false alarm rate threshold, when the false alarm rate of a certain scene exceeds the threshold for 3 consecutive days, automatically adjust the weight in the abnormality level determination formula, and at the same time trigger manual review of false alarm cases to correct the model determination logic.

Images