MES system operation industry data flow asynchronous processing method

Abstract

The application provides a MES system operation data flow asynchronous processing method, and belongs to the technical field of MES systems.The application collects material flow data by deploying multilevel sensors and distributes globally unique batch identifiers, calculates Lyapunov exponents, correlation dimensions and Kolmogorov entropy to identify and isolate abnormalities by using a phase space reconstruction anomaly detection module, calculates an entropy increase index for normal data and optimizes resource allocation by a double-layer game scheduling model, dynamically adjusts a back pressure threshold according to a resource allocation scheme and starts a fuse degradation strategy, performs cross-batch correlation analysis and corrects timing through a delay compensation mechanism, and adaptively adjusts a processing mode according to risk assessment parameters, thereby solving the technical problem of system stability decline caused by abnormal data propagation and pollution of downstream processing results in the asynchronous processing of batching operation data flow.

Description

Technical Field

[0001] This invention belongs to the field of MES system technology, and more specifically, relates to an asynchronous processing method for MES system operation data streams. Background Technology

[0002] In the batching operation of a manufacturing execution system (MES), asynchronous data stream processing technology improves system throughput by decoupling data acquisition and processing. Traditional methods use sliding window statistics to detect anomalies and use fixed-priority queues to schedule resources. However, parameters such as material flow rate, temperature, and pressure exhibit strong nonlinear and non-stationary characteristics during the batching process. The sliding window statistical method relies on the assumption that the data follows a specific probability distribution, resulting in high detection latency and a high false alarm rate under nonlinear and non-stationary data. Abnormal data is not isolated in time before entering subsequent processing flows. In existing technologies, due to the lack of an effective mechanism to block the propagation path of abnormal data, abnormal data is mixed with normal data in the asynchronous processing pipeline, contaminating intermediate results and triggering cascading errors, leading to inaccurate batch correlation analysis, resource scheduling disorder, and message queue avalanche. In other words, existing technologies suffer from the technical problem of abnormal data propagation during the asynchronous data stream processing of batching operations, contaminating downstream processing results and causing a decrease in system stability. Summary of the Invention

[0003] In view of this, the present invention provides an asynchronous processing method for batching operation data stream in an MES system, which can solve the technical problem in the prior art where abnormal data propagation during the asynchronous processing of batching operation data stream contaminates downstream processing results, leading to a decrease in system stability.

[0004] This invention is implemented as follows: It provides an asynchronous data flow processing method for batching operations in a MES system. This method involves deploying multi-level data acquisition nodes at the batching site to synchronously collect material flow data and assign globally unique batch identifiers and timestamps. The material flow data is input into a phase space reconstruction and anomaly detection module to reconstruct a high-dimensional phase space trajectory and calculate the Lyapunov exponent, correlation dimension, and Kolmogorov entropy. When the deviation of the high-dimensional phase space trajectory from the normal attraction domain exceeds a deviation threshold, it is marked as abnormal data and diverted to the anomaly processing channel. For material flow data not marked as abnormal data, a thermodynamic entropy increase index is calculated, and the evolution trend is predicted based on the Boltzmann distribution. When the Shannon entropy exceeds the Shannon entropy threshold, the scheduling priority is increased. A two-layer game scheduling model is established to optimize the allocation of asynchronous processing resources and solve the game equilibrium point to output the resource allocation scheme. The back pressure threshold of the message queue is dynamically adjusted according to the resource allocation scheme. When the message queue depth exceeds the back pressure threshold, the circuit breaker degradation strategy is initiated to reduce the sampling rate of non-critical data streams. Cross-batch correlation analysis is performed on batch data that has completed asynchronous processing, and batch lineage information is traced based on the event tracing model. When the data delay jitter between batches exceeds the delay jitter threshold, the delay compensation mechanism is triggered to correct the processing sequence. Risk assessment parameters are calculated, and the processing mode is adaptively adjusted according to the risk assessment parameters.

[0005] The multi-level data acquisition nodes include material flow sensors, temperature sensors, pressure sensors, and weighing sensors. The material flow sensors, temperature sensors, pressure sensors, and weighing sensors synchronously collect material flow data according to a set sampling period.

[0006] Among them, the phase space reconstruction anomaly detection module is based on the Tukens embedding theorem. It constructs a high-dimensional phase space from univariate time series data through delayed coordinate embedding. The reconstructed high-dimensional phase space is topologically equivalent to the original dynamic system phase space.

[0007] Among them, the Lyapunov index is used to quantify the sensitivity of high-dimensional phase space trajectories to initial conditions, the correlation dimension describes the fractal dimension of attractors in high-dimensional phase space, and the Kolmogorov entropy measures the information generation rate of the system.

[0008] Among them, the normal attraction domain is the clustering region of high-dimensional phase space trajectories in normal operating state in high-dimensional phase space, and the deviation is calculated as the normalized distance from the current high-dimensional phase space trajectory point to the centroid of the normal attraction domain.

[0009] Among them, the material flow data that was not marked as abnormal data was classified according to material type, process stage and batch correlation, and Shannon entropy and conditional entropy were calculated respectively. Shannon entropy is used to measure the information uncertainty of classified material flow data.

[0010] Among them, the entropy increase model treats the delay of classified material flow data as energy, predicts the probability of occurrence of different delay states through Boltzmann distribution, and infers the evolution trend of classified material flow data.

[0011] In the two-layer game scheduling model, the upper-layer model aims to minimize the overall latency of the data flow, while the lower-layer model aims to maximize the throughput of critical data flow processing. The upper-layer and lower-layer models achieve collaborative optimization through resource allocation coupling terms.

[0012] The game equilibrium point is solved using an iterative optimization algorithm. The upper-level model adjusts the initial resource allocation ratio based on the feedback from the lower-level model, and the lower-level model optimizes the processing strategy based on the new initial resource allocation ratio. This process is repeated until the strategies of the upper-level and lower-level models no longer change.

[0013] Among them, the back pressure threshold is dynamically adjusted according to the message queue depth. The back pressure mechanism is a strategy of backpropagating flow control signals. The back pressure signal is transmitted from the downstream processing node to the upstream producer and instructs the upstream producer to reduce the data sending rate.

[0014] Among them, the circuit breaker degradation strategy reduces the sampling rate of non-critical data streams. The adjustment of the sampling rate is negatively correlated with the message queue depth. Critical data streams maintain their original sampling rate and scheduling priority during the circuit breaker degradation strategy.

[0015] Among them, batch lineage information records the complete flow path of materials from raw material warehousing to finished product output, and the event tracing model regards batch state transitions as event sequences and persists them through distributed transaction logs.

[0016] Among them, the delay compensation mechanism identifies batch data with abnormal delays by comparing the timestamps of batch data with batch lineage information, and reconstructs the data association relationship using historical state information in the distributed transaction log.

[0017] The risk assessment parameters are calculated by combining message queue depth, inter-batch data latency jitter, and Shannon entropy. The message queue depth, inter-batch data latency jitter, and Shannon entropy are normalized and weighted according to their impact on system stability.

[0018] In this process, before the phase space reconstruction anomaly detection module reconstructs the high-dimensional phase space trajectory, the univariate time series data is preprocessed for denoising. The denoising preprocessing uses wavelet transform to filter out high-frequency noise and retain the main features of the univariate time series data.

[0019] Specifically, when calculating Shannon entropy in the entropy increase model, differentiated Shannon entropy thresholds are set for material flow data of different material types. The Shannon entropy threshold for reactive material flow data is lower than that for stable material flow data.

[0020] This invention maps univariate time-series data to a high-dimensional phase space through a phase space reconstruction anomaly detection module, and calculates dynamic characteristics such as the Lyapunov exponent, correlation dimension, and Kolmogorov entropy. It monitors the degree to which the trajectory deviates from the normal attraction domain in real time, completing anomaly identification within milliseconds and immediately diverting anomalous data to an isolation channel to prevent downstream propagation. The phase space reconstruction method, based on the Tukens embedding theorem, captures the intrinsic dynamic structure of nonlinear systems without assuming data distribution, overcoming the inherent delay and high false alarm rate of traditional statistical methods for non-stationary data. Once marked, anomalous data does not participate in subsequent entropy increase calculations and game-theoretic scheduling, ensuring resource allocation decisions are based on a clean dataset, preventing scheduling disorder and queue backlog caused by erroneous indicators, and guaranteeing the accuracy of batch correlation analysis and continuous stable system operation. In summary, this invention solves the technical problem mentioned in the background art where anomalous data propagation during asynchronous processing of batching operation data streams contaminates downstream processing results, leading to decreased system stability. Attached Figure Description

[0021] Figure 1 This is a flowchart of the method of the present invention.

[0022] Figure 2 A graph showing the latency comparison of data streams with different priorities.

[0023] Figure 3 This is a graph showing the dynamic changes in message queue depth and backpressure threshold.

[0024] Figure 4 This is a comparison chart showing the timing alignment effect before and after batch delay compensation. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below.

[0026] like Figure 1 The diagram shown is a flowchart of an asynchronous data stream processing method for MES system configuration operations provided by the present invention. This method includes the following steps:

[0027] S1. Deploy multi-level data acquisition nodes at the batching operation site. The multi-level data acquisition nodes include material flow sensors, temperature sensors, pressure sensors and weighing sensors. The material flow sensors, temperature sensors, pressure sensors and weighing sensors synchronously collect material flow data according to the set sampling period and assign a globally unique batch identifier and timestamp to each material flow data.

[0028] S2. Input the collected material flow data into the phase space reconstruction anomaly detection module. The phase space reconstruction anomaly detection module reconstructs the univariate time series data in the material flow data into a high-dimensional phase space trajectory and calculates the Lyapunov exponent, correlation dimension and Kolmogorov entropy of the high-dimensional phase space trajectory. When the deviation of the high-dimensional phase space trajectory from the normal attraction domain is greater than the deviation threshold, it is marked as abnormal data and diverted to the anomaly processing channel.

[0029] S3. Calculate the thermodynamic entropy increase index for material flow data that is not marked as abnormal data. After classifying the material flow data that is not marked as abnormal data according to material type, process stage and batch correlation, calculate Shannon entropy and conditional entropy respectively. Predict the evolution trend of each category of material flow data based on Boltzmann distribution and construct an entropy increase model. When the Shannon entropy of the category of material flow data is greater than the Shannon entropy threshold, increase the scheduling priority of the category of material flow data.

[0030] S4. Establish a two-layer game scheduling model to optimize the allocation of asynchronous processing resources. The upper-layer model of the two-layer game scheduling model aims to minimize the overall latency of the data flow, while the lower-layer model aims to maximize the throughput of critical data flow processing. The upper-layer model and the lower-layer model achieve collaborative optimization through resource allocation coupling terms. After solving the game equilibrium point of the two-layer game scheduling model, the resource allocation scheme of each processing node is output.

[0031] S5. Dynamically adjust the back pressure threshold of the message queue according to the resource allocation scheme of each processing node. When the message queue depth exceeds the back pressure threshold, the circuit breaker degradation strategy is activated. The circuit breaker degradation strategy implements sampling rate reduction processing for non-critical data streams. The sampling rate adjustment range of the sampling rate reduction processing is negatively correlated with the message queue depth.

[0032] S6. Perform cross-batch correlation analysis on batch data that has completed asynchronous processing. The cross-batch correlation analysis traces batch lineage information based on the event sourcing model and builds batch state transition logs. When the data delay jitter between batches exceeds the delay jitter threshold, the delay compensation mechanism is triggered. The delay compensation mechanism restores the data correlation relationship and corrects the processing sequence through the distributed transaction log.

[0033] S7. Calculate risk assessment parameters When risk assessment parameters The asynchronous data stream processing system is operating normally, when the risk assessment parameters... Returning to step S4, the two-layer game scheduling model is re-executed to optimize asynchronous processing resource allocation, when the risk assessment parameters... When an emergency response mode is activated, the sampling rate of non-critical data streams is reduced to the minimum sampling rate, and the number of processing nodes is increased.

[0034] The phase space reconstruction anomaly detection module is based on the Tukens embedding theorem, which states that after a univariate time series is embedded using delayed coordinates to construct a high-dimensional phase space, the reconstructed high-dimensional phase space is topologically equivalent to the original dynamic system phase space. For univariate time series data collected by material flow sensors, temperature sensors, pressure sensors, and weighing sensors, the embedding dimension and delay time parameters are selected to reconstruct the univariate time series data into a high-dimensional vector sequence. This high-dimensional vector sequence forms a high-dimensional phase space trajectory. The Lyapunov exponent is used to quantify the sensitivity of the high-dimensional phase space trajectory to initial conditions; a positive Lyapunov exponent indicates chaotic characteristics, while a negative Lyapunov exponent indicates stable convergence. The correlation dimension describes the fractal dimension of the attractor in the high-dimensional phase space and reflects the nonlinearity of the system. Kolmogorov entropy measures the information generation rate of the system and characterizes the randomness of the system. Under normal operating conditions, the univariate time-series data collected by the material flow sensor, temperature sensor, pressure sensor, and weighing sensor form a stable normal attraction domain structure in the high-dimensional phase space. The high-dimensional phase space trajectory moves around the normal attraction domain, and its geometry remains relatively stable. When the material flow sensor, temperature sensor, pressure sensor, or weighing sensor malfunctions, or when the process is disturbed, the high-dimensional phase space trajectory deviates significantly. The degree of deviation from the normal attraction domain serves as a quantitative indicator of the degree of anomaly.

[0035] By monitoring real-time changes in the sign of the Lyapunov exponent, abrupt changes in the correlation dimension, and abnormal fluctuations in Kolmogorov entropy, early identification of faults, abrupt changes in physical phenomena, and equipment performance degradation in material flow sensors, temperature sensors, pressure sensors, and weighing sensors is achieved. The phase space reconstruction anomaly detection module does not require the assumption that univariate time-series data follows a certain probability distribution and is adaptable to nonlinear and non-stationary batching process data. The phase space reconstruction anomaly detection module avoids the inherent delay of traditional sliding window statistical methods, directly reflecting system state changes through the real-time evolution of the high-dimensional phase space geometry, with detection latency controlled at the millisecond level. Once anomaly data is marked, it enters the anomaly processing channel and is not involved in the calculation of subsequent material flow data that is not marked as anomaly, thereby preventing the propagation of anomalies and contamination of downstream processing results, ensuring the quality and reliability of the batching operation data stream.

[0036] The entropy increase model is based on the second law of thermodynamics, mapping the disorder of categorized material flow data to the concept of thermodynamic entropy. Shannon entropy measures the information uncertainty of categorized material flow data. For discrete categorized material flow data, Shannon entropy is defined as the negative logarithmic weighted sum of the probabilities of each state; a higher Shannon entropy value indicates a greater amount of information or higher disorder in the categorized material flow data. Conditional entropy describes the residual uncertainty under conditions of known partial information and is used to assess the correlation strength between categorized material flow data. The Boltzmann distribution, originating from statistical physics, describes the probability distribution of a system under different energy states, with high-energy states corresponding to low probabilities. By treating the delay in categorized material flow data as energy, the probability of different delayed states is predicted using the Boltzmann distribution, thereby inferring the evolution trend of the categorized material flow data.

[0037] The core idea of ​​the entropy increase model is to treat high Shannon entropy classified material flow data as highly disordered regions of the system, prioritizing the allocation of processing resources to reduce overall disorder, similar to the mechanism of Maxwell's demon. Maxwell's demon is a hypothetical microscopic intelligent agent that selectively allows high-speed molecules to pass while blocking low-speed molecules, thereby reducing system entropy without consuming energy. In the scheduling of classified material flow data, the phase space reconstruction anomaly detection module and the entropy increase model act as Maxwell's demon. By calculating the Shannon entropy and conditional entropy of each classified material flow data, they identify classified material flow data with high information value or high urgency, dynamically adjusting the scheduling priority of classified material flow data in the message queue. For classified material flow data with Shannon entropy exceeding the Shannon entropy threshold, increasing the scheduling priority means allocating more processing resources and shorter waiting times, ensuring that critical information is delivered to downstream nodes first.

[0038] The technical effect of the entropy increase model lies in transforming the abstract characteristics of classified material flow data into quantifiable thermodynamic indicators, providing a theoretical basis and mathematical tools for asynchronous processing resource allocation. Intelligent sorting of classified material flow data is achieved through Shannon entropy threshold constraints, avoiding the situation where low-value classified material flow data occupies processing resources, leading to delays in high-value classified material flow data. This reduces the processing latency of key control data in batching operations, improves the system's response speed to sudden events and process changes, and simultaneously reduces the ineffective consumption of computing resources.

[0039] The two-layer game scheduling model models the asynchronous processing resource allocation problem as a game process between upper and lower level decision-makers. The upper-level objective function of the upper-level model is used to minimize the overall data flow latency of all categories of material flow data. The inputs of the upper-level objective function include the arrival rate of each category of material flow data, the computing power of the processing nodes, and the network transmission bandwidth. The output of the upper-level objective function is the initial resource allocation ratio of each processing node. The upper-level objective function normalizes the product of the weighted sum of the latency of each category of material flow data and the initial resource allocation ratio, and then finds its minimum value. The upper-level constraints of the upper-level model include total resource limits and load balancing constraints for each processing node.

[0040] The lower-level objective function of the lower-level model maximizes the processing throughput of the critical data stream. The inputs to the lower-level objective function include the critical data stream identifier, the initial resource allocation ratio, and the current queue state. The output of the lower-level objective function is the actual processing rate of the critical data stream. The lower-level objective function normalizes the resource allocation coupling term (the critical data stream processing throughput versus the initial resource allocation ratio) and then maximizes it. The lower-level constraints of the lower-level model include a processing rate cap and queue stability constraints. The resource allocation coupling term manifests in the fact that the initial resource allocation ratio allocated by the upper-level model directly affects the processing capacity of the lower-level model, while the feedback from the critical data stream processing throughput of the lower-level model influences the upper-level model's decision to adjust the initial resource allocation ratio.

[0041] An iterative optimization algorithm is used to find the game equilibrium point. The upper-level model adjusts the initial resource allocation ratio based on feedback from the lower-level model, and the lower-level model optimizes its processing strategy based on the new initial resource allocation ratio. This process is repeated until the strategies of the upper and lower-level models no longer change. The game equilibrium point corresponds to the optimal balance between system resource utilization and critical data stream processing efficiency. The two-layer game scheduling model avoids the uneven allocation of asynchronous processing resources caused by single-objective optimization. Through two-layer collaboration, it achieves a unified approach to control the overall latency of the data stream and ensure the throughput of critical data stream processing. This adapts to the dual requirements of batching operations, which require both overall process stability and timely critical control, thus improving the robustness and adaptability of the asynchronous processing system.

[0042] The backpressure threshold is dynamically adjusted based on the message queue depth. The backpressure mechanism is a strategy for backpropagating flow control signals. When downstream processing nodes experience a decrease in processing capacity due to computationally intensive tasks or database transaction blocking, message queue backlog grows rapidly. The backpressure signal is transmitted from the downstream processing nodes to the upstream producers, instructing them to reduce the data transmission rate. The backpressure threshold is the critical message queue depth that triggers the backpressure signal. Setting the backpressure threshold too low leads to frequent triggering, affecting system stability; setting it too high fails to prevent message queue overflow in a timely manner. The dynamic adjustment strategy calculates a reasonable backpressure threshold in real time based on the resource allocation scheme of each processing node and the current system load.

[0043] The circuit breaker and degradation strategy is activated when the message queue depth exceeds the backpressure threshold. This strategy reduces the sampling rate of non-critical data streams, discarding some data points or lowering the sampling frequency according to a set ratio. The magnitude of the sampling rate adjustment is negatively correlated with the message queue depth; the greater the message queue depth, the larger the adjustment. This sacrifices the integrity of non-critical data streams for overall system stability. Critical data streams maintain their original sampling rate and scheduling priority during the circuit breaker and degradation process, ensuring that core control signals for batching operations are unaffected. The circuit breaker and degradation strategy implements multi-level buffering and elastic scaling to avoid memory overflow or data loss caused by message queue backlog, preventing cascading collapses of the data pipeline and ensuring the continuous operation of the asynchronous batching system.

[0044] The batch lineage information records the complete flow path of materials from raw material warehousing to finished product output. The event sourcing model treats batch state transitions as event sequences. Each event in the sequence includes a timestamp, operation type, material attributes, and equipment status. The event sequence is persistently stored through a distributed transaction log. Cross-batch correlation analysis requires matching the correspondence between different batches in mixed processes. Due to the uncertain delay of asynchronous processing, batch data arriving at the cross-batch correlation analysis node at the same time comes from batches from different production times. Inter-batch data latency jitter refers to the random fluctuation of batch data processing latency, which reaches the second or minute level in batching operations.

[0045] The delay compensation mechanism identifies batches with abnormal delays by comparing their timestamps with batch lineage information and calculates their expected arrival times. It reconstructs data relationships using historical state information from the distributed transaction log. When inter-batch data delay jitter exceeds a threshold, the mechanism corrects the processing sequence, ensuring accuracy for quality traceability and process optimization analysis. This mechanism achieves eventual consistency across systems, overcoming the challenge of batch data time alignment in asynchronous processing and providing a reliable data foundation for causal link tracing and quality backtracking in batching operations.

[0046] The globally unique batch identifier is generated using the snowflake algorithm. This identifier includes a timestamp, data center identifier, and sequence number, ensuring its uniqueness within the distributed system. The normal attraction domain is a region of aggregation of high-dimensional phase space trajectories in normal operating state within the high-dimensional phase space. Its boundary range is determined through historical univariate time-series data statistics. The deviation is calculated as the normalized distance from the current high-dimensional phase space trajectory point to the centroid of the normal attraction domain. The Shannon entropy threshold is determined based on the Shannon entropy value distribution obtained from historical material flow data statistics, with the upper quartile of the Shannon entropy value distribution used as the threshold. The critical data streams include temperature control signals, pressure regulation commands, and formula parameter update data. These directly affect ingredient quality and safety. The resource allocation coupling term represents the product of the processor time slice and memory capacity allocated to the upper-level model and the actual critical data stream processing throughput of the lower-level model.

[0047] The negative correlation refers to the decrease in the sampling rate as the message queue depth increases, with the adjustment magnitude of the sampling rate decreasing according to a preset decreasing function. The distributed transaction log employs a write-ahead log mechanism, which ensures the atomicity and durability of event sequence recording. The calculation of the Lyapunov exponent requires analysis of the exponential divergence rate of neighboring orbits of the high-dimensional phase space trajectory. The correlation dimension is determined by the slope of the logarithmic coordinate graph of the correlation integral function. The Kolmogorov entropy measures the information generation rate of the high-dimensional phase space trajectory; a larger Kolmogorov entropy value indicates stronger system randomness. The Shannon entropy is calculated based on the probability distribution of the state of the categorized material flow data, obtained through frequency statistics of the categorized material flow data within a sliding window. The conditional entropy reflects the residual uncertainty of another category of material flow data given information about one category of material flow data. The energy parameter of the Boltzmann distribution is determined by the average delay and delay variance of the categorized material flow data.

[0048] The iterative optimization algorithm employs the alternating direction multiplier method, which either fixes the upper-level model variables to optimize the lower-level objective function or fixes the lower-level model variables to optimize the upper-level objective function. The load balancing constraint limits the resource utilization difference among processing nodes to no more than a set load balancing threshold. The upper limit of the processing rate is determined by the hardware performance of the processing nodes and the algorithm's time complexity. The write-ahead log mechanism writes operation records to the distributed transaction log before modifying batch data, ensuring batch data consistency during fault recovery. The decrement function uses an exponential decay form, and the decay rate is determined based on the message queue capacity and historical message queue backlog statistics. The risk assessment parameters... The Shannon entropy index is calculated by combining message queue depth, inter-batch data latency jitter, and Shannon entropy index. The message queue depth, inter-batch data latency jitter, and Shannon entropy index are normalized and weighted according to their impact on system stability.

[0049] Furthermore, in step S2, before the phase space reconstruction anomaly detection module reconstructs the high-dimensional phase space trajectory, it performs denoising preprocessing on the univariate time series data. The denoising preprocessing uses wavelet transform to filter out high-frequency noise and retain the main features of the univariate time series data.

[0050] Furthermore, in step S3, when the entropy increase model calculates Shannon entropy, it sets differentiated Shannon entropy thresholds for the classified material flow data of different material types. The Shannon entropy threshold for the classified material flow data of reactive materials is lower than that for the classified material flow data of stable materials.

[0051] Furthermore, in step S4, when the two-layer game scheduling model solves for the game equilibrium point, an upper limit is set for the number of iterations. When the number of iterations reaches the upper limit but does not converge, the current optimal solution is used as the resource allocation scheme for each processing node.

[0052] Furthermore, in step S5, the back pressure threshold adjustment cycle is synchronized with the system load monitoring cycle, and the system load monitoring cycle is dynamically determined according to the batching operation cycle.

[0053] Furthermore, in step S6, the batch lineage information is stored using a graph database structure, where nodes represent batch states and edges represent batch state transition events.

[0054] The sampling period is set according to the type of material flow sensor, temperature sensor, pressure sensor, and weighing sensor, and process requirements. The set sampling period for the material flow sensor is 100ms, for the temperature sensor it is 500ms, for the pressure sensor it is 200ms, and for the weighing sensor it is 1000ms. The embedding dimension is determined using the pseudo-nearest neighbor method, and the delay time parameter is selected using the mutual information method or the autocorrelation function method. The anomaly handling channel includes data isolation, source tracing analysis, and alarm notification functions. The data isolation function stores abnormal data in an anomaly database for subsequent fault diagnosis. The material types include reactive materials and stable materials. Reactive materials refer to materials that undergo chemical reactions during the batching process, while stable materials refer to materials that maintain stable physical properties during the batching process. The process stages include the material feeding stage, the mixing and stirring stage, and the temperature control stage. The material feeding stage refers to the process of adding raw materials to the batching container; the mixing and stirring stage refers to the process of uniformly dispersing the materials through mechanical stirring; and the temperature control stage refers to the process of controlling the batching temperature through heating or cooling. The batch correlation degree represents the degree of correlation between different batches of materials in the mixing process, and is calculated based on the mixing ratio and time overlap of the batch materials. The overall data flow latency is the sum of the time from acquisition to processing completion of all categorized material flow data. The critical data flow processing throughput is the amount of critical data flow data processed per unit time. The total resource limit refers to the sum of processor time slices and memory capacity not exceeding the upper limit of available system resources. The queue stability constraint refers to the fluctuation range of message queue depth not exceeding a set fluctuation threshold. The upstream producer refers to the data acquisition module that sends categorized material flow data to the message queue. The downstream processing node refers to the data processing module that receives categorized material flow data from the message queue and performs calculations. The minimum sampling rate is the minimum data acquisition frequency to ensure basic system monitoring functions. The emergency handling mode refers to a special operating mode activated when the system experiences severe processing latency; the emergency handling mode quickly restores system stability by significantly reducing the sampling rate of non-critical data flows and increasing the number of processing nodes.

[0055] The technical advantages of this invention lie in its ability to identify anomalies in the nonlinear batching process early through a phase space reconstruction anomaly detection module, avoiding the failure of traditional statistical methods under non-stationary single-variable time-series data. It transforms the scheduling of classified material flow data into a physical optimization problem through an entropy increase model, providing a theoretical basis and quantitative indicators for asynchronous processing resource allocation. A two-layer game scheduling model coordinates the conflicting demands of overall data flow delay control and critical data flow throughput assurance, achieving adaptive optimization of the asynchronous processing system. Backpressure mechanisms and circuit breaker degradation strategies prevent system crashes caused by message queue backlog, ensuring continuous and stable operation of the batching process. Event tracing models and delay compensation mechanisms overcome the challenge of batch data time alignment in asynchronous processing, providing accurate data support for quality traceability and process optimization.

[0056] Alternatively, the present invention also provides an asynchronous data stream processing system for batching operations in an MES system implemented by a computer. The computer is equipped with a readable storage medium that stores program instructions. When the program instructions are run in the computer, they execute the aforementioned asynchronous data stream processing method for batching operations in an MES system.

[0057] The specific implementation methods of the above steps are described in detail below.

[0058] The specific implementation of step S1 involves first installing a material flow sensor on the material conveying pipeline at the batching operation site, a temperature sensor on the wall of the batching container, a pressure sensor at the pipeline connection, and a weighing sensor at the bottom of the batching platform, forming a multi-level data acquisition node covering the entire material flow process. The material flow sensor uses an electromagnetic flowmeter to measure the volume of material passing through the pipeline per unit time; the temperature sensor uses a thermocouple to measure the temperature change inside the batching container; the pressure sensor uses a piezoresistive sensor to measure pressure fluctuations inside the pipeline; and the weighing sensor uses a strain gauge sensor to measure the total weight change of the batching platform. Based on the response speed of different sensors and process monitoring requirements, the sampling period for the material flow sensor is set to 100ms to capture rapid flow fluctuations; the sampling period for the temperature sensor is set to 500ms because temperature changes are relatively slow; the sampling period for the pressure sensor is set to 200ms to monitor pressure pulsations; and the sampling period for the weighing sensor is set to 1000ms to meet the requirements for weight accumulation monitoring. Each material flow data entry generates a globally unique batch identifier using the Snowflake algorithm at the moment of acquisition. The Snowflake algorithm converts the current timestamp into the high-order bits of a 64-bit integer, with the data center identifier occupying the middle 10 bits and the auto-incrementing sequence number occupying the low-order 12 bits, ensuring that the globally unique batch identifier will not be duplicated in a distributed acquisition environment. Simultaneously, a nanosecond-precision timestamp is appended to each material flow data entry, recording the absolute moment the data was generated, providing a time reference for subsequent batch-to-batch data latency and jitter analysis. The purpose of these steps is to establish a complete batching operation data acquisition system, providing a high-quality raw data source for asynchronous processing, and achieving comprehensive monitoring of the batching process through multi-level sensor collaboration.

[0059] The specific implementation of step S2 involves inputting the material flow data collected in step S1 into the phase space reconstruction anomaly detection module for real-time anomaly identification. First, the univariate time series data undergoes denoising preprocessing. Wavelet transform is used to decompose the univariate time series data into wavelet coefficients of multiple scales. A soft threshold function is used to suppress noise components in the high-frequency wavelet coefficients, and then the denoised univariate time series data is reconstructed. Then, the embedding dimension is determined using the pseudo-nearest neighbor method. This method gradually increases the embedding dimension and calculates the proportion of pseudo-nearest neighbors in the phase space. The dimension corresponding to the decrease in the proportion of pseudo-nearest neighbors to below 5% is the optimal embedding dimension. Typically, the embedding dimension ranges from 3 to 7. The delay time parameter is selected using the mutual information method. The mutual information method calculates the mutual information between the univariate time series data and its delayed version. The delay time corresponding to the first minimum value of the mutual information is the optimal delay time parameter. Using the determined embedding dimension and delay time parameter, the univariate time series data is reconstructed into a high-dimensional vector sequence, which forms a high-dimensional phase space trajectory in the high-dimensional phase space. When calculating the Lyapunov exponent of the high-dimensional phase space trajectory, the Wolf algorithm is used to track the exponential divergence rate of adjacent trajectories. A Lyapunov exponent greater than 0.1 indicates enhanced chaotic characteristics. The correlation dimension is obtained by calculating the slope of the correlation integral function on the logarithmic coordinate graph using the Grassberger-Procaccia algorithm. A sudden change in the correlation dimension exceeding 15% indicates a significant change in the system's nonlinearity. The Kolmogorov entropy is calculated by statistically analyzing the access frequency of different regions in the high-dimensional phase space. A fluctuation in Kolmogorov entropy exceeding 30% of the average indicates an abnormally enhanced randomness in the system. The distribution region of the high-dimensional phase space trajectory is statistically analyzed using historical univariate time-series data of normal operation to determine the boundary of the normal attraction domain. The centroid of the normal attraction domain is the geometric center of all normal high-dimensional phase space trajectory points. The Euclidean distance from the current high-dimensional phase space trajectory point to the centroid of the normal attraction domain is calculated, and the normalized deviation is obtained by dividing the Euclidean distance by the maximum radius of the normal attraction domain. The deviation threshold is determined based on the deviation distribution statistically analyzed from historical data, typically using the 95th percentile of the deviation distribution as the threshold, with a reference value of 0.75. When the deviation exceeds the threshold, the current material flow data is marked as abnormal and diverted to the anomaly handling channel. The anomaly handling channel uses data isolation to store the abnormal data in an anomaly database, traces the sensor node and time window that caused the anomaly through source analysis, and sends anomaly alarm information to maintenance personnel through alarm notification. These steps, based on the Tukens embedding theorem, map the dynamic characteristics of univariate time-series data to a high-dimensional phase space geometry. By monitoring the geometric shape and dynamic invariants of the high-dimensional phase space trajectory, early identification of sensor faults and process disturbances is achieved, preventing abnormal data from contaminating subsequent processing flows, with detection latency controlled within 10ms.

[0060] The specific implementation of step S3 involves calculating the thermodynamic entropy increase index of material flow data not marked as abnormal data and dynamically adjusting the scheduling priority. First, the material flow data not marked as abnormal data is classified according to material type, process stage, and batch correlation. Material types are distinguished as reactive materials and stable materials; process stages are distinguished as material delivery stage, mixing and stirring stage, and temperature regulation stage; batch correlation is calculated based on the mixing ratio and time overlap of batch materials. A sliding time window is constructed for each category of material flow data, with a window length set to 10 to 30 seconds. The state of the categorized material flow data within the window is discretized into multiple levels. The frequency of each state occurring within the window is calculated and normalized to a probability distribution. The probability distribution is substituted into the Shannon entropy calculation formula to obtain the Shannon entropy of the categorized material flow data. The Shannon entropy is calculated as the sum of the products of the probability of each state and its negative logarithm. For two related categories of material flow data, their joint probability distribution and marginal probability distribution are calculated. The conditional entropy equals the joint entropy minus the marginal entropy. The conditional entropy reflects the residual uncertainty of one category of material flow data after knowing the other category. The historical latency distribution of categorized material flow data is used as the energy distribution, with longer latency corresponding to higher energy. The probability of different latency states is calculated using the Boltzmann distribution formula. The energy parameter of the Boltzmann distribution is determined by the average latency and latency variance of the categorized material flow data. The Boltzmann distribution predicts the probability that the latency of categorized material flow data will exceed a threshold within a future time window. An entropy increase model is constructed by combining Shannon entropy and conditional entropy to assess the trend of disorder evolution in the categorized material flow data. The Shannon entropy threshold is determined based on the upper quartile of the Shannon entropy value distribution of historical categorized material flow data. The reference value for the Shannon entropy threshold is 2.8 for reactive material categorized material flow data and 3.5 for stable material categorized material flow data. When the Shannon entropy of categorized material flow data exceeds the Shannon entropy threshold, the scheduling priority of the categorized material flow data in the message queue is increased. This increase in scheduling priority is achieved by increasing the weight of the corresponding data in the message queue, with higher-weighted data being prioritized for downstream processing nodes. The steps described above introduce the second law of thermodynamics into data stream scheduling. By quantifying and classifying the information uncertainty and evolution trend of material flow data, a smart sorting effect similar to Maxwell's demon is achieved, ensuring that high-value and urgent classified material flow data are processed first and reducing the processing delay of critical control data.

[0061] The specific implementation of step S4 involves establishing a two-layer game scheduling model to optimize asynchronous processing resource allocation and solve for the game equilibrium point. The upper-layer model aims to minimize the overall data flow latency. The inputs to the upper-layer objective function include the arrival rate of data from each category of material flow, the computing power of processing nodes, and network transmission bandwidth. The upper-layer objective function normalizes the product of the weighted sum of the latency of each category of material flow data and the initial resource allocation ratio. This normalization is achieved by dividing the product by the latency cap and the total resource amount to unify the dimensions. The upper-layer objective function outputs the initial resource allocation ratio for each processing node. Upper-layer constraints include total resource limits and load balancing constraints. The total resource limit requires that the total processor time slices allocated to all processing nodes do not exceed the system's available processor time slice limit, and the total memory capacity does not exceed the system's available memory capacity limit. The load balancing constraint limits the difference in resource utilization among processing nodes to no more than a set load balancing threshold, with a reference value of 20%. The lower-level model aims to maximize the throughput of critical data stream processing. The inputs to the lower-level objective function include the critical data stream identifier, the initial resource allocation ratio output by the upper-level model, and the current queue state. The critical data stream identifier is used to filter temperature control signals, pressure regulation commands, and recipe parameter update data. The lower-level objective function normalizes the resource allocation coupling term between the critical data stream processing throughput and the initial resource allocation ratio. This normalization is achieved by dividing the coupling term by the product of the throughput cap and the resource ratio cap, unifying the dimensions. The lower-level objective function outputs the actual processing rate of the critical data stream. Lower-level constraints include a processing rate cap and queue stability constraints. The processing rate cap is determined by the CPU frequency of the processing node and the processing time per algorithm cycle. The queue stability constraint limits the fluctuation of the message queue depth to no more than a set fluctuation threshold, with a reference value of 30% of the message queue capacity. The resource allocation coupling term reflects how the processor time slices and memory capacity allocated by the upper-level model directly determine the parallel processing capability of the lower-level model, while the critical data stream processing throughput of the lower-level model influences the resource reallocation decisions of the upper-level model. The alternating direction multiplier method is used as an iterative optimization algorithm to solve for the game equilibrium point. The alternating direction multiplier method first fixes the lower-level model variables to solve for the optimal initial resource allocation ratio of the upper-level objective function, then fixes the upper-level model variables to solve for the optimal actual processing rate of the lower-level objective function. This alternating iteration continues until the change magnitude of the decision variables in both the upper and lower-level models is less than the convergence threshold, with a reference value of 0.01. An upper limit of 50 iterations is set. When the number of iterations reaches the upper limit but does not reach the convergence threshold, the solution with the optimal objective function value in the current iteration is used as the resource allocation scheme for each processing node. These steps achieve synergistic optimization of minimizing global latency and maximizing critical task throughput through two-level game modeling, avoiding resource allocation imbalances caused by single-objective optimization. The game equilibrium point corresponds to the Pareto optimal state of system performance.

[0062] The specific implementation of step S5 involves dynamically adjusting the backpressure threshold of the message queue based on the resource allocation scheme of each processing node output in step S4, and initiating a circuit breaker and degradation strategy when the queue is congested. The backpressure threshold is calculated based on the processor time slice and memory capacity allocated to each processing node to determine the theoretical processing capacity. The theoretical processing capacity equals the processor time slice divided by the average processing time per data item. The backpressure threshold is set to 80% of the message queue depth corresponding to the theoretical processing capacity, with a reference value of 1000 to 5000 data items. When the message queue depth exceeds the backpressure threshold, a backpressure signal is sent to the upstream producer. The backpressure signal is transmitted through a reverse control frame. After receiving the backpressure signal, the upstream producer reduces the data transmission rate to 50% to 70% of the original rate. Simultaneously, a circuit breaker and degradation strategy is initiated to reduce the sampling rate of non-critical data streams. Non-critical data streams refer to material flow data other than temperature control signals, pressure regulation commands, and formula parameter update data. The negative correlation between the sampling rate adjustment and message queue depth is described by a decreasing function, which adopts an exponential decay form. When the message queue depth is 1.2 times the backpressure threshold, the sampling rate decreases to 70% of the original sampling rate; when the message queue depth is 1.5 times the backpressure threshold, the sampling rate decreases to 40% of the original sampling rate; and when the message queue depth is 2 times the backpressure threshold, the sampling rate decreases to the minimum sampling rate of 10%. The decay rate of the exponential decay form is determined based on the message queue capacity and the maximum backlog depth in historical message queue statistics. The larger the message queue capacity, the smaller the decay rate, allowing for a longer buffer time. During the execution of the circuit breaker degradation strategy, the critical data flow maintains its original sampling rate and scheduling priority, ensuring the real-time transmission of temperature control signals, pressure regulation commands, and recipe parameter update data in the batching operation. The steps described above achieve reverse propagation from downstream processing capacity to upstream production rate through the backpressure mechanism. The circuit breaker degradation strategy sacrifices the integrity of non-critical data flows for overall stability when the system is overloaded, preventing data loss and system crashes caused by message queue overflow.

[0063] The specific implementation of step S6 involves performing cross-batch correlation analysis on the batch data that has completed asynchronous processing and triggering a delay compensation mechanism when delay anomalies occur. First, a globally unique batch identifier is extracted from the batch data. Based on this identifier, batch lineage information stored in a graph database structure is queried. Batch lineage information is stored in the form of nodes and edges. Nodes represent the state of a batch at different process stages, and edges represent batch state transition events. Edge attributes include timestamps, operation types, material attributes, and equipment status. A graph traversal algorithm is used to trace the complete flow path of a batch from raw material warehousing to its current state, identifying the batch's association with other batches in the mixed process. Inter-batch data delay jitter is calculated, equal to the difference in timestamps between different batches within the same association group arriving at the cross-batch correlation analysis node. The delay jitter threshold is determined based on process requirements and historical delay statistics, with a reference value of 5 to 30 seconds. When inter-batch data delay jitter exceeds the threshold, a delay compensation mechanism is triggered. This mechanism reads the historical state transition event sequence of the delayed batch from the distributed transaction log. The distributed transaction log uses a write-ahead log mechanism to ensure the atomicity and durability of event records. The expected arrival time of delayed batches is calculated based on the historical state transition event sequence. The timestamps of delayed batches are corrected using time offset to align them with those of associated batches. When reconstructing data associations, a nearest neighbor matching algorithm is used. This algorithm calculates the distance between the corrected timestamp batch data and the batch data to be associated in both time and attribute space, establishing an association between the batch data with the smallest distance. After correcting the processing sequence, the batch state transition log is updated to ensure that subsequent quality traceability and process optimization analysis are based on the correct batch associations. These steps record the complete batch lifecycle through an event sourcing model and overcome the uncertain delays of asynchronous processing through a delay compensation mechanism, achieving eventual consistency across systems and providing a reliable data foundation for causal link tracing in batching operations.

[0064] The specific implementation of step S7 involves calculating risk assessment parameters and executing corresponding system control strategies based on the value range of these parameters. The risk assessment parameters are calculated by integrating message queue depth, inter-batch data latency jitter, and Shannon entropy. Message queue depth is normalized to the ratio of message queue depth to message queue capacity; inter-batch data latency jitter is normalized to the ratio of inter-batch data latency jitter to a latency jitter threshold; and Shannon entropy is normalized to the ratio of Shannon entropy to a Shannon entropy threshold. The three normalized indicators are weighted and summed according to their impact on system stability to obtain the risk assessment parameters. The impact weight of message queue depth is set to 0.4, the impact weight of inter-batch data latency jitter is set to 0.3, and the impact weight of Shannon entropy is set to 0.3. The sum of these impact weights equals 1, ensuring that the risk assessment parameters range from 0 to 1. When the risk assessment parameters are within the range of 0 to 0.35, the asynchronous data stream processing system is considered to be operating normally and requires no adjustment. The system continues to operate according to the current resource allocation scheme and backpressure threshold for each processing node. When the risk assessment parameter is between 0.35 and 0.75, the system is suspected of having a processing bottleneck. The process returns to step S4 to re-execute the two-layer game scheduling model to optimize asynchronous processing resource allocation. During this re-optimization, the current message queue depth and inter-batch data latency jitter are used as boundary values ​​to guide the game model towards reducing latency. When the risk assessment parameter is between 0.75 and 1, the system is determined to have severe processing latency. An emergency processing mode is activated. This mode first forcibly reduces the sampling rate of all non-critical data streams to a minimum of 10%, ensuring basic system monitoring functions. Then, it requests an increase in the number of processing nodes from the cluster manager through the dynamic resource scheduling interface. The number of new processing nodes is calculated based on the current message queue backlog depth and single-node processing capacity, or more processor time slices and memory capacity are allocated to existing processing nodes to improve processing capacity. These steps quantify the system's operational health through risk assessment parameters, enabling graded responses based on risk levels. This prevents system performance degradation from causing data pipeline collapse and ensures the robustness and continuous operation of the asynchronous batching system.

[0065] It should be noted that the key technical ideas of this invention include phase space reconstruction anomaly detection, thermodynamic entropy increase scheduling, and two-layer game resource optimization. The phase space reconstruction anomaly detection technology is based on nonlinear dynamics theory, mapping univariate time-series data to a high-dimensional phase space. Chaotic features such as the Lyapunov exponent, correlation dimension, and Kolmogorov entropy characterize the system's dynamic behavior. Normal operating conditions correspond to stable attractor structures, while abnormal states cause attractor deviation or breakage. Millisecond-level anomaly identification is achieved by monitoring the phase space geometry. Compared to traditional anomaly detection methods based on statistical distribution assumptions, the phase space reconstruction method does not require assumptions that the data follows a Gaussian or other distributions. It has stronger generalization ability for nonlinear and non-stationary data in industrial settings, and can identify gradual faults and coupled anomalies that are difficult to capture by traditional methods. Its detection sensitivity and robustness are significantly better than sliding window statistical methods, avoiding the propagation and diffusion of anomalous data in asynchronous pipelines. The thermodynamic entropy-increasing scheduling technique introduces Shannon entropy from information theory and Boltzmann distribution from statistical physics into data stream scheduling. It quantifies the disorder and evolutionary trend of data streams into calculable thermodynamic indicators. High-entropy data streams correspond to high information value or high urgency, and intelligent priority allocation of data streams is achieved through entropy threshold constraints. Compared to traditional scheduling strategies based on static priority or simple queue length, thermodynamic entropy-increasing scheduling reveals the intrinsic link between data stream disorder and processing urgency from a physics perspective. It predicts the future evolution path of data streams through an entropy-increasing model, achieving proactive resource pre-allocation. A selective sorting mechanism similar to Maxwell's demon ensures that critical information passes through processing bottlenecks first. The scheduling strategy is adaptively adjusted when system load fluctuates, significantly reducing end-to-end latency of critical control data. The two-layer game-theoretic resource optimization technique models asynchronous processing resource allocation as a game problem between upper and lower layer decision-makers. The upper layer aims to minimize global latency, while the lower layer aims to maximize critical task throughput. Coordination between the two objectives is achieved through resource allocation coupling terms, and the game equilibrium point corresponds to the Pareto optimal solution. Compared to traditional single-objective optimization or weighted multi-objective optimization methods, the two-level game model clearly distinguishes the hierarchical relationship between system-level objectives and task-level objectives, avoiding objective conflicts and performance degradation caused by simple weighting. Through the iterative game process, it adaptively balances global performance and local requirements, prioritizing critical tasks without sacrificing overall stability when processing capacity is limited. Its resource utilization efficiency and system response performance are superior to traditional heuristic scheduling algorithms.The synergistic effect of these three technical approaches lies in constructing a complete asynchronous processing framework that extends from data quality assurance to intelligent scheduling and resource optimization. Phase space reconstruction filters out anomalies at the data entry point to ensure that subsequent processing is based on reliable data. Thermodynamic entropy increase scheduling achieves intelligent diversion at the data transmission layer to avoid queue congestion. Two-layer game dynamics optimizes the distribution of computing power at the resource allocation layer to adapt to load changes. These three elements form a deep protection system for asynchronous data flow processing. Compared with the passive response mechanism of traditional asynchronous processing methods, this invention achieves an organic unity of proactive anomaly isolation, predictive scheduling decision-making, and adaptive resource allocation. It maximizes processing efficiency while ensuring the integrity of batching operation data, significantly improving the stability and real-time performance of the MES system in high-concurrency, strongly coupled industrial scenarios.

[0066] It should be noted that this invention also solves the following technical problem: In asynchronous batching operations, due to fluctuations in data flow arrival rates and differences in processing node capabilities, uneven resource allocation leads to excessive delays in critical control data. This invention calculates the Shannon entropy and conditional entropy of each category of data using an entropy increase model, prioritizing resource allocation for high-entropy data as highly disordered regions, similar to Maxwell's demon mechanism for intelligent sorting. A two-layer game-theoretic scheduling model coordinates the conflicting objectives of minimizing overall delay and maximizing critical throughput. The upper-layer model allocates initial resource ratios, while the lower-layer model provides feedback on critical flow processing efficiency. Iteratively solving for the game equilibrium point outputs the optimal solution, avoiding the unintended consequences of single-objective optimization. This ensures priority processing of critical signals such as temperature and pressure while maintaining overall system smoothness, thus resolving the problem of uneven resource allocation. Furthermore, this invention also solves the problem of inaccurate quality traceability caused by the difficulty in aligning batch data time in asynchronous processing. It records the batch state transition sequence through an event sourcing model and persists it to a distributed transaction log. When performing cross-batch correlation analysis, it traces the complete flow path based on batch lineage information. When the latency jitter exceeds the threshold, a latency compensation mechanism is triggered. It uses historical states to reconstruct data correlation and correct the processing timing, thereby achieving eventual consistency guarantee and providing a reliable foundation for causal link tracing.

[0067] Specifically, the principle of this invention is as follows: This invention explicitly transforms the implicit dynamic characteristics of time-series data into a geometric topological structure through phase space reconstruction; the Lyapunov exponent quantifies the sensitivity of the trajectory to initial conditions; the correlation dimension characterizes the fractal dimension of the attractor; and the Kolmogorov entropy measures the information generation rate. These three elements together constitute a complete feature space of the system state. During normal operation, the trajectory moves around a stable attraction domain. Faults or disturbances cause abrupt changes in the trajectory's geometric shape. Identification can be completed at the anomaly bud stage by calculating the normalized distance from the trajectory point to the centroid of the attraction domain in real time. Once anomaly data is marked, it immediately enters an isolation channel, physically separated from the normal data stream. The subsequent entropy increase model only calculates Shannon entropy and conditional entropy for unmarked data. The input to the two-layer game scheduling model does not contain anomaly samples. Only verified, clean data flows through the message queue, eliminating the physical path of anomaly propagation from the source. This ensures that downstream batch correlation analysis, resource allocation decisions, and delay compensation mechanisms operate based on a trusted dataset, guaranteeing system stability.

[0068] The following provides a specific embodiment 1 of the present invention, and the specific implementation of each step in this embodiment 1 is described in detail below.

[0069] The specific implementation method of step S1 is the same as described above, and will not be repeated in detail here.

[0070] The specific implementation of step S2 is as follows: the phase space of the univariate time series data is reconstructed using the Tukens embedding theorem, and the formula for constructing the high-dimensional vector sequence is as follows:

[0071] ;

[0072] In the formula, For the first One reconstructed vector; For univariate time series data at time... The observed values ​​are in meters; This is the delay time parameter, in milliseconds; The embedding dimension is unitless. This is a time index, with values ​​ranging from 1 to the total number of observations. Embedding dimension. Determined using the pseudo-nearest neighbor method, with an empirical value of 3 to 10, the delay time parameter... Selected using the mutual information method, the empirical value is 1 to 5 times the sampling period. Lyapunov index. The calculation formula is expressed as follows:

[0073] ;

[0074] In the formula, The Lyapunov index is represented by the unit Lyapunov index. ; This represents the total number of trajectory points. This is the sampling time interval, in seconds; For the first Reference trajectory points at any given time; For the first The nearest trajectory points at any given time; For the first Reference trajectory points at any given time; For the first The nearest trajectory points at any given time; The reference trajectory point at the initial moment; These are the nearest trajectory points at the initial moment; is the Euclidean distance norm, in meters; The normalized length coefficient is set to 1 meter. (Related integral function) The formula is expressed as follows:

[0075] ;

[0076] In the formula, The values ​​are associated integral function values ​​and have no unit. This is the distance threshold, in meters. and For the trajectory point index in the correlation integral calculation; For the first One reconstructed vector; For the first One reconstructed vector; This is the Heaviside step function, which takes the value 1 when the input is greater than zero, and 0 otherwise. It has no unit. This is the distance normalization coefficient, with a value of 1 meter. Correlation dimension. The calculation formula is expressed as follows:

[0077] ;

[0078] In the formula, This is the correlation dimension, which has no unit. For reference distance, a value of 1 meter is used. Kolmogorov entropy. The calculation formula is expressed as follows:

[0079] ;

[0080] In the formula, Kolmogorov entropy, in units of ; For time steps; For precise division, the unit is meters; For the region index in Kolmogorov entropy calculation; To determine the number of regions; For the first Access probability for each region, unitless. Deviation. The calculation formula is expressed as follows:

[0081] ;

[0082] In the formula, Deviation degree, unitless; This refers to the current trajectory point in the high-dimensional phase space. The coordinates of the centroid of the normal attraction domain are in meters. The standard deviation of the normal attraction domain is expressed in meters. For reference length, a value of 1 meter is used. When greater than the deviation threshold Time markers are used to identify outlier data, and the deviation threshold is set. The experience value is 3.0.

[0083] The specific implementation of step S3 is as follows: Calculate the Shannon entropy for the classified material flow data. The formula is expressed as follows:

[0084] ;

[0085] In the formula, Shannon entropy, unitless; For the first The frequency of occurrence of a certain state, without units; Total number of states. Frequency of state occurrence. This was obtained through statistical analysis of material flow data categorized within a sliding window; the empirical value for the sliding window length is 100 data points. Conditional entropy. The calculation formula is expressed as follows:

[0086] ;

[0087] In the formula, Conditional entropy, which has no unit; and This refers to the state index used in conditional entropy calculation; This is a combined frequency, without units; These are edge frequencies, without units; and The number of states; For known categories of material flow data; The data represents the flow of materials to be evaluated. The formula for predicting the probability of delayed states based on the Boltzmann distribution is as follows:

[0088] ;

[0089] In the formula, For delay The corresponding probability has no unit. Data latency, in seconds; is the Boltzmann constant, with a value of 1 second; This is a temperature parameter; the unit is unitless. For all possible delay states, in seconds. Temperature parameter. The calculation formula is expressed as follows:

[0090] ;

[0091] In the formula, This is a temperature parameter, without units. The average delay is expressed in seconds. The standard deviation of the delay is expressed in seconds. This is the temperature normalization coefficient, with a value of 1 second. When... Greater than the Shannon entropy threshold Increase scheduling priority as needed.

[0092] The specific implementation of step S4 is as follows: upper-level objective function The formula is expressed as follows:

[0093] ;

[0094] In the formula, This is the value of the upper-level objective function, which has no unit. For the first Weight coefficients for each data stream, without units; For the first The latency of each data stream, in seconds; The maximum delay is set to 10 seconds. For the first The initial resource allocation ratio for each data stream, without units; Total number of data streams; This is an index for data streams. Upper-level constraints include total resource limits, expressed in the following formula:

[0095] ;

[0096] In the formula, the symbols have the same meaning as before. The formula for load balancing constraints is expressed as follows:

[0097] ;

[0098] In the formula, For the first Resource utilization rate of each processing node, in percentage; Average resource utilization rate, expressed as a percentage; This is the utilization rate normalization coefficient, with a value of 100%; This is the load balancing threshold, unitless, with an empirical value of 0.2. Lower-level objective function. The formula is expressed as follows:

[0099] ;

[0100] In the formula, This represents the value of the lower-level objective function, which is unitless. For the first Resource allocation coupling coefficients for key data streams, unitless; For the first Key data stream processing throughput, in units of ; To determine the maximum throughput, the value is [value to be filled in]. ; For the number of critical data streams; This is an index for critical data flows. Lower-level constraints include a processing rate ceiling, expressed in the following formula:

[0101] ;

[0102] In the formula, For the first The maximum processing rate of each key data stream, in units of The stability of the queue is determined by the hardware performance of the processing node. The formula for the queue stability constraint is as follows:

[0103] ;

[0104] In the formula, This represents the standard deviation of message queue depth fluctuation, expressed in rows. This represents the average message queue depth, expressed in messages. This is the queue depth normalization coefficient, with a value of 1000. This is the fluctuation threshold, unitless, with an empirical value of 0.3. Resource allocation coupling term. The formula is expressed as follows:

[0105] ;

[0106] In the formula, For the first The data stream and the first The resource allocation coupling term for each key data flow is unitless. Iterative optimization uses the alternating direction multiplier method to solve for the game equilibrium point. The alternating direction multiplier method is achieved by alternately fixing the upper-level variables to optimize the lower-level objective or fixing the lower-level variables to optimize the upper-level objective.

[0107] The specific implementation of step S5 is: back pressure threshold. The dynamic adjustment formula is expressed as follows:

[0108] ;

[0109] In the formula, This represents the back pressure threshold, expressed in bars. This represents the maximum capacity of the message queue, expressed in messages. Current system load, in percentage; This represents the maximum system load, set to 100%; This represents the average resource allocation ratio; it has no unit. The maximum resource allocation ratio is set to 1. Sampling rate adjustment range. The formula is expressed as follows:

[0110] ;

[0111] In the formula, To adjust the sampling rate, the unit is... ; The initial sampling rate, in units of ; This is the attenuation coefficient, which has no unit and an empirical value of 2.5. This represents the current message queue depth, in units of messages.

[0112] The specific implementation of step S6 is: inter-batch data delay jitter. The calculation formula is expressed as follows:

[0113] ;

[0114] In the formula, For delay jitter, no unit; For batch indexing; For the first The actual delay for each batch, in seconds; The average delay is expressed in seconds. The reference delay is 1 second. Total number of batches. Average delay. The calculation formula is expressed as follows:

[0115] ;

[0116] In the formula, the symbols have the same meaning as before. When Greater than the delay jitter threshold The delay compensation mechanism is triggered at certain times, and the delay jitter threshold is set. The empirical value is 0.15.

[0117] The specific implementation method of step S7 is: risk assessment parameters The calculation formula is expressed as follows:

[0118] ;

[0119] In the formula, This is a risk assessment parameter, without units. The weighting coefficients are unitless and satisfy the following conditions: The empirical values ​​are 0.4, 0.3, and 0.3 respectively; This represents the maximum delay jitter, which is dimensionless and has a value of 0.5. The maximum Shannon entropy is dimensionless and has a value of 5. According to The corresponding control strategy is executed based on the range of values.

[0120] The specific implementation of wavelet transform in the denoising preprocessing involves decomposing the univariate time-series data into low-frequency approximation coefficients and high-frequency detail coefficients using wavelet decomposition, filtering out high-frequency noise components, and then reconstructing the signal. The specific implementation of the differentiated Shannon entropy threshold setting involves using the Shannon entropy threshold of the reactive materials. The Shannon entropy threshold for stable materials is calculated by taking 0.8 times the upper quartile of historical data. The upper quartile of historical data is taken as 1.2 times. The specific implementation of the upper limit for the number of iterations is to use the current optimal solution when the game equilibrium point has not converged after 50 iterations. The specific implementation of the backpressure threshold adjustment cycle is to synchronize it with the system load monitoring cycle, which is dynamically determined based on the batching operation cycle time, with an empirical value of 500 milliseconds. The specific implementation of batch lineage information storage is to use a graph database structure, where nodes represent batch states, edges represent state transition events, and edge attributes include timestamps and operation types.

[0121] To better understand and implement this invention, a specific application scenario of the invention is provided below as Example 2: To verify the effectiveness of the invention, technicians built a chemical batching operation test environment. This environment simulates the batching workshop of a fine chemical production line, where the batching system needs to mix reactive and stable materials to produce chemical products. The test environment deployed 12 material flow sensors, 8 temperature sensors, 6 pressure sensors, and 4 weighing sensors, collecting data at sampling periods of 100ms, 500ms, 200ms, and 1000ms, respectively. The test ran continuously for 72 hours, generating approximately 8.5 million material flow data records. Each record was assigned a globally unique batch identifier generated by the snowflake algorithm and a nanosecond-level timestamp.

[0122] During the anomaly detection phase space reconstruction stage, technicians performed wavelet transform denoising preprocessing on the collected univariate time-series data, using the db4 wavelet basis function to filter out high-frequency noise. For the flow data collected by the material flow sensor, the embedding dimension was determined to be 7 using the pseudo-nearest neighbor method, and the delay time parameter was determined to be 8 sampling periods using the mutual information method. The reconstructed high-dimensional phase space trajectory exhibits a stable spiral structure around the centroid of the normal attraction domain under normal operating conditions. Technicians calculated the Lyapunov exponent to be -0.032, the correlation dimension to be 2.47, and the Kolmogorov entropy to be 0.0156. During the 38th hour of testing, a temperature sensor experienced a sudden change in reading due to poor contact. The corresponding high-dimensional phase space trajectory rapidly deviated from the normal attraction domain, with the deviation jumping from 0.12 in the normal range to 0.89, exceeding the set deviation threshold of 0.65. The system immediately marked this data stream as abnormal and diverted it to the anomaly processing channel. The anomaly detection latency was only 230ms, far earlier than the detection latency of traditional sliding window statistical methods. The abnormal data was isolated and stored in the anomaly database, triggering an alarm notification. Technicians promptly replaced the faulty sensor, preventing abnormal data from contaminating downstream processing results.

[0123] During the entropy increase model calculation phase, technicians categorized unmarked material flow data according to material type, process stage, and batch correlation. Material types were divided into reactive and stable materials, and process stages included material input, mixing, and temperature control. As shown in Table 1, the Shannon entropy and conditional entropy calculation results for each category of material flow data indicate that reactive materials exhibit the highest Shannon entropy (4.82) during the mixing stage. This significantly exceeds the Shannon entropy threshold of 3.65 set for reactive materials. .

[0124] Table 1. Entropy Index Statistics of Categorized Material Flow Data

[0125] Based on Boltzmann distribution predictions, the data stream of reactive materials during the mixing and stirring stage shows an increasing probability of high latency, with the average latency predicted to increase from the normal 120ms to 280ms. The system automatically upgrades the scheduling priority of this data stream from priority 3 to priority 1 to ensure priority transmission of critical reaction control signals. Figure 2 As shown, the actual average latency of the data stream after priority was increased stabilized at 135ms, effectively curbing the latency growth trend.

[0126] In the two-layer game-theoretic scheduling model, the upper-layer model aims to minimize the overall latency of the data flow. The input includes six categories of material flow data arriving at rates of 150, 180, 165, 130, 155, and 140 data points per second, respectively. The computing power of the eight processing nodes is 2.4 GHz, 2.6 GHz, 2.8 GHz, 2.5 GHz, 2.7 GHz, 2.9 GHz, 2.6 GHz, and 2.5 GHz, respectively, with a network bandwidth of 10 Gbps. The upper-layer model calculates the initial resource allocation ratios for each processing node as 0.142, 0.156, 0.168, 0.148, 0.162, 0.174, 0.156, and 0.148, respectively. The lower-layer model aims to maximize the throughput of critical data flows, including temperature control signals, pressure regulation commands, and recipe parameter update data. The engineers used the alternating direction multiplier method for iterative optimization, setting an upper limit of 200 iterations and a load balancing threshold of 0.15. After 127 iterations, a game equilibrium was reached, the resource utilization difference among processing nodes converged to 0.13, and the throughput of critical data streams reached 2150 messages per second.

[0127] During the backpressure mechanism and circuit breaker / degradation strategy phase, technical personnel dynamically adjust the backpressure threshold of the message queue based on the resource allocation scheme of each processing node. For example... Figure 3 As shown, during normal operation, the message queue depth is maintained between 450 and 650, and the backpressure threshold is set to 800. In the 51st hour of testing, a downstream processing node experienced a decrease in processing capacity due to database transaction lock waiting, causing the message queue depth to rapidly increase to 1020, exceeding the backpressure threshold and triggering a circuit breaker and degradation strategy. The system implemented a reduced sampling rate for non-critical data streams. For stable materials, the data stream sampling rate during the material delivery phase was reduced from once every 100ms to once every 300ms. The adjustment range of the sampling rate was negatively correlated with the message queue depth, and the exponential decay function used had a decay rate of 0.0018. Critical data streams maintained their original sampling rate, and the message queue depth fell back to a safe range of 720 within 180 seconds, avoiding the risk of system crash.

[0128] In the cross-batch correlation analysis phase, technicians used a graph database structure to store batch lineage information, with nodes representing batch states and edges representing batch state transition events. As shown in Table 2, batch B2045 is associated with batches B2047 and B2049 in the mixing process, with batch correlation degrees of 0.42 and 0.58, respectively. Due to the uncertain delay of asynchronous processing, batch B2045 completes the mixing stage at timestamp 1735024856.345, while batch B2047 completes it at timestamp 1735024859.128, resulting in a 2.783-second inter-batch data delay jitter, exceeding the set delay jitter threshold of 1.5 seconds.

[0129] Table 2 Batch Association and Delay Statistics

[0130] The system triggers a delay compensation mechanism, which reconstructs data relationships by comparing the timestamps of batch data with batch lineage information and utilizing the event sequences recorded in the distributed transaction log. For example... Figure 4 As shown, the delay compensation mechanism calculated the expected arrival time of batch B2047 to be 1735024856.871. After correcting the processing sequence, the causal link between batches was restored, ensuring the accuracy of quality traceability analysis. The distributed transaction log adopts a write-ahead log mechanism, writing operation records to the log before modifying batch data to ensure data consistency during fault recovery.

[0131] During the risk assessment parameter calculation phase, technical personnel comprehensively considered message queue depth, inter-batch data latency jitter, and Shannon entropy, and performed a normalized weighted summation with weights of 0.35, 0.40, and 0.25, respectively. During normal operation, the risk assessment parameters... The system remained within the range of 0.18 to 0.28, indicating normal system operation. During the 51st hour message queue backlog event, the risk assessment parameters... The value rises to 0.62, at which point the system reverts to a two-level game scheduling model to re-execute resource allocation optimization, adjusting the risk assessment parameters accordingly. Reduced to 0.31. In the 64th hour, multiple processing nodes simultaneously experienced computationally intensive tasks, and the risk assessment parameters... The value spiked to 0.83, exceeding the emergency handling mode threshold of 0.75. The system then activated the emergency handling mode, reducing the sampling rate of non-critical data streams to the minimum sampling rate of once every 1000ms, while dynamically adding 2 processing nodes, restoring system stability within 450 seconds.

[0132] The advancements of this invention over traditional methods are mainly reflected in the following aspects. Traditional batching operation data stream processing relies on sliding window statistical methods for anomaly detection. This method assumes that the data follows a normal distribution or other probability distributions, which is poorly adaptable to nonlinear and non-stationary batching process data, and inherent window delays lead to lag in anomaly identification. This invention employs a phase space reconstruction anomaly detection module, which reconstructs univariate time-series data into high-dimensional phase space trajectories based on the Tukens embedding theorem. It quantifies the system state using nonlinear dynamic indicators such as the Lyapunov exponent, correlation dimension, and Kolmogorov entropy. It can capture the chaotic characteristics and fractal structure of the batching process without assuming data distribution, and the detection delay is controlled at the millisecond level, enabling early anomaly identification. Traditional resource allocation strategies typically use static priority or simple first-come, first-served queues, which cannot balance the dual requirements of overall process stability and the timeliness of critical control. This invention maps the data stream scheduling problem to a thermodynamic optimization problem using an entropy increase model. It quantifies the information value and urgency of data streams using Shannon entropy and conditional entropy, and achieves intelligent resource allocation through a selective scheduling mechanism similar to Maxwell's demon. Furthermore, it combines a two-layer game-theoretic scheduling model to coordinate conflicts between upper and lower layer decision-making objectives, finding the optimal balance between resource utilization and processing efficiency through iterative optimization. Traditional systems lack elastic response mechanisms when message queues are backlogged, easily leading to memory overflow or data loss and system crashes. This invention introduces a dynamic backpressure mechanism and a circuit breaker degradation strategy. The backpressure threshold is adjusted in real time based on the actual load of processing nodes. When overloaded, the sampling rate of non-critical data streams is reduced through an exponential decay function, sacrificing some data integrity for system stability while ensuring that critical data streams are unaffected, achieving multi-level buffering and elastic scaling. Traditional asynchronous processing systems struggle to solve batch data time alignment problems in cross-batch correlation analysis; latency jitter leads to causal link breaks, affecting the accuracy of quality traceability. This invention establishes a batch state transition log based on an event sourcing model, uses the write-ahead mechanism of distributed transaction logs to ensure the atomicity and durability of event sequences, and uses a delay compensation mechanism to estimate the expected arrival time of batches and reconstruct data associations, thereby achieving eventual consistency across systems and providing a reliable data foundation for causal link tracing and quality backtracking in batching operations.

[0133] It should be noted that the variables involved in this invention are explained in detail in Tables 3 and 4.

[0134] Table 3. Variable Explanation Table (Part 1)

[0135]

[0136] Table 4. Variable Explanation Table (Part Two)

[0137]

[0138] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for asynchronous processing of operation data streams in an MES system, characterized in that, This includes deploying multi-level data acquisition nodes at the batching site to synchronously collect material flow data and assign globally unique batch identifiers and timestamps. The material flow data is then input into a phase space reconstruction and anomaly detection module to reconstruct high-dimensional phase space trajectories and calculate the Lyapunov index, correlation dimension, and Kolmogorov entropy. When the deviation of the high-dimensional phase space trajectory from the normal attraction domain exceeds a deviation threshold, it is marked as abnormal data and diverted to the anomaly processing channel. For material flow data not marked as abnormal, thermodynamic entropy increase indices are calculated, and evolution trends are predicted based on Boltzmann distribution. When the Shannon entropy exceeds the Shannon entropy threshold, scheduling is escalated. Prioritize and establish a two-layer game scheduling model to optimize asynchronous processing resource allocation and solve for the game equilibrium point to output the resource allocation scheme. Dynamically adjust the back pressure threshold of the message queue according to the resource allocation scheme. When the message queue depth exceeds the back pressure threshold, activate the circuit breaker degradation strategy to reduce the sampling rate of non-critical data streams. Perform cross-batch correlation analysis on batch data that has completed asynchronous processing and trace batch lineage information based on the event tracing model. When the data latency jitter between batches exceeds the latency jitter threshold, trigger the latency compensation mechanism to correct the processing sequence. Calculate risk assessment parameters and adaptively adjust the processing mode according to the risk assessment parameters.

2. The method according to claim 1, characterized in that, The multi-level data acquisition nodes include material flow sensors, temperature sensors, pressure sensors, and weighing sensors. The material flow sensors, temperature sensors, pressure sensors, and weighing sensors synchronously collect material flow data according to a set sampling period.

3. The method according to claim 2, characterized in that, The phase space reconstruction anomaly detection module is based on the Tukens embedding theorem. It constructs a high-dimensional phase space from univariate time series data through delayed coordinate embedding. The reconstructed high-dimensional phase space is topologically equivalent to the original dynamic system phase space.

4. The method according to claim 3, characterized in that, Lyapunov exponents are used to quantify the sensitivity of high-dimensional phase space trajectories to initial conditions, correlation dimension describes the fractal dimension of attractors in high-dimensional phase space, and Kolmogorov entropy measures the information generation rate of a system.

5. The method according to claim 4, characterized in that, The normal attraction domain is the region where high-dimensional phase space trajectories in normal operating state are clustered in high-dimensional phase space. The deviation is calculated as the normalized distance from the current high-dimensional phase space trajectory point to the centroid of the normal attraction domain.

6. The method according to claim 5, characterized in that, Material flow data not marked as abnormal data are classified according to material type, process stage, and batch correlation. Shannon entropy and conditional entropy are calculated separately. Shannon entropy is used to measure the information uncertainty of classified material flow data.

7. The method according to claim 6, characterized in that, The entropy increase model treats the delay in classified material flow data as energy, predicts the probability of occurrence of different delay states through Boltzmann distribution, and infers the evolution trend of classified material flow data.

8. The method according to claim 7, characterized in that, The upper-level model of the two-layer game scheduling model aims to minimize the overall latency of the data flow, while the lower-level model aims to maximize the throughput of critical data flow processing. The upper-level and lower-level models achieve collaborative optimization through resource allocation coupling terms.

9. The method according to claim 8, characterized in that, The game equilibrium point is solved using an iterative optimization algorithm. The upper-level model adjusts the initial resource allocation ratio based on the feedback from the lower-level model, and the lower-level model optimizes the processing strategy based on the new initial resource allocation ratio. This process is repeated until the strategies of the upper-level and lower-level models no longer change.

10. The method according to claim 9, characterized in that, The backpressure threshold is dynamically adjusted based on the message queue depth. The backpressure mechanism is a strategy for backpropagating flow control signals. The backpressure signal is transmitted from the downstream processing node to the upstream producer and instructs the upstream producer to reduce the data sending rate.

Images