Microservice flow anomaly monitoring method and device based on traffic data

Abstract

The embodiment of the present application provides a method and device for monitoring microservice traffic anomalies based on traffic data. The method includes: receiving traffic data including text data and image data continuously uploaded by road cameras, storing and processing the traffic data by setting microservices, obtaining traffic log information when the microservices process the traffic data, calculating the volatility and change trend of the time series data, and constructing a model training data set using a sliding time window method based on the traffic log information; collecting the traffic log information of the microservices every minute through a parallel mechanism and inputting it into a trained traffic data flow analysis model to obtain the peak ratio, change rate and duration output by the traffic data flow analysis model; and processing different levels of warnings using log recording, alarm push and automatic flow limiting measures respectively; the present application can effectively improve the efficiency and accuracy of microservice traffic anomaly monitoring of traffic data.

Description

Technical Field

[0001] The present application relates to the field of data processing, and specifically to a method and device for monitoring microservice traffic anomalies based on traffic data. Background Art

[0002] Traffic data flow anomaly monitoring is an important part of the intelligent transportation system. With the increasing complexity of urban transportation systems and the rapid growth of data volume, traditional monitoring methods can no longer meet the needs of modern traffic management. At present, solutions based on microservice architecture are becoming a new development direction, but they also face many challenges.

[0003] In terms of data collection and processing, existing systems generally have the problem of unstable data quality. The data collected by road cameras may be affected by factors such as weather conditions and equipment failures, resulting in noisy, missing or distorted data. At the same time, delays and packet loss during data transmission will also affect the performance of the monitoring system. In addition, the data formats and quality standards from different sources are inconsistent, which increases the complexity of data processing. Summary of the invention

[0004] In response to the problems in the prior art, the present application provides a method and device for monitoring microservice flow anomalies based on traffic data, which can effectively improve the efficiency and accuracy of monitoring microservice flow anomalies based on traffic data.

[0005] In order to solve at least one of the above problems, the present application provides the following technical solutions:

[0006] In a first aspect, the present application provides a method for monitoring microservice traffic anomalies based on traffic data, comprising:

[0007] Receive traffic data including text data and image data continuously uploaded by road cameras, store and process the traffic data by setting a microservice, obtain traffic log information when the microservice processes the traffic data, perform wavelet transform noise reduction and exponential smoothing on the traffic log information, calculate the volatility and change trend of the time series data, remove abnormal fluctuation points that exceed a preset standard deviation multiple according to the volatility and the change trend, obtain processed traffic log information, and construct a model training data set according to the traffic log information using a sliding time window method;

[0008] Input the model training data set into the bidirectional long short-term memory-attention network to train the traffic data flow analysis model, and build a model parameter update buffer pool. When the amount of new data reaches a preset ratio, use the new data to train the model online, and update the attention layer weights and bidirectional long short-term memory layer parameters to obtain the traffic data flow analysis model after model training;

[0009] Traffic log information of the microservice is collected every minute through a parallel mechanism and input into a trained traffic data flow analysis model to obtain the peak ratio, change rate and duration output by the traffic data flow analysis model, wherein the peak ratio is the ratio of the current flow to the average flow in the same period of the past 24 hours, the change rate is the percentage of change in the flow in adjacent time windows, and the duration is the number of minutes the abnormal state lasts; when the peak ratio exceeds a preset threshold or the change rate exceeds a preset duration, it is determined as a first-level warning; when the peak ratio exceeds the preset threshold and the duration exceeds the preset duration, it is determined as a second-level warning; when the peak ratio exceeds the preset threshold and the duration exceeds the preset duration, it is determined as a third-level warning; different levels of warnings are processed by log recording, alarm push and automatic flow limiting measures respectively.

[0010] Furthermore, the traffic log information when the microservice processes the traffic data is obtained and the traffic log information is subjected to wavelet transform noise reduction and exponential smoothing processing, and then the volatility and change trend of the time series data are calculated, including:

[0011] Performing multi-scale wavelet decomposition on the traffic log information to obtain low-frequency coefficients and high-frequency coefficients, calculating a threshold of the high-frequency coefficients, setting the high-frequency coefficients above the threshold to zero, and performing wavelet reconstruction using the processed high-frequency coefficients and the original low-frequency coefficients to obtain the traffic log information after noise reduction;

[0012] The denoised traffic log information is divided into multiple time series according to the time window. Each time series is processed using the two-parameter exponential smoothing model. The volatility is obtained by calculating the ratio of the absolute value of the difference between adjacent data points in each time series to the mean. The first-order difference of the data points in each time series is calculated to obtain the change trend.

[0013] Furthermore, the abnormal fluctuation points exceeding a preset standard deviation multiple are eliminated according to the fluctuation rate and the change trend to obtain processed traffic log information, including:

[0014] Calculate the mean and standard deviation of the volatility data in each time window, mark the data points whose volatility values ​​exceed the product of the mean and the preset standard deviation multiple as candidate abnormal points, and select the data points whose consecutive occurrence times are less than the preset threshold as abnormal volatility points from the marked candidate abnormal points;

[0015] Calculate the mean and standard deviation of the changing trend in each time window, mark the data points whose absolute value of the changing trend exceeds the product of the mean and the preset standard deviation as fluctuation anomaly points, remove the fluctuation anomaly points from the traffic log information, and use the linear interpolation method to fill the removed data points to obtain the processed traffic log information.

[0016] Furthermore, constructing a model training data set using a sliding time window method according to the traffic log information includes:

[0017] A fixed-size time window is set based on the processed traffic log information, and the traffic data in the time window is used as the input feature, and the traffic data at a moment after the time window is used as the output label. A sample sequence is constructed by sliding the time window on the time axis, and the sample sequence is normalized by the maximum and minimum values;

[0018] Feature extraction is performed on the standardized sample sequence, and the mean feature, variance feature and kurtosis feature in each time window are calculated. The extracted features are concatenated with the original sample sequence to form an enhanced feature vector, and the enhanced feature vector is divided into a training data set and a validation data set according to a preset ratio.

[0019] Furthermore, the traffic log information of the microservice is collected every minute through the parallel mechanism and input into the trained traffic data flow analysis model to obtain the peak ratio, change rate and duration output by the traffic data flow analysis model.

[0020] Create multiple independent data collection threads. Each thread is responsible for the log collection task of a specific microservice node. The collected log information is cached in the message queue. The message consumption thread obtains the log information from the queue and converts the data format. The converted data is organized into the same feature structure as the training data set.

[0021] Based on the converted feature data, the trained traffic data flow analysis model is called to obtain the prediction results. The prediction results are compared with the actual flow data to calculate the peak ratio. The change rate is calculated based on the flow changes within the preset time interval. The cumulative time that the flow index exceeds the normal range is counted to obtain the duration.

[0022] Further, when the peak ratio exceeds a preset threshold or the change rate exceeds a preset duration, it is determined as a first-level warning; when the peak ratio exceeds a preset threshold and the duration exceeds a preset duration, it is determined as a second-level warning; when the peak ratio exceeds a preset threshold and the duration exceeds a preset duration, it is determined as a third-level warning, and further includes:

[0023] The benchmark thresholds of the peak ratio and the change rate are set based on the statistical distribution of historical data. The threshold parameters are dynamically updated according to the real-time traffic fluctuation using an adaptive adjustment algorithm. The updated threshold parameters are compared with the current peak ratio and change rate, and the level of triggering the warning is determined based on the comparison results.

[0024] Set a benchmark threshold group for duration, build a cumulative counter based on time decay to record the duration of the abnormal state, make segmented judgments on the cumulative value of the counter, determine the final warning level based on the interval in which the cumulative value is located, and write the warning level information into the warning status table.

[0025] Furthermore, the different levels of warnings are processed by log recording, alarm push and automatic flow limiting measures, including:

[0026] Construct a data structure of warning information, write the warning level, trigger time, abnormal indicators and their values ​​into the structure, push the warning information to the alarm processing module through the distributed message middleware, and the alarm processing module selects the corresponding push channel and push method according to the warning level, and stores the warning information persistently in the warning log database;

[0027] Set the current limiting rules based on the warning level, use the token bucket algorithm to control the request passing rate, dynamically adjust the token generation rate and bucket capacity parameters, return the current limiting response to the request exceeding the current limiting threshold, record the current limiting events and update the current limiting counter, and restore the normal request processing logic when the warning is lifted.

[0028] In a second aspect, the present application provides a microservice traffic anomaly monitoring device based on traffic data, comprising:

[0029] In a third aspect, the present application provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the method for monitoring microservice traffic anomalies based on traffic data are implemented.

[0030] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the microservice traffic anomaly monitoring method based on traffic data.

[0031] In a fifth aspect, the present application provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the microservice traffic anomaly monitoring method based on traffic data.

[0032] It can be seen from the above technical scheme that the present application provides a method and device for monitoring microservice traffic anomalies based on traffic data. The method receives traffic data including text data and image data continuously uploaded by road cameras, stores and processes traffic data by setting microservices, obtains traffic log information when microservices process traffic data, calculates the volatility and change trend of time series data, and constructs a model training data set using a sliding time window method based on the traffic log information. The method collects traffic log information of microservices every minute through a parallel mechanism and inputs it into a trained traffic data flow analysis model to obtain the peak ratio, change rate and duration output by the traffic data flow analysis model. Different levels of warnings are processed using log recording, alarm push and automatic flow limiting measures, thereby effectively improving the efficiency and accuracy of microservice traffic anomaly monitoring of traffic data. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0034] Figure 1 This is one of the flow charts of the microservice traffic anomaly monitoring method based on traffic data in an embodiment of the present application;

[0035] Figure 2 This is a second flow chart of a method for monitoring abnormal microservice traffic based on traffic data in an embodiment of the present application;

[0036] Figure 3 This is a flow chart of the third method for monitoring abnormal microservice traffic based on traffic data in an embodiment of the present application;

[0037] Figure 4 This is a fourth flow chart of a method for monitoring abnormal microservice traffic based on traffic data in an embodiment of the present application;

[0038] Figure 5 This is a fifth flow chart of a method for monitoring abnormal microservice traffic based on traffic data in an embodiment of the present application;

[0039] Figure 6 This is a sixth flow chart of a method for monitoring abnormal microservice traffic based on traffic data in an embodiment of the present application;

[0040] Figure 7 This is a flow chart of the seventh embodiment of the microservice traffic anomaly monitoring method based on traffic data in the embodiment of the present application;

[0041] Figure 8 This is a structural diagram of a microservice traffic anomaly monitoring device based on traffic data in an embodiment of the present application;

[0042] Fig. 9 It is a schematic diagram of the structure of an electronic device in an embodiment of the present application.

[0043] Reference numerals:

[0044] Electronic device 9600, central processing unit 9100, memory 9140, communication module 9110, input unit 9120, audio processor 9130, display 9160, power supply 9170, buffer memory 9141, application / function storage unit 9142, data storage unit 9143, driver program storage unit 9144, antenna 9111, speaker 9131, microphone 9132. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0046] The acquisition, storage, use, and processing of data in the technical solution of this application comply with the relevant provisions of national laws and regulations.

[0047] In view of the problems existing in the prior art, the present application provides a method and device for monitoring microservice traffic anomalies based on traffic data. The method receives traffic data including text data and image data continuously uploaded by road cameras, stores and processes the traffic data by setting microservices, obtains traffic log information when the microservices process the traffic data, calculates the volatility and change trend of the time series data, and constructs a model training data set using a sliding time window method based on the traffic log information. The method collects the traffic log information of the microservices every minute through a parallel mechanism and inputs it into a trained traffic data flow analysis model to obtain the peak ratio, change rate and duration output by the traffic data flow analysis model. Different levels of warnings are processed by log recording, alarm push and automatic flow limiting measures, thereby effectively improving the efficiency and accuracy of microservice traffic anomaly monitoring of traffic data.

[0048] In order to effectively improve the efficiency and accuracy of microservice traffic anomaly monitoring based on traffic data, the present application provides an embodiment of a microservice traffic anomaly monitoring method based on traffic data. Figure 1The microservice traffic anomaly monitoring method based on traffic data specifically includes the following contents:

[0049] Step S101: receiving traffic data including text data and image data continuously uploaded by road cameras, storing and processing the traffic data by setting a microservice, obtaining traffic log information when the microservice processes the traffic data, performing wavelet transform noise reduction and exponential smoothing on the traffic log information, calculating the volatility and change trend of the time series data, eliminating abnormal fluctuation points that exceed a preset standard deviation multiple according to the volatility and the change trend, obtaining processed traffic log information, and constructing a model training data set according to the traffic log information using a sliding time window method;

[0050] Optionally, in this embodiment, this step constructs a complete data preprocessing process, covering multiple links such as data collection, storage, noise reduction, smoothing and outlier processing, aiming to provide a high-quality training data set for subsequent traffic analysis models.

[0051] In the data collection phase, the system receives real-time data streams from multiple road cameras through a distributed architecture. These data include text data such as traffic flow and speed, as well as vehicle image data. The application of microservice architecture ensures high availability and scalability of the system. Each microservice node is responsible for processing data in a specific area or type, and optimizes resource allocation through a load balancing mechanism.

[0052] When storing the collected raw data, the system adopts a distributed storage architecture to store the data in pieces on different nodes to ensure efficient data reading and writing and disaster recovery backup. Detailed operation logs will be recorded during the storage process, including data processing time, processing results and other information. These log information is an important basis for subsequent analysis.

[0053] In the signal processing stage, the system first uses wavelet transform to reduce noise. Wavelet transform can analyze in both time and frequency domains at the same time, by decomposing the signal into wavelet coefficients of different scales, then thresholding the high-frequency noise, and finally reconstructing the denoised signal. This method has better local characteristics than the traditional Fourier transform and can better maintain the mutation characteristics of the signal.

[0054] Next, the system performs exponential smoothing on the denoised data. Exponential smoothing gives greater weight to recent data, while the weight of long-term data decays exponentially over time. This processing method not only retains the trend of data changes, but also reduces the impact of random fluctuations. By setting smoothing parameters reasonably, a good balance can be achieved between maintaining signal response speed and suppressing noise.

[0055] The system calculates volatility and change trends based on the processed data. Volatility reflects the relative magnitude of data change, while the change trend reflects the direction of data development. By setting a standard deviation multiple threshold based on statistical principles, the system can identify and eliminate abnormal fluctuation points, which may come from equipment failures or emergencies.

[0056] Finally, the system uses the sliding time window method to construct the training data set. By setting the appropriate window size and sliding step size, the continuous time series data is converted into a sample sequence with a fixed dimension. This method not only maintains the time series characteristics of the data, but also facilitates the training of subsequent deep learning models.

[0057] This step solves technical problems such as large noise, many outliers, and complex time series characteristics in the original traffic data. Through the systematic data preprocessing process, the data quality is significantly improved, providing a reliable data basis for subsequent traffic anomaly monitoring. At the same time, the automation and parallelization of the preprocessing process significantly improves the processing efficiency, enabling the system to meet the needs of real-time data processing.

[0058] The processed data set has the characteristics of low noise, good continuity, and few outliers. It can truly reflect the changing laws of traffic flow and provide high-quality input data for subsequent model training, thereby improving the accuracy and reliability of the entire anomaly monitoring system. In addition, the preprocessing method has strong versatility and can adapt to the traffic data processing needs in different scenarios.

[0059] Step S102: input the model training data set into the bidirectional long short-term memory-attention network to train the traffic data flow analysis model, and construct a model parameter update buffer pool. When the amount of new data reaches a preset ratio, the model is trained online using the new data, and the attention layer weights and bidirectional long short-term memory layer parameters are updated to obtain the traffic data flow analysis model after model training;

[0060] Optionally, in this embodiment, this step focuses on describing the training process of the traffic data flow analysis model based on deep learning, adopts a deep learning architecture of a bidirectional long short-term memory network (Bi-LSTM) combined with an attention mechanism, and implements an online update mechanism of the model.

[0061] The core advantage of the bidirectional long short-term memory network is that it can simultaneously consider the forward and backward dependencies of sequence data. When processing time series data such as traffic flow, the forward LSTM layer can capture the impact of historical information on the current moment, while the backward LSTM layer can obtain feedback from future moments on the current prediction. The fusion of information in the two directions provides a more complete representation of time series features.

[0062] The introduction of the attention mechanism enables the model to adaptively assign weights to features at different time steps. For traffic flow data, the importance of different time periods is often different. For example, data during peak hours may be more valuable for prediction. The attention layer calculates the similarity between the feature vector and the query vector and generates attention weights, allowing the model to pay more attention to information at key time points.

[0063] In order to enable the model to adapt to the dynamic changes in traffic flow, the system builds a model parameter update buffer pool. This buffer pool stores newly collected training samples. When the number of accumulated new samples reaches a preset ratio, the model's online training process is triggered. Online training uses a small batch gradient descent method to update model parameters using new data, including the weight matrix of the attention layer and the gate unit parameters of the Bi-LSTM layer.

[0064] During online training, the system adopts a progressive learning strategy, that is, mixed training of new and old data. This approach not only maintains the model's memory of historical patterns, but also learns new traffic characteristics. In order to prevent the model from over-adapting to new data and forgetting the learned patterns, the system introduces an elastic weight decay mechanism during training to appropriately constrain the update range of model parameters.

[0065] This step solves the technical problem that traditional static models cannot adapt to the dynamic changes of traffic flow. Through the bidirectional information processing capability of the Bi-LSTM network, the feature importance weighting of the attention mechanism, and the dynamic update mechanism of online training, the system can accurately capture the time-varying characteristics and long-term dependencies of traffic flow.

[0066] The trained model shows excellent prediction performance and adaptability. In terms of prediction accuracy, the combination of the bidirectional structure and the attention mechanism enables the model to better understand the temporal pattern of traffic flow. In terms of real-time response capability, the online training mechanism enables the model to adapt to new traffic patterns in a timely manner. In terms of computational efficiency, the design of the parameter update buffer pool avoids frequent model updates, reducing computational overhead while ensuring model timeliness.

[0067] In addition, the model has good generalization ability and robustness, and can process traffic flow data of different time scales and different characteristic distributions, providing a reliable analysis basis for subsequent abnormal monitoring. The adaptive characteristics of the model also enable it to play a role in traffic monitoring in different cities and different road sections.

[0068] Step S103: Traffic log information of the microservice is collected every minute through a parallel mechanism and input into a trained traffic data flow analysis model to obtain the peak ratio, change rate and duration output by the traffic data flow analysis model, wherein the peak ratio is the ratio of the current flow to the average flow in the same period of the past 24 hours, the change rate is the percentage of change in the flow in adjacent time windows, and the duration is the number of minutes the abnormal state lasts; when the peak ratio exceeds a preset threshold or the change rate exceeds a preset duration, it is determined as a first-level warning; when the peak ratio exceeds a preset threshold and the duration exceeds a preset duration, it is determined as a second-level warning; when the peak ratio exceeds a preset threshold and the duration exceeds a preset duration, it is determined as a third-level warning; different levels of warnings are processed by log recording, alarm push and automatic flow limiting measures respectively.

[0069] Optionally, in this embodiment, this step describes the specific implementation process of a real-time traffic monitoring and multi-level warning processing system, which realizes accurate identification and graded processing of abnormal traffic conditions through parallel computing and multi-dimensional indicator analysis.

[0070] The system uses a parallel mechanism to collect traffic log information from microservices every minute. The parallel processing mechanism processes information from multiple data sources simultaneously through multi-threading or distributed computing, significantly improving data collection and processing efficiency. The collected data is directly input into the trained traffic data flow analysis model, achieving seamless connection from data collection to analysis.

[0071] The system has designed three core indicators to quantify traffic anomalies: peak ratio, change rate, and duration. The peak ratio can effectively identify abnormal traffic peaks by comparing the current traffic with the average traffic in the same period of the past 24 hours. This comparison method takes into account the periodic characteristics of traffic flow and can accurately reflect whether the current traffic deviates significantly from the normal level. The change rate focuses on the changes in traffic in adjacent time windows and can capture sudden traffic changes in a timely manner. The statistics of duration help the system determine the severity of the abnormal state.

[0072] The warning mechanism adopts a three-level classification standard and makes judgments based on the combination of different indicators. The first-level warning is mainly for short-term traffic anomalies, which is triggered when the peak ratio or change rate exceeds the preset threshold. The second-level warning focuses on persistent traffic anomalies, requiring the peak ratio and duration to exceed the threshold at the same time. The third-level warning is the highest level of warning, indicating that the system has detected a serious and persistent abnormality.

[0073] The system has designed corresponding treatment measures for different levels of warnings. For the first-level warning, the system will log and save detailed information of the abnormal situation for subsequent analysis. The second-level warning will trigger the alarm push mechanism and send warning information to relevant personnel through the preset communication channel. The third-level warning will start the automatic flow restriction measures to ease traffic pressure by adjusting the timing of traffic lights and limiting traffic flow.

[0074] This step solves the technical problems of the traditional traffic monitoring system, such as delayed response, inaccurate warning, and single treatment measures. Through real-time parallel monitoring and multi-dimensional indicator analysis, the system can quickly and accurately identify abnormal conditions. The multi-level warning mechanism provides a more refined abnormal classification, which helps to take more targeted treatment measures.

[0075] The implementation effect is remarkable. The system has achieved a monitoring response speed of milliseconds and the accuracy of early warning has been significantly improved. Multi-level early warning and corresponding treatment measures have formed a complete closed-loop management mechanism, which can effectively prevent and alleviate traffic congestion. The parallel processing mechanism ensures the stability and reliability of the system when processing large-scale data.

[0076] From the above description, it can be seen that the microservice traffic anomaly monitoring method based on traffic data provided in the embodiment of the present application can receive traffic data including text data and image data continuously uploaded by road cameras, store and process traffic data by setting microservices, obtain traffic log information when the microservice processes traffic data, calculate the volatility and change trend of time series data, and construct a model training data set using a sliding time window method based on the traffic log information; collect traffic log information of the microservice every minute through a parallel mechanism and input it into the trained traffic data flow analysis model to obtain the peak ratio, change rate and duration output by the traffic data flow analysis model; and use log recording, alarm push and automatic flow limiting measures to process different levels of warnings, thereby effectively improving the efficiency and accuracy of microservice traffic anomaly monitoring of traffic data.

[0077] In one embodiment of the microservice traffic anomaly monitoring method based on traffic data of the present application, see Figure 2 , and can also include the following:

[0078] Step S201: performing multi-scale wavelet decomposition on the traffic log information to obtain low-frequency coefficients and high-frequency coefficients, calculating a threshold of the high-frequency coefficients, setting the high-frequency coefficients above the threshold to zero, and performing wavelet reconstruction using the processed high-frequency coefficients and the original low-frequency coefficients to obtain the traffic log information after noise reduction;

[0079] Step S202: Divide the traffic log information after noise reduction into multiple time series according to the time window, use the two-parameter exponential smoothing model to process each time series, calculate the ratio of the absolute value of the difference between adjacent data points in each time series to the mean to obtain the volatility, and calculate the first-order difference of the data points in each time series to obtain the change trend.

[0080] Optionally, in this embodiment, these two steps describe in detail the signal processing process of the traffic log information, achieve noise reduction through wavelet transform, and extract data features through exponential smoothing and statistical analysis, forming a complete signal preprocessing system.

[0081] In step S201, the system first performs multi-scale wavelet decomposition on the traffic log information. The core idea of ​​wavelet transform is to decompose the signal into different frequency components. By scaling and translating the mother wavelet function, the signal characteristics can be analyzed at different scales. Multi-scale decomposition divides the signal into low-frequency coefficients (approximate coefficients) and high-frequency coefficients (detail coefficients). The low-frequency coefficients reflect the overall trend of the signal, while the high-frequency coefficients contain local details and noise information.

[0082] The system uses the threshold method to process high-frequency coefficients, which is a noise reduction method based on statistical principles. By setting a reasonable threshold, valid signals and noise can be distinguished. Setting high-frequency coefficients above the threshold to zero actually removes noise components while retaining the main features of the signal. The choice of threshold is usually based on the variance of the signal and the number of sampling points, and a balance needs to be struck between noise removal and signal fidelity.

[0083] After threshold processing, the system uses the processed high-frequency coefficients and the original low-frequency coefficients for wavelet reconstruction. The reconstruction process is the inverse process of decomposition. The coefficients of different scales are recombined through inverse wavelet transform to obtain the denoised signal. This reconstruction method ensures the integrity and continuity of the signal.

[0084] In step S202, the system first segments the denoised data according to time windows. The selection of the size of the time window needs to consider the time characteristics of the data and the analysis requirements, ensuring that the amount of data in the window is sufficient for statistical analysis and that the timeliness of the analysis results is guaranteed.

[0085] For each time series, the system uses a two-parameter exponential smoothing model for processing. The model contains two parameters, the level term and the trend term, which can capture the basic level and change trend of the data at the same time. By giving a larger weight to recent data and a smaller weight to long-term data, the gradual forgetting of historical data is achieved, so that the processed data better reflects the current state.

[0086] The volatility is calculated by the ratio of the absolute value of the difference between adjacent data points to the mean. This method can effectively quantify the relative volatility of the data. The trend of change is obtained by calculating the first-order difference, which can intuitively reflect the upward or downward trend of the data.

[0087] These two steps solve the technical problems of large noise interference, strong random fluctuations, and unclear trends in the original traffic data. Through wavelet noise reduction and exponential smoothing processing, the signal-to-noise ratio of the data is significantly improved, making the changing characteristics of the data clearer.

[0088] The processed data has the characteristics of low noise, obvious trends and prominent features, providing high-quality input for subsequent anomaly detection. The processing effect of the system is not only reflected in the improvement of data quality, but also in the optimization of computing efficiency. The combination of multi-scale decomposition and parallel processing enables the system to quickly process large amounts of real-time data.

[0089] In addition, this processing method has strong adaptability and can adjust the scale of wavelet decomposition, threshold selection strategy and smoothing parameter value according to actual needs, so as to adapt to the data processing needs in different scenarios. The entire processing flow forms a complete signal processing system, which lays the foundation for subsequent feature extraction and pattern recognition.

[0090] In one embodiment of the microservice traffic anomaly monitoring method based on traffic data of the present application, see Figure 3 , and can also include the following:

[0091] Step S301: Calculate the mean and standard deviation of the volatility data in each time window, mark the data points whose volatility values ​​exceed the product of the mean and the preset standard deviation multiple as candidate abnormal points, and select the data points whose consecutive occurrence times are less than the preset threshold from the marked candidate abnormal points as abnormal volatility points;

[0092] Step S302: Calculate the mean and standard deviation of the change trend in each time window, mark the data points whose absolute value of the change trend exceeds the product of the mean and the preset standard deviation multiple as fluctuation anomalies, remove the fluctuation anomalies from the traffic log information, and use the linear interpolation method to fill the removed data points to obtain the processed traffic log information.

[0093] Optionally, in this embodiment, these two steps describe in detail the outlier detection and processing flow based on statistical methods, and achieve accurate identification and correction of abnormal data through analysis of volatility and change trends.

[0094] Step S301 first performs statistical analysis on the volatility data in each time window. The system calculates the mean and standard deviation of the volatility, which respectively reflect the central tendency and dispersion of the data. The method of adding or subtracting the standard deviation multiple is used to set the abnormality judgment threshold. This method is based on the assumption of normal distribution of data and can effectively identify data points that significantly deviate from the normal range.

[0095] In the process of screening candidate abnormal points, the system introduces a continuity judgment mechanism. By setting a threshold for the number of consecutive occurrences, it is possible to distinguish between real abnormal fluctuations and random fluctuations. The principle of this method is based on the fact that abnormal events usually have a certain degree of continuity, while random fluctuations are often short-lived. Through this dual screening mechanism, the system can more accurately identify real abnormal fluctuation points.

[0096] Step S302 focuses on the changing trend characteristics of the data. The system calculates the mean and standard deviation of the changing trend and uses similar statistical methods to identify trend anomalies. Unlike volatility analysis, here the focus is on the rate and direction of data change. By calculating the absolute value of the trend and comparing it with the statistical threshold, the system can capture drastic trend changes.

[0097] For the identified outliers, the system uses a removal and filling strategy to process them. The removal operation directly removes the outliers, while the linear interpolation reconstructs the removed data using the information of the adjacent normal data points. The linear interpolation method assumes that there is a linear relationship between adjacent data points, which can maintain the continuity and smoothness of the data.

[0098] These two steps mainly solve technical problems such as the difficulty in accurately identifying abnormal fluctuations in traffic data and data distortion caused by improper abnormal data processing. The accuracy of anomaly detection is improved by combining statistical analysis and continuity judgment. The use of linear interpolation ensures the continuity and availability of data after processing.

[0099] The processed traffic log information shows better data quality and reliability. The accuracy of anomaly detection is significantly improved, and the false alarm rate and missed alarm rate are effectively controlled. The continuity and smoothness of the data are guaranteed, providing a more reliable basis for subsequent analysis and prediction.

[0100] The processing effect of the system is also reflected in the computational efficiency. The anomaly detection algorithm based on statistical methods has a small amount of calculation and can process a large amount of data in real time. The linear interpolation method also has the characteristics of simple calculation and easy implementation, which ensures the real-time performance of the system.

[0101] In addition, the processing method has strong adaptability and scalability. By adjusting parameters such as standard deviation multiples and continuity thresholds, the system can adapt to the anomaly detection needs of different scenarios. The modular design of the processing flow also makes it easy to integrate other anomaly detection algorithms or data filling methods into the system.

[0102] The entire processing flow forms a complete anomaly detection and processing system, which not only improves data quality but also provides reliable data support for subsequent traffic flow analysis.

[0103] In one embodiment of the microservice traffic anomaly monitoring method based on traffic data of the present application, see Figure 4 , and can also include the following:

[0104] Step S401: a fixed-size time window is set based on the processed traffic log information, the traffic data in the time window is used as the input feature, the traffic data at a moment after the corresponding time window is used as the output label, a sample sequence is constructed by sliding the time window on the time axis, and the sample sequence is normalized by the maximum and minimum values;

[0105] Step S402: extract features from the standardized sample sequence, calculate the mean feature, variance feature and kurtosis feature in each time window, concatenate the extracted features with the original sample sequence to form an enhanced feature vector, and divide the enhanced feature vector into a training data set and a validation data set according to a preset ratio.

[0106] Optionally, in this embodiment, these two steps describe the feature engineering and dataset construction process of traffic flow data, and prepare high-quality datasets for subsequent model training through sliding windows and feature enhancement techniques.

[0107] Step S401 first introduces a sliding window mechanism to construct a sample sequence. A fixed-size time window can capture the temporal dependency of data. The traffic data within the window is used as the input feature, and the traffic data at the moment after the window is used as the output label. This design reflects the essence of time series prediction. As the sliding window moves on the time axis, new samples are continuously generated to form a complete sample sequence.

[0108] Maximum and minimum value normalization is a key step in data preprocessing. By mapping the data into a specified range, the dimensional differences between different features are eliminated, making the data distribution more uniform. Normalization also helps improve the stability and convergence speed of model training, and is an important pre-step for deep learning model training.

[0109] Step S402 focuses on feature extraction and data set construction. The system calculates three important statistical features: the mean feature reflects the overall level of the data, the variance feature describes the degree of fluctuation of the data, and the kurtosis feature represents the degree of sharpness of the data distribution. These statistical features can describe the characteristics of traffic data from different angles and provide rich data representation.

[0110] The extracted features are concatenated with the original sample sequence to form an enhanced feature vector. This feature enhancement method not only retains the time series information of the original data, but also introduces a global perspective of statistical features, making the data expression more comprehensive and rich. The construction of the feature vector takes into account multiple dimensions of the data, which helps to improve the predictive ability of the model.

[0111] The data set is divided into a training set and a validation set according to a preset ratio. This division method ensures the independence of model training and validation, which helps to evaluate the generalization ability of the model. A reasonable division ratio can balance the adequacy of training data and the reliability of validation.

[0112] These two steps mainly solve technical problems such as insufficient features of original traffic data, uneven data distribution, and unreasonable construction of training samples. Through feature engineering and data enhancement, the expressiveness and quality of data are significantly improved.

[0113] The processed data set is characterized by rich features, uniform distribution and reasonable structure. The sliding window mechanism ensures the continuity and integrity of the samples, the introduction of statistical features enhances the expressiveness of the data, and the standardization process improves the distribution characteristics of the data.

[0114] The processing effect of the system is also reflected in the practicality of the data set. The constructed data set contains both the necessary time series information and the global perspective of statistical features, providing high-quality training materials for subsequent model training. The division of the data set also ensures the reliability of model evaluation.

[0115] In addition, the processing method has strong versatility and scalability. By adjusting parameters such as window size, sliding step size, feature type, etc., the system can adapt to the modeling requirements of different scenarios. The modular design of the processing flow also makes it easy to integrate new feature extraction methods or data enhancement techniques into the system.

[0116] In one embodiment of the microservice traffic anomaly monitoring method based on traffic data of the present application, see Figure 5 , and can also include the following:

[0117] Step S501: Create multiple independent data collection threads, each thread is responsible for the log collection task of a specific microservice node, the collected log information is cached through the message queue, the message consumption thread obtains the log information from the queue and converts the data format, and organizes the converted data into the same feature structure as the training data set;

[0118] Step S502: Based on the converted feature data, the trained traffic data flow analysis model is called to obtain the prediction result, the prediction result is compared with the actual flow data to calculate the peak ratio, the change rate is calculated based on the flow change within the preset time interval, and the cumulative time that the flow index exceeds the normal range is calculated to obtain the duration.

[0119] Optionally, in this embodiment, these two steps describe in detail the real-time data collection and analysis process in a distributed system, and realize real-time monitoring and anomaly detection of traffic flow through multi-threaded parallel processing and model prediction.

[0120] Step S501 adopts a multi-threaded parallel collection architecture, and each collection thread runs independently and is responsible for the log collection of a microservice node. This design is based on the producer-consumer model. The collection thread acts as a producer and writes the collected log information into the message queue. The message queue is implemented using a high-performance memory queue, which supports asynchronous writing and reading and can effectively buffer fluctuations in data traffic. The design of the queue needs to take into account memory limitations, and usually a circular queue or a blocking queue with capacity limitations is used to ensure the stability of the system under high load conditions.

[0121] The message consumption thread adopts a batch processing strategy when reading data from the queue, which can reduce thread switching overhead. The data format conversion process includes field parsing, type conversion, timestamp standardization and other steps. The converted data needs to maintain the same feature structure as the training data set, including the consistency of the time window size, feature dimension and data standardization method. The alignment of feature structure usually involves operations such as data resampling, feature calculation and standardization.

[0122] Step S502 first calls the pre-trained traffic analysis model for prediction. The model calling process includes feature data preprocessing, model reasoning, and result post-processing. The preprocessing stage needs to ensure that the input data meets the requirements of the model, including data format, dimension, and normalization. The model reasoning process needs to consider computational efficiency, and batch processing can be used to improve throughput.

[0123] The comparison and analysis between the prediction results and the actual traffic data uses multiple indicators. The calculation of the peak ratio takes into account the maximum ratio of the predicted value to the actual value. This indicator reflects the amplitude of traffic fluctuations. The calculation of the rate of change is based on the flow difference within the preset time interval, and the selection of the time window and smoothing processing need to be considered. The duration statistics record the cumulative time of the abnormal state, which requires setting a reasonable threshold to determine the normal range.

[0124] These two steps mainly solve technical problems such as asynchronous data collection, large processing delay, and insufficient prediction accuracy in distributed systems. Through multi-threaded parallel processing and message queue buffering, the system's data processing capability and real-time performance are significantly improved.

[0125] The technical effects achieved by the system are reflected in many aspects: first, the multi-threaded collection architecture ensures the real-time and integrity of data collection, and the use of message queues effectively solves the problem of data traffic fluctuations; second, data format conversion and feature alignment ensure the input quality of the prediction model; finally, multi-dimensional indicator analysis provides comprehensive traffic monitoring capabilities.

[0126] The performance of the processing flow includes: the delay of data collection is controlled at the millisecond level, the queue buffer capacity can cope with short-term traffic surges, and the response time predicted by the model meets the needs of real-time monitoring. The scalability of the system is also guaranteed, supporting the dynamic addition of collection nodes and adjustment of processing parameters.

[0127] In addition, the system also has fault recovery capabilities. Abnormalities in the collection thread can be detected and restarted in a timely manner, the persistence mechanism of the message queue ensures that data will not be lost, and the deployment of the prediction model supports hot updates. These features all improve the reliability of the system.

[0128] In one embodiment of the microservice traffic anomaly monitoring method based on traffic data of the present application, see Figure 6 , and can also include the following:

[0129] Step S601: Based on the statistical distribution of historical data, a reference threshold value of the peak ratio and a reference threshold value of the change rate are set, and the threshold parameters are dynamically updated according to the real-time traffic fluctuation using an adaptive adjustment algorithm. The updated threshold parameters are compared with the current peak ratio and change rate, and the level of triggering the warning is determined according to the comparison results;

[0130] Step S602: Set a benchmark threshold group for duration, construct a cumulative counter based on time decay to record the duration of the abnormal state, perform segmented judgment on the cumulative value of the counter, determine the final warning level according to the interval of the cumulative value, and write the warning level information into the warning status table.

[0131] Optionally, in this embodiment, these two steps describe in detail the dynamic threshold setting and warning level determination process of the traffic flow warning mechanism, and realize the intelligence and precision of the warning system through the adaptive algorithm and time decay mechanism.

[0132] Step S601 first determines the initial threshold based on the statistical distribution of historical data. This process requires analyzing a large amount of historical data and calculating the distribution characteristics of the peak ratio and the rate of change, including statistics such as the mean, standard deviation, and quantile. The initial threshold is usually set using the quantile method. For example, the 95% quantile can be selected as the benchmark threshold, which can exclude the influence of extreme outliers.

[0133] The adaptive adjustment algorithm is the core of this step. The algorithm first sets a sliding time window and continuously monitors the traffic indicators within the window. By calculating the statistical characteristics of the indicators in the window, such as mean, variance, kurtosis, etc., the current traffic fluctuation degree is evaluated. The algorithm dynamically adjusts the threshold according to the degree of fluctuation. When the fluctuation is large, the threshold is appropriately relaxed, and when the fluctuation is small, the threshold is tightened. The adjustment process uses a smoothing factor to control the speed of change and avoid drastic fluctuations in the threshold.

[0134] The determination of the warning level adopts a multi-level threshold mechanism. The system sets multiple threshold intervals corresponding to different warning levels. When the indicator value falls into a certain interval, the corresponding level of warning is triggered. The judgment process needs to consider the combined effect of multiple indicators, and the weighted sum or decision tree method can be used to comprehensively evaluate the warning level.

[0135] Step S602 introduces a cumulative counting mechanism based on time decay. The benchmark threshold group contains multiple time thresholds corresponding to different warning levels. The cumulative counter is designed using a time decay function, commonly used are exponential decay or linear decay functions. The update rules of the counter include: the count increases in abnormal states, the count decays in normal states, and the count value has upper and lower limit constraints.

[0136] The segmented judgment process maps the cumulative value of the counter to the warning level. The system sets multiple cumulative value intervals, each of which corresponds to a warning level. The change trend of the count value needs to be considered when judging, and the change rate can be introduced as an auxiliary judgment basis. The final warning level determination needs to comprehensively consider multiple factors, including the current cumulative value, change trend and historical warning conditions.

[0137] These two steps mainly solve technical problems such as fixed warning threshold, insufficient warning sensitivity, and high false alarm rate. The accuracy and practicality of the early warning system are significantly improved through adaptive threshold and time decay mechanism.

[0138] The technical effects achieved by the system include: the warning threshold can be dynamically adjusted according to the actual traffic conditions and has strong adaptability; the warning level determination is more accurate, and the false alarm rate and missed alarm rate are effectively controlled; the time decay mechanism makes the system more sensitive to persistent anomalies.

[0139] The performance of the processing flow includes: good real-time adjustment of thresholds, which can quickly respond to traffic changes; low counter update overhead, which supports high-frequency status updates; and efficient data structure for storage of warning status, which supports fast query and update.

[0140] In addition, the system is configurable and maintainable. Threshold parameters, attenuation functions, judgment rules, etc. can be flexibly adjusted according to actual needs. The system supports dynamic updates of warning rules, which makes it easier for operation and maintenance personnel to optimize warning strategies according to actual conditions.

[0141] In one embodiment of the microservice traffic anomaly monitoring method based on traffic data of the present application, see Figure 7 , and can also include the following:

[0142] Step S701: construct a data structure of warning information, write the warning level, trigger time, abnormal indicators and their values ​​into the structure, push the warning information to the alarm processing module through the distributed message middleware, the alarm processing module selects the corresponding push channel and push method according to the warning level, and stores the warning information persistently in the warning log database;

[0143] Step S702: Set current limiting rules based on the warning level, use the token bucket algorithm to control the request passing rate, dynamically adjust the token generation rate and bucket capacity parameters, return a current limiting response to requests that exceed the current limiting threshold, record the current limiting event and update the current limiting counter, and restore the normal request processing logic when the warning is lifted.

[0144] Optionally, in this embodiment, these two steps describe in detail the processing of warning information and the flow control mechanism, and achieve reliable alarm and flow control of the system through distributed message push and adaptive flow control algorithm.

[0145] Step S701 first designs the data structure of the warning information. The structure contains multiple key fields: the warning level is represented by an integer, corresponding to different levels of urgency; the trigger time is in timestamp format, accurate to milliseconds; the abnormal indicator field stores the specific indicator name that triggers the warning; the value field records the specific value of the abnormal indicator. The design of the structure needs to consider the balance between serialization efficiency and storage space.

[0146] Distributed message middleware adopts the publish-subscribe model, supports reliable message delivery and retry upon failure. The selection of middleware needs to consider performance, reliability and scalability. Messages are sent asynchronously to avoid blocking the main process. To ensure message delivery, the system implements a message confirmation mechanism and a retry strategy upon failure.

[0147] The alarm processing module implements a multi-channel push strategy. Different warning levels correspond to different push priorities and channel combinations. High-level warnings may trigger emails, SMS, and system notifications at the same time, while low-level warnings may only send system notifications. The push process includes steps such as message formatting, channel selection, and sending confirmation.

[0148] Step S702 implements an adaptive current limiting mechanism based on the token bucket algorithm. The core parameters of the token bucket algorithm include the token generation rate and bucket capacity. The system sets different parameter values ​​according to the warning level. The higher the warning level, the stricter the current limiting. Token generation uses a fixed time interval or a smooth generation method to avoid traffic fluctuations caused by sudden increases in tokens.

[0149] The implementation of the token bucket takes into account the issue of concurrent access and uses atomic operations to ensure thread safety. The status information of the bucket includes the current number of tokens, the last update time, etc. When a request arrives, the system first calculates the current number of available tokens and then decides whether to release the request. For requests that are restricted, the system returns a specific response code and prompt information.

[0150] The current limiting event is recorded asynchronously, including fields such as request information, current limiting reason, and timestamp. The current limiting counter uses a sliding window mechanism to count current limiting situations at different time scales. When the warning is lifted, the system gradually restores normal processing capacity through a smooth transition to avoid sudden increases in traffic.

[0151] These two steps mainly solve technical problems such as unreliable early warning information transmission, fixed current limiting strategy, and uneven system recovery. Through reliable message push and adaptive current limiting mechanism, the reliability and stability of the system are significantly improved.

[0152] The technical effects achieved by the system include: the transmission of warning information is more reliable and supports multi-channel push; the flow limiting mechanism is more flexible and can be dynamically adjusted according to the warning level; the system recovery process is smoother, avoiding the problem of sudden traffic surges.

[0153] The performance of the processing flow includes: the delay of message push is controlled at the millisecond level; the response time of current limiting judgment is at the microsecond level; the system supports the processing of high-concurrency requests, and the accuracy of current limiting counts is guaranteed.

[0154] In addition, the system has good maintainability and scalability. Warning push channels can be easily added and configured; current limiting parameters support dynamic adjustment; monitoring indicators are complete, which facilitates system operation and optimization.

[0155] The entire processing flow forms a complete early warning processing and flow control system, which organically combines message push, flow control and monitoring records, providing reliable guarantee for the stable operation of the system. The design of the system fully considers the actual operation and maintenance needs, and ensures the reliability and maintainability of the system while ensuring the functional integrity.

[0156] In order to effectively improve the efficiency and accuracy of microservice traffic anomaly monitoring based on traffic data, the present application provides an embodiment of a microservice traffic anomaly monitoring device based on traffic data for implementing all or part of the content of the microservice traffic anomaly monitoring method based on traffic data, see Figure 8 The microservice traffic anomaly monitoring device based on traffic data specifically includes the following contents:

[0157] The traffic data processing module 10 is used to receive the traffic data including text data and image data continuously uploaded by the road camera, store and process the traffic data by setting a microservice, obtain the traffic log information when the microservice processes the traffic data, perform wavelet transform noise reduction and exponential smoothing on the traffic log information, calculate the volatility and change trend of the time series data, remove abnormal fluctuation points that exceed the preset standard deviation multiple according to the volatility and the change trend, obtain the processed traffic log information, and construct the model training data set according to the traffic log information using the sliding time window method;

[0158] The traffic model construction module 20 is used to input the model training data set into the bidirectional long short-term memory-attention network to train the traffic data flow analysis model, and to construct a model parameter update buffer pool. When the amount of new data reaches a preset ratio, the model is trained online using the new data, and the attention layer weights and the bidirectional long short-term memory layer parameters are updated to obtain the traffic data flow analysis model after model training;

[0159] The traffic flow warning module 30 is used to collect the traffic log information of the microservice every minute through a parallel mechanism and input it into the trained traffic data flow analysis model to obtain the peak ratio, change rate and duration output by the traffic data flow analysis model, wherein the peak ratio is the ratio of the current flow to the average flow in the same period of the past 24 hours, the change rate is the percentage change of the flow in adjacent time windows, and the duration is the number of minutes the abnormal state lasts; when the peak ratio exceeds the preset threshold or the change rate exceeds the preset duration, it is determined as a first-level warning; when the peak ratio exceeds the preset threshold and the duration exceeds the preset duration, it is determined as a second-level warning; when the peak ratio exceeds the preset threshold and the duration exceeds the preset duration, it is determined as a third-level warning; different levels of warnings are processed by log recording, alarm push and automatic flow limiting measures respectively.

[0160] From the above description, it can be seen that the microservice flow anomaly monitoring device based on traffic data provided in the embodiment of the present application can receive traffic data including text data and image data continuously uploaded by road cameras, store and process traffic data by setting microservices, obtain traffic log information when the microservice processes traffic data, calculate the volatility and change trend of time series data, and construct a model training data set using a sliding time window method based on the traffic log information; collect the traffic log information of the microservice every minute through a parallel mechanism and input it into the trained traffic data flow analysis model to obtain the peak ratio, change rate and duration output by the traffic data flow analysis model; and use log recording, alarm push and automatic flow limiting measures to process different levels of warnings, thereby effectively improving the efficiency and accuracy of microservice flow anomaly monitoring of traffic data.

[0161] From the hardware level, in order to effectively improve the efficiency and accuracy of microservice traffic anomaly monitoring of traffic data, the present application provides an embodiment of an electronic device for implementing all or part of the content of the microservice traffic anomaly monitoring method based on traffic data, and the electronic device specifically includes the following content:

[0162] Processor, memory, communication interface and bus; wherein the processor, memory and communication interface communicate with each other through the bus; the communication interface is used to realize information transmission between the microservice traffic anomaly monitoring device based on traffic data and related devices such as core business systems, user terminals and related databases; the logic controller can be a desktop computer, a tablet computer and a mobile terminal, etc., and the present embodiment is not limited thereto. In the present embodiment, the logic controller can be implemented with reference to the embodiment of the microservice traffic anomaly monitoring method based on traffic data and the embodiment of the microservice traffic anomaly monitoring device based on traffic data in the embodiment, and the contents thereof are incorporated herein, and the repeated parts are not repeated.

[0163] It is understandable that the user terminal may include a smart phone, a tablet electronic device, a network set-top box, a portable computer, a desktop computer, a personal digital assistant (PDA), a vehicle-mounted device, a smart wearable device, etc. Among them, the smart wearable device may include smart glasses, a smart watch, a smart bracelet, etc.

[0164] In practical applications, part of the microservice traffic anomaly monitoring method based on traffic data can be executed on the electronic device side as described above, or all operations can be completed in the client device. The specific selection can be based on the processing capability of the client device and the limitations of the user's usage scenario. This application does not limit this. If all operations are completed in the client device, the client device may also include a processor.

[0165] The client device may have a communication module (i.e., a communication unit) that can communicate with a remote server to achieve data transmission with the server. The server may include a server on the task scheduling center side, and other implementation scenarios may also include a server on an intermediate platform, such as a server on a third-party server platform that has a communication link with the task scheduling center server. The server may include a single computer device, or a server cluster consisting of multiple servers, or a server structure of a distributed device.

[0166] Fig. 9 FIG. 9 is a schematic block diagram of the system structure of the electronic device 9600 according to an embodiment of the present application. Fig. 9 As shown, the electronic device 9600 may include a central processor 9100 and a memory 9140; the memory 9140 is coupled to the central processor 9100. It is worth noting that Fig. 9 is exemplary; other types of structures may also be used to supplement or replace this structure to implement telecommunication functions or other functions.

[0167] In one embodiment, the function of the microservice traffic anomaly monitoring method based on traffic data can be integrated into the central processor 9100. The central processor 9100 can be configured to perform the following control:

[0168] Step S101: receiving traffic data including text data and image data continuously uploaded by road cameras, storing and processing the traffic data by setting a microservice, obtaining traffic log information when the microservice processes the traffic data, performing wavelet transform noise reduction and exponential smoothing on the traffic log information, calculating the volatility and change trend of the time series data, eliminating abnormal fluctuation points that exceed a preset standard deviation multiple according to the volatility and the change trend, obtaining processed traffic log information, and constructing a model training data set according to the traffic log information using a sliding time window method;

[0169] Step S102: input the model training data set into the bidirectional long short-term memory-attention network to train the traffic data flow analysis model, and construct a model parameter update buffer pool. When the amount of new data reaches a preset ratio, the model is trained online using the new data, and the attention layer weights and bidirectional long short-term memory layer parameters are updated to obtain the traffic data flow analysis model after model training;

[0170] Step S103: Traffic log information of the microservice is collected every minute through a parallel mechanism and input into a trained traffic data flow analysis model to obtain the peak ratio, change rate and duration output by the traffic data flow analysis model, wherein the peak ratio is the ratio of the current flow to the average flow in the same period of the past 24 hours, the change rate is the percentage of change in the flow in adjacent time windows, and the duration is the number of minutes the abnormal state lasts; when the peak ratio exceeds a preset threshold or the change rate exceeds a preset duration, it is determined as a first-level warning; when the peak ratio exceeds a preset threshold and the duration exceeds a preset duration, it is determined as a second-level warning; when the peak ratio exceeds a preset threshold and the duration exceeds a preset duration, it is determined as a third-level warning; different levels of warnings are processed by log recording, alarm push and automatic flow limiting measures respectively.

[0171] From the above description, it can be seen that the electronic device provided in the embodiment of the present application receives traffic data including text data and image data continuously uploaded by road cameras, stores and processes the traffic data by setting microservices, obtains traffic log information when the microservice processes traffic data, calculates the volatility and change trend of time series data, and constructs a model training data set using a sliding time window method based on the traffic log information; collects traffic log information of the microservice every minute through a parallel mechanism and inputs it into a trained traffic data flow analysis model to obtain the peak ratio, change rate and duration output by the traffic data flow analysis model; and uses log recording, alarm push and automatic flow limiting measures to process different levels of warnings, thereby effectively improving the efficiency and accuracy of microservice flow anomaly monitoring of traffic data.

[0172] In another embodiment, the microservice flow anomaly monitoring device based on traffic data can be configured separately from the central processor 9100. For example, the microservice flow anomaly monitoring device based on traffic data can be configured as a chip connected to the central processor 9100, and the function of the microservice flow anomaly monitoring method based on traffic data can be implemented through the control of the central processor.

[0173] like Fig. 9 As shown, the electronic device 9600 may also include: a communication module 9110, an input unit 9120, an audio processor 9130, a display 9160, and a power supply 9170. It is worth noting that the electronic device 9600 does not necessarily have to include Fig. 9 In addition, the electronic device 9600 may also include Fig. 9 For components not shown, reference may be made to the prior art.

[0174] like Fig. 9 As shown, the central processing unit 9100 is sometimes also referred to as a controller or an operation control, and may include a microprocessor or other processor device and / or logic device. The central processing unit 9100 receives input and controls the operation of various components of the electronic device 9600.

[0175] The memory 9140 may be, for example, one or more of a cache, a flash memory, a hard drive, a removable medium, a volatile memory, a non-volatile memory or other suitable devices. The above-mentioned information related to the failure may be stored, and a program for executing the relevant information may also be stored. The CPU 9100 may execute the program stored in the memory 9140 to implement information storage or processing, etc.

[0176] The input unit 9120 provides input to the central processing unit 9100. The input unit 9120 is, for example, a key or a touch input device. The power supply 9170 is used to provide power to the electronic device 9600. The display 9160 is used to display display objects such as images and texts. The display may be, for example, an LCD display, but is not limited thereto.

[0177] The memory 9140 may be a solid-state memory, such as a read-only memory (ROM), a random access memory (RAM), a SIM card, etc. It may also be a memory that saves information even when the power is off, can be selectively erased, and is provided with more data, examples of which are sometimes referred to as EPROMs, etc. The memory 9140 may also be some other type of device. The memory 9140 includes a buffer memory 9141 (sometimes referred to as a buffer). The memory 9140 may include an application / function storage unit 9142, which is used to store application programs and function programs or processes for executing the operation of the electronic device 9600 through the central processor 9100.

[0178] The memory 9140 may also include a data storage unit 9143 for storing data, such as contacts, digital data, pictures, sounds, and / or any other data used by the electronic device. The driver storage unit 9144 of the memory 9140 may include various drivers for communication functions of the electronic device and / or for executing other functions of the electronic device (such as messaging applications, address book applications, etc.).

[0179] The communication module 9110 is a transmitter / receiver that sends and receives signals via the antenna 9111. The communication module 9110 (transmitter / receiver) is coupled to the central processor 9100 to provide input signals and receive output signals, which may be the same as the case of a conventional mobile communication terminal.

[0180] Based on different communication technologies, multiple communication modules 9110 may be provided in the same electronic device, such as a cellular network module, a Bluetooth module and / or a wireless LAN module. The communication module 9110 (transmitter / receiver) is also coupled to a speaker 9131 and a microphone 9132 via an audio processor 9130 to provide an audio output via the speaker 9131 and receive an audio input from the microphone 9132, thereby realizing a common telecommunication function. The audio processor 9130 may include any suitable buffer, decoder, amplifier, etc. In addition, the audio processor 9130 is also coupled to the central processor 9100, so that recording can be performed on the local machine through the microphone 9132, and the sound stored on the local machine can be played through the speaker 9131.

[0181] The embodiments of the present application also provide a computer-readable storage medium capable of implementing all the steps of the microservice traffic anomaly monitoring method based on traffic data in the above-mentioned embodiment, where the execution subject is a server or a client. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, all the steps of the microservice traffic anomaly monitoring method based on traffic data in the above-mentioned embodiment are implemented. For example, when the processor executes the computer program, the following steps are implemented:

[0182] Step S101: receiving traffic data including text data and image data continuously uploaded by road cameras, storing and processing the traffic data by setting a microservice, obtaining traffic log information when the microservice processes the traffic data, performing wavelet transform noise reduction and exponential smoothing on the traffic log information, calculating the volatility and change trend of the time series data, eliminating abnormal fluctuation points that exceed a preset standard deviation multiple according to the volatility and the change trend, obtaining processed traffic log information, and constructing a model training data set according to the traffic log information using a sliding time window method;

[0183] Step S102: input the model training data set into the bidirectional long short-term memory-attention network to train the traffic data flow analysis model, and construct a model parameter update buffer pool. When the amount of new data reaches a preset ratio, the model is trained online using the new data, and the attention layer weights and bidirectional long short-term memory layer parameters are updated to obtain the traffic data flow analysis model after model training;

[0184] Step S103: Traffic log information of the microservice is collected every minute through a parallel mechanism and input into a trained traffic data flow analysis model to obtain the peak ratio, change rate and duration output by the traffic data flow analysis model, wherein the peak ratio is the ratio of the current flow to the average flow in the same period of the past 24 hours, the change rate is the percentage of change in the flow in adjacent time windows, and the duration is the number of minutes the abnormal state lasts; when the peak ratio exceeds a preset threshold or the change rate exceeds a preset duration, it is determined as a first-level warning; when the peak ratio exceeds a preset threshold and the duration exceeds a preset duration, it is determined as a second-level warning; when the peak ratio exceeds a preset threshold and the duration exceeds a preset duration, it is determined as a third-level warning; different levels of warnings are processed by log recording, alarm push and automatic flow limiting measures respectively.

[0185] From the above description, it can be seen that the computer-readable storage medium provided in the embodiment of the present application receives traffic data including text data and image data continuously uploaded by road cameras, stores and processes traffic data by setting microservices, obtains traffic log information when the microservice processes traffic data, calculates the volatility and change trend of time series data, and constructs a model training data set using a sliding time window method based on the traffic log information; collects traffic log information of the microservice every minute through a parallel mechanism and inputs it into a trained traffic data flow analysis model to obtain the peak ratio, change rate and duration output by the traffic data flow analysis model; and uses log recording, alarm push and automatic flow limiting measures to process different levels of warnings, thereby effectively improving the efficiency and accuracy of microservice flow anomaly monitoring of traffic data.

[0186] The embodiments of the present application also provide a computer program product capable of implementing all the steps of the microservice traffic anomaly monitoring method based on traffic data in the above embodiments, where the execution subject is a server or a client. When the computer program / instruction is executed by a processor, the steps of the microservice traffic anomaly monitoring method based on traffic data are implemented. For example, the computer program / instruction implements the following steps:

[0187] Step S101: receiving traffic data including text data and image data continuously uploaded by road cameras, storing and processing the traffic data by setting a microservice, obtaining traffic log information when the microservice processes the traffic data, performing wavelet transform noise reduction and exponential smoothing on the traffic log information, calculating the volatility and change trend of the time series data, eliminating abnormal fluctuation points that exceed a preset standard deviation multiple according to the volatility and the change trend, obtaining processed traffic log information, and constructing a model training data set according to the traffic log information using a sliding time window method;

[0188] Step S102: input the model training data set into the bidirectional long short-term memory-attention network to train the traffic data flow analysis model, and construct a model parameter update buffer pool. When the amount of new data reaches a preset ratio, the model is trained online using the new data, and the attention layer weights and bidirectional long short-term memory layer parameters are updated to obtain the traffic data flow analysis model after model training;

[0189] Step S103: Traffic log information of the microservice is collected every minute through a parallel mechanism and input into a trained traffic data flow analysis model to obtain the peak ratio, change rate and duration output by the traffic data flow analysis model, wherein the peak ratio is the ratio of the current flow to the average flow in the same period of the past 24 hours, the change rate is the percentage of change in the flow in adjacent time windows, and the duration is the number of minutes the abnormal state lasts; when the peak ratio exceeds a preset threshold or the change rate exceeds a preset duration, it is determined as a first-level warning; when the peak ratio exceeds a preset threshold and the duration exceeds a preset duration, it is determined as a second-level warning; when the peak ratio exceeds a preset threshold and the duration exceeds a preset duration, it is determined as a third-level warning; different levels of warnings are processed by log recording, alarm push and automatic flow limiting measures respectively.

[0190] From the above description, it can be seen that the computer program product provided in the embodiment of the present application receives traffic data including text data and image data continuously uploaded by road cameras, stores and processes traffic data by setting microservices, obtains traffic log information when the microservice processes traffic data, calculates the volatility and change trend of time series data, and constructs a model training data set using a sliding time window method based on the traffic log information; collects traffic log information of the microservice every minute through a parallel mechanism and inputs it into a trained traffic data flow analysis model to obtain the peak ratio, change rate and duration output by the traffic data flow analysis model; and uses log recording, alarm push and automatic flow limiting measures to process different levels of warnings, thereby effectively improving the efficiency and accuracy of microservice flow anomaly monitoring of traffic data.

[0191] It should be understood by those skilled in the art that embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0192] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (apparatus), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0193] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0194] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0195] The present invention uses specific embodiments to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A microservice traffic anomaly monitoring method based on traffic data, characterized in that: The method comprises: Receive traffic data including text data and image data continuously uploaded by road cameras, store and process the traffic data by setting a microservice, obtain traffic log information when the microservice processes the traffic data, perform wavelet transform noise reduction and exponential smoothing on the traffic log information, calculate the volatility and change trend of the time series data, remove abnormal fluctuation points that exceed a preset standard deviation multiple according to the volatility and the change trend, obtain processed traffic log information, and construct a model training data set according to the traffic log information using a sliding time window method; Input the model training data set into the bidirectional long short-term memory-attention network to train the traffic data flow analysis model, and build a model parameter update buffer pool. When the amount of new data reaches a preset ratio, use the new data to train the model online, and update the attention layer weights and bidirectional long short-term memory layer parameters to obtain the traffic data flow analysis model after model training; Traffic log information of the microservice is collected every minute through a parallel mechanism and input into a trained traffic data flow analysis model to obtain the peak ratio, change rate and duration output by the traffic data flow analysis model, wherein the peak ratio is the ratio of the current flow to the average flow in the same period of the past 24 hours, the change rate is the percentage of change in the flow in adjacent time windows, and the duration is the number of minutes the abnormal state lasts; when the peak ratio exceeds a preset threshold or the change rate exceeds a preset duration, it is determined as a first-level warning; when the peak ratio exceeds the preset threshold and the duration exceeds the preset duration, it is determined as a second-level warning; when the peak ratio exceeds the preset threshold and the duration exceeds the preset duration, it is determined as a third-level warning; different levels of warnings are processed by log recording, alarm push and automatic flow limiting measures respectively.

2. The microservice traffic anomaly monitoring method based on traffic data according to claim 1 is characterized in that: The obtaining of traffic log information when the microservice processes the traffic data and performing wavelet transform noise reduction and exponential smoothing on the traffic log information, and calculating the volatility and change trend of the time series data, includes: Performing multi-scale wavelet decomposition on the traffic log information to obtain low-frequency coefficients and high-frequency coefficients, calculating a threshold of the high-frequency coefficients, setting the high-frequency coefficients above the threshold to zero, and performing wavelet reconstruction using the processed high-frequency coefficients and the original low-frequency coefficients to obtain the traffic log information after noise reduction; The denoised traffic log information is divided into multiple time series according to the time window. Each time series is processed using the two-parameter exponential smoothing model. The volatility is obtained by calculating the ratio of the absolute value of the difference between adjacent data points in each time series to the mean. The first-order difference of the data points in each time series is calculated to obtain the change trend.

3. The microservice traffic anomaly monitoring method based on traffic data according to claim 1 is characterized in that: The abnormal fluctuation points exceeding the preset standard deviation multiples are eliminated according to the fluctuation rate and the change trend to obtain the processed traffic log information, including: Calculate the mean and standard deviation of the volatility data in each time window, mark the data points whose volatility values ​​exceed the product of the mean and the preset standard deviation multiple as candidate abnormal points, and select the data points whose consecutive occurrence times are less than the preset threshold as abnormal volatility points from the marked candidate abnormal points; Calculate the mean and standard deviation of the changing trend in each time window, mark the data points whose absolute value of the changing trend exceeds the product of the mean and the preset standard deviation as fluctuation anomaly points, remove the fluctuation anomaly points from the traffic log information, and use the linear interpolation method to fill the removed data points to obtain the processed traffic log information.

4. The microservice traffic anomaly monitoring method based on traffic data according to claim 1 is characterized in that: The method of constructing a model training data set using a sliding time window method according to the traffic log information includes: A fixed-size time window is set based on the processed traffic log information, and the traffic data in the time window is used as the input feature, and the traffic data at a moment after the time window is used as the output label. A sample sequence is constructed by sliding the time window on the time axis, and the sample sequence is normalized by the maximum and minimum values; Feature extraction is performed on the standardized sample sequence, and the mean feature, variance feature and kurtosis feature in each time window are calculated. The extracted features are concatenated with the original sample sequence to form an enhanced feature vector, and the enhanced feature vector is divided into a training data set and a validation data set according to a preset ratio.

5. The microservice traffic anomaly monitoring method based on traffic data according to claim 1 is characterized in that: The traffic log information of the microservice is collected every minute through a parallel mechanism and input into a trained traffic data flow analysis model to obtain the peak ratio, change rate and duration output by the traffic data flow analysis model, including: Create multiple independent data collection threads. Each thread is responsible for the log collection task of a specific microservice node. The collected log information is cached in the message queue. The message consumption thread obtains the log information from the queue and converts the data format. The converted data is organized into the same feature structure as the training data set. Based on the converted feature data, the trained traffic data flow analysis model is called to obtain the prediction results. The prediction results are compared with the actual flow data to calculate the peak ratio. The change rate is calculated based on the flow changes within the preset time interval. The cumulative time that the flow index exceeds the normal range is counted to obtain the duration.

6. The microservice traffic anomaly monitoring method based on traffic data according to claim 1 is characterized in that: When the peak ratio exceeds a preset threshold or the change rate exceeds a preset duration, it is determined as a first-level warning; when the peak ratio exceeds a preset threshold and the duration exceeds a preset duration, it is determined as a second-level warning; when the peak ratio exceeds a preset threshold and the duration exceeds a preset duration, it is determined as a third-level warning, further comprising: The benchmark thresholds of the peak ratio and the change rate are set based on the statistical distribution of historical data. The threshold parameters are dynamically updated according to the real-time traffic fluctuation using an adaptive adjustment algorithm. The updated threshold parameters are compared with the current peak ratio and change rate, and the level of triggering the warning is determined based on the comparison results. Set a benchmark threshold group for duration, build a cumulative counter based on time decay to record the duration of the abnormal state, make segmented judgments on the cumulative value of the counter, determine the final warning level based on the interval in which the cumulative value is located, and write the warning level information into the warning status table.

7. The microservice traffic anomaly monitoring method based on traffic data according to claim 1 is characterized in that: The different levels of warnings are handled by log recording, alarm push and automatic flow control measures, including: Construct a data structure of warning information, write the warning level, trigger time, abnormal indicators and their values ​​into the structure, push the warning information to the alarm processing module through the distributed message middleware, and the alarm processing module selects the corresponding push channel and push method according to the warning level, and stores the warning information persistently in the warning log database; Set the current limiting rules based on the warning level, use the token bucket algorithm to control the request passing rate, dynamically adjust the token generation rate and bucket capacity parameters, return the current limiting response to the request exceeding the current limiting threshold, record the current limiting events and update the current limiting counter, and restore the normal request processing logic when the warning is lifted.

8. A microservice traffic anomaly monitoring device based on traffic data, characterized in that: The device comprises: A traffic data processing module is used to receive traffic data including text data and image data continuously uploaded by road cameras, store and process the traffic data by setting a microservice, obtain traffic log information when the microservice processes the traffic data, perform wavelet transform noise reduction and exponential smoothing on the traffic log information, calculate the volatility and change trend of the time series data, remove abnormal fluctuation points that exceed a preset standard deviation multiple according to the volatility and the change trend, obtain processed traffic log information, and construct a model training data set according to the traffic log information using a sliding time window method; A traffic model construction module is used to input the model training data set into the bidirectional long short-term memory-attention network to train the traffic data flow analysis model, and to construct a model parameter update buffer pool. When the amount of new data reaches a preset ratio, the model is trained online using the new data, and the attention layer weights and bidirectional long short-term memory layer parameters are updated to obtain a traffic data flow analysis model after model training; The traffic flow warning module is used to collect the traffic log information of the microservice every minute through a parallel mechanism and input it into the trained traffic data flow analysis model to obtain the peak ratio, change rate and duration output by the traffic data flow analysis model, wherein the peak ratio is the ratio of the current flow to the average flow in the same period of the past 24 hours, the change rate is the percentage change of the flow in adjacent time windows, and the duration is the number of minutes the abnormal state lasts; when the peak ratio exceeds the preset threshold or the change rate exceeds the preset duration, it is determined as a first-level warning; when the peak ratio exceeds the preset threshold and the duration exceeds the preset duration, it is determined as a second-level warning; when the peak ratio exceeds the preset threshold and the duration exceeds the preset duration, it is determined as a third-level warning; different levels of warnings are processed by log recording, alarm push and automatic flow limiting measures respectively.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the microservice traffic anomaly monitoring method based on traffic data according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the microservice traffic anomaly monitoring method based on traffic data as described in any one of claims 1 to 7 are implemented.

Images