Full-link monitoring method and system for state application in edge distribution scene

Abstract

The invention discloses a full-link monitoring method and system for state application in an edge distribution scene, and the method comprises the steps: S1, deploying a monitoring agent at an edge calculation node, and collecting the state data and performance indexes of edge equipment; s2, sending the collected data to a central monitoring system in real time through an efficient data transmission protocol; s3, storing and managing synchronous data between the edge nodes and the center node by using a distributed database; s4, preprocessing the collected data by applying a real-time data stream processing technology; s5, performing anomaly detection and mode recognition on the preprocessed data, and generating intelligent early warning; s6, adopting a full-link tracking tool to monitor and record a full-link path requested in the system, and identifying and analyzing a performance bottleneck and a fault point; and S7, optimizing a data transmission process by using a message queue system, and implementing a data encryption and access control strategy. The method has the advantages of low delay, high data security and high system response speed.

Description

Technical Field

[0001] The present invention relates to the field of full-link monitoring technology, and in particular to a full-link monitoring method and system for state applications in edge distribution scenarios. Background Art

[0002] With the rise of edge computing, sinking functions such as data processing, content caching, and application services to the edge of the network has become an effective solution to meet the needs of delay-sensitive and bandwidth-intensive applications in modern networks. However, the distributed and dynamic nature of the edge computing environment also brings new challenges to system monitoring and management. In this environment, traditional centralized monitoring systems are difficult to meet the requirements of real-time performance and accuracy due to high latency and high bandwidth consumption.

[0003] In existing technologies, many systems rely on centralized data processing and analysis, which not only increases the bandwidth demand for central data centers, but also increases the delay in data transmission. Specifically, these traditional methods have the following defects:

[0004] 1. High latency problem: In an edge computing environment, data needs to be transmitted to a remote server for processing and analysis, which results in longer response time and makes it difficult to meet the demand for low-latency applications.

[0005] 2. Bandwidth limitation: As the number of edge devices increases, continuously sending large amounts of data to the central server will consume a lot of network bandwidth, affecting the overall performance and efficiency of the system.

[0006] 3. Data security and privacy issues: During data transmission, data may be intercepted or tampered with, especially in distributed networks, where data security issues are particularly important.

[0007] 4. Lack of real-time data processing capabilities: Traditional monitoring systems often find it difficult to process and analyze large-scale real-time data, and are unable to promptly detect and warn of potential system problems or security threats.

[0008] Therefore, how to provide a full-link monitoring method and system for status applications in edge distribution scenarios is a problem that technical personnel in this field urgently need to solve. Summary of the invention

[0009] One purpose of the present invention is to propose a full-link monitoring method for state applications in edge distribution scenarios. The present invention combines edge computing, distributed system technology, artificial intelligence and real-time data processing, and describes in detail how to improve the real-time and accuracy of monitoring through full-link tracking and intelligent anomaly detection. It has the advantages of low latency, strong data security and fast system response speed.

[0010] A full-link monitoring method for state applications in an edge distribution scenario according to an embodiment of the present invention is characterized by comprising the following steps:

[0011] S1. Deploy monitoring agents on edge computing nodes to collect status data and performance indicators of edge devices.

[0012] S2, through efficient data transmission protocol, the collected data is sent to the central monitoring system in real time;

[0013] S3, using a distributed database to store and manage data synchronized between edge nodes and central nodes;

[0014] S4, apply real-time data stream processing technology to pre-process the collected data;

[0015] S5. Combine artificial intelligence technology to perform anomaly detection and pattern recognition on pre-processed data to generate intelligent warnings;

[0016] S6. Use full-link tracking tools to monitor and record the full-link path of requests in the system, identify and analyze performance bottlenecks and failure points;

[0017] S7. Use the message queue system to optimize the data transmission process and implement data encryption and access control strategies.

[0018] Optionally, the S1 specifically includes:

[0019] S11. Select an edge computing node and configure the monitoring agent M according to its hardware configuration H and network connection status C, where H represents detailed parameters of the hardware configuration and C represents the network bandwidth and latency of the node;

[0020] S12, installing monitoring agent software M on the selected node, and accessing the operating system and application interface of the edge node;

[0021] S13, setting monitoring parameters P, including data collection frequency f, data type T and data retention period D, where f represents the time interval for data collection, T represents the type of collected data, and D represents the storage time of the collected data on the edge node;

[0022] S14, start the monitoring agent M and verify its data connection and synchronization function with the central monitoring system;

[0023] S15. Integrate a fault detection module F in the monitoring agent M to preliminarily analyze the collected data and preliminarily identify any potential performance issues and anomalies locally;

[0024] S16. The monitoring agent M will automatically adjust the data collection frequency f and type T according to the real-time data and the preset performance baseline B, where B represents the performance threshold set based on historical data;

[0025] S17. Implement data encryption E, encrypt all transmitted monitoring data before leaving the edge node, where E represents encryption technology and key management strategy.

[0026] Optionally, the S2 specifically includes:

[0027] S21. Determine the data transmission protocol P t , where P t Used to transmit data between edge computing nodes and central monitoring systems;

[0028] S22. Configure data compression mechanism C m , using compression algorithms to reduce the size of transmitted data, where C m Select compression algorithm for real-time data stream;

[0029] S23. Establishing encrypted channel E c , creating a secure data transmission environment between edge nodes and the central system, where E c Including the public key K p and private key K s Use of;

[0030] S24. Implement data serialization d , converting the various data formats collected by monitoring into a uniform format that can be transmitted, where S d For different types of data d Apply the corresponding serialization method;

[0031] S25. Deploy data buffer B d , buffer size S b :

[0032] S b =R g ×Δt;

[0033] Among them, R g represents the data generation rate, and “Δt” represents the residence time of the data in the buffer;

[0034] S26, monitoring data transmission status M t , real-time tracking of data packet transmission, where M t Use network monitoring tools to assess network conditions and adjust transmission parameters;

[0035] S27, optimize transmission path O p , path efficiency P eBy the path length L p and delay D l Decide:

[0036]

[0037] Among them, L p Indicates the length of the selected network path, "D l ” represents the average delay through the path.

[0038] Optionally, the S3 specifically includes:

[0039] S31. Configure distributed database D d , supports high-concurrency data access and elastic data storage, the number of data copies R n :

[0040]

[0041] Where N represents the total number of nodes in the system;

[0042] S32. Use weighted average synchronization to implement data synchronization mechanism S m :

[0043]

[0044] Among them, D i represents the data status of the i-th node, w i is the weight of the ith node, and N represents the total number of nodes participating in synchronization;

[0045] S33, Application Data Version Control V c , manage historical versions and updates of data:

[0046] V c =V n +Δt×r;

[0047] Among them, V n represents the current version number, Δt represents the time since the last update, and r is the version update frequency;

[0048] S34. Deploy data redundancy strategy R s , reduce the risk of data loss, calculate the redundancy level L r :

[0049]

[0050] Among them, S c Indicates the critical data storage capacity, S a Indicates the total available storage capacity;

[0051] S35. Implement data compression C p and Encryption E p Function, data compression rate C r and encryption strength E s They are:

[0052]

[0053] Among them, p(x i ) represents the probability distribution of the i-th element in the data;

[0054] S36. Monitor data access and modification operations, implement access control lists (ACLs) and data access logs d :

[0055] ACL={user_id:access_level};

[0056] L d ={timestamp, operation, user_id};

[0057] Among them, user_id is the identity of the accessing user, access_level is the user's access permission level, timestamp is the timestamp of the operation, and operation is the specific operation type performed.

[0058] Optionally, the S4 specifically includes:

[0059] S41, deploying a real-time data stream processing engine Ep to process the continuous data stream collected from the edge nodes;

[0060] S42. Remove invalid and erroneous data records and implement data cleaning C w :

[0061] C w =N total -N invalid ;

[0062] Among them, N total is the total number of originally collected data, N invalid is the number of data identified as invalid;

[0063] S43, application data conversion T f , formatting the raw data into a unified format, including timestamp standardization and value range adjustment, conversion efficiency η t :

[0064]

[0065] Among them, N processedIndicates the number of data items that were successfully processed and converted;

[0066] S44. Use the aggregate function f a , perform data aggregation A g , aggregate and summarize the data to reduce the processing load:

[0067]

[0068] Among them, N group Indicates the number of data groups grouped by specific conditions, D i represents the data of the i-th group;

[0069] S45, deploy flow window W s , perform data analysis within a set time window, the window length is L w Set according to the real-time requirements of the data:

[0070] W s =CurrentTime-WindowStartTime=L w ;

[0071] Among them, "WindowStartTime" is the time point when the window starts;

[0072] S46. Implementing anomaly detection D e , apply the anomaly detection model to the preprocessed data, mark the abnormal data, and the model sensitivity S d Controlled by the error threshold θ:

[0073] D e ={x|x>μ+k×σorx<μ-k×σ};

[0074] Where x is a single data point, μ is the mean of the data, σ is the standard deviation, and k is a constant that controls sensitivity;

[0075] S47, output pre-processed data to monitoring system O m :

[0076] O m =f output (D preprocessed );

[0077] Among them, f output is the data output function, D preprocessed It is the preprocessed data set.

[0078] Optionally, the S5 specifically includes:

[0079] S51. Deployment of long short-term memory network model M LSTM, analyze the preprocessed data and perform anomaly detection and pattern recognition;

[0080] S52. Select time series feature x t And input to the LSTM model:

[0081] x t =[x(tT),x(t-T+1),…,x(t-1)];

[0082] Where x(t) represents the data at time point t, and T is the length of the time window considered;

[0083] S53. Train the LSTM model to identify anomalies and trends in data:

[0084]

[0085] θ new =θ old +Δθ;

[0086] Among them, θ represents the parameter set of the LSTM model, η is the learning rate, and J(θ; x, y) is the loss function, which measures the model prediction The error between y and the true label y;

[0087] The cross entropy loss function is used to complete the classification task:

[0088]

[0089] Among them, y c is a one-hot encoded true label vector, p c (x; θ) is the model’s prediction of the probability that input x belongs to category c based on the current parameters θ, where C is the total number of categories;

[0090] S54. Apply the trained LSTM model M LSTM Real-time monitoring of new data x new :

[0091]

[0092] Among them, x new is the input feature vector at the current moment, M LSTM (x new ; θ) is the model’s response to the input x based on the parameter θ new The output of , σ is the activation function, z is the output of the LSTM layer, and θ is the threshold used to determine anomalies;

[0093] S55, generate warning signal S w ,when Indicates an exception when triggered:

[0094]

[0095] in, is the predicted value obtained by the model through calculation, θ is the set threshold, when If the value of is greater than or equal to θ, the system will generate an "Alert" signal, indicating that potential abnormal behavior has been detected. If it is less than θ, no warning is triggered and "NoAlert" is output;

[0096] S56. Continuously optimize and adjust the LSTM model M LSTM Improve accuracy and responsiveness based on newly collected data:

[0097]

[0098] Among them, θ old and θ new Respectively represent the model parameters before and after the update, η is the learning rate, is based on the latest data x new and the actual label y new The gradient of the loss function J.

[0099] Optionally, the S6 specifically includes:

[0100] S61. Deployment of full-link tracking system T sys , track and record the request path between edge computing nodes and the central monitoring system in real time;

[0101] S62. Mark each data request R i The corresponding response time t i :

[0102] T mark (R i )=(ID i , t start , t end );

[0103] Among them, ID i is a unique identifier for the request, t start and t end Respectively represent the start and end time of the request;

[0104] S63. Calculate the network delay Δt for each request i and processing time p i :

[0105] Δt i =t response -t request ;

[0106] pi =t end -t start ;

[0107] Among them, t response is the time when the response is received, t request is the time when the request was sent;

[0108] S64, Use Performance Analysis P analysis , analyze the performance bottleneck in the request path:

[0109]

[0110] Where R represents the set of all traced requests, and n is the number of requests;

[0111] S65, optimize the request path to reduce delay and improve data processing efficiency, optimize strategy O strategy for:

[0112]

[0113] Where n is the total number of traced requests, Δt i and p i are the network delay and processing time of the i-th request respectively;

[0114] S66. Record and analyze tracking data and generate performance reports performance :

[0115] R performance ={ID i , Δt i , p i , Efficiency i};

[0116]

[0117] Among them, the efficiency of each request is i The ratio of useful work to total time.

[0118] Optionally, the S7 specifically includes:

[0119] S71. Deploy the message queue system Q s :

[0120]

[0121] Among them, N msg is the number of messages processed per unit time, T proc is the time required to process these messages;

[0122] S72, using efficient communication protocol P comm , its transmission efficiency E trans for:

[0123]

[0124] Among them, D trans is the amount of data successfully transmitted, T trans is the transmission time, T delay is the delay time during transmission;

[0125] S73. Implementing data flow priority sorting priority :

[0126] S priority (D i )=w1×I i +w2×R i ;

[0127] Among them, D i is a data item, I i is its importance score, R i is the response time requirement, w1 and w2 are weight coefficients;

[0128] S74, dynamic buffer mechanism B dyn , buffer time T buffer for:

[0129]

[0130] Among them, Q len is the queue length, C net is the transmission capacity of the network, T max is the maximum allowed cache time;

[0131] S75. Implement data encryption and secure transmission measures E secure :

[0132]

[0133] Among them, L orig is the original data length, L enc is the length of the encrypted data;

[0134] S76, monitor data transmission status and performance M trans :

[0135]

[0136] Among them, E error is the error rate percentage, E trans is the transmission efficiency.

[0137] A full-link monitoring system for state applications in edge distribution scenarios, including:

[0138] The data collection unit is deployed on each edge computing node and is responsible for collecting the status data and performance indicators of each node;

[0139] The data synchronization and storage unit uses distributed database technology to synchronize and store data collected from edge nodes;

[0140] Real-time data processing and anomaly detection unit, which applies long short-term memory network to perform real-time processing and anomaly detection on the collected data;

[0141] The full-link tracing and performance optimization unit uses full-link tracing tools to monitor and record the complete path of data requests and analyze and optimize system performance;

[0142] The security and communication unit uses efficient communication protocols and data encryption technology to ensure data security and integrity during transmission;

[0143] Centralized monitoring and management interface provides a centralized user interface that enables administrators to effectively monitor network status and respond to potential security incidents through real-time data views, alerts, and performance reports.

[0144] The beneficial effects of the present invention are:

[0145] (1) The present invention provides real-time monitoring and intelligent early warning of system status by combining edge computing and full-link monitoring technology, so that the system can reduce data transmission delays and bandwidth consumption in a decentralized edge computing environment, and effectively address common data synchronization and consistency issues in distributed networks. At the same time, using artificial intelligence and machine learning technologies, such as long short-term memory networks (LSTM), it can efficiently process and analyze a large amount of edge-generated data and accurately extract key performance and safety information from it.

[0146] (2) The present invention realizes the complete path monitoring of data requests through the full-link tracking and performance optimization unit, and can accurately identify and analyze the performance bottlenecks and potential failure points in the system. This not only speeds up the problem solving, but also improves the data processing process through dynamic optimization technology, thereby greatly improving the overall performance and stability of the system.

[0147] (3) The security and communication unit of the present invention ensures the security and integrity of data during transmission through efficient communication protocols and advanced data encryption measures. Combined with the real-time monitoring and management interface, the system administrator can monitor the network status in real time and respond quickly to potential security incidents, greatly enhancing the network's security defense capabilities and ability to respond to external threats. At the same time, the system's centralized management interface provides administrators with a powerful tool to more effectively manage and configure security policies, improving the efficiency and effectiveness of security incident handling. BRIEF DESCRIPTION OF THE DRAWINGS

[0148] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0149] Figure 1 A schematic diagram of the overall architecture of the full-link monitoring method and system for state applications in edge distribution scenarios proposed by the present invention;

[0150] Figure 2 A workflow diagram of a data collection unit of a full-link monitoring method and system for state applications in an edge distribution scenario proposed by the present invention;

[0151] Figure 3 This is a structural diagram of the data synchronization and storage unit of the full-link monitoring method and system for status applications in edge distribution scenarios proposed by the present invention. DETAILED DESCRIPTION

[0152] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, which only illustrate the basic structure of the present invention in a schematic manner, and therefore only show the components related to the present invention.

[0153] refer to Figure 1-Figure 3 A full-link monitoring method for state applications in an edge distribution scenario, characterized in that it includes the following steps:

[0154] S1. Deploy monitoring agents on edge computing nodes to collect status data and performance indicators of edge devices.

[0155] In this implementation, S1 specifically includes:

[0156] S11. Select an edge computing node and configure the monitoring agent M according to its hardware configuration H and network connection status C, where H represents detailed parameters of the hardware configuration and C represents the network bandwidth and latency of the node;

[0157] S12, installing monitoring agent software M on the selected node, and accessing the operating system and application interface of the edge node;

[0158] S13, setting monitoring parameters P, including data collection frequency f, data type T and data retention period D, where f represents the time interval for data collection, T represents the type of collected data, and D represents the storage time of the collected data on the edge node;

[0159] S14, start the monitoring agent M and verify its data connection and synchronization function with the central monitoring system;

[0160] S15. Integrate a fault detection module F in the monitoring agent M to preliminarily analyze the collected data and preliminarily identify any potential performance issues and anomalies locally;

[0161] S16. The monitoring agent M will automatically adjust the data collection frequency f and type T according to the real-time data and the preset performance baseline B, where B represents the performance threshold set based on historical data;

[0162] S17. Implement data encryption E, encrypt all transmitted monitoring data before leaving the edge node, where E represents encryption technology and key management strategy.

[0163] S2, through efficient data transmission protocol, the collected data is sent to the central monitoring system in real time;

[0164] In this implementation, S2 specifically includes:

[0165] S21. Determine the data transmission protocol P t , where P t Used to transmit data between edge computing nodes and central monitoring systems;

[0166] S22. Configure data compression mechanism C m , using compression algorithms to reduce the size of transmitted data, where C m Select compression algorithm for real-time data stream;

[0167] S23. Establishing encrypted channel E c , creating a secure data transmission environment between edge nodes and the central system, where E c Including the public key K p and private key K s Use of;

[0168] S24. Implement data serialization d , converting the various data formats collected by monitoring into a uniform format that can be transmitted, where S d For different types of data d Apply the corresponding serialization method;

[0169] S25. Deploy data buffer B d, buffer size S b :

[0170] S b =R g ×Δt;

[0171] Among them, R g represents the data generation rate, and “Δt” represents the residence time of the data in the buffer;

[0172] S26, monitoring data transmission status M t , real-time tracking of data packet transmission, where M t Use network monitoring tools to assess network conditions and adjust transmission parameters;

[0173] S27, optimize transmission path O p , path efficiency P e By the path length L p and delay D l Decide:

[0174]

[0175] Among them, L p Indicates the length of the selected network path, "D l ” represents the average delay through the path.

[0176] S3, using a distributed database to store and manage data synchronized between edge nodes and central nodes;

[0177] In this implementation, S3 specifically includes:

[0178] S31. Configure distributed database D d , supports high-concurrency data access and elastic data storage, the number of data copies R n :

[0179]

[0180] Where N represents the total number of nodes in the system;

[0181] S32. Use weighted average synchronization to implement data synchronization mechanism S m :

[0182]

[0183] Among them, D i represents the data status of the i-th node, w i is the weight of the ith node, and N represents the total number of nodes participating in synchronization;

[0184] S33, Application Data Version Control Vc , manage historical versions and updates of data:

[0185] V c =V n +Δt×r;

[0186] Among them, V n represents the current version number, Δt represents the time since the last update, and r is the version update frequency;

[0187] S34. Deploy data redundancy strategy R s , reduce the risk of data loss, calculate the redundancy level L r :

[0188]

[0189] Among them, S c Indicates the critical data storage capacity, S a Indicates the total available storage capacity;

[0190] S35. Implement data compression C p and Encryption E p Function, data compression rate C r and encryption strength E s They are:

[0191]

[0192] Among them, p(x i ) represents the probability distribution of the i-th element in the data;

[0193] S36. Monitor data access and modification operations, implement access control lists (ACLs) and data access logs d :

[0194] ACL={user_id:access_level};

[0195] L d ={timestamp, operation, user_id};

[0196] Among them, user_id is the identity of the accessing user, access_level is the user's access permission level, timestamp is the timestamp of the operation, and operation is the specific operation type performed.

[0197] S4, apply real-time data stream processing technology to pre-process the collected data;

[0198] In this implementation, S4 specifically includes:

[0199] S41. Deploy real-time data stream processing engine E p ,processing the continuous data stream collected from the edge nodes;

[0200] S42. Remove invalid and erroneous data records and implement data cleaning C w :

[0201] C w =N total -N invalid ;

[0202] Among them, N total is the total number of originally collected data, N invalid is the number of data identified as invalid;

[0203] S43, application data conversion T f , formatting the raw data into a unified format, including timestamp standardization and value range adjustment, conversion efficiency η t :

[0204]

[0205] Among them, N processed Indicates the number of data items that were successfully processed and converted;

[0206] S44. Use the aggregate function f a , perform data aggregation A g , aggregate and summarize the data to reduce the processing load:

[0207]

[0208] Among them, N group Indicates the number of data groups grouped by specific conditions, D i represents the data of the i-th group;

[0209] S45, deploy flow window W s , perform data analysis within a set time window, the window length is L w Set according to the real-time requirements of the data:

[0210] W s =CurrentTime-WindowStartTime=L w ;

[0211] Among them, "WindowStartTime" is the time point when the window starts;

[0212] S46. Implementing anomaly detection D e , apply the anomaly detection model to the preprocessed data, mark the abnormal data, and the model sensitivity S dControlled by the error threshold θ:

[0213] D e ={x|x>μ+k×σorx<μ-k×σ};

[0214] Where x is a single data point, μ is the mean of the data, σ is the standard deviation, and k is a constant that controls sensitivity;

[0215] S47, output pre-processed data to monitoring system O m :

[0216] O m =f output (D preprocessed );

[0217] Among them, f output is the data output function, D preprocessed It is the preprocessed data set.

[0218] S5. Combine artificial intelligence technology to perform anomaly detection and pattern recognition on pre-processed data to generate intelligent warnings;

[0219] In this implementation, S5 specifically includes:

[0220] S51. Deployment of long short-term memory network model M LSTM , analyze the preprocessed data and perform anomaly detection and pattern recognition;

[0221] S52. Select time series feature x t And input to the LSTM model:

[0222] x t =[x(tT),x(t-T+1),…,x(t-1)];

[0223] Where x(t) represents the data at time point t, and T is the length of the time window considered;

[0224] S53. Train the LSTM model to identify anomalies and trends in data:

[0225]

[0226] θ new =θ old +Δθ;

[0227] Among them, θ represents the parameter set of the LSTM model, η is the learning rate, and J(θ; x, y) is the loss function, which measures the model prediction The error between y and the true label y;

[0228] The cross entropy loss function is used to complete the classification task:

[0229]

[0230] Among them, y c is a one-hot encoded true label vector, p c (x; θ) is the model’s prediction of the probability that input x belongs to category c based on the current parameters θ, where C is the total number of categories;

[0231] S54. Apply the trained LSTM model M LSTM Real-time monitoring of new data x new :

[0232]

[0233] Among them, x new is the input feature vector at the current moment, M LSTM (x new ; θ) is the model’s response to the input x based on the parameter θ new The output of , σ is the activation function, z is the output of the LSTM layer, and θ is the threshold used to determine anomalies;

[0234] S55, generate warning signal S w ,when Indicates an exception when triggered:

[0235]

[0236] in, is the predicted value obtained by the model through calculation, θ is the set threshold, when If the value of is greater than or equal to θ, the system will generate an "Alert" signal, indicating that potential abnormal behavior has been detected. If it is less than θ, no warning is triggered and "NoAlert" is output;

[0237] S56. Continuously optimize and adjust the LSTM model M LSTM Improve accuracy and responsiveness based on newly collected data:

[0238]

[0239] Among them, θ old and θ new Respectively represent the model parameters before and after the update, η is the learning rate, is based on the latest data x new and the actual label y new The gradient of the loss function J.

[0240] S6. Use full-link tracking tools to monitor and record the full-link path of requests in the system, identify and analyze performance bottlenecks and failure points;

[0241] In this implementation, S6 specifically includes:

[0242] S61. Deployment of full-link tracking system T sys , track and record the request path between edge computing nodes and the central monitoring system in real time;

[0243] S62. Mark each data request R i The corresponding response time t i :

[0244] T mark (R i )=(ID i , t start , t end );

[0245] Among them, ID i is a unique identifier for the request, t start and i end Respectively represent the start and end time of the request;

[0246] S63. Calculate the network delay Δt for each request i and processing time p i :

[0247] Δt i =t response -t request ;

[0248] p i =t end -t start ;

[0249] Among them, t response is the time when the response is received, t request is the time when the request was sent;

[0250] S64, Use Performance Analysis P analysis , analyze the performance bottleneck in the request path:

[0251]

[0252] Where R represents the set of all traced requests, and n is the number of requests;

[0253] S65, optimize the request path to reduce delay and improve data processing efficiency, optimize strategy O strategy for:

[0254]

[0255] Where n is the total number of traced requests, Δt i and pi are the network delay and processing time of the i-th request respectively;

[0256] S66. Record and analyze tracking data and generate performance reports Rperformance:

[0257] R performance ={ID i ,Δt i , p i , Efficiency i};

[0258]

[0259] Among them, the efficiency of each request is i The ratio of useful work to total time.

[0260] S7. Use the message queue system to optimize the data transmission process and implement data encryption and access control strategies.

[0261] In this implementation, S7 specifically includes:

[0262] S71. Deploy the message queue system Q s :

[0263]

[0264] Among them, N msg is the number of messages processed per unit time, T proc is the time required to process these messages;

[0265] S72, using efficient communication protocol P comm , its transmission efficiency E trans for:

[0266]

[0267] Among them, D trans is the amount of data successfully transmitted, T trans is the transmission time, T delay is the delay time during transmission;

[0268] S73. Implementing data flow priority sorting priority :

[0269] S priority (D i )=w1×I i +w2×R i ;

[0270] Among them, D i is a data item, Ii is its importance score, R i is the response time requirement, w1 and w2 are weight coefficients;

[0271] S74, dynamic buffer mechanism B dyn , buffer time T buffer for:

[0272]

[0273] Among them, Q len is the queue length, C net is the transmission capacity of the network, T max is the maximum allowed cache time;

[0274] S75. Implement data encryption and secure transmission measures E secure :

[0275]

[0276] Among them, L orig is the original data length, L enc is the length of the encrypted data;

[0277] S76, monitor data transmission status and performance M trans :

[0278]

[0279] Among them, E error is the error rate percentage, E trans is the transmission efficiency.

[0280] A full-link monitoring system for state applications in edge distribution scenarios, including:

[0281] The data collection unit is deployed on each edge computing node and is responsible for collecting the status data and performance indicators of each node;

[0282] The data synchronization and storage unit uses distributed database technology to synchronize and store data collected from edge nodes;

[0283] Real-time data processing and anomaly detection unit, which applies long short-term memory network to perform real-time processing and anomaly detection on the collected data;

[0284] The full-link tracing and performance optimization unit uses full-link tracing tools to monitor and record the complete path of data requests and analyze and optimize system performance;

[0285] The security and communication unit uses efficient communication protocols and data encryption technology to ensure data security and integrity during transmission;

[0286] Centralized monitoring and management interface provides a centralized user interface that enables administrators to effectively monitor network status and respond to potential security incidents through real-time data views, alerts, and performance reports.

[0287] Embodiment 1:

[0288] In order to verify the effectiveness of the present invention, we selected a large telecom operator for example analysis. The company faces severe network management and security challenges due to its wide network scale and highly decentralized business. In particular, in terms of real-time data monitoring and security management of edge computing nodes, the traditional centralized monitoring system cannot meet the company's needs due to high latency and untimely data processing.

[0289] After deploying the full-link monitoring system of the present invention, the system implements a series of operations such as data collection, real-time processing, and anomaly detection on multiple edge computing nodes. The system specifically implements the following steps: Deploy monitoring agents at edge nodes across the country to collect key performance indicators and security data in real time. Use a distributed database to synchronize and store data at local nodes to improve the efficiency and accuracy of data processing. Analyze data through an integrated LSTM network to identify and warn of potential security threats and performance issues in real time. Monitor the complete path of data requests, accurately identify performance bottlenecks, and make timely optimization adjustments. In order to specifically illustrate the improvements brought about by the present invention, the following is the performance comparison data before and after the deployment of the system:

[0290] Table 1 Telecommunications company network security and performance status improvement report

[0291] Data Category Before deploying the system After deploying the system Number of security incidents per month 85 times 3 times Real-time monitoring effect - 25 times Isolation Forest Algorithm Effect - 40 times Accuracy 78.5％ 99.1％ False Positive Rate 5.2％ 0.9％ Response time 45 minutes 3 minutes Data Breach Prevention - Significantly reduce

[0292] As can be seen from the table above, after deploying the present invention, the network security of the telecommunications company has been significantly improved, the number of security incidents per month has been reduced from 85 to 3, the false alarm rate of the system has also been reduced from 5.2% to 0.9%, and the response time has been shortened from 45 minutes to 3 minutes. This improvement not only improves the security of the network, but also enhances the system's ability to respond quickly to emergencies, greatly improving operational efficiency and customer service satisfaction.

[0293] As can be seen from the table above, after deploying the present invention, the network security of the telecommunications company has been significantly improved, the number of security incidents per month has been reduced from 85 to 3, the false alarm rate of the system has also been reduced from 5.2% to 0.9%, and the response time has been shortened from 45 minutes to 3 minutes. This improvement not only improves the security of the network, but also enhances the system's ability to respond quickly to emergencies, greatly improving operational efficiency and customer service satisfaction.

[0294] In a specific example, the system successfully blocked a large-scale distributed denial of service (DDoS) attack. Through real-time data processing and anomaly detection units, the system accurately identified abnormal traffic patterns within a few minutes of the attack, immediately triggered an alarm and automatically activated the preset defense mechanism. This rapid response effectively mitigated the impact of the attack on the network infrastructure, ensuring the continuous operation of critical services and the security of data.

[0295] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A full-link monitoring method for state applications in edge distribution scenarios, characterized in that: The steps include: S1. Deploy monitoring agents on edge computing nodes to collect status data and performance indicators of edge devices. S2, through efficient data transmission protocol, the collected data is sent to the central monitoring system in real time; S3, using a distributed database to store and manage data synchronized between edge nodes and central nodes; S4, apply real-time data stream processing technology to pre-process the collected data; S5. Combine artificial intelligence technology to perform anomaly detection and pattern recognition on pre-processed data to generate intelligent warnings; S6. Use full-link tracking tools to monitor and record the full-link path of requests in the system, identify and analyze performance bottlenecks and failure points; S7. Use the message queue system to optimize the data transmission process and implement data encryption and access control strategies.

2. According to the full-link monitoring method of a state application in an edge distribution scenario according to claim 1, it is characterized in that: The S1 specifically includes: S11. Select an edge computing node and configure the monitoring agent M according to its hardware configuration H and network connection status C, where H represents detailed parameters of the hardware configuration and C represents the network bandwidth and latency of the node; S12, installing monitoring agent software M on the selected node, and accessing the operating system and application interface of the edge node; S13, setting monitoring parameters P, including data collection frequency f, data type T and data retention period D, where f represents the time interval for data collection, T represents the type of collected data, and D represents the storage time of the collected data on the edge node; S14, start the monitoring agent M and verify its data connection and synchronization function with the central monitoring system; S15. Integrate a fault detection module F in the monitoring agent M to preliminarily analyze the collected data and preliminarily identify any potential performance issues and anomalies locally; S16. The monitoring agent M will automatically adjust the data collection frequency f and type t according to the real-time data and the preset performance baseline B, where B represents the performance threshold set based on historical data; S17. Implement data encryption E, encrypt all transmitted monitoring data before leaving the edge node, where E represents encryption technology and key management strategy.

3. According to the full-link monitoring method of a state application in an edge distribution scenario according to claim 2, it is characterized in that: The S2 specifically includes: S21. Determine the data transmission protocol P t , where P t Used to transmit data between edge computing nodes and central monitoring systems; S22. Configure data compression mechanism C m , using compression algorithms to reduce the size of transmitted data, where C m Select compression algorithm for real-time data stream; S23. Establishing encrypted channel E c , creating a secure data transmission environment between edge nodes and the central system, where E c Including the public key K p and private key K s Use of; S24. Implement data serialization d , converting the various data formats collected by monitoring into a uniform format that can be transmitted, where S d For different types of data d Apply the corresponding serialization method; S25. Deploy data buffer B d , buffer size S b : S b =R g ×Δt; Among them, R g represents the data generation rate, "Δt" represents the residence time of the data in the buffer; S26, monitoring data transmission status M t , real-time tracking of data packet transmission, where M t Use network monitoring tools to assess network conditions and adjust transmission parameters; S27, optimize transmission path O p , path efficiency P e By the path length L p and delay D l Decide: Among them, L p Indicates the length of the selected network path, "D l ” represents the average delay through the path.

4. According to the full-link monitoring method of a state application in an edge distribution scenario according to claim 3, it is characterized in that: The S3 specifically includes: S31. Configure distributed database D d , supports high-concurrency data access and elastic data storage, the number of data copies R n : Where N represents the total number of nodes in the system; S32. Use weighted average synchronization to implement data synchronization mechanism S m : Among them, D i represents the data status of the i-th node, w i is the weight of the ith node, and N represents the total number of nodes participating in synchronization; S33, Application Data Version Control V c , manage historical versions and updates of data: V c =V n +Δt×r; Among them, V n represents the current version number, Δt represents the time since the last update, and r is the version update frequency; S34. Deploy data redundancy strategy R s , reduce the risk of data loss, calculate the redundancy level L r : Among them, S c represents the critical data storage capacity, S a Indicates the total available storage capacity; S35. Implement data compression C p and Encryption E p Function, data compression rate C r and encryption strength E s They are: Among them, p(x i ) represents the probability distribution of the i-th element in the data; S36. Monitor data access and modification operations, implement access control lists (ACLs) and data access logs d : ACL={user_id:access_level}; L d ={timestamp,operation,user_id}; Among them, user_id is the identity of the accessing user, access_level is the user's access permission level, timestamp is the timestamp of the operation, and operation is the specific operation type performed.

5. According to the full-link monitoring method of a state application in an edge distribution scenario of claim 4, it is characterized in that: The S4 specifically includes: S41. Deploy real-time data stream processing engine E p ,processing the continuous data stream collected from the edge nodes; S42. Remove invalid and erroneous data records and implement data cleaning C w : C w =N total -N invalid ; Among them, N total is the total number of originally collected data, N invalid is the number of data identified as invalid; S43, application data conversion T f , formatting the raw data into a unified format, including timestamp standardization and value range adjustment, conversion efficiency η t : Among them, N procrssed Indicates the number of data items that were successfully processed and converted; S44. Use the aggregate function f a , perform data aggregation A g , summarize and aggregate the data to reduce the processing load: Among them, N group Indicates the number of data groups grouped by specific conditions, D i represents the data of the i-th group; S45, deploy flow window W s , perform data analysis within a set time window, the window length is L w Set according to the real-time requirements of the data: W s =CurrentTime-WindowStratTime=L w ; Among them, "WindowStartTime" is the time point when the window starts; S46. Implementing anomaly detection D e , apply the anomaly detection model to the preprocessed data, mark the abnormal data, and the model sensitivity S d Controlled by the error threshold θ: D e ={x∣x>μ+k×σorx<μ-k×σ}; Where x is a single data point, μ is the mean of the data, σ is the standard deviation, and k is a constant that controls sensitivity; S47, output pre-processed data to monitoring system O m : O m =f output (D preprocessed ); Among them, f output is the data output function, D preprocessed It is the preprocessed data set.

6. The full-link monitoring method for state applications in an edge distribution scenario according to claim 5 is characterized in that: The S5 specifically includes: S51. Deployment of long short-term memory network model M LSTM , analyze the preprocessed data and perform anomaly detection and pattern recognition; S52. Select time series feature x t And input to the LSTM model: x t =[x(t-T),x(t-T+1),…,x(t-1)]; Where x(t) represents the data at time point t, and T is the length of the time window considered; S53. Train the LSTM model to identify anomalies and trends in data: i new =θ old +Δθ; Among them, θ represents the parameter set of the LSTM model, η is the learning rate, and J(θ; x, y) is the loss function, which measures the model prediction The error between y and the true label y; The cross entropy loss function is used to complete the classification task: Among them, y c is a one-hot encoded true label vector, p c (x; θ) is the model’s prediction of the probability that input x belongs to category c based on the current parameters θ, where C is the total number of categories; S54. Apply the trained LSTM model M LSTM Real-time monitoring of new data x new : Among them, x new is the input feature vector at the current moment, M LSTM (x new ; θ) is the model’s response to the input x based on the parameter θ new The output of , σ is the activation function, z is the output of the LSTM layer, and θ is the threshold used to determine anomalies; S55, generate warning signal S w ,when Indicates an exception when triggered: in, is the predicted value obtained by the model through calculation, θ is the set threshold, when If the value of is greater than or equal to θ, the system will generate an "Alert" signal, indicating that potential abnormal behavior has been detected. If it is less than θ, no warning is triggered and "NoAlert" is output; S56. Continuously optimize and adjust the LSTM model M LSTM Improve accuracy and responsiveness based on newly collected data: Among them, θ old and θ new Respectively represent the model parameters before and after the update, η is the learning rate, is based on the latest data x new and the actual label y new The gradient of the loss function J.

7. The full-link monitoring method for state applications in an edge distribution scenario according to claim 6 is characterized in that: The S6 specifically includes: S61. Deployment of full-link tracking system T sys , track and record the request path between edge computing nodes and the central monitoring system in real time; S62. Mark each data request R i The corresponding response time t i : T mark (R i )=(ID i ,t start ,t end ); Among them, ID i is a unique identifier for the request, t start and t end Respectively represent the start and end time of the request; S63. Calculate the network delay Δt for each request i and processing time p i : Δt i =t response -t request ; p i =t end -t start ; Among them, t response is the time when the response is received, t request is the time when the request was sent; S64, Use Performance Analysis P analysis , analyze the performance bottleneck in the request path: Where R represents the set of all traced requests, and n is the number of requests; S65, optimize the request path to reduce delay and improve data processing efficiency, optimize strategy O stategy for: Where n is the total number of traced requests, Δt i and p i are the network delay and processing time of the i-th request respectively; S66. Record and analyze tracking data and generate performance reports performance : R performance ={ID i ,Δt i ,p i ,Efficiency i }; Among them, the efficiency of each request is i The ratio of useful work to total time.

8. The full-link monitoring method for state applications in edge distribution scenarios according to claim 7 is characterized in that: The S7 specifically includes: S71. Deploy the message queue system Q s : Among them, N msg is the number of messages processed per unit time, T proc is the time required to process these messages; S72, using efficient communication protocol P comm , its transmission efficiency E trans for: Among them, D trans is the amount of data successfully transmitted, T trans is the transmission time, T delay is the delay time during transmission; S73. Implementing data flow priority sorting priority : S priority (D i )=w1×I i +w2×R i ; Among them, D i is a data item, I i is its importance score, R i is the response time requirement, w1 and w2 are weight coefficients; S74, dynamic buffer mechanism B dyn , buffer time T buffer for: Among them, Q len is the queue length, C net is the transmission capacity of the network, T max is the maximum allowed cache time; S75. Implement data encryption and secure transmission measures E secure : Among them, L orig is the original data length, L enc is the length of the encrypted data; S76, monitor data transmission status and performance M trans : Among them, E error is the error rate percentage, E trans is the transmission efficiency.

9. A full-link monitoring method and system for a state application in an edge distribution scenario according to any one of claims 1 to 8, characterized in that: include: The data collection unit is deployed on each edge computing node and is responsible for collecting the status data and performance indicators of each node; The data synchronization and storage unit uses distributed database technology to synchronize and store data collected from edge nodes; The real-time data processing and anomaly detection unit uses a long short-term memory network to perform real-time processing and anomaly detection on the collected data; The full-link tracing and performance optimization unit uses full-link tracing tools to monitor and record the complete path of data requests and analyze and optimize system performance; The security and communication unit uses efficient communication protocols and data encryption technology to ensure data security and integrity during transmission; Centralized monitoring and management interface provides a centralized user interface that enables administrators to effectively monitor network status and respond to potential security incidents through real-time data views, alerts, and performance reports.

Images