Data processing method, device, medium and program product

Abstract

The present application provides a data processing method, device, medium, and program product that can be applied in the field of computer technology. The data processing method includes: analyzing performance indicators from a gateway device to determine an alarm indicator; processing alarm data related to the alarm indicator based on a kernel density estimation method to obtain a predicted alarm probability, wherein the alarm data includes an alarm time, an alarm location, and an alarm level, and the kernel density estimation method is used to constrain the balance between the alarm data and the predicted alarm probability; adjusting the frequency of the gateway device's collection of the alarm indicator based on the historical probability density and the predicted alarm probability to obtain a target collection frequency; and sending the target collection frequency to the gateway device.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and more specifically to a data processing method, device, medium, and program product. Background Art

[0002] With traditional centralized monitoring, edge servers connect directly to cloud-based monitoring centers. This often leads to high bandwidth usage and one-way network issues. Furthermore, deploying monitoring agents locally on edge servers not only consumes computing resources but also only supports the passive collection of fixed metrics, preventing dynamic adjustment of monitoring policies. Summary of the Invention

[0003] In view of the above problems, the present application provides a data processing method, device, medium and program product.

[0004] According to the first aspect of the present application, a data processing method is provided, which is applied to a cloud server, including: analyzing performance indicators from a gateway device to determine alarm indicators; processing alarm data related to the alarm indicators based on a kernel density estimation method to obtain a predicted alarm probability, the alarm data including the alarm time, alarm location and alarm level, and the kernel density estimation method is used to constrain the balance relationship between the alarm data and the predicted alarm probability; based on the historical probability density and predicted alarm probability of the alarm indicators, adjusting the collection frequency of the gateway device for the alarm indicators to obtain a target collection frequency; and sending the target collection frequency to the gateway device.

[0005] According to the second aspect of the present application, a data processing method is provided, which is applied to a gateway device, including: in response to receiving a monitoring task from a cloud server, executing a monitoring operation corresponding to the monitoring task to obtain monitoring data, wherein the monitoring task includes collecting monitoring data related to the alarm indicator at a target collection frequency; and sending the monitoring data to the cloud server.

[0006] The third aspect of the present application provides an electronic device, comprising: one or more processors; a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the above method.

[0007] The fourth aspect of the present application further provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, causes the processor to execute the above method.

[0008] The fifth aspect of the present application further provides a computer program product, comprising a computer program, which implements the above method when executed by a processor.

[0009] According to the data processing method, equipment, medium and program product provided by this application, by analyzing the performance indicators from the gateway device, determining the alarm indicators, and processing the alarm data related to the alarm indicators based on the kernel density estimation method, the predicted alarm probability is obtained, which can effectively handle the complex distribution of alarm data, accurately reflect the possibility of alarm occurrence, and reduce false alarms and missed alarms. In addition, by adopting data such as alarm time, alarm level, and alarm location, the accuracy of anomaly detection is increased, the false alarm rate of alarms is reduced, and the accuracy of alarm positioning is improved. Based on the historical probability density and predicted alarm probability of the alarm indicators, the collection frequency can be dynamically adjusted without manual intervention, which improves the flexibility of the system, avoids the waste of resources of fixed collection frequency, and reduces the load and energy consumption of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The above contents and other objects, features and advantages of the present application will become more apparent through the following description of the embodiments of the present application with reference to the accompanying drawings, in which:

[0011] Figure 1 A diagram showing an application scenario of the data processing method, device, medium, and program product according to an embodiment of the present application is shown;

[0012] Figure 2 A flow chart showing a data processing method applied to a cloud server according to an embodiment of the present application is shown;

[0013] Figure 3 A flow chart showing a data processing method applied to a gateway device according to an embodiment of the present application is shown;

[0014] Figure 4 A diagram showing a communication architecture for monitoring agent deployment according to an embodiment of the present application is shown;

[0015] Figure 5 The following is an architectural diagram showing a data processing method and apparatus according to an embodiment of the present application;

[0016] Figure 6 A structural block diagram of a data processing device applied to a cloud server according to an embodiment of the present application is shown;

[0017] Figure 7 A structural block diagram of a data processing device applied to a gateway device according to an embodiment of the present application is shown;

[0018] Figure 8 A block diagram of an electronic device suitable for implementing a data processing method according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0019] Hereinafter, embodiments of the present application will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present application. In the detailed description below, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present application. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessarily confusing the concepts of the present application.

[0020] The terms used herein are only for describing specific embodiments and are not intended to limit the present application. The terms "comprise," "include," etc. used herein indicate the presence of features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0021] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0022] When expressions such as "at least one of A, B, and C, etc." are used, they should generally be interpreted in accordance with the meaning commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0023] With the surge in the number of IoT devices and the increasing demand for real-time performance, traditional centralized cloud computing-based monitoring models are facing significant bottlenecks. Centralized servers must process massive amounts of data, resulting in high network bandwidth pressure and latency, making it difficult to meet the responsiveness requirements of real-time monitoring scenarios. The large number of edge servers and their widespread distribution pose a significant challenge in effectively monitoring and managing them. In this context, edge computing, by bringing computing power to the edge of the network, enables localized data processing and analysis, effectively addressing the performance bottlenecks of centralized cloud-based processing.

[0024] In traditional centralized monitoring, edge servers connect directly to a cloud-based monitoring center, which monitors and manages the edge servers. The edge servers upload all data (device assets, status, performance, logs, and metrics). This technology has the following drawbacks.

[0025] 1) Large amounts of data transmission lead to network congestion, especially in high-throughput scenarios involving large amounts of data collection, which results in high bandwidth usage.

[0026] 2) Centralized architecture often reduces the frequency of data collection. Low-frequency data collection may result in insufficient real-time performance of indicator data.

[0027] 3) Since edge servers are often used in special environments and can only be accessed through the local area network, remote servers cannot be directly connected and accessed.

[0028] For edge-side lightweight monitoring agents, the monitoring agent is deployed locally on the edge server and only collects and uploads some indicators. This technology has the following disadvantages.

[0029] 1) Resource preemption: Monitoring programs occupy edge server computing resources, affecting business performance.

[0030] 2) Single function: It only supports the passive collection of fixed indicators and cannot dynamically adjust the monitoring strategy.

[0031] 3) Device discovery relies on manual configuration: New devices or changes in network topology cannot be automatically identified.

[0032] In view of this, an embodiment of the present application provides a data processing method, including: analyzing performance indicators from a gateway device to determine alarm indicators; processing alarm data related to the alarm indicators based on a kernel density estimation method to obtain a predicted alarm probability, the alarm data including the alarm time, alarm location and alarm level, and the kernel density estimation method is used to constrain the balance relationship between the alarm data and the predicted alarm probability; based on the historical probability density and predicted alarm probability of the alarm indicators, adjusting the collection frequency of the gateway device for the alarm indicators to obtain the target collection frequency; and sending the target collection frequency to the gateway device.

[0033] Figure 1 The present invention illustrates an application scenario diagram of a data processing method, device, medium, and program product according to an embodiment of the present application.

[0034] like Figure 1 As shown, an application scenario 100 of the data processing method, device, medium, and program product of an embodiment of the present application includes a cloud server 110, a gateway device 120, and an edge server 130. The gateway device 120 provides communication between the cloud server 110 and the edge server 130. The cloud server 110 can receive edge server information uploaded by the gateway device 120, manage it, and send monitoring task instructions to the gateway device 120. The gateway device can deploy a lightweight monitoring agent program, use the monitoring agent program to monitor the gateway device 120, and send the collected monitoring data to the cloud server 110. The edge server 130 can be a server deployed at the edge of the network and capable of monitoring.

[0035] It should be understood that Figure 1The number of edge servers and gateway devices in the embodiment is only for illustration. Any number of edge servers and gateway devices may be provided according to implementation requirements.

[0036] The following will be based on Figure 1 The scene described by Figures 2 to 5 The data processing method of the disclosed embodiment is described in detail.

[0037] Figure 2 A flow chart of a data processing method applied to a cloud server according to an embodiment of the present application is shown.

[0038] like Figure 2 As shown, the data processing method of this embodiment includes operations S210 to S240.

[0039] In operation S210 , the performance indicators from the gateway device are analyzed to determine an alarm indicator.

[0040] In operation S220 , the alarm data related to the alarm indicator is processed based on a kernel density estimation method to obtain a predicted alarm probability.

[0041] In operation S230 , based on the historical probability density and the predicted alarm probability of the alarm indicator, the frequency of collecting the alarm indicator by the gateway device is adjusted to obtain a target collection frequency.

[0042] In operation S240 , the target acquisition frequency is sent to the gateway device.

[0043] According to embodiments of the present application, performance indicators may include, but are not limited to, device status indicators, resource usage-related indicators, business logic-related indicators, and indicators such as temperature and humidity in the device's operating environment. Embodiments of the present application do not limit the types of performance indicators. Analysis of performance indicators from gateway devices may determine alarm indicators. Specifically, among the performance indicators collected from gateway devices, thresholds set for corresponding performance indicators may be used to determine alarm indicators.

[0044] According to an embodiment of the present application, the alarm data may include the alarm time, alarm location, and alarm level corresponding to when the alarm indicator occurs, and the kernel density estimation method can be used to constrain the balance between the alarm data and the predicted alarm probability. Based on the kernel density estimation method, the alarm data related to the alarm indicator is processed to obtain the predicted alarm probability. For example, the alarm indicator is the CPU usage rate, the alarm time is the timestamp of the alarm occurrence, the alarm location is the network location of the alarm device, and the alarm level can be divided into three levels: low, medium, and high. Based on the kernel density estimation method, the encoded features of the alarm data are processed to obtain the predicted alarm probability.

[0045] According to an embodiment of the present application, based on the historical probability density and predicted alarm probability of the alarm indicator, the frequency of collecting alarm indicators by the gateway device is adjusted to obtain a target collection frequency, and the obtained target collection frequency is sent to the gateway device.

[0046] According to the embodiments of the present application, by analyzing the performance indicators from the gateway device, determining the alarm indicators, and processing the alarm data related to the alarm indicators based on the kernel density estimation method, a predicted alarm probability is obtained, which can effectively handle the complex distribution of alarm data, accurately reflect the possibility of alarm occurrence, and reduce false alarms and missed alarms. In addition, by using data such as alarm time, alarm level, and alarm location, the accuracy of anomaly detection is increased, the false alarm rate of alarms is reduced, and the accuracy of alarm positioning is improved. Based on the historical probability density and predicted alarm probability of the alarm indicators, the acquisition frequency can be dynamically adjusted without manual intervention, which improves the flexibility of the system, avoids the waste of resources of a fixed acquisition frequency, and reduces the load and energy consumption of the system.

[0047] According to an embodiment of the present application, based on the historical probability density and predicted alarm probability of the alarm indicator, the collection frequency of the gateway device for the alarm indicator is adjusted to obtain the target collection frequency, including: obtaining the historical probability density of the alarm indicator, the historical probability density includes a first historical threshold, a second historical threshold and a historical benchmark threshold; when the predicted alarm probability is greater than or equal to the first historical threshold, the target collection frequency is the first target collection frequency; when the predicted alarm probability is less than the first historical threshold and greater than the second historical threshold, the target collection frequency is the second target collection frequency; when the predicted alarm probability is less than or equal to the second historical threshold, the target collection frequency is the third target collection frequency; wherein, the first target collection frequency is less than the second target collection frequency, and the second target collection frequency is less than the third target collection frequency.

[0048] Predicting the probability of an alarm When the threshold is greater than or equal to the first historical threshold, it indicates that the alarm event is in a high-density area. The target acquisition frequency can be set to the first target acquisition frequency. The first target acquisition frequency can be the minimum acquisition interval, such as 1 second. When the value is less than or equal to the second historical threshold, it indicates that the alarm event is in a low-density area. The target collection frequency can be set to a third target collection frequency, which can be the maximum collection interval, for example, 10 minutes. When the target acquisition frequency is less than the first historical threshold and greater than the second historical threshold, the target acquisition frequency may be the second target acquisition frequency, which may be greater than the first target acquisition frequency and less than the third target acquisition frequency, for example, 5 minutes. The embodiments of the present application do not limit the specific value of the target acquisition frequency.

[0049] According to the embodiments of the present application, the collection frequency can be dynamically adjusted to avoid the waste of resources of a fixed collection frequency. In addition, multi-dimensional historical indicators are compared with the predicted alarm probability. Multi-dimensional historical indicators such as the first historical threshold, the second historical threshold and the historical benchmark threshold increase the accuracy of anomaly detection. The dynamic decision-making of probability density reduces the false alarm rate of the alarm and improves the accuracy of fault location.

[0050] According to an embodiment of the present application, the second target acquisition frequency is determined based on the following method: the second target acquisition frequency is determined based on the first target acquisition frequency and the transition frequency threshold. The transition frequency threshold is obtained by using a smoothing function to process the first difference between the third target acquisition frequency and the first target acquisition frequency and the second difference between the predicted alarm probability and the historical benchmark threshold. The transition frequency threshold is used to achieve continuity in the acquisition of alarm indicators.

[0051] The second target acquisition frequency can be determined based on the first target acquisition frequency and the transition frequency threshold. The transition frequency threshold is obtained by using a smoothing function to process the first difference between the third target acquisition frequency and the first target acquisition frequency, and the second difference between the predicted alarm probability and the historical benchmark threshold. The transition frequency threshold can be used to achieve continuity in the acquisition of alarm indicators. The relationship between the target acquisition frequency and the predicted alarm probability is shown in Formula (1).

[0052] (1);

[0053] Where T represents the target acquisition frequency, represents the predicted alarm probability, Indicates the first target acquisition frequency, Indicates the third target acquisition frequency, They represent the first historical threshold, the second historical threshold, and the historical benchmark threshold respectively. k is the slope of the control curve, which is usually between 5 and 10.

[0054] According to the embodiments of the present application, by setting multi-stage target acquisition frequencies, a first target acquisition frequency can be set in areas with high alarm density, and a third target acquisition frequency can be set in areas with low alarm density. In the transition area, the second target acquisition frequency can be determined based on methods such as smoothing functions, thereby ensuring that the continuity of data acquisition is not affected when the target acquisition frequency changes, and at the same time avoiding the waste of resources of a fixed acquisition frequency.

[0055] According to an embodiment of the present application, alarm data related to alarm indicators are processed based on a kernel density estimation method to obtain a predicted alarm probability, including: extracting features of the alarm time, alarm location, and alarm level respectively to obtain alarm time features, alarm space features, and alarm level features; processing the alarm time features, alarm space features, and alarm level features based on a kernel density estimation method, optimizing the first distance between the alarm time feature and the historical alarm time feature, the second distance between the alarm space feature and the historical alarm space feature, and the third distance between the alarm level feature and the historical alarm level feature to obtain a predicted alarm probability.

[0056] The features of alarm time, alarm location and alarm level can be extracted respectively to obtain alarm time feature, alarm space feature and alarm level feature. The alarm time feature, alarm space feature and alarm level feature are processed based on the kernel density estimation method to optimize the first distance between the alarm time feature and the historical alarm time feature, the second distance between the alarm space feature and the historical alarm space feature, and the third distance between the alarm level feature and the historical alarm level feature to obtain the predicted alarm probability, as shown in formula (2).

[0057] (2);

[0058] in, represents the predicted alarm probability, t represents the historical alarm time characteristics; x represents the historical alarm space characteristics; s represents the historical alarm level characteristics; Indicates the alarm time characteristics; Indicates the alarm space characteristics; Indicates the alarm level characteristics; n indicates the number of alarm indicators, bandwidth parameters is the smoothing parameter of the control kernel, which is a fixed value. Represents the kernel function.

[0059] According to an embodiment of the present application, the data processing method also includes: analyzing the monitoring data from the gateway device to obtain the edge server type; based on the edge server type, determining the number of the target edge server from the edge server list from the gateway device, the edge server list including the edge server number, the communication protocol type corresponding to the edge server number, and the status identifier; when the status identifier corresponding to the number of the target edge server represents that it is online, sending a monitoring task instruction to the gateway device based on the communication protocol type of the target edge server.

[0060] The monitoring data may include the operating status, performance indicators, log information, etc. of the device. The edge server list may include the edge server number, the communication protocol type corresponding to the edge server number, and a status identifier. The communication protocol type may include HyperText Transfer Protocol (HTTP), Secure Shell (SSH), Transmission Control Protocol / Internet Protocol (TCP / IP), etc., but is not limited to these. This application does not limit the communication protocol type. The status identifier may include whether the edge server is online or offline. The edge server type can be obtained by analyzing the monitoring data from the gateway device. Based on the edge server type, the number of the target edge server can be determined from the edge server list from the gateway device. When the status identifier corresponding to the number of the target edge server indicates that it is online, a monitoring task instruction can be sent to the gateway device based on the communication protocol type of the target edge server.

[0061] According to an embodiment of the present application, the edge server type includes at least one of the following: a storage server, a computing server, an image processor server, and a file server.

[0062] Figure 3 A flow chart of a data processing method applied to a gateway device according to an embodiment of the present application is shown.

[0063] like Figure 3 As shown, the steps of this embodiment include step S310 to step S320.

[0064] Step S310: In response to receiving the monitoring task from the cloud server, executing the monitoring operation corresponding to the monitoring task to obtain monitoring data.

[0065] Step S320: Send the monitoring data to the cloud server.

[0066] A monitoring task may include collecting monitoring data related to an alarm indicator at a target collection frequency. In response to receiving a monitoring task from a cloud server, the monitoring operation corresponding to the monitoring task is executed to obtain monitoring data. The collected monitoring data may be filtered to remove invalid data, and then sent to the cloud server, which may analyze the monitoring data to determine, for example, the target collection frequency.

[0067] According to an embodiment of the present application, the data processing method also includes: deploying the monitoring agent program in a container on the gateway device, and starting the monitoring agent program to establish a connection with the cloud server based on a two-way communication protocol; and the monitoring agent program establishes a connection with the edge server in the local area network.

[0068] After being deployed as a container plugin (Linux Containers, LXC) on the gateway device, the monitoring agent runs continuously as a service. Upon startup, it continuously attempts to connect to the cloud server. Successful connections complete registration and establish a two-way channel. If the connection is abnormally disconnected, the monitoring agent continues trying. Through this two-way channel, the monitoring agent reports device information to the cloud server, which in turn issues control commands.

[0069] Due to the storage resource limitations of the gateway device, the monitoring agent program needs to be strictly limited and its size can be controlled within 5M to save resources and facilitate updates.

[0070] The two-way communication protocol may include a Websocket protocol, and the monitoring agent may establish a connection with the cloud server based on the two-way communication protocol; and the monitoring agent may scan the edge server under the current local area network and establish a connection with the edge server.

[0071] According to the embodiment of the present application, by introducing a gateway device, the gateway device can run a monitoring agent program as an independent device, reducing system resource usage while being able to process and save device information in real time, thereby ensuring data reliability.

[0072] According to an embodiment of the present application, the data processing method further includes: using a monitoring agent program to scan edge servers in the local area network to generate an edge server list; and sending the edge server list to a cloud server.

[0073] By using a monitoring agent to scan edge servers in the local area network, a list of edge servers can be generated. The list of edge servers can be sent to the cloud server so that the cloud server can manage the edge servers. The list of edge servers can also include gateway device information, such as Internet Protocol (IP) and Media Access Control (MAC).

[0074] According to an embodiment of the present application, the data processing method also includes: when the communication between the monitoring agent program and the cloud server is disconnected, storing the monitoring data in the cache of the monitoring agent program; when the communication between the monitoring agent program and the cloud server is connected, using the monitoring agent program to send the monitoring data in the cache to the cloud server.

[0075] A built-in cache can be built into the monitoring agent to store the most recent monitoring data. If the monitoring agent loses communication with the cloud server, the monitoring data can be stored in the monitoring agent's cache to continue working. If the monitoring agent is connected to the cloud server, the monitoring agent can send the monitoring data in the cache to the cloud server in a timely manner.

[0076] According to an embodiment of the present application, the data processing method further includes: when the monitoring agent program monitors that a failure occurs in the edge server, the monitoring agent program controls the edge server to restart.

[0077] Compared with deploying the monitoring agent in the edge server, this method can ensure that the monitoring agent is not affected when the edge server fails, and can detect it in real time so that the monitoring agent can try to recover the operation and control the edge server to restart through the monitoring agent to achieve fault recovery.

[0078] According to an embodiment of the present application, the monitoring data is sent to a cloud server, including: preprocessing the monitoring data to obtain target monitoring data, the preprocessing including at least one of data cleaning, data aggregation and data compression; encrypting the target monitoring data and sending it to the cloud server.

[0079] After receiving monitoring tasks from the cloud, the monitoring agent can parse the tasks and perform corresponding monitoring operations, such as collecting monitoring data and checking edge server status. The monitoring agent can preprocess the collected monitoring data to obtain target monitoring data. This preprocessing can include at least one of data cleaning, data aggregation, and data compression. The target monitoring data is then uploaded to the cloud server via an encrypted channel. The cloud server then analyzes the target monitoring data to identify anomalies and trigger alerts.

[0080] Figure 4 A diagram showing a communication architecture for deploying a monitoring agent according to an embodiment of the present application is shown.

[0081] like Figure 4 As shown, gateway hardware is added to the edge server side as an independent hardware device. The gateway hardware can access both cloud servers and edge servers. For example, the first edge server 450 can communicate with the first gateway hardware 420, the second edge server 460 can communicate with the second gateway hardware 430, and the third edge server 470 can communicate with the third gateway hardware 440. The first gateway hardware 420, the second gateway hardware 430, and the third gateway hardware 440 can all communicate with the cloud server 110.

[0082] Cloud server 110 has device management, task scheduling, and data analysis capabilities. Device management involves storing a list of edge servers, which may include the edge server number, the corresponding communication protocol type, and status identifiers. Task scheduling can involve sending monitoring task instructions to target edge servers based on a target collection frequency. Data analysis involves the cloud server analyzing and organizing monitoring data sent by gateway devices.

[0083] The first gateway hardware 420, the second gateway hardware 430 and the third gateway hardware 440 all have an external network interface and an internal network interface. The external network interface can be connected to the cloud server, and the internal network interface can be connected to the edge server, providing hardware system and network communication support for the monitoring agent program.

[0084] Figure 5 The figure shows an architecture diagram of a data processing method and device according to an embodiment of the present application.

[0085] like Figure 5 As shown, gateway device 120 can connect to gateway monitoring platform 510 and cloud server 110. Upon startup, gateway device 120 automatically connects to gateway monitoring platform 510. Gateway monitoring platform 510 can control gateway device 120 via Message Queuing Telemetry Transport (MQTT). The monitoring agent is deployed as an LXC container, enabling gateway monitoring platform 510 to monitor LXC container status in real time and control plugin restarts and updates.

[0086] After the LXC plug-in is deployed via the gateway monitoring platform 510, the monitoring agent is directly started. The monitoring agent directly connects to the cloud server 110 via the WebSocket protocol and establishes a secure two-way communication channel. Through this channel, the monitoring agent can report the edge server list to the cloud server 110, and the cloud server 110 can also issue control instructions.

[0087] After the monitoring agent is started, it will simultaneously scan the first edge server 450, the second edge server 460, and the third edge server 470 under the local area network 520, establish connections with the first edge server 450, the second edge server 460, and the third edge server 470, and establish communication channels. Through the communication channels, data can be collected from the first edge server 450, the second edge server 460, and the third edge server 470 and control instructions can be sent.

[0088] Based on the above data processing method, this application also provides a data processing device. Figure 6 The device is described in detail.

[0089] Figure 6 A structural block diagram of a data processing device applied to a cloud server according to an embodiment of the present application is shown.

[0090] like Figure 6 As shown, the data processing device 600 of this embodiment includes an analysis module 610 , a predicted alarm probability obtaining module 620 , a target acquisition frequency module 630 and a first sending module 640 .

[0091] The analysis module 610 is configured to analyze the performance indicators from the gateway device and determine the alarm indicators. In one embodiment, the analysis module 610 may be configured to perform the operation S210 described above, which will not be described in detail here.

[0092] The predicted alarm probability obtaining module 620 is used to process the alarm data related to the alarm indicator based on the kernel density estimation method to obtain the predicted alarm probability. In one embodiment, the predicted alarm probability obtaining module 620 can be used to perform the operation S220 described above, which will not be repeated here.

[0093] The target collection frequency module 630 is used to adjust the frequency at which the gateway device collects alarm indicators based on the historical probability density and predicted alarm probability of the alarm indicators to obtain a target collection frequency. In one embodiment, the target collection frequency module 630 can be used to perform the operation S230 described above, which will not be repeated here.

[0094] The first sending module 640 is configured to send the target acquisition frequency to the gateway device. In one embodiment, the first sending module 640 may be configured to execute the operation S240 described above, which will not be described in detail herein.

[0095] By analyzing performance indicators from gateway devices to determine alarm indicators, and processing alarm data related to these indicators using kernel density estimation, the system derives predicted alarm probabilities. This effectively handles the complex distribution of alarm data, accurately reflects the likelihood of an alarm occurring, and reduces false alarms and missed alarms. Furthermore, by utilizing data such as alarm time, alarm level, and alarm location, the accuracy of anomaly detection is increased, the false alarm rate is reduced, and the accuracy of alarm location is improved. Based on the historical probability density of alarm indicators and the predicted alarm probability, the collection frequency can be dynamically adjusted without manual intervention, increasing system flexibility, avoiding the resource waste of a fixed collection frequency, and reducing system load and energy consumption.

[0096] According to an embodiment of the present application, the target acquisition frequency module 630 includes: an acquisition unit, a first target acquisition frequency unit, a second target acquisition frequency unit, and a third target acquisition frequency unit.

[0097] An acquiring unit is used to acquire a historical probability density of the alarm indicator, where the historical probability density includes a first historical threshold, a second historical threshold, and a historical benchmark threshold.

[0098] The first target acquisition frequency unit is configured to set the target acquisition frequency to a first target acquisition frequency when the predicted alarm probability is greater than or equal to the first historical threshold.

[0099] The second target acquisition frequency unit is configured to set the target acquisition frequency to a second target acquisition frequency when the predicted alarm probability is less than the first historical threshold and greater than the second historical threshold.

[0100] The third target acquisition frequency unit is configured to set the target acquisition frequency to a third target acquisition frequency when the predicted alarm probability is less than or equal to the second historical threshold.

[0101] The first target acquisition frequency is smaller than the second target acquisition frequency, and the second target acquisition frequency is smaller than the third target acquisition frequency.

[0102] According to an embodiment of the present application, the second target acquisition frequency unit includes a second target acquisition frequency subunit.

[0103] The second target acquisition frequency subunit is used to determine the second target acquisition frequency based on the first target acquisition frequency and a transition frequency threshold. The transition frequency threshold is obtained by using a smoothing function to process the first difference between the third target acquisition frequency and the first target acquisition frequency and the second difference between the predicted alarm probability and the historical benchmark threshold. The transition frequency threshold is used to achieve continuity in the acquisition of the alarm indicator.

[0104] According to an embodiment of the present application, the predicted alarm probability obtaining module 620 includes: a feature extraction unit and an optimization unit.

[0105] The feature extraction unit is used to extract features of the alarm time, the alarm location and the alarm level respectively to obtain alarm time features, alarm space features and alarm level features.

[0106] An optimization unit is used to process the alarm time feature, the alarm space feature and the alarm level feature based on a kernel density estimation method, optimize the first distance between the alarm time feature and the historical alarm time feature, the second distance between the alarm space feature and the historical alarm space feature, and the third distance between the alarm level feature and the historical alarm level feature, so as to obtain a predicted alarm probability.

[0107] According to an embodiment of the present application, the data processing device 600 further includes: an edge server type obtaining module, a number determination module, and a monitoring task instruction module.

[0108] The edge server type obtaining module is used to analyze the monitoring data from the gateway device to obtain the edge server type.

[0109] The number determination module is used to determine the number of the target edge server from the edge server list from the gateway device based on the edge server type, the edge server list including the edge server number, the communication protocol type corresponding to the edge server number, and the status identifier.

[0110] The monitoring task instruction module is configured to send a monitoring task instruction to the gateway device based on the communication protocol type of the target edge server when the status identifier corresponding to the number of the target edge server indicates that the server is online.

[0111] According to an embodiment of the present application, the edge server type includes at least one of the following: a storage server, a computing server, an image processor server, and a file server.

[0112] Figure 7 The figure shows a structural block diagram of a data processing device applied to a gateway device according to an embodiment of the present application.

[0113] like Figure 7 As shown, the data processing device 700 of this embodiment includes a monitoring data obtaining module 710 and a second sending module 720 .

[0114] The monitoring data obtaining module 710 is used to, in response to receiving the monitoring task from the cloud server, perform the monitoring operation corresponding to the monitoring task and obtain monitoring data. In one embodiment, the monitoring data obtaining module 710 can be used to perform the operation S310 described above, which will not be repeated here.

[0115] The second sending module 720 is used to send the monitoring data to the cloud server. In one embodiment, the second sending module 720 can be used to perform the operation S320 described above, which will not be repeated here.

[0116] The data processing device 700 includes a first connection module and a second connection module.

[0117] The first connection module is used to deploy the monitoring agent program in a container on the gateway device and start the monitoring agent program to establish a connection with the cloud server based on a two-way communication protocol.

[0118] The second connection module is used to monitor the establishment of a connection between the agent program and the edge server in the local area network.

[0119] The data processing device 700 includes: a list generating module and a list sending module.

[0120] The list generation module is used to use the monitoring agent program to scan the edge servers in the local area network and generate an edge server list.

[0121] The list sending module is used to send the edge server list to the cloud server.

[0122] The data processing device 700 includes: a storage module and a monitoring data sending module.

[0123] The storage module is used to store the monitoring data in the cache of the monitoring agent program when the communication between the monitoring agent program and the cloud server is disconnected.

[0124] The monitoring data sending module is used to use the monitoring agent program to send the cached data to the cloud server when the monitoring agent program and the cloud server are in communication connection.

[0125] The data processing device 700 includes: a restart module.

[0126] The restart module is used to control the edge server to restart when the monitoring agent detects that the edge server has failed.

[0127] The second sending module 720 includes: a pre-processing unit and an encryption unit.

[0128] The preprocessing unit is used to preprocess the monitoring data to obtain target monitoring data, where the preprocessing includes at least one of data cleaning, data aggregation and data compression.

[0129] The encryption unit is used to encrypt the target monitoring data and send it to the cloud server.

[0130] Any of the following modules: analysis module 610, predicted alarm probability acquisition module 620, target acquisition frequency module 630, first sending module 640, monitoring data acquisition module 710, and second sending module 720 may be implemented in a single module, or any of these modules may be split into multiple modules. Alternatively, at least part of the functionality of one or more of these modules may be combined with at least part of the functionality of other modules and implemented in a single module. At least one of analysis module 610, predicted alarm probability acquisition module 620, target acquisition frequency module 630, first sending module 640, monitoring data acquisition module 710, and second sending module 720 may be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or may be implemented in hardware or firmware through any other suitable means of circuit integration or packaging, or implemented in any one of software, hardware, and firmware, or any suitable combination of these. Alternatively, at least one of the analysis module 610, the predicted alarm probability acquisition module 620, the target acquisition frequency module 630, the first sending module 640, the monitoring data acquisition module 710 and the second sending module 720 can be at least partially implemented as a computer program module, which can perform corresponding functions when the computer program module is run.

[0131] Figure 8 A block diagram of an electronic device suitable for implementing a data processing method according to an embodiment of the present application is shown.

[0132] like Figure 8 As shown, an electronic device according to an embodiment of the present application includes a processor 801, which can perform various appropriate actions and processes based on a program stored in a read-only memory (ROM) 802 or a program loaded from a storage unit 808 into a random access memory (RAM) 803. The processor 801 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a dedicated microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 801 may also include onboard memory for caching purposes. The processor 801 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present application.

[0133] Various programs and data required for the operation of the electronic device are stored in RAM 803. The processor 801, ROM 802, and RAM 803 are connected to each other via a bus 804. The processor 801 performs various operations of the method flow according to the embodiment of the present application by executing the programs in ROM 802 and / or RAM 803. It should be noted that the programs may also be stored in one or more memories other than ROM 802 and RAM 803. The processor 801 may also perform various operations of the method flow according to the embodiment of the present application by executing the programs stored in one or more memories.

[0134] The electronic device may also include an input / output (I / O) interface 805, which is also connected to the bus 804. The electronic device may also include one or more of the following components connected to the I / O interface 805: an input section 806 including a keyboard, mouse, etc.; an output section 807 including devices such as a cathode ray tube (CRT), liquid crystal display (LCD), and speakers; a storage section 808 including a hard disk; and a communication section 809 including a network interface card such as a LAN card or modem. The communication section 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to the I / O interface 805 as needed. Removable media 811, such as a magnetic disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed in the drive 810 as needed, so that computer programs read from the removable media can be installed into the storage section 808 as needed.

[0135] This application also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments, or may exist independently and not be incorporated into the device / apparatus / system. The computer-readable storage medium carries one or more programs, and when the one or more programs are executed, the data processing method according to the embodiments of this application is implemented.

[0136] The computer-readable storage medium may be a non-volatile computer-readable storage medium, and may include, for example, but is not limited to, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present application, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, the computer-readable storage medium may include the ROM 802 and / or RAM 803 described above, and / or one or more memories other than ROM 802 and RAM 803.

[0137] The embodiments of the present application also include a computer program product, which includes a computer program containing program code for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program code is used to enable the computer system to implement the data processing method provided in the embodiments of the present application.

[0138] The computer program executes the above functions defined in the system / device of the embodiment of the present application when the computer program is executed by the processor 801. The above-described systems, devices, modules, units, etc. can be implemented by computer program modules.

[0139] In one embodiment, the computer program may be stored on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may be transmitted and distributed in the form of a signal on a network medium, downloaded and installed via the communication portion 809, and / or installed from a removable medium 811. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to wireless, wired, or any suitable combination thereof.

[0140] In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 809 and / or installed from a removable medium 811. When the computer program is executed by the processor 801, the above-described functions defined in the system of the embodiment of the present application are performed. The systems, devices, apparatuses, modules, units, etc. described above can be implemented by computer program modules.

[0141] The program code for executing the computer program provided by the embodiments of the present application can be written in any combination of one or more programming languages. Specifically, these computer programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, Python, "C" language, or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect via the Internet).

[0142] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the above-mentioned module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of the boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0143] Those skilled in the art will appreciate that the features described in the various embodiments of this application may be combined and / or coupled in various ways, even if such combinations or couplings are not explicitly described in this application. In particular, the features described in the various embodiments of this application may be combined and / or coupled in various ways without departing from the spirit and teachings of this application. All such combinations and / or couplings fall within the scope of this application.

[0144] The embodiments of the present application have been described above. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of the present application. Although each embodiment has been described separately above, this does not mean that the measures in each embodiment cannot be advantageously used in combination. Without departing from the scope of the present application, those skilled in the art may make various substitutions and modifications, and these substitutions and modifications should all fall within the scope of the present application.

Claims

1. A data processing method, characterized in that: Applied to a cloud server, the method includes: Analyze performance indicators from gateway devices and determine alarm indicators; Processing alarm data related to the alarm indicator based on a kernel density estimation method to obtain a predicted alarm probability, the alarm data including an alarm time, an alarm location, and an alarm level, the kernel density estimation method being used to constrain a balance between the alarm data and the predicted alarm probability; Obtaining a historical probability density of the alarm indicator, where the historical probability density includes a first historical threshold, a second historical threshold, and a historical benchmark threshold; When the predicted alarm probability is greater than or equal to the first historical threshold, the target acquisition frequency is the first target acquisition frequency; When the predicted alarm probability is less than the first historical threshold and greater than the second historical threshold, the target acquisition frequency is the second target acquisition frequency; When the predicted alarm probability is less than or equal to the second historical threshold, the target acquisition frequency is the third target acquisition frequency, the first target acquisition frequency is less than the second target acquisition frequency, the second target acquisition frequency is less than the third target acquisition frequency, and the second target acquisition frequency is determined based on the following method: determining the second target acquisition frequency based on the first target acquisition frequency and a transition frequency threshold, wherein the transition frequency threshold is obtained by processing a first difference between the third target acquisition frequency and the first target acquisition frequency and a second difference between the predicted alarm probability and the historical benchmark threshold using a smoothing function, and the transition frequency threshold is used to ensure continuity in the acquisition of the alarm indicator; and The target acquisition frequency is sent to the gateway device.

2. The method according to claim 1, characterized in that The kernel density estimation method is used to process the alarm data related to the alarm indicator to obtain a predicted alarm probability, including: Extracting features of the alarm time, the alarm location, and the alarm level to obtain alarm time features, alarm space features, and alarm level features; The alarm time feature, the alarm space feature and the alarm level feature are processed based on the kernel density estimation method, and the first distance between the alarm time feature and the historical alarm time feature, the second distance between the alarm space feature and the historical alarm space feature, and the third distance between the alarm level feature and the historical alarm level feature are optimized to obtain the predicted alarm probability.

3. The method according to claim 1, characterized in that The method further comprises: Analyzing monitoring data from the gateway device to obtain an edge server type; Determining, based on the edge server type, an ID of a target edge server from an edge server list from the gateway device, the edge server list including an edge server ID, a communication protocol type corresponding to the edge server ID, and a status identifier; When the status identifier corresponding to the number of the target edge server indicates that the server is online, a monitoring task instruction is sent to the gateway device based on the communication protocol type of the target edge server.

4. The method according to claim 3, characterized in that The edge server type includes at least one of the following: Storage servers, computing servers, image processor servers, and file servers.

5. A data processing method, characterized in that: Applied to a gateway device, the method includes: In response to receiving a monitoring task from a cloud server, executing a monitoring operation corresponding to the monitoring task to obtain monitoring data, wherein the monitoring task includes collecting monitoring data related to the alarm indicator at a target collection frequency; The monitoring data is sent to the cloud server so that the cloud server executes the method according to any one of claims 1 to 4.

6. The method according to claim 5, characterized in that The method further comprises: Deploying a monitoring agent program in a container on the gateway device, and starting the monitoring agent program to establish a connection with the cloud server based on a two-way communication protocol; and The monitoring agent establishes a connection with an edge server in the local area network.

7. The method according to claim 6, characterized in that The method further comprises: Scanning edge servers in the local area network using the monitoring agent to generate an edge server list; Send the edge server list to the cloud server.

8. The method according to claim 6, characterized in that The method further comprises: When the monitoring agent program loses communication with the cloud server, storing the monitoring data in a cache of the monitoring agent program; and In a case where the monitoring agent program is in communication with the cloud server, the monitoring data in the cache is sent to the cloud server by using the monitoring agent program.

9. The method according to claim 6, characterized in that The method further comprises: When the monitoring agent program monitors that a failure occurs on the edge server, the monitoring agent program controls the edge server to restart.

10. The method according to claim 5, characterized in that The sending of the monitoring data to the cloud server includes: Preprocessing the monitoring data to obtain target monitoring data, wherein the preprocessing includes at least one of data cleaning, data aggregation, and data compression; The target monitoring data is encrypted and sent to the cloud server.

11. An electronic device comprising: one or more processors; a memory for storing one or more computer programs, It is characterized in that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 10.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 10 are implemented.

13. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 10 are implemented.

Images