Load identification method and system of cloud edge collaborative architecture

Abstract

The invention provides a load identification method and system for a cloud edge collaborative architecture, and the method comprises the steps: dividing a calculation task into a lightweight task and a complex calculation task through introducing a distributed calculation model; the edge device is responsible for executing lightweight tasks such as time sequence waveform data preprocessing and feature extraction, effectively filtering noise data and reducing transmission of redundant information; the cloud server is responsible for complex model reasoning, global optimization and deep analysis of historical data, so that cooperative processing of edge computing and cloud computing is achieved, a task allocation strategy is flexibly adjusted through a dynamic task scheduling mechanism according to the real-time resource state, the task priority and the computing complexity, and the task allocation efficiency is improved. And the network bandwidth pressure and the cloud computing power load are further reduced. The method has the advantages that the real-time performance of task processing in the Internet of Things system, the stability of the system and the accuracy of an identification result are improved.

Description

Technical Field

[0001] The present application relates to the field of Internet of Things, and specifically to a load identification method and system for a cloud-edge collaborative architecture. Background Art

[0002] With the rapid development of Internet of Things (IoT) technology, hundreds of millions of sensor devices are widely used in smart grids, industrial automation, smart cities and other fields. These devices generate massive amounts of data in real time, posing a huge challenge to the computing, storage and transmission capabilities of the system. The traditional centralized computing model relies on the high-performance computing capabilities of the cloud to transmit all collected data to the cloud for processing. However, with the increase in data volume and computational complexity, this model has exposed problems such as low efficiency in computing resource allocation, high network bandwidth consumption, and large noise data interference in terms of real-time, reliability and economy.

[0003] Cloud computing, with its powerful centralized computing power, can play its advantages in massive data processing, complex model reasoning and global optimization. However, traditional cloud computing architecture has a series of problems, such as the centralized computing model easily becomes a performance bottleneck in a high-concurrency environment, lacks a flexible load balancing mechanism, and when the network transmission delay is high, the cloud computing's ability to support real-time response decreases.

[0004] Load identification is an important task in IoT applications, especially in smart grids. By analyzing the load characteristics of electrical equipment, power optimization, energy conservation and emission reduction, and equipment failure prediction can be achieved. In traditional load identification methods, all waveform data is directly uploaded to the cloud for centralized processing. This model has significant computing and transmission bottlenecks. With the rapid growth of the number of IoT devices, solutions that rely solely on the cloud are difficult to meet actual needs. Summary of the invention

[0005] The present application provides a load identification method and system for a cloud-edge collaborative architecture to solve the problem that existing load identification methods have significant computing and transmission bottlenecks.

[0006] The present application provides a load identification method based on a cloud-edge collaborative architecture, which specifically includes a data collection step, a computing task evaluation step, an edge computing step, and a cloud computing step.

[0007] The data collection step is to collect the timing waveform data of different sensor devices and obtain more than two groups of timing waveform data; the computing task evaluation step is to evaluate and classify the computing tasks performed on the timing waveform data. If the computing task is a lightweight computing task, the edge computing step is executed; if the computing task is a complex computing task, the cloud computing step is executed; the edge computing step is to pre-process and extract features of the acquired timing waveform data after the computing tasks are classified, and to execute lightweight computing tasks, output feature timing waveform data and initial recognition results, compress the feature timing waveform data and the initial recognition results, and obtain compressed data and compression results; the cloud computing step is to perform complex computing tasks on the obtained compressed data and compression results, and output recognition results.

[0008] Furthermore, the computing task evaluation step specifically includes a first judgment step, a second judgment step, a third judgment step and a real machine experiment step.

[0009] The first judgment step is used to judge whether the computing task includes ultra-large-scale matrix operations. If so, the computing task is determined to be a complex computing task, if not, the next step is executed; the second judgment step is used to judge whether the number of data points in the working set of the computing task is not less than the first threshold value. If so, the computing task is determined to be a complex computing task, if not, the next step is executed; the third judgment step is used to judge whether the number of nested loop layers of the computing task is not less than the second threshold value. If so, the computing task is determined to be a complex computing task, if not, the next step is executed; the real machine experiment step is to select N segments of data from the timing waveform data to execute the edge computing step and the cloud computing step, the length of the nth segment of data is tn, the time required for the N segments of data to execute the edge computing step and the cloud computing step is t′n, and the time consumption accounts for an=t′n / tn. If there is no k such that ak>1, and the number of k satisfying ak>0.5 does not exceed a threshold range, then the computing task is determined to be a lightweight computing task, otherwise the computing task is determined to be a complex computing task.

[0010] Furthermore, the edge computing step specifically includes a data filtering step, a feature extraction step, a lightweight computing step and a compression processing step.

[0011] The data filtering step is to perform efficient noise filtering on the collected timing waveform data, first filter the data segments with smaller power in the timing waveform data, and then filter the data segments with smaller power jumps according to the power jump amplitude, and extract the waveform segments with valid information; the feature extraction step is to slice the waveform segments with valid information at different lengths to obtain slice data, perform feature extraction on the slice data, and extract active power, reactive power, active distortion, reactive distortion, local maximum, local minimum, standard deviation and gradient as characteristic timing waveform data of the slice data; the lightweight calculation step performs lightweight calculation tasks on the characteristic timing waveform data of the slice data to obtain an initial recognition result; the compression processing step is to encode the characteristic timing waveform data and the initial recognition result using a variable length coding method to obtain the compressed data and the compression result.

[0012] Furthermore, the cloud computing step specifically includes a credibility calculation step, a credibility judgment step and an identification result optimization step.

[0013] The credibility calculation step is to perform load identification on the compressed data, obtain an identification result, and calculate the credibility of the identification result; the credibility judgment step is used to judge whether the credibility is greater than a credibility threshold. If so, the identification result is output; if not, the next step is executed; the identification result optimization step is based on the identification result and the compression result, adjust the algorithm parameters, reduce the deviation, and improve the accuracy of load identification.

[0014] The present application also provides a load identification system based on a cloud-edge collaborative architecture, including an edge device and a cloud server.

[0015] The edge device is connected to different sensor devices to collect timing waveform data and obtain characteristic timing waveform data and initial recognition results of the timing waveform data through calculation; the cloud server is used to receive compressed data and compression results transmitted by the edge device, and perform complex computing tasks on the compressed data and compression results.

[0016] Furthermore, the cloud server includes an algorithm management unit, a load identification unit, a load monitoring unit and a load balancing unit.

[0017] The algorithm management unit is used to realize the coexistence of different versions of load identification algorithms, and supports automatic deployment and dynamic switching in distributed computing environments; the load identification unit calculates the average frequency of the initial identification results based on historical time series waveform data, predicts the cloud task load, and allocates initial computing resources to the computing tasks; the load monitoring unit is used to monitor the operating status and available computing power margin of each computing node in real time, and monitor the changes in the average load identification event frequency; the load balancing unit automatically reallocates computing resources and adjusts the task scheduling strategy when the frequency change of the initial identification result exceeds a frequency threshold.

[0018] Furthermore, the algorithm management unit is used to perform the following steps: deploy or update the algorithm version between different nodes in the cloud through an automated tool; dynamically select algorithm clusters of different versions according to the scenario or requirements in which the edge device is installed.

[0019] Furthermore, the load balancing unit is used to perform the following steps: adjust the task distribution of cloud computing nodes in real time to ensure load balance of each node; smoothly migrate tasks to other computing nodes without affecting the tasks being processed based on changes in average event frequency; and automatically issue operation and maintenance alarm notifications when computing resources are insufficient.

[0020] The present application provides a load identification method and system for a cloud-edge collaborative architecture, which divides computing tasks into lightweight tasks and complex computing tasks by introducing a distributed computing model; the edge device is responsible for performing lightweight tasks such as time series waveform data preprocessing and feature extraction, effectively filtering noise data and reducing the transmission of redundant information; the cloud server is responsible for complex model reasoning, global optimization, and in-depth analysis of historical data, thereby realizing the collaborative processing of edge computing and cloud computing, and through a dynamic task scheduling mechanism, flexibly adjusting the task allocation strategy according to real-time resource status, task priority, and computing complexity, further reducing network bandwidth pressure and cloud computing load. The present invention improves the real-time performance of task processing, the stability of the system, and the accuracy of identification results, and is particularly suitable for data-intensive and computing-intensive IoT application scenarios, solving the problem that the existing load identification method has significant computing and transmission bottlenecks due to the rapid growth in the number of IoT devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0022] Figure 1is a schematic diagram of a load identification system based on a cloud-edge collaborative architecture described in this embodiment;

[0023] Figure 2 is a flow chart of a load identification method based on a cloud-edge collaborative architecture described in this embodiment;

[0024] Figure 3 is a flowchart of the computing task evaluation steps described in this embodiment;

[0025] Figure 4 is a flow chart of edge computing steps described in this embodiment;

[0026] Figure 5 is a flow chart of the cloud computing steps described in this embodiment. DETAILED DESCRIPTION

[0027] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0028] like Figure 1 As shown, the present application also provides a load identification system based on a cloud-edge collaborative architecture, including an edge device and a cloud server.

[0029] The edge device is connected to different sensor devices to collect timing waveform data and obtain characteristic timing waveform data and initial recognition results of the timing waveform data through calculation; the cloud server is used to receive compressed data and compression results transmitted by the edge device, and perform complex computing tasks on the compressed data and compression results.

[0030] In this embodiment, the task scheduling module is used to achieve efficient collaboration between edge devices and cloud servers, and load prediction and dynamic load balancing technology are combined to further improve system performance and reliability.

[0031] Furthermore, the cloud server includes an algorithm management unit, a load identification unit, a load monitoring unit and a load balancing unit.

[0032] The algorithm management unit is used to realize the coexistence of different versions of load identification algorithms and support automatic deployment and dynamic switching in a distributed computing environment.

[0033] Furthermore, the algorithm management unit is used to perform the following steps: deploy or update the algorithm version between different nodes in the cloud through an automated tool; dynamically select algorithm clusters of different versions according to the scenario or requirements in which the edge device is installed.

[0034] In this embodiment, the algorithm management unit in the cloud server uses a network file system (NFS) to distribute different versions of algorithm models and algorithm model parameters, and uses a distributed message queue system to distribute the correspondence between algorithms and points. An agent program is run on each computing node to perform dynamic version switching. When a request is received from an administrator or other unit, the corresponding update instruction is issued to the agent program through the distributed message queue system.

[0035] The load identification unit calculates the average frequency of the initial identification result based on the historical time series waveform data, predicts the cloud task load, and allocates initial computing resources to the computing task.

[0036] In this embodiment, the initial identification result is an electrical switch event. The load identification unit stores historical timing waveform data through a timing database, receives external requests through a distributed message queue, and calculates the predicted load value in real time when receiving a request from an administrator or other unit. While returning the predicted value to the requesting party, it submits it to the automatic load balancing unit.

[0037] The actual load value is the ratio of the number of feature data segments received per second to 1000 times the number of CPU cores of the cluster node. That is, for a 1-core CPU node, the load value reaches an upper limit of 100% when 1000 feature data segments are received per second. The predicted load value is the product of the current actual load value and the load value growth rate in the past 15 seconds plus 1.

[0038] The load monitoring unit is used to monitor the operating status and available computing power margin of each computing node in real time, and monitor changes in the average load identification event frequency.

[0039] The load balancing unit automatically reallocates computing resources and adjusts task scheduling strategies when the frequency change of the initial recognition result exceeds a frequency threshold.

[0040] Furthermore, the load balancing unit is used to perform the following steps: adjust the task distribution of cloud computing nodes in real time to ensure load balance of each node; smoothly migrate tasks to other computing nodes without affecting the tasks being processed based on changes in average event frequency; and automatically issue operation and maintenance alarm notifications when computing resources are insufficient.

[0041] In this embodiment, the historical predicted load values ​​and the historical actual load levels of each computing node are stored through a time series database, and the computing power of each computing node is stored through a relational database. The expected load level and actual load level of the current node are calculated every 15 seconds. When the actual load level of any node is greater than 75%, the task is automatically migrated to other computing nodes with a load level lower than 75%. During migration, the current buffer and status of the task are stored through object serialization technology, and the calculation is suspended. After migration, the data lost during the migration period is retrieved from the online time series database, stored in the buffer, and then the calculation is resumed. When a request is received from an administrator or other unit, the load identification unit is first requested for a predicted load value. After receiving the result, the computing node with the lowest current actual load and no less than 5% of system resources remaining after the predicted load value is met is automatically searched. After finding a computing node that meets the requirements, a start algorithm instruction is sent to the algorithm management unit to obtain a load identification result. If no computing node that meets the requirements is found, an operation and maintenance alarm is sent and an error is reported to the requesting party.

[0042] like Figure 2 As shown, the load identification system based on cloud-edge collaborative architecture is used to execute a load identification method based on cloud-edge collaborative architecture, which specifically includes step S1) data collection step, step S2) computing task evaluation step, step S3) edge computing step and step S4) cloud computing step.

[0043] Step S1) Data collection step, collecting the timing waveform data of different sensor devices to obtain more than two groups of timing waveform data.

[0044] Step S2) is a computing task evaluation step, in which the computing task performed on the timing waveform data is evaluated and graded. If the computing task is a lightweight computing task, an edge computing step is executed; if the computing task is a complex computing task, a cloud computing step is executed.

[0045] like Figure 3 As shown, step S2) the computing task evaluation step specifically includes step S21) a first judgment step, step S22) a second judgment step, step S23) a third judgment step and step S24) a real machine experiment step.

[0046] Step S21) The first judgment step is to judge whether the computing task includes ultra-large-scale matrix operations. If so, the computing task is determined to be a complex computing task. If not, the next step is executed.

[0047] Step S22) The second judgment step is to judge whether the number of data points in the working set of the computing task is not less than the first threshold. If so, the computing task is determined to be a complex computing task. If not, the next step is executed.

[0048] Step S23) The third judgment step is to judge whether the number of nested loop layers of the computing task is not less than the second threshold. If so, the computing task is determined to be a complex computing task. If not, the next step is executed.

[0049] Step S24) Real machine experiment step, the timing waveform data selects N segments of data to execute the edge computing step and the cloud computing step, the length of the nth segment of data is tn, the time required for the N segments of data to execute the edge computing step and the cloud computing step is t′n, and the time consumption accounts for an=t′n / tn. If there is no k that satisfies ak>1, and the number of k that satisfies ak>0.5 does not exceed a threshold range, then the computing task is determined to be a lightweight computing task, otherwise the computing task is determined to be a complex computing task.

[0050] In this embodiment, the number of k does not exceed a threshold range of 4.

[0051] Step S3) Edge computing step, after grading the computing tasks, preprocessing and feature extraction are performed on the acquired timing waveform data, and lightweight computing tasks are performed, characteristic timing waveform data and initial recognition results are output, and the characteristic timing waveform data and the initial recognition results are compressed to obtain compressed data and compression results.

[0052] Furthermore, step S3) the edge computing step specifically includes step S31) a data filtering step, step S32) a feature extraction step, step S33) a lightweight computing step and step S34) a compression processing step.

[0053] Step S31) Data filtering step, performing efficient noise filtering on the collected timing waveform data, first filtering the data segments with smaller power in the timing waveform data, and then filtering the data segments with smaller power jumps according to the power jump amplitude, to extract the waveform segments with valid information.

[0054] In this embodiment, data filtering first confirms the validity of the input data, filters out invalid data in special circumstances such as initial power-on, device plugging and unplugging, and surges, then applies a low-pass filter to reduce input signal jitter caused by external factors such as sensor noise and circuit technology, and finally filters data with low power or no power jumps.

[0055] Step S32) Feature extraction step, slicing the waveform segment of the effective information into slices of different lengths to obtain slice data, performing feature extraction on the slice data, extracting active power, reactive power, active distortion, reactive distortion, local maximum, local minimum, standard deviation and gradient as characteristic time series waveform data of the slice data.

[0056] In this embodiment, the filtered data is stored in a data buffer. First, slices of different lengths are performed according to the requirements of the algorithm model. The sliced ​​data is subjected to feature extraction. Active power, reactive power, and distortion, local maximum value, local minimum value, standard deviation, and gradient of active and reactive power are extracted as feature data of the data segment. The edge end performs initial identification based on the feature data to obtain the edge end load identification result.

[0057] Step S33) A lightweight calculation step, performing a lightweight calculation task on the characteristic time series waveform data of the slice data to obtain an initial recognition result, wherein the initial recognition result indicates the recognized turned-on electrical appliances.

[0058] Step S34) Compression processing step, using variable length coding method to encode the characteristic timing waveform data and the initial recognition result to obtain the compressed data and the compression result.

[0059] Step S4) Cloud computing step, performing complex computing tasks on the obtained compressed data and compression results, and outputting the recognition results.

[0060] like Figure 5 As shown, step S4) the cloud computing step specifically includes step S41) a credibility calculation step, step S42) a credibility judgment step and step S43) a recognition result optimization step.

[0061] Step S41) Credibility calculation step, performing load identification on the compressed data, obtaining an identification result, and calculating the credibility of the identification result.

[0062] In this embodiment, the credibility is obtained by dividing the maximum power value of the characteristic data by the preset power constant C to obtain a coefficient a, and then multiplying the coefficient a by the load identification accuracy of the edge device in the past 15 days to obtain the final credibility.

[0063] Step S42) Credibility determination step, determining whether the credibility is greater than a credibility threshold, the credibility threshold is 95%, if so, outputting the recognition result, if not, executing the next step.

[0064] Step S43) Identification result optimization step, based on the identification result and the compression result, adjust the algorithm parameters to reduce the deviation and improve the load identification accuracy.

[0065] In this embodiment, adjusting the algorithm parameters is an automatic process. Different algorithm models are selected according to the credibility and the maximum power of the feature data, and the feature data is input into the selected algorithm model to obtain the final recognition result.

[0066] The present application provides a load identification method and system for a cloud-edge collaborative architecture, which divides computing tasks into lightweight tasks and complex computing tasks by introducing a distributed computing model; the edge device is responsible for performing lightweight tasks such as time series waveform data preprocessing and feature extraction, effectively filtering noise data and reducing the transmission of redundant information; the cloud server is responsible for complex model reasoning, global optimization, and in-depth analysis of historical data, thereby realizing the collaborative processing of edge computing and cloud computing, and through a dynamic task scheduling mechanism, flexibly adjusting the task allocation strategy according to real-time resource status, task priority, and computing complexity, further reducing network bandwidth pressure and cloud computing load. The present invention improves the real-time performance of task processing, the stability of the system, and the accuracy of identification results, and is particularly suitable for data-intensive and computing-intensive IoT application scenarios, solving the problem that the existing load identification method has significant computing and transmission bottlenecks due to the rapid growth in the number of IoT devices.

[0067] The above provides a load identification method and system for a cloud-edge collaborative architecture for the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for general technical personnel in this field, based on the ideas of the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A load identification method based on cloud-edge collaborative architecture, characterized in that: The specific steps include: A data collection step, collecting time series waveform data of different sensor devices to obtain more than two groups of time series waveform data; A computing task evaluation step, evaluating and grading the computing task performed on the timing waveform data, and if the computing task is a lightweight computing task, executing an edge computing step; if the computing task is a complex computing task, executing a cloud computing step; The edge computing step, after classifying the computing tasks, preprocesses and extracts features from the acquired time series waveform data, executes lightweight computing tasks, outputs feature time series waveform data and initial recognition results, compresses the feature time series waveform data and the initial recognition results, and obtains compressed data and compression results; as well as The cloud computing step performs complex computing tasks on the compressed data and compression results, and outputs the recognition results.

2. The load identification method based on cloud-edge collaborative architecture according to claim 1, characterized in that: The computing task evaluation step specifically includes the following steps: The first judgment step is to judge whether the computing task includes ultra-large-scale matrix operations. If so, the computing task is determined to be a complex computing task. If not, the next step is executed. A second judgment step is to judge whether the number of data points in the working set of the computing task is not less than a first threshold value, if so, the computing task is determined to be a complex computing task, if not, the next step is executed; A third judgment step, judging whether the number of nested loop layers of the computing task is not less than a second threshold, if so, determining that the computing task is a complex computing task, if not, executing the next step; as well as The real machine experiment steps are as follows: N segments of data are selected from the timing waveform data to execute the edge computing step and the cloud computing step. The length of the nth segment of data is tn. The time required for the N segments of data to execute the edge computing step and the cloud computing step is t′n. The time consumption accounts for an=t′n / tn. If there is no k such that ak>1, and the number of k satisfying ak>0.5 does not exceed a threshold range, then the computing task is determined to be a lightweight computing task, otherwise the computing task is determined to be a complex computing task.

3. The load identification method based on cloud-edge collaborative architecture according to claim 1, characterized in that: The edge computing step specifically includes the following steps: The data filtering step is to perform efficient noise filtering on the collected time series waveform data, first filter the data segments with smaller power in the time series waveform data, and then filter the data segments with smaller power jumps according to the power jump amplitude, so as to extract the waveform segments with valid information; The feature extraction step is to slice the waveform segment of the effective information into slices of different lengths to obtain slice data, and to extract features from the slice data to extract active power, reactive power, active distortion, reactive distortion, local maximum, local minimum, standard deviation and gradient as feature time series waveform data of the slice data; a light-weight calculation step, performing a light-weight calculation task on the characteristic time-series waveform data of the slice data to obtain an initial recognition result; and The compression processing step uses a variable length coding method to encode the characteristic time series waveform data and the initial recognition result to obtain the compressed data and the compression result.

4. The load identification method based on cloud-edge collaborative architecture according to claim 1, characterized in that: The cloud computing step specifically includes the following steps: a credibility calculation step of performing load identification on the compressed data to obtain an identification result, and calculating the credibility of the identification result; A credibility judgment step, judging whether the credibility is greater than a credibility threshold, if so, outputting the recognition result, if not, executing the next step; as well as The identification result optimization step adjusts the algorithm parameters based on the identification result and the compression result to reduce the deviation, improve the load identification accuracy, and obtain the final identification result.

5. A load identification system based on cloud-edge collaborative architecture, characterized in that: include: An edge device, which is connected to different sensor devices, is used to collect time series waveform data and obtain characteristic time series waveform data and initial recognition results of the time series waveform data through calculation; as well as The cloud server receives the compressed data and compression results transmitted by the edge device, and performs complex computing tasks on the compressed data and compression results.

6. The load identification system based on cloud-edge collaborative architecture according to claim 5, characterized in that: The cloud server includes: Algorithm management unit, which enables the coexistence of different versions of load identification algorithms and supports automatic deployment and dynamic switching in distributed computing environments; The load identification unit calculates the average frequency of the initial identification results based on the historical time series waveform data, predicts the cloud task load, and allocates initial computing resources for the computing tasks; A load monitoring unit that monitors the operating status and available computing power margin of each computing node in real time, and monitors changes in the frequency of average load identification events; and The load balancing unit automatically reallocates computing resources and adjusts the task scheduling strategy when the frequency change of the initial recognition result exceeds a frequency threshold.

7. The load identification system based on cloud-edge collaborative architecture according to claim 5, characterized in that: The algorithm management unit is used to perform the following steps: Deploy or update algorithm versions between different nodes in the cloud through automated tools; Different versions of algorithm clusters are dynamically selected according to the scenario or requirement in which the edge device is installed.

8. The load identification system based on cloud-edge collaborative architecture according to claim 5, characterized in that: The load balancing unit is used to perform the following steps: Adjust the task distribution of cloud computing nodes in real time to ensure load balance among nodes; Based on the changes in the average event frequency, smoothly migrate tasks to other computing nodes without affecting the tasks being processed; When computing resources are insufficient, operation and maintenance alarm notifications are automatically issued.

Images