Cloud edge interactive operation power dispatching method, system and equipment based on lightweight time sequence database and medium

Abstract

The invention is suitable for the technical field of cloud-side interaction, and provides a cloud-side interaction operation power dispatching method, system and device based on a lightweight time sequence database and a medium, and the method comprises the steps that terminal equipment transmits collected initial data to an edge end through a communication protocol; the edge end carries out data coding on the received initial data; the edge end identifies the encoded time sequence data by using a sliding window algorithm, filters the identified abnormal data, and compresses the identified normal data; and the edge end decodes the compressed data to generate a time sequence database, and performs data synchronization with a cloud end based on the time sequence database, so that the cloud end completes task scheduling according to service requirements. According to the method, the abnormal data and the normal data in the time sequence data are identified, the normal data are further compressed, cloud edge data interaction is performed based on the generated time sequence database, and the efficiency of cloud edge interaction data scheduling is effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of cloud-edge interaction technology, and in particular to a cloud-edge interactive operation power dispatching method, system, device and medium based on a lightweight time series database. Background Art

[0002] The hardware heterogeneity and geographical location differences between cloud centers and edge devices result in different applicable tasks for the two. Traditional cloud computing has powerful centralized computing and storage capabilities, and is more suitable for long-term, non-real-time data processing. Edge computing is closer to the user's geographical location, but is limited by computing and communication endurance, so it is more suitable for short-term, real-time data processing. For most IoT application scenarios such as smart cities and smart factories, the traditional cloud center architecture solution cannot meet the requirements of application real-time and privacy, especially for the storage of IoT sensor data. If the massive time series data generated by cloud-edge interaction is directly transmitted to the central cloud for storage without processing, it will inevitably bring huge pressure to the communication network and cloud center load.

[0003] In the prior art, heterogeneous or abnormal data in time series data is identified and processed. For example, in an industrial Internet cloud-edge-end data fusion system and method disclosed in China's public invention patent application CN116260698A, it is recorded that the collaborative transmission module includes a streaming processing unit, a batch processing unit, a data extraction unit, and a data governance unit. The streaming processing unit performs streaming calculations on structured data, the batch processing unit performs batch processing on structured data, the data extraction unit extracts unstructured data into structured data, the data governance unit identifies the data to obtain label definitions, and performs lineage analysis on the derived data. There is no mention of how to identify normal and abnormal data after the converted structured data is identified, and how to further quantify and compress the data. The multi-source demand data fusion and demand management method and system based on edge computing disclosed in China's public invention patent application CN116167013A states: Establish and configure unified coding rules for demand data from different sources, demand identification engine, and unified naming engine for demand baselines on the cloud center server, unify the common attributes and uses of demand specification features, and perform data fusion and data exchange based on demand data archiving and classification. It is mentioned here that when processing heterogeneous demand data, coding rules are used to schedule demand data. In this method, data scheduling is more dependent on manually set fusion rules, which makes the data processing process less accurate and affects the accuracy of the data. Summary of the invention

[0004] The embodiments of the present application provide a cloud-edge interactive power scheduling method, system, device and medium based on a lightweight time series database, which are used to solve the problem of low data processing efficiency during the interaction between the edge and the cloud.

[0005] A first aspect of an embodiment of the present application provides a cloud-edge interactive operation power dispatching method based on a lightweight time series database, comprising:

[0006] The terminal device transmits the collected initial data to the edge end through the communication protocol;

[0007] The edge end performs data encoding on the received initial data;

[0008] The edge end uses a sliding window algorithm to identify the encoded time series data, filters the identified abnormal data, and compresses the identified normal data;

[0009] The edge decodes the compressed data to generate a time series database, and synchronizes data with the cloud based on the time series database, so that the cloud can complete task scheduling according to business needs.

[0010] Furthermore, the edge end performs data encoding on the received initial data, including:

[0011] The encoder is connected, and the encoder determines whether the initial data conforms to the encoding specification. If so, verification calculation and escape operation are performed, and after the operation is completed, the time series data is returned to the edge end.

[0012] Furthermore, the edge end uses a sliding window algorithm to identify the encoded time series data, including:

[0013] Dividing the time series data into a plurality of subsequences of preset lengths;

[0014] Calculating the slope between any two adjacent data points of the target subsequence in the plurality of subsequences, and calculating the confidence interval distance radius of the slope;

[0015] It is determined whether the confidence interval distance radius of the slope is within a threshold range. If so, it is determined that the target subsequence is abnormal time series data; if not, it is determined that the target subsequence is normal time series data.

[0016] Furthermore, the expressions for calculating the slope between any two adjacent data points of the target subsequence in the plurality of subsequences are as follows:

[0017]

[0018] Where: k i is the slope, X j (i) and X j (i―1) are two adjacent data points in the target subsequence, t i is the acquisition time of the target subsequence.

[0019] Furthermore, the expression for the confidence interval distance radius for calculating the slope is as follows:

[0020]

[0021] Where: d j is the confidence interval distance radius, is the upper confidence limit, θ j is the lower confidence limit, is the mean of the slope of the jth subsequence, σ j is the mean square error of the slope of the jth subsequence, the random variable Z satisfies the normal distribution of N(0,1), α is the confidence level, and α=0.05 is taken.

[0022] Furthermore, the expression for compressing the recognized normal data is as follows:

[0023]

[0024] Where A is the performance index for normal data compression, X is the normal data sequence, λ is the data compression amount, G is the performance parameter, τ is the normal data sample regression parameter, d max is the maximum value of normal time series data, d min This is the minimum value of normal time series data.

[0025] Furthermore, the data synchronization based on the time series database and the cloud includes:

[0026] The cloud sets an interface according to the configuration of the time series database, and the edge synchronizes data through the interface. The data is a data type customized by the cloud according to the business logic.

[0027] A second aspect of an embodiment of the present application provides a cloud-edge interactive operation power dispatching system based on a lightweight time series database, including:

[0028] An initial data transmission unit is used for the terminal device to transmit the collected initial data to the edge end through the communication protocol;

[0029] An initial data encoding unit, used for the edge end to encode the received initial data;

[0030] A time series data identification unit, used for the edge end to identify the encoded time series data using a sliding window algorithm, filter the identified abnormal data, and compress the identified normal data;

[0031] The time series database generation unit is used for the edge end to decode the compressed data to generate a time series database, and synchronize data with the cloud based on the time series database, so that the cloud can complete task scheduling according to business needs.

[0032] A third aspect of the embodiments of the present application provides a computer device, including:

[0033] memories, transceivers, processors, and bus systems;

[0034] Wherein, the memory is used to store programs;

[0035] The processor is used to execute the program in the memory, including executing the methods of the above aspects;

[0036] The bus system is used to connect the memory and the processor so that the memory and the processor can communicate with each other.

[0037] A fourth aspect of an embodiment of the present application provides a computer-readable storage medium, comprising instructions, which, when executed on a computer, enables the computer to execute the methods provided in the above aspects.

[0038] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0039] In this application, the terminal device transmits the collected initial data to the edge through the communication protocol, and the edge encodes the received initial data, and then uses the sliding window algorithm to identify the encoded time series data, filters the identified abnormal data, and compresses the identified normal data. Finally, the edge decodes the compressed data to generate a time series database, and synchronizes data with the cloud based on the time series database, so that the cloud can complete task scheduling according to business needs. This application divides the abnormal data and normal data in the time series data through a sliding window algorithm, and further compresses the normal data after division, and generates a time series database based on the compressed data. Data scheduling is performed directly through the time series database during the cloud-edge interaction process, which improves the data processing efficiency of the cloud-edge interaction process, thereby improving the cloud-edge interaction data scheduling efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 A schematic diagram of an embodiment of a cloud-edge interactive operation power dispatching method based on a lightweight time series database in the present invention;

[0041] Figure 2 It is a flowchart of another embodiment of a cloud-edge interactive operation power dispatching method based on a lightweight time series database in the present invention;

[0042] Figure 3 It is a structural block diagram of an embodiment of a cloud-edge interactive operation power dispatching system based on a lightweight time series database in the present invention;

[0043] Figure 4 The figure is a structural block diagram of a computer device in the present invention. DETAILED DESCRIPTION

[0044] The embodiments of the present application provide a cloud-edge interactive power dispatching method, system, device and medium based on a lightweight time series database, which solves the problem of low efficiency of system scheduling data during cloud-edge interaction.

[0045] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein, for example. In addition, the terms "including" and "corresponding to" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0046] Embodiment 1

[0047] The following will introduce the cloud-edge interactive operation power dispatching method based on lightweight time series database in this application from the perspective of data interaction between terminal devices, edge terminals and cloud terminals. Figure 1 , the method provided in the embodiment of the present application comprises the following steps:

[0048] S11. The terminal device transmits the collected initial data to the edge end through the communication protocol;

[0049] In this embodiment, the terminal device refers to an external field device, such as an IoT sensor device and a gateway device, etc., which collects initial time series data in real time through the sensor field. The initial data here refers to different data forms and structures according to different fields. After collecting the data, the terminal device uses different communication protocols, such as MQ, MQTT, AMQP, HTTP, TCP, HJT212 and NB, etc., to access the edge end through data transmission, so that the edge end performs subsequent data processing. By setting the corresponding communication protocol, the security of data transmission between the terminal external field device and the edge end can be improved.

[0050] S12. The edge end encodes the received initial data;

[0051] In this embodiment, the edge end encodes the received initial data by:

[0052] The encoder is connected to determine whether the initial data conforms to the encoding specification. If so, verification calculation and escape operation are performed. After the operation is completed, the time series data is returned to the edge end.

[0053] Specifically, an encoder is connected to the edge, where the type of encoder is not limited, and the encoding template of the encoder is set according to the timing data. During the encoding process, the interface is first set at the edge, and the encoder determines whether the received timing data is serializable, and then creates a ByteBuf instance to store the encoder data, obtains a specific instance and determines whether the length meets the encoding specification. If the protocol ID, command word and content all meet the specification, a crc check is performed, and the timing data is written to the ByteBuf instance through a specific instance, and finally a specific interface is called to send the ByteBuf. If the buffer does not contain bytes that can be written, the encoded ByteBuf is released, and an empty ByteBuf is written to the specific interface. After the sending operation is completed, the ByteBuf object located in the encoding buffer needs to be released. In this way, the timing data is encoded to facilitate further processing of the data.

[0054] S13. The edge uses a sliding window algorithm to identify the encoded time series data, filters the identified abnormal data, and compresses the identified normal data;

[0055] In this embodiment, the edge uses a sliding window algorithm to identify abnormal data and normal data in the above-mentioned encoded time series. The abnormal data here refers to the data generated by the terminal device due to sudden changes in the external environment during the data collection process. When abnormal data is identified, the edge system marks the abnormal data, and the edge discards the abnormal data, and sets the number of abnormal warning conditions. When the marked and discarded abnormal data reaches a certain preset number of times within a certain period of time, it means that the system or related field equipment has a fault. The edge will then feed back the alarm information to the terminal device, so that the terminal device can perform further detection and operation and maintenance based on the alarm information.

[0056] Specifically, time series data generally has a large amount of data, a small range of value changes, and a low unit value. Here, the time series data is compressed according to the data amount and value changes. The specific expression is as follows:

[0057]

[0058] Where A is the performance index for normal data compression, X is the normal data sequence, λ is the data compression amount, G is the performance parameter, τ is the normal data sample regression parameter, d maxis the maximum value of normal time series data, d min This is the minimum value of normal time series data.

[0059] In addition, compression conditions are set during the data compression process, a compression threshold range and a timestamp range of the received data points are set, and the threshold range is set based on the first received data point. When the received data compression amount is greater than or equal to the threshold range, there is no time limit here, and the corresponding performance indicator compression is calculated according to the above formula; if the compression amount is less than the threshold range, it is necessary to further determine whether the time from the receipt of the data point to the previous data point exceeds the set timestamp range. When the timestamp exceeds the set range, data compression is also required.

[0060] S14. The edge decodes the compressed data to generate a time series database, and synchronizes data with the cloud based on the time series database, so that the cloud can complete task scheduling according to business needs.

[0061] In this embodiment, data synchronization between the time series database and the cloud includes:

[0062] The cloud sets up an interface based on the configuration of the time series database, and the edge synchronizes data through the interface. The data is the data type customized by the cloud based on the business logic.

[0063] Specifically, the compressed time series data is decoded, wherein the decoding calls the decoder in the same way. The decoding operation process is similar to the encoding process, which will not be described here. When the decoding is completed, a time series database is generated based on the time series data generated by the decoding. In practical applications, there are two ways of cloud-edge interaction scheduling: bottom-up and top-down. Here, the edge end directly synchronizes with the cloud when generating the time series database. When the cloud needs to allocate data to the edge end for data processing, the time series database is also applicable to the data processing of this method. After the allocated data is subjected to the above data identification, abnormal data filtering, and normal data compression, the corresponding time series database is generated. However, the time series database here is different from the time series database generated from the terminal device, and its generation method is the same. During the cloud-edge interaction process, the cloud sets the corresponding interface according to the configuration of the time series database. Through this interface, the cloud end synchronizes the data of the time series database with the edge end. The cloud end can retrieve the data of the time series database according to the data type set by the specific business logic.

[0064] By building a time series database on the edge, an editing and query function is set up in the database. When the edge interacts with the cloud, the cloud can directly call the data in the database, or the edge can directly perform data scheduling, data storage and data query through the database, which can avoid the trouble of data processing on the edge and effectively improve the efficiency of data transmission between the cloud and the edge.

[0065] Embodiment 2

[0066] See also Figure 2 In this application, the edge end uses a sliding window algorithm to identify the encoded time series data, and the following steps are also included:

[0067] S131. Divide the time series data into several subsequences of preset length;

[0068] Specifically, after receiving the time series data, the edge end divides the data into several sub-sequences of preset length, where the preset length can be equal length or unequal length. Here, taking the equal length as an example, the time series with a time length of m can be expressed as X=(x(t1),x(t2),…,x(t m )), where x(t1) is the data received at time t1. Use a sliding window of length l (l<<m) to split the time series into equal lengths, then move back a step length r, slide continuously (m―l) / r times, and generate (m―l) / r+1 equal length subsequence segments.

[0069] S132. Calculate the slope between any two adjacent data points of the target subsequence in the plurality of subsequences, and calculate the confidence interval distance radius of the slope;

[0070] Set the target subsequence to a random subsequence among several subsequences, and set the subsequence to X j (1≤j≤(m―l) / r+1, then calculate the slope k between any two adjacent data points i , the expression is as follows:

[0071]

[0072] Where: X j (i) and X j (i―1) are two adjacent data points in the target subsequence, t i is the acquisition time of the target subsequence.

[0073] Calculate the confidence interval distance radius d for the slope j , the expression is as follows:

[0074]

[0075] Where: is the upper confidence limit, θ j is the lower confidence limit, is the mean of the slope of the jth subsequence, σ j is the mean square error of the slope of the jth subsequence, the random variable Z satisfies the normal distribution of N(0,1), α is the confidence level, and α=0.05 is taken.

[0076] S133. Determine whether the confidence interval distance radius of the slope is within the threshold range;

[0077] S134. When the confidence interval distance radius of the slope is within the threshold range, the target subsequence is determined to be abnormal time series data.

[0078] S135. When the confidence interval distance radius of the slope is not within the threshold range, the target subsequence is determined to be normal time series data.

[0079] The confidence interval radius of the slope calculated above is used to determine whether it is within the threshold range. γ is set as the threshold. If d j >γ, it means that the subsequence is an abnormal time series data sequence. If d j ≤γ, it means that the subsequence is a normal time series data sequence.

[0080] The above embodiment identifies the data type of the received data through a sliding window algorithm, provides an effective data basis for subsequent abnormal data filtering and alarming and normal data compression, and can improve the efficiency of data processing at the edge.

[0081] Embodiment 3

[0082] See also Figure 3 , an embodiment of a cloud-edge interactive operation power dispatching system based on a lightweight time series database in the present invention includes the following steps:

[0083] An initial data transmission unit 101 is used for the terminal device to transmit the collected initial data to the edge end through a communication protocol;

[0084] The initial data encoding unit 102 is used for the edge end to encode the received initial data;

[0085] The time series data identification unit 103 is used to identify the encoded time series data using a sliding window algorithm at the edge, filter the identified abnormal data, and compress the identified normal data;

[0086] The time series database generation unit 104 is used to decode the compressed data at the edge to generate a time series database, and synchronize data with the cloud based on the time series database, so that the cloud can complete task scheduling according to business needs.

[0087] For the specific definition of the system, please refer to the definition of the method above, which will not be repeated here. Each module in the above system can be implemented in whole or in part by software, hardware and a combination thereof. The above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0088] Embodiment 4

[0089] See also Figure 4 The present invention provides a computer device, including a memory, a processor, and computer-readable instructions stored in the memory and executable on the processor. When the processor executes the computer-readable instructions, the steps of the above method are implemented.

[0090] Those of ordinary skill in the art will appreciate that the units of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition of each example has been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0091] In the embodiments provided by the present invention, it should be understood that the division of units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units can be combined into one unit, one unit can be split into multiple units, or some features can be ignored. In addition, each functional unit in each embodiment of the present invention can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units.

[0092] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-0nlyMemory), random access memory (RAM, RandomAccessMemory), mobile hard disk, magnetic disk or optical disk, etc., which can store program code.

[0093] It can be understood that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein by equivalents. These modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention, and they should all be included in the scope of the claims and specification of the present invention.

Claims

1. A cloud-edge interactive operation power dispatching method based on a lightweight time series database, characterized in that: include: The terminal device transmits the collected initial data to the edge end through the communication protocol; The edge end performs data encoding on the received initial data; The edge end uses a sliding window algorithm to identify the encoded time series data, filters the identified abnormal data, and compresses the identified normal data; The edge decodes the compressed data to generate a time series database, and synchronizes data with the cloud based on the time series database, so that the cloud can complete task scheduling according to business needs.

2. The cloud-edge interactive operation power dispatching method based on a lightweight time series database according to claim 1 is characterized in that: The edge end performs data encoding on the received initial data, including: The encoder is connected, and the encoder determines whether the initial data conforms to the encoding specification. If so, verification calculation and escape operation are performed, and after the operation is completed, the time series data is returned to the edge end.

3. The cloud-edge interactive operation power dispatching method based on lightweight time series database according to claim 1 is characterized in that: The edge end uses a sliding window algorithm to identify the encoded time series data, including: Dividing the time series data into a plurality of subsequences of preset lengths; Calculating the slope between any two adjacent data points of the target subsequence in the plurality of subsequences, and calculating the confidence interval distance radius of the slope; It is determined whether the confidence interval distance radius of the slope is within a threshold range. If so, it is determined that the target subsequence is abnormal time series data; if not, it is determined that the target subsequence is normal time series data.

4. The cloud-edge interactive operation power dispatching method based on a lightweight time series database according to claim 3 is characterized in that: The expressions for calculating the slope between any two adjacent data points of the target subsequence in the plurality of subsequences are as follows: Where: k i is the slope, X j (i) and X j (i-1) are two adjacent data points in the target subsequence, t i is the acquisition time of the target subsequence.

5. The cloud-edge interactive operation power dispatching method based on a lightweight time series database according to claim 4 is characterized in that: The expression for the confidence interval distance radius of the slope calculation is as follows: Where: d j is the confidence interval distance radius, is the upper confidence limit, θ j is the lower confidence limit, is the mean of the slope of the jth subsequence, σ j is the mean square error of the slope of the jth subsequence, the random variable Z satisfies the normal distribution of N(0,1), α is the confidence level, and α=0.05 is taken.

6. The cloud-edge interactive operation power dispatching method based on lightweight time series database according to claim 1 is characterized in that: The expression for compressing the recognized normal data is as follows: Where A is the performance index for normal data compression, X is the normal data sequence, λ is the data compression amount, G is the performance parameter, τ is the normal data sample regression parameter, d max is the maximum value of normal time series data, d min This is the minimum value of normal time series data.

7. The cloud-edge interactive operation power dispatching method based on lightweight time series database according to claim 1 is characterized in that: The step of synchronizing data with the cloud based on the time series database includes: The cloud sets an interface according to the configuration of the time series database, and the edge synchronizes data through the interface. The data is a data type customized by the cloud according to the business logic.

8. A cloud-edge interactive power dispatching system based on a lightweight time series database, characterized in that: include: An initial data transmission unit is used for the terminal device to transmit the collected initial data to the edge end through the communication protocol; An initial data encoding unit, used for the edge end to encode the received initial data; A time series data identification unit, used for the edge end to identify the encoded time series data using a sliding window algorithm, filter the identified abnormal data, and compress the identified normal data; The time series database generation unit is used for the edge end to decode the compressed data to generate a time series database, and synchronize data with the cloud based on the time series database, so that the cloud can complete task scheduling according to business needs.

9. A computer device, characterized in that: include: memories, transceivers, processors, and bus systems; Wherein, the memory is used to store programs; The processor is used to execute the program in the memory, including executing the cloud-edge interactive operation power dispatching method based on a lightweight time series database according to any one of claims 1 to 7; The bus system is used to connect the memory and the processor so that the memory and the processor can communicate with each other.

10. A readable storage medium, characterized in that: It includes instructions, which, when running on a computer, enable the computer to execute the cloud-edge interactive power dispatching method based on a lightweight time series database as described in any one of claims 1 to 7.

Images