Data processing method, device, apparatus, storage medium, program product and system

Abstract

The application provides a data processing method, device, equipment, storage medium, program product and system. The method comprises: generating compression parameters of each edge device according to system environment parameters of the edge device; sending the compression parameters to the edge device, so that the edge device compresses original data collected based on the compression parameters to obtain compressed data; and receiving the compressed data uploaded by the edge device. In the embodiment of the application, the compression parameters of each edge device are generated according to the system environment parameters of the edge device, and the edge device compresses the collected data according to the corresponding compression parameters, so that the data compression can be performed based on the actual situation, and the data transmission overhead and distortion rate are reduced.

Description

Technical Field

[0001] This application relates to the field of Internet of Things (IoT) technology, and more specifically, to a data processing method, apparatus, device, storage medium, program product, and system. Background Technology

[0002] With the development of the global green economy, the demand for green energy is constantly expanding both domestically and internationally, and more and more enterprises are beginning to demand green and low-carbon energy. Zero-carbon industrial park planning and design can effectively promote the development of energy systems towards cleaner energy production, more efficient energy allocation, electrified end-consumer energy, and flexible grid-load interaction, thus driving cities to gradually evolve from "high-carbon" to "low-carbon" and "zero-carbon".

[0003] The zero-carbon park includes multiple edge devices, whose data is sent to a cloud platform for unified management. To ensure the integrity and accuracy of the edge device data and enable more accurate cloud platform monitoring, the current solution requires edge devices to report collected data frame by frame. This results in excessive communication overhead between the edge devices and the cloud platform, easily causing communication link congestion. Compressing the collected data using random sampling would lead to data distortion. Summary of the Invention

[0004] The purpose of this application is to provide a data processing method, apparatus, device, storage medium, program product, and system to reduce communication overhead between edge devices and digital energy management platforms, and to reduce data distortion rate.

[0005] In a first aspect, embodiments of this application provide a data processing method, which can be applied to a digital energy management platform, the method comprising:

[0006] The compression parameters for each edge device are generated based on the system environment parameters of the edge devices; the compression parameters are sent to the edge devices so that the edge devices can compress the collected raw data based on the compression parameters to obtain compressed data;

[0007] Receive compressed data uploaded by edge devices.

[0008] In this embodiment, the compression parameters for each edge device are generated based on the system environment parameters of the edge device. The edge device compresses the collected data according to the corresponding compression parameters, thereby compressing the data based on the actual situation, reducing data transmission overhead and distortion rate.

[0009] In any embodiment, system environment parameters include the topology of edge devices, and compression parameters include a sparse matrix; generating the sparse matrix in the compression parameters of each edge device based on the system environment parameters of the edge devices includes:

[0010] Based on the topological relationships, obtain the relevant variables between the data collected by the edge devices;

[0011] Determine the correlation between data based on relevant variables;

[0012] Construct a sparse matrix based on correlation.

[0013] This application embodiment constructs a sparse matrix by utilizing the correlation between data collected by edge devices, eliminates invalid data, achieves efficient data compression, reduces communication overhead, and improves the accuracy of data recovery on the digital energy management platform.

[0014] In any embodiment, system environment parameters include the topology of edge devices and network environment parameters; compression parameters include a measurement matrix; generating the measurement matrix in the compression parameters of each edge device based on the system environment parameters of the edge devices includes:

[0015] The first objective function is constructed based on the topology of the edge devices, network environment parameters, and measurement functions.

[0016] The first objective function is solved using a preset optimization algorithm to obtain the sampling rate corresponding to each edge device;

[0017] The measurement matrix is ​​determined based on the sampling rate and the sparse matrix; wherein each row of the measurement matrix includes a non-zero element.

[0018] The embodiments of this application construct the measurement matrix in the form of the sparsest matrix. By having each row of the measurement matrix contain only one non-zero element, the complexity of the matrix is ​​reduced, which reduces communication overhead while maintaining the accuracy of data recovery.

[0019] In any embodiment, the measurement function includes a first communication load function, a first energy consumption cost function, and a first data value function; a first objective function is constructed based on the topology of the edge device, network environment measurement, and the measurement function, including:

[0020] The first communication load function is constructed based on the data transmission volume under the predicted sampling rate, the current network bandwidth, and the time weighting coefficient.

[0021] The first energy cost function is constructed based on the equipment energy consumption, energy price, and electricity price fluctuation coefficient under the predicted sampling rate.

[0022] The first data value function is constructed based on the predicted data change amount, the predicted change weight coefficient, and the sampling rate penalty coefficient;

[0023] The first objective function is generated based on the first communication load function, the first energy consumption cost function, and the first data value function.

[0024] The embodiments of this application determine a first objective function through a first communication load function, a first energy consumption cost function, and a first data value function. By obtaining the optimal first objective function, a sampling rate is determined. When data compression is performed using this sampling rate, the communication overhead of data transmission is reduced on the one hand, and the accuracy of data recovery is improved on the other hand.

[0025] In any embodiment, the compression parameters further include a sampling frequency; the sampling frequency is determined by the following method:

[0026] A second objective function is constructed based on the topology of edge devices, network environment parameters, and measurement functions.

[0027] The second objective function is solved using a preset optimization algorithm to obtain the sampling frequency corresponding to each edge device.

[0028] This application embodiment constructs a second objective function and solves the second objective function to obtain the sampling frequency, so that the edge device collects data according to the sampling frequency. By reasonably setting the sampling frequency, the monitoring efficiency is improved and the energy consumption is reduced while satisfying the effective monitoring of the edge device.

[0029] In any embodiment, the measurement function includes a second communication load function, a second energy consumption cost function, and a second data value function; the second objective function is constructed based on the topology of the edge device, network environment measurement, and the measurement function, including:

[0030] A second communication load function is constructed based on the communication load and average communication of each edge device at different time periods;

[0031] A second energy consumption cost function is constructed based on the energy consumption of each edge device at the predicted sampling frequency.

[0032] A second data value function is constructed based on the importance and sampling frequency of the data collected by each edge device within a preset time period;

[0033] The second objective function is generated based on the second communication load function, the second energy consumption cost function, and the second data value function.

[0034] In this embodiment, a second objective function is determined by a second communication load function, a second energy consumption cost function, and a second data value function. The sampling frequency is determined by obtaining the optimal second objective function. When data is collected at this sampling frequency, the communication overhead of data transmission is reduced on the one hand, and the accuracy of data recovery is improved on the other hand.

[0035] In any embodiment, the compression parameters include a sparse matrix and a measurement matrix, and after receiving the compressed data, the method further includes:

[0036] The compressed data is recovered based on the sparse matrix and the measurement matrix to obtain the recovered data.

[0037] The embodiments of this application determine the sparse solution through the sparse matrix and the measurement matrix, and complete the recovery of the compressed data based on the sparse solution reconstruction algorithm, thereby obtaining the accuracy of the recovered data.

[0038] In any embodiment, the method further includes:

[0039] The collected monitoring parameters are input into the environmental perception model to obtain prediction results;

[0040] If the prediction results meet the update strategy conditions, the compression parameters are regenerated and sent to the edge devices.

[0041] This application embodiment dynamically adjusts compression parameters through environmental perception, enabling edge devices to collect and compress data appropriately. This reduces data transmission overhead and the distortion rate of compressed data.

[0042] Secondly, embodiments of this application provide another data processing method applied to an edge device, the method comprising:

[0043] Receive compression parameters from the digital energy management platform; the compression parameters are generated by the digital energy management platform based on system environment parameters.

[0044] The collected raw data is compressed according to the compression parameters to obtain compressed data;

[0045] Upload compressed data to the digital energy management platform.

[0046] This application embodiment receives compression parameters issued by a digital energy management platform and collects and compresses data according to these parameters. Since the compression parameters are generated by the digital energy management platform based on system environment parameters, data compression can be performed based on actual conditions, reducing data transmission overhead and distortion rate.

[0047] In any embodiment, the compression parameters include a sparse matrix and a measurement matrix;

[0048] The collected raw data is compressed according to the compression parameters, including:

[0049] The collected raw data is compressed based on the sparse matrix and the measurement matrix.

[0050] The embodiments of this application construct a sparse matrix and a measurement matrix based on the correlation of data collected by edge devices. On the one hand, this achieves compression of the original data and reduces the overhead of data transmission. On the other hand, the compressed data can be accurately recovered by the digital energy management platform.

[0051] In any embodiment, the compression parameters further include a sampling frequency; before compressing the acquired raw data according to the sparse matrix and the measurement matrix, the method further includes:

[0052] Raw data is collected based on the sampling frequency.

[0053] This application embodiment collects raw data according to the sampling frequency, thereby allowing the collection to be performed according to actual needs and thus collecting more useful data.

[0054] Thirdly, embodiments of this application provide a data processing apparatus, including:

[0055] The strategy generation module is used to generate compression parameters for each edge device based on the system environment parameters of the edge devices.

[0056] The strategy delivery module is used to send compression parameters to edge devices so that the edge devices can compress the collected raw data based on the compression parameters to obtain compressed data.

[0057] The data receiving module is used to receive compressed data uploaded by edge devices.

[0058] Fourthly, embodiments of this application provide an electronic device, including: a processor, a memory, and a bus, wherein,

[0059] The processor and the memory communicate with each other via the bus;

[0060] The memory stores program instructions that can be executed by the processor, and the processor can execute the method of the first aspect by calling the program instructions.

[0061] Fifthly, embodiments of this application provide a non-transitory computer-readable storage medium, comprising:

[0062] The non-transitory computer-readable storage medium stores computer instructions that cause the computer to perform the method of the first aspect.

[0063] In a sixth aspect, embodiments of this application provide a computer program product, including computer program instructions, which, when read and executed by a processor, perform the method of the first aspect.

[0064] In a seventh aspect, embodiments of this application provide a data processing system, including a digital energy management platform and an edge device; the digital energy management platform is communicatively connected to the edge device;

[0065] The digital energy management platform is used to perform the method described in the first aspect;

[0066] The edge device is used to perform the method described in the second aspect.

[0067] Other features and advantages of this application will be set forth in the following description and will be apparent in part from the description or may be learned by practicing embodiments of this application. The objectives and other advantages of this application may be realized and obtained by means of the structures particularly pointed out in the written description, claims, and drawings. Attached Figure Description

[0068] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0069] Figure 1 This is a schematic flowchart of a data processing method provided in an embodiment of this application;

[0070] Figure 2 This is a schematic diagram of another data processing method provided in an embodiment of this application;

[0071] Figure 3 This is a schematic diagram of a data processing device structure provided in an embodiment of this application;

[0072] Figure 4 This is a schematic diagram of the physical structure of an electronic device provided in an embodiment of this application;

[0073] Figure 5 This application provides a schematic diagram of a data processing system structure.

[0074] Figure 6 A schematic diagram of a data processing system network architecture provided in this application embodiment;

[0075] Figure 7 A topology diagram of a zero-carbon park scenario provided in this application embodiment;

[0076] Figure 8 This is a signaling interaction diagram for zero-carbon park data processing provided in an embodiment of this application. Detailed Implementation

[0077] The embodiments of the technical solution of this application will now be described in detail with reference to the accompanying drawings. These embodiments are only used to more clearly illustrate the technical solution of this application and are therefore merely examples, and should not be used to limit the scope of protection of this application.

[0078] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the application; the terms “comprising” and “having”, and any variations thereof, in the specification, claims, and foregoing description of the drawings are intended to cover non-exclusive inclusion.

[0079] In the description of the embodiments of this application, technical terms such as "first" and "second" are used only to distinguish different objects and should not be construed as indicating or implying relative importance or implicitly specifying the number, specific order, or primary and secondary relationship of the indicated technical features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly defined.

[0080] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0081] In the description of the embodiments in this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship.

[0082] In the description of the embodiments of this application, the term "multiple" refers to two or more (including two), similarly, "multiple sets" refers to two or more (including two sets), and "multiple pieces" refers to two or more (including two pieces).

[0083] In the description of the embodiments of this application, unless otherwise expressly specified and limited, technical terms such as "installation," "connection," "joining," and "fixing" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. For those skilled in the art, the specific meaning of the above terms in the embodiments of this application can be understood according to the specific circumstances.

[0084] In the Internet of Things (IoT) field, there are two main approaches to processing data collected by edge devices: one is for each edge device to process its own data, and the other is to send the collected data to a cloud platform for unified processing. The first approach avoids communication overhead, but applying the data locally can lead to data inconsistencies. For the second approach, existing data transmission methods include edge devices sending all collected data to the cloud platform, or using random sampling to report the data. Sending all data results in significant communication overhead during data transmission, easily causing network congestion. Random sampling leads to data distortion for the cloud platform.

[0085] To address the aforementioned technical problems, embodiments of this application provide a data processing method, apparatus, device, storage medium, program product, and system. By setting compression parameters for each edge device, the edge devices collect and compress data according to the corresponding compression parameters, thereby reducing communication overhead during data transmission and lowering the distortion rate of the compressed data.

[0086] It is understood that the data processing method provided in this application embodiment can be applied to data processing between various edge devices and the cloud platform in a zero-carbon park, and can also be used for data processing in a cloud-based battery management system. It can also be applied to other scenarios where data needs to be compressed before transmission; this application embodiment does not limit specific application scenarios.

[0087] Figure 1 This is a schematic flowchart of a data processing method provided in an embodiment of this application, such as... Figure 1 As shown, this method can be applied to a digital energy management platform, which can run in the cloud, at the edge, or on a terminal, or integrate modules that include different functions of the cloud, edge, and terminal. The method includes:

[0088] Step 101: Generate compression parameters for each edge device based on the system environment parameters of the edge devices;

[0089] Step 102: Send compression parameters to the edge device so that the edge device can compress the collected raw data based on the compression parameters to obtain compressed data;

[0090] Step 103: Receive compressed data uploaded by the edge device.

[0091] In practical implementation, edge devices refer to physical or virtual devices located at the network edge, close to the data source or end user, and capable of generating data. For zero-carbon parks for new energy, edge devices can include at least one of the following: energy storage devices, photovoltaic devices, load units (including various electrical devices such as lighting equipment), and charging devices.

[0092] System environment parameters refer to the relevant parameters of a system comprised of multiple edge devices and a digital energy management platform. These parameters include at least one of the following: the topology between the edge devices, network environment parameters, device performance, and remaining power. Different edge devices have different system environment parameters.

[0093] Compression parameters refer to the parameters used by edge devices to compress the raw data they collect. Each edge device corresponds to one compression parameter, which is generated by the digital energy management platform based on system environment parameters. After generating the corresponding compression parameters for each edge device, the digital energy management platform sends the compression parameters to the respective edge device. Upon receiving the compression parameters, the edge device collects and compresses the data according to the parameters. It should be noted that different edge devices may collect different data. For example, energy storage devices collect data including the temperature, voltage, and current of each battery cell. Photovoltaic devices collect data including the electricity generated by each solar photovoltaic panel.

[0094] After receiving the compressed data, the edge device sends the compressed data to the digital energy management platform.

[0095] In this embodiment, the compression parameters for each edge device are generated based on the system environment parameters of the edge device. The edge device compresses the collected data according to the corresponding compression parameters, thereby compressing the data based on the actual situation, reducing data transmission overhead and distortion rate.

[0096] Based on the above embodiments, the system environment parameters include the topology of the edge devices, and the compression parameters include a sparse matrix; generating the sparse matrix in the compression parameters of each edge device according to the system environment parameters of the edge devices includes:

[0097] Based on the topological relationships, obtain the relevant variables between the data collected by the edge devices;

[0098] Determine the correlation between data based on relevant variables;

[0099] Construct a sparse matrix according to the relevance.

[0100] In a specific implementation process, the topological relationship of edge devices refers to the connection relationship of each component in the edge devices. Taking an energy storage battery as an edge device as an example: its topological relationship is used to characterize that the energy storage battery includes 1 battery cabinet, a battery cluster with multiple battery packs, each battery pack includes multiple battery modules, and each battery module includes multiple battery cells. And the connection relationships between multiple battery modules and between multiple battery cells, etc. In addition, the energy storage battery may further include a battery management system, an energy storage inverter, an energy storage meter, as well as corresponding air-cooling equipment and fire-fighting equipment. Taking a photovoltaic device as an edge device as another example, its topological relationship is used to characterize that the photovoltaic device includes multiple solar photovoltaic panels, and the connection relationship between multiple solar photovoltaic panels. The photovoltaic device may also include a photovoltaic inverter, a photovoltaic meter, a supporting meteorological station, a distribution box and other equipment.

[0101] For the sake of easy understanding, first introduce the compressive sensing technology:

[0102] Compressive sensing is an innovative data acquisition and compression technology, which is significantly different from the traditional Nyquist sampling theory. It is based on a core concept: there is often redundancy in actual data, and its essential features can be efficiently captured through sparse representation.

[0103] For a series of discrete device monitoring data x[n], 1 ≤ n ≤ N, assume that there exists a sparse representation basis Ψ N×N , which can convert the measured data vector into a sparse vector s that only contains K non-zero elements, expressed as s = Ψ - 1 x. The measured data x that satisfies this characteristic can be called a K-order sparse signal.

[0104] Within the framework of compressive sensing theory, the sparse representation s corresponding to an N-dimensional K-order sparse signal x can be accurately recovered with a high probability from the M (M < N)-dimensional random sampling values y of x in the space R N , and this process can be expressed by the formula y = Φx = ΦΨ s . Where y is the M-dimensional random sampling value, and Φ is an M × N random measurement matrix. At this time, the reconstruction of the original measured data will be transformed into solving the optimization problem of minimizing the l1 - norm, and the optimization objective is expressed as:

[0105]

[0106] The accuracy and stability of data reconstruction depend on whether the measurement matrix satisfies the Restricted Isometric Property (RIP) condition. This is represented as sparse data s∈R. N If ||S|| = 0, then there exists δ. k ∈(0,1), such that the measurement matrix R M×N satisfy Then the measurement matrix Φ is said to have a RIP condition of order K.

[0107] Compression parameters include sparse matrices, which are matrices where most elements are zero and only a few are non-zero. In compressed sensing, sparse matrices are typically used to represent the sparsity of a signal under a specific basis. A sparse matrix reflects the sparsity characteristics of a signal, meaning that only a few coefficients are significant in a certain transform domain (such as the frequency domain or wavelet domain), while most coefficients are close to zero. This sparsity allows signals to be compressed and sampled at much lower rates than traditional sampling methods, while still preserving the original structure and information of the signal.

[0108] For edge devices whose collected data is inherently sparse, their compression parameters can contain only a sparse matrix. After sparsification using the sparse matrix, compression can be performed.

[0109] In zero-carbon park scenarios, IoT device monitoring data typically exhibits significant spatiotemporal correlations. This application's embodiments capture the correlations between data collected by edge devices based on a Gaussian joint distribution and represent them sparsely using correlation coefficients. It should be noted that the correlation between data can characterize the correlation between data collected by the same edge device at different time points, and also the correlation between data collected by different edge devices at the same time point.

[0110] Specifically, this application's embodiments employ a Gaussian distribution joint model and utilize kernel functions to describe the correlation between data from edge devices. For example: for x i and x j The correlation between them can be expressed as: Among them, d i,j The can be used to represent x i and x jThe relevant physical quantities include, for example, the series-parallel relationship between two battery cells, and the spatial distance between two photovoltaic panels; σ is a hyperparameter called bandwidth, which controls the "width" of the Gaussian kernel function and determines the rate at which the function value changes with the distance between data points. When the bandwidth is small, the "width" of the Gaussian kernel function is narrow, and only data points that are very close to each other will have a high similarity measure value. In this case, the function is very sensitive to changes in the distance between data points. Conversely, when the bandwidth is large, the "width" of the function is wide, and data points that are far apart may also have a relatively high similarity measure value. The function is relatively less sensitive to changes in distance.

[0111] The solution process for σ is as follows: (taking the maximum likelihood function as an example)

[0112] Determine the likelihood function: Suppose we have a set of training data (x1, x2, ..., x...). n Models built upon Gaussian kernel functions (such as SVM) provide certain probability distribution assumptions about these data (e.g., in classification problems, the probability of data belonging to each category). Taking Gaussian process regression as an example, suppose the observed data y = [y1, y2, ..., y...]. n ] T It follows a Gaussian distribution with mean μ = [μ1, μ2, ..., μ] n ] T The elements of the covariance matrix ∑ ij K(x i ,x j The likelihood function L(σ;y) is the probability of observing data y given the hyperparameter σ. According to the probability density function formula of the Gaussian distribution, the likelihood function can be expressed as:

[0113]

[0114] Differentiating the likelihood function and setting it to zero: To find the value of σ that maximizes the likelihood function, differentiate the likelihood function L(σ; y) with respect to σ. After a series of differentiation operations, setting the derivative to zero yields an equation in σ.

[0115] Solving the equations yields the estimates: Solving the equations about σ obtained above yields the maximum likelihood estimates of the hyperparameters. In practice, since the equations may be complex, numerical methods (such as Newton's method, gradient descent, etc.) may be needed to solve them.

[0116] After obtaining the correlation, a sparse matrix can be constructed based on the correlation. Each element in this sparse matrix reflects the correlation strength between two corresponding data points, and can be represented as follows:

[0117]

[0118] To obtain a more efficient data representation, the correlation matrix G can be diagonalized, represented as G = ψΛψ -1 Here, Ψ is an orthogonal matrix called the eigenvalue matrix, containing the orthogonal eigenvectors of matrix G as column vectors; Λ is a diagonal matrix, whose diagonal values ​​are the eigenvalues ​​of matrix G. These eigenvalues ​​represent the variance contribution of the data under the orthogonal eigenvalue basis. The eigenvalues ​​are arranged in descending order, reflecting the degree of contribution of different features to data variation. This sparse representation not only captures the essential characteristics of the data but also significantly reduces the dimensionality of the data representation, thereby reducing the communication resources required during data acquisition and transmission.

[0119] This application embodiment constructs a sparse matrix by utilizing the correlation between data collected by edge devices, eliminates invalid data, achieves efficient data compression, reduces communication overhead, and improves the accuracy of data recovery on the digital energy management platform.

[0120] Based on the above embodiments, the system environment parameters include the topology of the edge devices and network environment parameters; the compression parameters include a measurement matrix; generating the measurement matrix in the compression parameters of each edge device according to the system environment parameters of the edge devices includes:

[0121] The first objective function is constructed based on the topology of the edge devices, network environment parameters, and measurement functions.

[0122] The first objective function is solved using a preset optimization algorithm to obtain the sampling rate corresponding to each edge device;

[0123] The measurement matrix is ​​determined based on the sampling rate and the sparse matrix; wherein each row of the measurement matrix includes a non-zero element.

[0124] In the specific implementation process, network environment parameters include the maximum communication load that the total bandwidth of the entire system can handle, the current network bandwidth usage, etc.

[0125] The measurement matrix (also known as the sampling matrix) is a key tool in compressed sensing technology for achieving signal compression sampling. It is a matrix, typically non-square, used to map signals from a high-dimensional space to a low-dimensional space, generating observation vectors. If the data collected by the edge device is sufficiently sparse, then sparsity processing is unnecessary, and compression can be performed directly using the measurement matrix.

[0126] When constructing the measurement matrix, a first objective function is first built based on the topology of the edge devices, network environment parameters, and the measurement function. The measurement function is determined according to actual needs, i.e., based on the main factors of concern at present, such as communication load, energy consumption cost, and data value. This embodiment uses a measurement function including communication load, energy consumption cost, and data value as an example. The first objective function can be expressed as: min r (ω1C r +ω2E r +ω3V r ), Where ω1, ω2, and ω3 are weighting coefficients, r is a sampling rate vector representing the sampling rate of different edge devices; C r E is the first communication load function, measuring the impact of the sampling rate on network load. r The first energy cost function reflects the impact of the sampling rate on energy consumption; V r The first data value function is the sampling importance based on environmental data change prediction; r min The minimum sampling rate; r max C is the maximum value of the sampling rate. max E represents the maximum communication load that the system's total bandwidth can handle; max This refers to the maximum energy consumption that can be provided or allocated based on the current energy consumption situation within the park.

[0127] The first communication load function aims to minimize peak network load and ensure smooth and stable data transmission. This function considers the following factors:

[0128] Data transmission volume: The amount of data generated by each device is calculated based on the sampling rate r.

[0129] Network bandwidth usage: Monitor current network bandwidth usage and predict future bandwidth demand.

[0130] Time weighting: During peak network usage periods, the weight of communication load is increased to reduce data transmission during these periods. Represented as: C r =∑ i (r i ·d i +αB t ), where r i d is the sampling rate of the i-th edge device; i The amount of data generated per unit of time; α is the time weighting coefficient; B t This represents the network bandwidth usage at time t.

[0131] The first energy cost function is used to evaluate energy consumption under different compression parameters and to find the sampling configuration with the lowest energy consumption. This function considers the following factors:

[0132] Equipment energy consumption model: Calculate equipment energy consumption based on sampling rate.

[0133] Energy price fluctuations: Reduce the sampling rate during periods of high electricity prices to decrease energy costs.

[0134] Energy supply status: Increase the sampling rate when energy supply is sufficient and decrease the sampling rate when supply is tight. Represented as: E r =∑ i (r i ·c i +βP t ), where r i c is the sampling rate of the i-th edge device; i The energy consumption of the i-th device is sparse; β is the electricity price fluctuation coefficient; P t Let t be the energy price at time t.

[0135] The first data value function is used to assess the importance of collected data to support future decision-making, encouraging a higher sampling rate when data value is high. This function considers the following factors:

[0136] Environmental prediction model: Using time series prediction models to predict the changing trends of environmental data.

[0137] Event identification: Identify upcoming events or anomalies and predict their impact on data value.

[0138] Historical data analysis: Analyze historical data to determine the correlation between data changes and value.

[0139] The first data value function can be expressed as: V r =∑ i (γΔ it -δ·r i ), where r i γ is the sampling rate of the i-th edge device; γ is the weighting coefficient for the predicted change; Δ it Let be the predicted data change of the i-th device at time t; δ is the penalty coefficient for the sampling rate to balance the sampling frequency and data value.

[0140] By using the aforementioned first communication load function, first energy consumption cost function, and first data value function, a convex optimization problem that comprehensively considers communication load, energy consumption, and data value can be constructed. Specifically, it can be solved using a Markov decision model or a reinforcement learning model. The sampling rate can be obtained by calculating the optimal solution of the first objective function. A measurement matrix is ​​then constructed based on the sampling rate.

[0141] In traditional data recovery methods, to achieve recovery accuracy, each row of the sparse random projection matrix is ​​typically required to have at least 10N non-zero elements. While this method can meet the accuracy requirements, its savings in communication overhead are often limited because it does not fully utilize the sparsity of the data. In contrast, this application's embodiment constructs the measurement matrix Φ using the sparsest matrix form, meaning each row of the measurement matrix contains only one non-zero element. The value of each element in the measurement matrix Φ is determined as follows:

[0142]

[0143] Where, r i Let r be an independent random distribution index that satisfies r i <r i+1 ,r i ∈[1,N], thus, the compressed sensing random sampling vector x k It can be represented as:

[0144]

[0145] Where x1,x2,...,x N This is the original data. The embodiments of this application take into account the spatiotemporal correlation and inherent sparsity of the data collected by edge devices, thereby constructing the sparsest matrix as the measurement matrix. This not only maintains high data accuracy but also reduces communication overhead.

[0146] In another embodiment, if the data collected by the edge device is not sparse enough, the compression parameters may include a sparse matrix and a measurement matrix. It is understood that the construction methods of the sparse matrix and measurement matrix are as described in the above embodiments. After obtaining the compression parameters, the edge device first performs sparsity processing on the collected raw data using the sparse matrix, and then compresses the sparsed data using the measurement matrix to obtain compressed data.

[0147] Based on the above embodiments, the compression parameters also include the sampling frequency; the sampling frequency is determined by the following method:

[0148] A second objective function is constructed based on the topology of edge devices, network environment calculations, and calculation functions.

[0149] The second objective function is solved using a preset optimization algorithm to obtain the sampling frequency corresponding to each edge device.

[0150] In practical implementation, sampling frequency refers to the frequency at which edge devices collect data. The sampling frequency affects communication bandwidth usage and energy consumption. Furthermore, in certain critical scenarios, the sampling frequency can be appropriately increased. For example, for photovoltaic (PV) equipment, sunlight is strongest in the mid-afternoon, resulting in higher electricity generation. Therefore, the sampling frequency can be increased during this period, while it can be appropriately decreased during other times when sunlight is weaker.

[0151] When determining the sampling frequency, various factors can be considered comprehensively. A second objective function can be constructed based on the topology of the edge devices, network environment parameters, and measurement functions. Then, an optimization algorithm can be used to solve the second objective function to obtain the sampling frequency corresponding to each edge device.

[0152] The calculation function can include functions corresponding to various factors, such as: the second communication load function, the second energy consumption cost function, and the second data value function.

[0153] The second communication load function refers to the degree of communication load balancing between each edge device and the digital energy management platform. To avoid some communication links being overloaded while others are idle, the communication load from each edge device to the digital energy management platform should be as balanced as possible across different time periods. The degree of communication load balancing can be measured in the following ways:

[0154]

[0155] in, J1 represents the average communication load of all edge devices within time period j. The goal is to minimize J1, that is, to minimize the sum of squared deviations between the communication load of each device and the average communication load in different time periods, in order to achieve communication load balancing.

[0156] The second energy consumption cost function refers to the total energy consumption required for all edge devices to collect data at a given sampling frequency. Considering the need to reduce energy consumption, the system aims to minimize overall energy consumption. Since the energy consumption of different edge devices is related to the sampling frequency, the following energy consumption objective function is constructed to achieve the desired total energy consumption for all devices at a given sampling frequency:

[0157]

[0158] The second data value function is used to assess the importance of the collected data to support future decision-making, encouraging higher sampling frequencies when data value is high. To ensure effective perception of park environmental data, the sampling frequency needs to be set reasonably according to the importance of the data collected by different edge devices. The effectiveness of environmental data perception can be measured by maximizing J2, i.e., having edge devices with high importance collect data at higher sampling frequencies to ensure sufficient perception of environmental data.

[0159]

[0160] Combining the above three objectives, the optimization problem regarding the adaptive sampling frequency can be formulated as a multi-objective optimization problem, with the objective function being:

[0161] J = ω1J1 + ω2J2 + ω3J3

[0162] Wherein, ω1, ω2, and ω3 are weighting coefficients used to balance the importance of the three different objectives in the optimization process, and satisfy ω1+ω2+ω3=1; ω1≥0; ω2≥0; ω3≥0.

[0163] The constraints for solving the above objective function are as follows:

[0164] Sampling frequency upper and lower limits constraints: The sampling frequency of each edge device should be within a reasonable range, that is:

[0165] f i,min ≤f i ≤f i,max , where f i,min and f i,max These are the lower and upper limits of the sampling frequency for the i-th edge device, respectively, which are determined by factors such as the device's performance and data acquisition requirements.

[0166] Communication load constraint: To ensure the normal operation of the communication link, the communication load of each edge device at different times should not exceed the maximum carrying capacity of its communication link, i.e.: C ij ≤C ij,max C ij,max This represents the maximum capacity of the communication link for the i-th edge device within time period j.

[0167] Energy Constraints: Overall energy consumption should not exceed the upper limit of energy consumption that the park can withstand, which is determined by factors such as the park's energy supply and energy conservation targets. This is expressed as: J² ≤ E max .

[0168] The variables that appear in the above modeling process are defined as follows:

[0169] f iThis represents the sampling frequency of the i-th edge device (or a certain type of environmental data monitoring point), where i = 1, 2, ..., n, and n is the total number of devices or monitoring points that need to be monitored.

[0170] C ij This represents the communication load from the i-th edge device to the digital energy management platform, when the sampling frequency is f. j The value of time is taken within the time period j, where j = 1, 2, ..., m, and m is the total number of time periods (for example, time periods can be divided by hour, day, etc.).

[0171] E i (f i ) indicates that the i-th edge device is at a sampling frequency of f i The energy consumption function at time is usually about f i It is a monotonically increasing function because the higher the sampling frequency, the higher the energy consumption of equipment operation and data acquisition.

[0172] D ij This indicates the importance of the data collected by the i-th edge device for environmental data perception within time period j. The value range can be [0,1], with higher values ​​indicating greater importance.

[0173] It should be noted that the above embodiments only use the second communication load function, the second energy consumption cost function, and the second data value function as influencing factors to construct the second objective function. In practical applications, only one or two of the influencing factors may be considered, or other influencing factors may be added. This application embodiment does not specifically limit it.

[0174] This application embodiment constructs a second objective function and solves the second objective function to obtain the sampling frequency, so that the edge device collects data according to the sampling frequency. By reasonably setting the sampling frequency, the monitoring efficiency is improved and the energy consumption is reduced while satisfying the effective monitoring of the edge device.

[0175] Based on the above embodiments, the compression parameters include a sparse matrix and a measurement matrix. After receiving the compressed data, the method further includes:

[0176] The compressed data is recovered based on the sparse matrix and the measurement matrix to obtain the recovered data.

[0177] In the specific implementation process, the compression parameters sent by the digital energy management platform to the edge device include a sparse matrix and a measurement matrix. The generation methods of the sparse matrix and the measurement matrix are described in the above embodiment and will not be repeated here. After the edge device collects and compresses data according to the compression parameters, it returns the compressed data to the digital energy management platform, which then needs to perform high-precision recovery of the compressed data.

[0178] In the data recovery process, this application embodiment needs to address the l1 constructed during the compression process described above. - The norm minimization problem can be solved using algorithms such as basis pursuit or orthogonal matching pursuit to obtain an estimated vector of the sparse signal, such that:

[0179] Where y is the compressed data received by the digital energy management platform, and Φ is the measurement matrix. Once the estimated vector s is obtained, the original data can be recovered using appropriate reconstruction algorithms, such as least squares or iterative thresholding algorithms.

[0180] The embodiments of this application determine the sparse solution through the sparse matrix and the measurement matrix, and complete the recovery of the compressed data based on the sparse solution reconstruction algorithm, thereby obtaining the accuracy of the recovered data.

[0181] Based on the above embodiments, the method further includes:

[0182] The collected monitoring parameters are input into the environmental perception model to obtain prediction results;

[0183] If the prediction results meet the update strategy conditions, the compression parameters are regenerated and sent to the edge devices.

[0184] In practical implementation and application, changes in environmental data can affect the collected data. For example, a malfunction in an edge device or a sudden change in weather may occur. To adapt to these environmental changes, the compression parameters in this embodiment also need to be adjusted accordingly.

[0185] The digital energy management platform can collect monitoring parameters in real time or at preset time intervals. These parameters include environmental data and operational data from edge devices. Environmental data includes at least one of the following: temperature, humidity, light intensity, and wind speed. Edge device operational data includes whether the edge devices are functioning normally. For real-time monitoring parameters, real-time data stream processing frameworks such as Kafka and Flink can be used for data processing.

[0186] After collecting monitoring parameters, features can be extracted to reveal key information reflecting environmental changes, such as trends, seasonal patterns, and extreme values. These extracted features are then input into an environmental perception model for analysis, yielding prediction results. It is understood that the environmental perception model can be built and trained using random forests, gradient boosting trees, or deep learning techniques (recurrent neural networks, long short-term memory networks, or convolutional neural networks).

[0187] The prediction results are used to characterize changes in environmental parameters over a future period. For example, they can predict whether a certain edge device will fail, or predict future light intensity. If the prediction results meet the update strategy conditions, the digital energy management platform regenerates the compression parameters and sends the newly generated compression parameters to the edge device. It should be noted that the update strategy conditions are pre-set. For example, the update strategy conditions are met when an edge device fails; the update strategy conditions are also met when a new edge device is added; furthermore, the update strategy conditions are also considered met when the light intensity meets a preset level.

[0188] It should be noted that the accuracy and reliability of the environmental perception model can be evaluated periodically, and methods such as cross-validation and A / B testing can be used for model tuning to improve the accuracy of the environmental perception model's predictions. Furthermore, by developing event recognition algorithms, the digital energy management platform can identify anomalies or critical events in environmental data, such as extreme weather changes or equipment malfunctions, thereby adjusting compression parameters in a timely manner. Ultimately, the predicted information is used as feedback to update compression parameters in real time, ensuring that the sampling rate is increased before environmental changes occur to capture key data. The implementation of this series of steps not only improves the relevance and timeliness of data collection but also significantly enhances the adaptability and intelligence level of the digital energy management platform, providing strong technical support for achieving efficient and accurate data collection.

[0189] Figure 2 This is a schematic diagram of another data processing method provided in an embodiment of this application, such as... Figure 2 As shown, this method is applied to edge devices, which can be devices installed in zero-carbon parks, such as energy storage devices, photovoltaic devices, load units, charging devices, etc. The method includes:

[0190] Step 201: Receive compression parameters from the digital energy management platform; the compression parameters are generated by the digital energy management platform based on system environment parameters;

[0191] Step 202: Compress the collected raw data according to the compression parameters to obtain compressed data;

[0192] Step 203: Upload the compressed data to the digital energy management platform.

[0193] In practical implementation, the compression parameters issued by the digital energy management platform may include at least one of a sparse matrix, a measurement matrix, and a sampling frequency, and may also include other parameters. Furthermore, the method for generating the compression parameters can be found in the above embodiments, and will not be repeated here.

[0194] After receiving the compression parameters, the edge device will collect raw data according to the sampling frequency if the compression parameters include a sampling frequency; otherwise, it will collect raw data according to a preset frequency. The method for determining the sampling frequency can be found in the above embodiment and will not be repeated here.

[0195] The compression parameters may also include a sparse matrix and / or a measurement matrix.

[0196] A sparse matrix (or dictionary matrix) is used to represent original data in a sparse form within a specific transformation domain. It can transform the original data so that the transformed data exhibits sparse characteristics in the "coordinate system" defined by the matrix, meaning that most elements are zero or close to zero, with only a few non-zero elements.

[0197] Assuming the sparse representation matrix of the original data x is G, then it can be obtained through... The transformation can make It possesses sparsity properties. Essentially, it helps to uncover the fundamental characteristics of data. This is because, in sparse representation, the key information of the data is often concentrated on the "directions" corresponding to a few non-zero elements.

[0198] Measurement matrix: A matrix used to measure the sparsely represented data. Its function is to transform high-dimensional sparse data into low-dimensional data through matrix multiplication with the sparsely represented data.

[0199] Measured values. For the data after the above sparse representation... Define the measurement matrix Φ, then the measured values ​​can be obtained through... Obtained. Due to It is inherently sparse, and the design of the measurement matrix allows for the use of relatively few measurements (i.e., the dimension of y is much lower than...). By considering the dimensionality of the sparse representation matrix (Φ), sufficient information can be obtained to reconstruct the original data x. Furthermore, the measurement matrix needs to meet certain conditions to guarantee the reconstructability of the original data. One important condition is that the measurement matrix Φ and the sparse representation matrix G must satisfy the so-called "incoherence" condition. Simply put, the measurement matrix and the sparse representation matrix cannot be too similar or correlated; otherwise, it may lead to the inability to accurately reconstruct the original data.

[0200] If the raw data collected by the edge device is sufficiently sparse, the compression parameters can include a measurement matrix, which can then be used to compress the raw data. If the raw data collected by the edge device is not sparse enough, the compression parameters can include a sparse matrix and a measurement matrix. The edge device can first use the sparse matrix to sparsify the raw data, and then use the measurement matrix to compress the sparsified data to obtain the compressed data.

[0201] After obtaining the compressed data, the compressed data is uploaded to the digital resource management platform.

[0202] This application embodiment receives compression parameters issued by a digital energy management platform and collects and compresses data according to these parameters. Since the compression parameters are generated by the digital energy management platform based on system environment parameters, data compression can be performed based on actual conditions, reducing data transmission overhead and distortion rate.

[0203] Figure 3 This is a schematic diagram of a data processing device structure provided in an embodiment of this application. The device can be a module, program segment, or code on an electronic device. It should be understood that this device is similar to the one described above. Figure 1 The method implementation corresponds to this and can be executed. Figure 1 The specific functions of the device involved in the method embodiment can be found in the description above; to avoid repetition, detailed descriptions are omitted here. The device includes: a policy generation module 301, a policy distribution module 302, and a data receiving module 303, wherein:

[0204] The strategy generation module 301 is used to generate compression parameters for each edge device based on the system environment parameters of the edge devices;

[0205] The strategy distribution module 302 is used to send compression parameters to the edge device so that the edge device can compress the collected raw data based on the compression parameters to obtain compressed data.

[0206] The data receiving module 303 is used to receive compressed data uploaded by the edge device.

[0207] Based on the above embodiments, the system environment parameters include the topology of the edge devices, and the compression parameters include a sparse matrix; the policy generation module 301 is specifically used for:

[0208] Based on the topological relationship, obtain the relevant variables between the data collected by the edge devices;

[0209] The correlation between the data is determined based on the aforementioned relevant variables;

[0210] The sparse matrix is ​​constructed based on the correlation.

[0211] Based on the above embodiments, the system environment parameters include the topology of the edge device and network environment parameters; the compression parameters include a measurement matrix; the policy generation module 301 is specifically used for:

[0212] A first objective function is constructed based on the topology of the edge devices, network environment parameters, and calculation functions.

[0213] The first objective function is solved using a preset optimization algorithm to obtain the sampling rate corresponding to each edge device;

[0214] The measurement matrix is ​​determined based on the sampling rate and the sparse matrix; wherein each row of the measurement matrix includes a non-zero element.

[0215] Based on the above embodiments, the calculation function includes a first communication load function, a first energy consumption cost function, and a first data value function; the strategy generation module 301 is specifically used for:

[0216] The first communication load function is constructed based on the data transmission volume under the predicted sampling rate, the current network bandwidth, and the time weighting coefficient.

[0217] The first energy cost function is constructed based on the equipment energy consumption, energy price, and electricity price fluctuation coefficient under the predicted sampling rate.

[0218] The first data value function is constructed based on the predicted data change amount, the predicted change weight coefficient, and the sampling rate penalty coefficient;

[0219] The first objective function is generated based on the first communication load function, the first energy consumption cost function, and the first data value function.

[0220] Based on the above embodiments, the compression parameters further include a sampling frequency; the sampling frequency is determined by the following method:

[0221] A second objective function is constructed based on the topology of the edge devices, network environment calculations, and calculation functions.

[0222] The second objective function is solved using a preset optimization algorithm to obtain the sampling frequency corresponding to each edge device.

[0223] Based on the above embodiments, the calculation function includes a second communication load function, a second energy consumption cost function, and a second data value function;

[0224] Based on the above embodiments, the strategy generation module 301 is specifically used for:

[0225] The second communication load function is constructed based on the communication load and average communication of each edge device at different time periods;

[0226] The second energy consumption cost function is constructed based on the energy consumption of each edge device at the predicted sampling frequency.

[0227] The second data value function is constructed based on the importance and sampling frequency of the data collected by each edge device within a preset time period;

[0228] The second objective function is generated based on the second communication load function, the second energy consumption cost function, and the second data value function.

[0229] Based on the above embodiments, the compression parameters include a sparse matrix and a measurement matrix, and the device further includes a data recovery module for:

[0230] The compressed data is recovered based on the sparse matrix and the measurement matrix to obtain the recovered data.

[0231] Based on the above embodiments, the device further includes a policy update module, used for:

[0232] The collected monitoring parameters are input into the environmental perception model to obtain prediction results;

[0233] If the prediction result meets the update strategy conditions, the compression parameters are regenerated and sent to the edge device.

[0234] Figure 4 This is a schematic diagram of the physical structure of the electronic device provided in the embodiments of this application, such as... Figure 4 As shown, the electronic device includes: a processor 401, a memory 402, and a bus 403; wherein,

[0235] The processor 401 and the memory 402 communicate with each other through the bus 403;

[0236] The processor 401 is used to call program instructions in the memory 402 to execute the methods provided in the above-described method embodiments, including, for example: generating compression parameters for each edge device based on the system environment parameters of the edge device; sending the compression parameters to the edge device so that the edge device compresses the collected raw data based on the compression parameters to obtain compressed data; and receiving the compressed data uploaded by the edge device.

[0237] Processor 401 can be an integrated circuit chip with signal processing capabilities. The processor 401 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor.

[0238] The memory 402 may include, but is not limited to, random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc.

[0239] This embodiment discloses a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions, and when the program instructions are executed by a computer, the computer can execute the methods provided in the above-described method embodiments, such as: generating compression parameters for each edge device based on system environment parameters of the edge device; sending the compression parameters to the edge device so that the edge device compresses the collected raw data based on the compression parameters to obtain compressed data; and receiving the compressed data uploaded by the edge device.

[0240] This embodiment provides a non-transitory computer-readable storage medium storing computer instructions that cause the computer to execute the methods provided in the above-described method embodiments, such as: generating compression parameters for each edge device based on system environment parameters of the edge device; sending the compression parameters to the edge device so that the edge device compresses the collected raw data based on the compression parameters to obtain compressed data; and receiving the compressed data uploaded by the edge device.

[0241] Figure 5 This is a schematic diagram of a data processing system structure provided in an embodiment of this application, such as... Figure 5 As shown, the system includes a digital energy management platform 501 and an edge device 502. The digital energy management platform 501 and the edge device 502 are communicatively connected. There is at least one edge device 502. The digital energy management platform 501 is used to execute the methods provided in the above embodiments for generating, distributing, and recovering compression parameters. The edge device 502 is used to execute the methods in the above embodiments for receiving compression parameters, processing data based on the compression parameters, and uploading compressed data.

[0242] Figure 6 This application provides a schematic diagram of a data processing system network architecture, as shown in the embodiments below. Figure 6 As shown, the system comprises an infrastructure layer, a platform layer, and a business layer. Edge devices reside in the infrastructure layer, and the data they collect communicates with the platform layer via controllers / IoT gateways and physically defined message channels (MQTT, Zigbee, Bluetooth, Wi-Fi, etc.). The platform layer includes a digital energy management platform, comprising functional modules such as scene configuration, IoT platform, predictive simulation, data analysis, and data processing. This application's embodiments primarily address data transmission between the infrastructure layer and the platform layer.

[0243] Figure 7 A topology diagram of a zero-carbon park scenario provided in this application embodiment, such as... Figure 7 As shown, it includes one energy storage unit, one photovoltaic unit, one load unit, and one charging unit. It should be noted that in practical applications, a zero-carbon park may include other units, or only some of the listed units. The energy storage unit contains one battery cabinet (containing a battery cluster of five battery packs), one battery management system (BMS), two energy storage converters (PCS), one local energy management system (LEMS), one energy storage meter, and corresponding air-cooling and fire-fighting equipment. The photovoltaic unit consists of nine solar photovoltaic panels, one photovoltaic inverter (INV), one photovoltaic meter, and supporting equipment such as a weather station and distribution box. The load unit includes a main meter, lighting meters, air conditioning and other electrical equipment, as well as corresponding smart meters. The charging unit includes three V2G-enabled AC charging piles and one main charging pile meter. Each of the above devices and units is assigned a unique device ID after instantiation, facilitating device identification during data reporting. It should be noted that in actual application scenarios, the specific components included in each of the above units may differ, and the embodiments in this application are merely examples.

[0244] In the intelligent data acquisition and monitoring system of the zero-carbon park, the equipment begins operation after completing initialization and self-test procedures. First, the battery management system (BMS) and power converter (PCS) of the energy storage unit are activated, monitoring battery status and energy flow. Simultaneously, the solar photovoltaic panels and inverters (INV) of the photovoltaic unit also begin operation, converting sunlight into electricity. The lighting and air conditioning systems of the load unit start according to a predetermined schedule, while the AC charging stations of the charging unit are ready to provide charging services for electric vehicles.

[0245] Figure 8 A zero-carbon park data processing signaling interaction diagram is provided for embodiments of this application, such as... Figure 8 As shown.

[0246] After the edge device is initialized, the digital energy management platform issues a data collection command to it to enable data reporting. At this time, the digital energy management platform generates compression parameters and reports them based on the current channel measurement results and the topology relationship of the edge device. The formulation of compression parameters needs to consider multiple factors and is derived by solving the optimization problems of the first and second objective functions described in the above embodiments. For edge devices that lack the capability for data compression, their status data is reported to the Local Energy Management System (LEMS) in real time via their device ID. The LEMS applies the data compression method provided in this application to perform real-time compression processing on this data, removing unnecessary redundant information and retaining only key data points. The compressed data is then reported to the digital energy management platform. This process ensures that data transmission efficiency is maintained even with large data volumes, while reducing network load. For edge devices capable of data compression, the collected data can be compressed before being uploaded to the digital energy management platform.

[0247] Specifically, for example, the cells within a battery cluster in the energy storage unit will report their voltage, current, and temperature according to the following frame structure. It is foreseeable that cells within the same battery cluster will exhibit a certain correlation in their voltage, current, and temperature data. This correlation is a result of the combined effects of the series-parallel connection methods between cells, BMS management, and thermal management, which provides the basis for the sparse compression parameters provided in this application embodiment. The measurement matrix generated based on the characteristics of the cell and battery cluster data will be used for cell data sampling. It is anticipated that the sampling matrix corresponding to the cell data will have a large number of missing values, which can all be accurately filled in by relevant data in the cloud based on their inherent relationships.

[0248] Device ID Individual voltage Current temperature Timestamp

[0249] Furthermore, taking the photovoltaic units in the park as an example, the system increases the data sampling frequency at midday when the photovoltaic panels receive the strongest sunlight to accurately capture peak energy output. Conversely, in the evening when sunlight intensity decreases, the system reduces the sampling frequency to adapt to the decline in energy output. The system can dynamically adjust the data acquisition frequency based on changes in daytime sunlight intensity and equipment usage patterns through adaptive compression parameters. Simultaneously, an environmental perception model monitors changes in the external environment in real time, such as fluctuations in temperature and humidity. The system uses machine learning and deep learning models to predict the potential impact of these environmental changes on energy output and consumption. Based on these predictions, the system intelligently adjusts energy allocation and equipment operation strategies to optimize energy efficiency.

[0250] The compressed data received by the digital energy management platform is recovered through data restoration. The spatiotemporal relationships of missing data are captured using the sparse matrix in this embodiment. The recovery methods include, but are not limited to, traditional optimization algorithms or deep learning schemes. It should be noted that the sparse matrix and measurement matrix differ between different edge devices, and will be determined based on factors such as device characteristics and reporting frequency. The recovered data will be used for data analysis and device status monitoring to provide accurate information. These analysis results are then used for energy management decisions, such as adjusting energy storage strategies, optimizing load allocation, and scheduling the use of charging stations.

[0251] This method reduces problems such as excessive network load and high communication overhead between the edge and cloud in the data reporting of park equipment, laying the foundation for real-time monitoring and optimized control of park equipment.

[0252] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0253] Furthermore, the units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0254] Furthermore, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0255] In this document, relational terms such as first and second are used only to distinguish one entity or operation from another entity or operation, without necessarily requiring or implying any such actual relationship or order between these entities or operations.

[0256] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A data processing method, characterized in that, The method is applied to a digital energy management platform, and the method includes: Generate compression parameters for each edge device based on the system environment parameters of the edge devices; The compression parameters are sent to the edge device so that the edge device can compress the collected raw data based on the compression parameters to obtain compressed data; Receive the compressed data uploaded by the edge device.

2. The method according to claim 1, characterized in that, The system environment parameters include the topology of the edge devices, and the compression parameters include a sparse matrix; generating the sparse matrix in the compression parameters of each edge device based on the system environment parameters of the edge devices includes: Based on the topological relationship, obtain the relevant variables between the data collected by the edge devices; The correlation between the data is determined based on the aforementioned relevant variables; The sparse matrix is ​​constructed based on the correlation.

3. The method according to claim 2, characterized in that, The system environment parameters include the topology and network environment parameters of the edge devices; the compression parameters include a measurement matrix; generating the measurement matrix in the compression parameters of each edge device based on the system environment parameters of the edge devices includes: A first objective function is constructed based on the topology of the edge devices, network environment parameters, and calculation functions. The first objective function is solved using a preset optimization algorithm to obtain the sampling rate corresponding to each edge device. The measurement matrix is ​​determined based on the sampling rate and the sparse matrix; wherein each row of the measurement matrix includes a non-zero element.

4. The method according to claim 3, characterized in that, The calculation function includes a first communication load function, a first energy consumption cost function, and a first data value function; the construction of the first objective function based on the topology of the edge device, network environment calculation, and calculation function includes: The first communication load function is constructed based on the data transmission volume under the predicted sampling rate, the current network bandwidth, and the time weighting coefficient. The first energy cost function is constructed based on the equipment energy consumption, energy price, and electricity price fluctuation coefficient under the predicted sampling rate. The first data value function is constructed based on the predicted data change amount, the predicted change weight coefficient, and the sampling rate penalty coefficient; The first objective function is generated based on the first communication load function, the first energy consumption cost function, and the first data value function.

5. The method according to claim 1, characterized in that, The compression parameters include the sampling frequency; the sampling frequency is determined by the following method: A second objective function is constructed based on the topology of the edge devices, network environment calculations, and calculation functions. The second objective function is solved using a preset optimization algorithm to obtain the sampling frequency corresponding to each edge device.

6. The method according to claim 5, characterized in that, The calculation function includes a second communication load function, a second energy consumption cost function, and a second data value function; the construction of the second objective function based on the topology of the edge device, network environment calculation, and calculation function includes: The second communication load function is constructed based on the communication load and average communication of each edge device at different time periods; The second energy consumption cost function is constructed based on the energy consumption of each edge device at the predicted sampling frequency. The second data value function is constructed based on the importance and sampling frequency of the data collected by each edge device within a preset time period; The second objective function is generated based on the second communication load function, the second energy consumption cost function, and the second data value function.

7. The method according to claim 1, characterized in that, The compression parameters include a sparse matrix and a measurement matrix. After receiving the compressed data, the method further includes: The compressed data is recovered based on the sparse matrix and the measurement matrix to obtain the recovered data.

8. The method according to any one of claims 1-7, characterized in that, The method further includes: The collected monitoring parameters are input into the environmental perception model to obtain prediction results; If the prediction result meets the update strategy conditions, the compression parameters are regenerated and sent to the edge device.

9. A data processing method, characterized in that, Applied to edge devices, the method includes: Receive compression parameters from the digital energy management platform; the compression parameters are generated by the digital energy management platform based on system environment parameters; The collected raw data is compressed according to the compression parameters to obtain compressed data; The compressed data is uploaded to the digital energy management platform.

10. The method according to claim 9, characterized in that, The compression parameters include a sparse matrix and a measurement matrix; The step of compressing the collected raw data according to the compression parameters includes: The collected raw data is compressed based on the sparse matrix and the measurement matrix.

11. The method according to claim 10, characterized in that, The compression parameters also include the sampling frequency; before compressing the acquired raw data according to the sparse matrix and the measurement matrix, the method further includes: The raw data is collected according to the sampling frequency.

12. A data processing apparatus, characterized in that, include: The strategy generation module is used to generate compression parameters for each edge device based on the system environment parameters of the edge devices. The strategy delivery module is used to send the compression parameters to the edge device, so that the edge device can compress the collected raw data based on the compression parameters to obtain compressed data; A data receiving module is used to receive the compressed data uploaded by the edge device.

13. An electronic device, characterized in that, include: Processor, memory, and bus, among which, The processor and the memory communicate with each other via the bus; The memory stores program instructions that can be executed by the processor, and the processor can execute the method as described in any one of claims 1-8 by calling the program instructions.

14. A non-transitory computer-readable storage medium, characterized in that, The non-transitory computer-readable storage medium stores computer instructions that, when executed by a computer, cause the computer to perform the method as described in any one of claims 1-8.

15. A computer program product, characterized in that, It includes computer program instructions, which, when read and executed by a processor, perform the method as described in any one of claims 1-8.

16. A data processing system, characterized in that, It includes a digital energy management platform and edge devices; the digital energy management platform is communicatively connected to the edge devices; The digital energy management platform is used to perform the method as described in any one of claims 1-8; The edge device is used to perform the method as described in any one of claims 9-11.

Images