Electricity data compression methods, transmission systems, devices and electronic equipment

Abstract

This invention relates to the field of power data storage and transmission technology, and discloses a method, transmission system, device, and electronic equipment for compressing electricity consumption data. The method includes: acquiring raw electricity consumption data; generating an electricity consumption data matrix based on the raw electricity consumption data; determining whether the data in the electricity consumption data matrix is ​​valid and generating matrix markers; filtering valid data based on the matrix markers and compressing the valid data using a compression algorithm; and generating compressed data blocks by combining the matrix markers. This invention transforms domain knowledge into compression rules, overcomes the shortcomings of general algorithms in utilizing the characteristics of structured data, significantly reduces the transmission bandwidth and storage cost of massive amounts of electricity consumption data, and simultaneously meets the real-time requirements of high-frequency acquisition scenarios.

Description

Technical Field

[0001] This invention relates to the field of power data storage and transmission technology, specifically to a method for compressing power consumption data, a transmission system, a device, and electronic equipment. Background Technology

[0002] With the widespread adoption of smart grids, smart meters collect electricity consumption data every 15 minutes, generating 96 records per day. Each record contains multiple electricity consumption fields (such as total consumption, rate-based consumption, and reverse consumption). A single meter generates approximately 7KB of data daily, placing pressure on transmission bandwidth and storage costs due to the massive volume of meter data. Simultaneously, supporting high-frequency data collection and remote transmission places high demands on the efficiency and real-time performance of compression algorithms. While traditional general-purpose compression algorithms offer some compression effectiveness, they do not fully utilize the row and column characteristics of electricity consumption data, leaving room for improvement in compression efficiency and real-time performance. This not only reduces compression efficiency and real-time performance requirements but also increases data transmission bandwidth and storage costs.

[0003] Therefore, there is an urgent need for a method to compress electricity consumption data that can overcome the above problems. Summary of the Invention

[0004] This invention provides a method, transmission system, device, and electronic device for compressing electricity data, in order to solve the problems of reduced compression efficiency and real-time performance in traditional electricity data compression methods.

[0005] In a first aspect, the present invention provides a method for compressing electricity consumption data, the method comprising:

[0006] Obtain raw electricity consumption data;

[0007] An electricity consumption data matrix is ​​generated based on the original electricity consumption data;

[0008] Determine if the data in the electricity consumption data matrix is ​​valid, and generate matrix markers;

[0009] Valid data is filtered based on matrix labeling, and the valid data is compressed using a compression algorithm. The compressed data blocks are then generated by combining the matrix labeling.

[0010] This invention provides a method for compressing electricity consumption data. By analyzing the unique row-column sparsity pattern of electricity consumption data, an adaptive compressed data block is designed, which can accurately identify and encode only valid non-zero data. This method transforms domain knowledge into compression rules, overcoming the shortcomings of general algorithms in not fully utilizing the features of structured data. It significantly reduces the transmission bandwidth and storage cost of massive amounts of electricity consumption data, while meeting the real-time requirements of high-frequency acquisition scenarios, thus solving the problems of reduced compression efficiency and real-time performance in traditional electricity consumption data compression methods.

[0011] In one optional implementation, generating an electricity consumption data matrix based on the original electricity consumption data includes:

[0012] The system acquires raw electricity consumption data at multiple points within a preset time period, using a preset time interval as the time point. The raw electricity consumption data includes multiple time points and various electricity consumption types.

[0013] Generate an electricity consumption data matrix with each time point as a row and electricity consumption type as a column.

[0014] The present invention provides an electricity consumption data compression method that organizes massive, discrete raw electricity consumption data into a standardized matrix structure with time points as rows and electricity consumption types as columns. This allows the inherent spatiotemporal correlation and type dimension of the data to be explicitly presented, laying a structured foundation for subsequent accurate identification of row and column sparsity patterns and the realization of adaptive compression based on data distribution characteristics. This transforms domain knowledge into computable and operable compression logic.

[0015] In one optional implementation, determining whether the data in the electricity consumption data matrix is ​​valid and generating matrix markers includes:

[0016] Determine if a row in the electricity consumption data matrix has been logically deleted or is invalid, and generate a row deletion marker;

[0017] If a column in the electricity consumption data matrix contains all preset invalid values, a column enable flag is generated.

[0018] This invention provides a method for compressing electricity consumption data. By identifying and marking invalid rows and preset invalid value columns, the implicit row and column sparsity in the electricity consumption data is transformed into explicit and computable compression guidance information. This not only eliminates the interference of invalid data on compression efficiency, allowing subsequent processing to focus only on valid non-zero data, but also replaces the storage and transmission of the original data with a lightweight marker bitmap. While maintaining the integrity of the data's logical structure, it provides a precise adaptive basis for achieving extremely high compression ratios, thus effectively solving the compression efficiency bottleneck problem caused by general algorithms ignoring the characteristics of domain data.

[0019] In one optional implementation, determining which rows of data in the electricity consumption data matrix have been logically deleted or are invalid, and generating a row deletion marker, includes:

[0020] Iterate through each row of the electricity consumption data matrix and determine whether the current row of data meets the preset invalidity condition;

[0021] If the preset invalidity condition is met, the value at the corresponding position of the row deletion marker is set to the first value indicating logical deletion or data abnormality; otherwise, it is set to the second value indicating valid data.

[0022] This invention provides a method for compressing electricity consumption data. Before compression, it proactively identifies and marks invalid data rows through rule-based judgment, ensuring that the compression process accurately focuses on valid data and avoiding redundant calculations and storage overhead. This mechanism integrates data validity verification into the compression process, maintaining low complexity while ensuring the quality of the processed data from the source, thus improving the overall effectiveness of compression.

[0023] In one optional implementation, determining a column in the electricity consumption data matrix where all data are preset invalid values ​​and generating a column enable flag includes:

[0024] Iterate through each column of the electricity consumption data matrix and determine whether all data elements in the current column are preset invalid values;

[0025] If all values ​​are preset invalid values, the value at the corresponding position in the column enable flag is set to the third value indicating that the column is disabled; otherwise, it is set to the fourth value indicating that the column is enabled.

[0026] This invention provides a method for compressing electricity consumption data. By performing a full invalid value check on each column of a matrix, it accurately identifies empty columns that do not contain valid data and sets a disabled flag for them. This allows all subsequent compression processes to skip these empty columns and directly focus on the valid columns containing actual electricity consumption information, thereby significantly reducing unnecessary data scanning and calculation overhead, and further improving the targeting and execution efficiency of the compression process from the column dimension.

[0027] In one optional implementation, valid data is filtered based on matrix tags, and a compression algorithm is used to compress the valid data, generating compressed data blocks in combination with the matrix tags, including:

[0028] A metadata area is constructed based on row deletion flags and column enable flags;

[0029] For each valid column indicated by the column enable flag, based on the distribution characteristics of the valid column data, determine whether to generate a column data description flag for the corresponding column, and construct a column description area based on the generated column data description flag;

[0030] Based on the row deletion flag, column enable flag, and column data description flag, filter out all valid data that needs to be compressed from the electricity consumption data matrix;

[0031] The selected valid data is compressed and encoded using a compression algorithm, and the encoding result is used to construct a compressed data area.

[0032] The compressed data block is obtained by summarizing the metadata area, column description area, and compressed data area.

[0033] This invention provides a method for compressing electricity consumption data. By systematically transforming the extracted effective data (row / column markers) into a hierarchical, adaptive compressed data structure, it achieves a closed loop from data feature analysis to efficient storage. The metadata area provides global oversight, the column description area provides detailed descriptions, and the compressed data area focuses on storing effective information. The collaboration of these three elements ensures that the compression process can fully utilize the row and column sparsity of the data to achieve an extremely high compression ratio while maintaining complete structural information for lossless restoration, thus achieving an optimal balance between compression efficiency and data integrity.

[0034] In one optional implementation, a compression algorithm is used to compress and encode the selected valid data, and the encoding result is used to construct a compressed data area, including:

[0035] The selected valid data is used as input, and a compression algorithm is called to encode the data, generating the corresponding compressed data stream.

[0036] The compressed data stream is sequentially written into the storage space reserved for compressed data to construct the compressed data area.

[0037] This invention provides a method for compressing electricity consumption data. By using a compression algorithm optimized for small integers to encode a highly purified set of valid non-zero data after multi-level screening, it ensures that computing resources are fully concentrated on the conversion of valuable information, thereby maximizing storage density. Simultaneously, the compact data stream generated by encoding is sequentially written into reserved space, guaranteeing the structural integrity and reading efficiency of the compressed data area, completing the final crucial transformation from data feature identification to efficient binary storage.

[0038] Secondly, the present invention provides an electrical data transmission system, the system comprising:

[0039] The cloud platform and the compression subsystem of the power consumption data compression method applied to the first aspect above or any corresponding embodiment thereof; the compression subsystem includes terminal equipment and edge equipment;

[0040] Terminal equipment used to acquire electricity consumption data;

[0041] Edge devices are used to generate an electricity consumption data matrix based on raw electricity consumption data; determine whether the data in the electricity consumption data matrix is ​​valid and generate matrix tags; filter valid data based on matrix tags, compress the valid data using a compression algorithm, and generate compressed data blocks in combination with matrix tags;

[0042] The cloud platform connects to the edge devices. The cloud platform receives compressed data blocks output by the edge devices, decompresses the compressed data blocks, and restores the power consumption data matrix.

[0043] Thirdly, the present invention provides an electrical data compression device, the device comprising:

[0044] The electricity consumption data acquisition module is used to acquire raw electricity consumption data.

[0045] The electricity consumption data matrix generation module is used to generate an electricity consumption data matrix based on the original electricity consumption data.

[0046] The matrix marker generation module is used to determine whether the data in the electricity consumption data matrix is ​​valid and to generate matrix markers.

[0047] The compressed data block generation module is used to filter valid data based on matrix markers, compress the valid data using a compression algorithm, and generate compressed data blocks in combination with matrix markers.

[0048] Fourthly, the present invention provides an electronic device, comprising: a memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the computer instructions to perform the power data compression method of the first aspect or any corresponding embodiment described above.

[0049] Fifthly, the present invention provides a computer-readable storage medium storing computer instructions for causing a computer to execute the power consumption data compression method of the first aspect or any corresponding embodiment thereof.

[0050] In a sixth aspect, the present invention provides a computer program product, including computer instructions for causing a computer to execute the power consumption data compression method of the first aspect or any corresponding embodiment thereof. Attached Figure Description

[0051] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0052] Figure 1 This is a schematic diagram of an application scenario according to an embodiment of the present invention;

[0053] Figure 2 This is a schematic diagram of the first process of the power consumption data compression method according to an embodiment of the present invention;

[0054] Figure 3 This is a schematic diagram of a second process for a power consumption data compression method according to an embodiment of the present invention;

[0055] Figure 4 This is a schematic diagram of the third process of the power consumption data compression method according to an embodiment of the present invention;

[0056] Figure 5 This is a structural block diagram of an electrical data compression device according to an embodiment of the present invention;

[0057] Figure 6 This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0058] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0059] It is understood that before using the technical solutions disclosed in the various embodiments of the present invention, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in the present invention and their authorization should be obtained in accordance with relevant laws and regulations through appropriate means.

[0060] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0061] As an optional application scenario of this invention, such as Figure 1 The figure shows an electricity data transmission system according to an embodiment of the present invention. The system includes a compression subsystem, comprising a terminal device and an edge device. The terminal device is used to acquire electricity data; the edge device is used to generate an electricity data matrix based on the original electricity data; determine whether the data in the electricity data matrix is ​​valid and generate matrix markers; filter valid data based on the matrix markers and compress the valid data using a compression algorithm, and generate compressed data blocks in combination with the matrix markers.

[0062] The system also includes network transmission equipment, a cloud platform, and a big data platform for electricity consumption.

[0063] The terminal device, namely the smart meter, collects raw electricity consumption data every 15 minutes, including total electricity consumption (E), electricity consumption at each tariff rate (E1-E8), total reverse electricity consumption (RE), and electricity consumption at each reverse tariff rate (RE1-RE8).

[0064] Edge devices are typically deployed within data concentrators in distribution areas, responsible for the localized preprocessing of massive amounts of meter data, significantly reducing network transmission pressure. Edge devices include a raw data cache, a compression module, and a compressed data cache. The raw data cache temporarily stores daily electricity consumption data collected from multiple meters; the compression module preprocesses, analyzes, and compresses the meter data; and the compressed data cache stores processed compressed data blocks, awaiting upload.

[0065] Network transmission equipment includes a wireless communication module (such as NB-IoT, 4G / 5G, LoRa, etc.) responsible for providing the uplink and transmitting highly compressed data processed by edge devices to the cloud platform. Because the data volume is drastically reduced after compression, it can significantly save communication traffic, reduce the probability of network congestion, and improve transmission speed.

[0066] The cloud platform connects to edge devices, providing data reception and decompression services. It receives compressed data blocks uploaded from numerous edge concentrators, parses the compressed data block information, decodes the compressed data area based on algorithm identifiers, recovers the list of non-zero values, and obtains metadata and column description information. Based on the metadata and column description information, the decompressed non-zero values ​​are filled into the correct positions in an all-zero matrix, reconstructing the original electricity consumption data matrix.

[0067] The electricity data big data platform stores the decompressed complete data (i.e., the reconstructed original electricity data matrix) for subsequent advanced applications such as electricity bill calculation, electricity behavior analysis, load forecasting, and big data dashboard display.

[0068] The core workflow of the power data transmission system is as follows:

[0069] 1) Collect raw electricity consumption data: The smart meter collects raw electricity consumption data every 15 minutes, including total electricity consumption (E), electricity consumption at each tariff rate (E1-E8), total reverse electricity consumption (RE), and electricity consumption at each reverse tariff rate (RE1-RE8).

[0070] It should be noted that four-rate meters only include E and E1-E4, eight-rate meters include E and E1-E8, and eight-rate reverse meters include E, E1-E8, RE, and RE1-RE8.

[0071] 2) Data concentrator stores data: An edge data concentrator receives raw electricity consumption data from multiple meters and stores the raw electricity consumption data in the raw data cache.

[0072] 3) The compression processing module processes the raw electricity consumption data, including: analyzing the electricity consumption data and generating the metadata area content of the compressed block; analyzing the data in each column and marking columns with all zeros as disabled; counting the number of zero values ​​in data columns to determine whether to enable the column description flag; after all the above operations are completed, generating the column description area content of the compressed block; compressing and encoding non-zero electricity consumption data using the Varint (or Simple8b algorithm); and assembling the metadata area, the column description area of ​​enabled columns, and the compressed electricity consumption data into a complete compressed data block.

[0073] 4) Network transmission equipment: All compressed data blocks from the edge data concentrators are transmitted to the cloud platform via the wireless communication module.

[0074] 5) Cloud platform decompresses data: The cloud platform parses the compressed data block, reads the metadata area information, decodes the compressed data area according to the algorithm identifier, and fills the decompressed non-zero values ​​into the correct position of an all-zero matrix according to the metadata and column description information, thus reconstructing the original 96-row × 18-column electricity consumption data matrix.

[0075] 6) Electricity consumption data big data platform storage for subsequent applications: The electricity consumption data big data platform stores the decompressed complete data (i.e., the reconstructed electricity consumption data matrix) for subsequent advanced applications such as electricity bill calculation, electricity consumption behavior analysis, load forecasting, and big data dashboard display.

[0076] According to an embodiment of the present invention, an embodiment of a method for compressing electrical data is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0077] This embodiment provides a method for compressing electricity consumption data, which can be used in the aforementioned electricity consumption data compression system. Figure 2 This is a flowchart of an electricity data compression method according to an embodiment of the present invention, such as... Figure 2 As shown, the process includes the following steps:

[0078] Step S201: Obtain raw electricity consumption data.

[0079] Specifically, the smart meter collects raw electricity consumption data every 15 minutes, including total energy (E), energy consumption at each tariff rate (Tariff 1 Energy - Tariff 8 Energy, E1-E8), reverse total energy (RE), and energy consumption at each reverse tariff rate (RE1-RE8).

[0080] It should be noted that four-rate meters only include E and E1-E4, eight-rate meters include E and E1-E8, and eight-rate reverse meters include E, E1-E8, RE, and RE1-RE8.

[0081] Furthermore, analysis of the raw electricity consumption data format:

[0082] Each time point is 15 minutes long. A 4-byte integer is used to represent 100 times the electricity consumption data (to save bytes by converting two decimal places to integers). Daily electricity consumption data contains 96 time point records (24 hours × 4). Each record contains the following fields:

[0083] DeleteFlag: 1 byte;

[0084] E (Total Battery Level): 4 bytes;

[0085] E1-E8 (Rate and Power Consumption): 8 × 4 = 32 bytes;

[0086] RE (Reverse Total Power): 4 bytes;

[0087] RE1-RE8 (Reverse Rate Electricity): 8 × 4 = 32 bytes;

[0088] Length of a single record: 1 + 4 + 32 + 4 + 32 = 73 bytes;

[0089] Total daily data volume for a single table: 96 × 73 = 7008 bytes.

[0090] Analysis of characteristics of raw electricity consumption data:

[0091] Small numerical range: Since it is the power consumption in 15 minutes, most power values ​​can actually be represented by 1-2 bytes, and in extreme cases, no more than 3 bytes. On average, 1.5 bytes are used for calculation.

[0092] Valid data: The Total Energy E / Reverse Total Energy RE fields only have data when using / charging electricity; otherwise, they are 0. The data exhibits characteristics of continuous data presence and continuous absence of data.

[0093] Rate field: Approximately 1 / 3 of the data points in the rate field are not 0. If the data does not fall within this rate, this rate field will be 0. (For example, assuming it is four-rate electricity consumption data, the peak data is from 8 to 10 am, then only E1 has data during this period, while E2 to E4 are all 0).

[0094] In summary, the electricity consumption data exhibits a clear sparse row and column pattern and continuous zero-value blocks.

[0095] Step S202: Generate an electricity consumption data matrix based on the original electricity consumption data.

[0096] Specifically, the electricity consumption data matrix is ​​the core structured data model of the entire compression method. It is an intermediate data representation in the form of a two-dimensional table that reorganizes the original electricity consumption data. Its purpose is to maximize the exposure and utilization of the row and column sparsity of the data, thereby driving subsequent efficient compression.

[0097] Step S203: Determine whether the data in the electricity consumption data matrix is ​​valid and generate matrix markers.

[0098] Specifically, matrix labeling is a general term referring to the set of navigation information generated during the compression process by analyzing the electricity consumption data matrix, which guides all subsequent compression operations. Matrix labeling includes:

[0099] Row deletion marker: In the time dimension, an entire row of data (all electricity consumption types) may be entirely zero due to no electricity consumption, forming a continuous period of blank space. Specifically, a row deletion marker (12-byte bitmap) is used to identify rows that are invalid due to all zeros or data anomalies.

[0100] Column Enable Flag: In the electricity consumption type dimension, an entire column of data (all time points) may be entirely zero due to equipment or billing rules, resulting in a blank column. Specifically, this is represented by an 18-bit column enable flag. This flag identifies valid columns where the data is not entirely zero. For example, for users without photovoltaic power generation, their RE and the entire RE1-RE8 column set would be marked as "disabled".

[0101] Column data description flag for zero-value blocks within a column: Within a single valid column, zero values ​​appear in blocks due to business rules (such as rate switching), rather than being randomly scattered. For example, during non-peak rate periods, the E1 (peak electricity consumption) column may be continuously 0. Specifically, this is represented by the column data description flag (enabled on demand, maximum 12 bytes per column). It is enabled when the number of zero values ​​in a column exceeds a threshold (e.g., 12), and is used to precisely describe which rows in that column have zero values.

[0102] Numerical range characteristics: The electricity consumption value for a single 15-minute period is very small, typically requiring only 1-2 bytes of storage, but the original format is fixed at 4 bytes. This is manifested in the choice of compression algorithm (such as Varint algorithm). Variable-length encoding for small integers utilizes the sparsity of numerical values.

[0103] Step S204: Filter valid data based on matrix marking, compress the valid data using a compression algorithm, and generate compressed data blocks by combining matrix marking.

[0104] Specifically, the compressed data block is the final output and core deliverable. It is a carefully designed, self-contained structured data packet containing all the valid information extracted from the original electricity consumption data matrix and encapsulated in high-density data for transmission and storage. The final compressed data block includes a metadata area, a column description area, and a compressed data area.

[0105] The electricity consumption data compression method provided in this embodiment analyzes the unique row-column sparsity pattern of electricity consumption data and designs adaptive compressed data blocks, which can accurately identify and encode only valid non-zero data. This method transforms domain knowledge into compression rules, overcomes the shortcomings of general algorithms in not fully utilizing the features of structured data, significantly reduces the transmission bandwidth and storage cost of massive amounts of electricity consumption data, and meets the real-time requirements of high-frequency acquisition scenarios, solving the problems of reduced compression efficiency and real-time performance in traditional electricity consumption data compression methods.

[0106] This embodiment provides a method for compressing electricity consumption data, which can be used in the aforementioned electricity consumption data compression system. Figure 3 This is a flowchart of an electricity data compression method according to an embodiment of the present invention, such as... Figure 3 As shown, the process includes the following steps:

[0107] Step S301: Obtain raw electricity consumption data. For details, please refer to [link to relevant documentation]. Figure 2 Step S201 of the illustrated embodiment will not be described again here.

[0108] Step S302: Generate an electricity consumption data matrix based on the original electricity consumption data.

[0109] Specifically, step S302 includes:

[0110] Step S3021: Using a preset time interval as the time point, obtain the raw electricity consumption data at multiple time points within the preset time period; the raw electricity consumption data includes multiple time points and multiple electricity consumption types.

[0111] Specifically, the smart meter collects raw electricity consumption data every 15 minutes. Using a preset time interval (e.g., 15 minutes) as the time point, it obtains raw electricity consumption data at multiple time points within a preset time period (e.g., one day). The raw electricity consumption data includes values ​​for multiple electricity consumption types (e.g., total electricity consumption, electricity consumption at each rate) corresponding to each time point.

[0112] Step S3022: Generate an electricity consumption data matrix with each time point as a row and electricity consumption type as a column.

[0113] Specifically, an electricity consumption data matrix is ​​generated by using each time point as a row of the matrix and each type of electricity consumption as a column of the matrix.

[0114] Step S303: Determine whether the data in the electricity consumption data matrix is ​​valid and generate matrix markers.

[0115] Specifically, step S303 includes:

[0116] Step S3031: Determine the row data in the electricity consumption data matrix that has been logically deleted or is invalid, and generate a row deletion mark.

[0117] In an optional implementation, step S3031 includes:

[0118] Step a1: Traverse each row of the electricity consumption data matrix and determine whether the current row of data meets the preset invalidity condition.

[0119] Specifically, the preset invalid conditions include at least one of the following: the value of all data elements in the row is zero;

[0120] This row contains abnormal data marked as a collection error or outside the reasonable range.

[0121] Iterate through each row of the electricity consumption data matrix and determine whether the current row of data satisfies at least one of the above-mentioned preset invalid conditions.

[0122] Step a2: If the preset invalidity condition is met, the value at the corresponding position of the row deletion marker is set to the first value indicating logical deletion or data abnormality; otherwise, it is set to the second value indicating valid data.

[0123] Specifically, the first data can be set to 1, and the second data can be set to 0.

[0124] The row deletion flag is represented by DeleteFlag, which is 12 bytes and is used to identify logically deleted rows. If the row data is all 0 or contains abnormal data, the DeleteFlag is marked on the row data during data cleaning.

[0125] Step S3032: Determine the columns in the electricity consumption data matrix where all data are preset invalid values, and generate column enable flags.

[0126] In an optional implementation, step S3032 includes:

[0127] Step b1: Traverse each column of the electricity consumption data matrix and determine whether all data elements in the current column are preset invalid values.

[0128] Specifically, zero is used as a preset invalid value, which means iterating through each column of the electricity consumption data matrix. For each column, it checks whether all the data values ​​in all its rows are zero.

[0129] In step b2, if all values ​​are preset invalid values, the value at the corresponding position in the column enable flag is set to the third value indicating that the column is disabled; otherwise, it is set to the fourth value indicating that the column is enabled.

[0130] Specifically, if all values ​​are zero, the corresponding binary bit in the column enable flag is set to a disabled value (e.g., 0), indicating that the column is not enabled and its data does not need to participate in subsequent compressed storage; if at least one non-zero value exists, it is set to an enabled value (e.g., 1).

[0131] The column enable flag, consisting of 18 bits, indicates whether the 18 electricity data columns E, E1-E8, RE, and RE1-RE8 are enabled. If there is no data in the column, it is not enabled.

[0132] Step S304: Valid data is filtered based on matrix markers, and a compression algorithm is used to compress the valid data. Compressed data blocks are then generated based on the matrix markers. For details, please refer to [link to relevant documentation]. Figure 2 Step S204 of the illustrated embodiment will not be described again here.

[0133] The electricity consumption data compression method provided in this embodiment actively identifies and marks invalid rows and columns of completely invalid values, transforming the implicit row and column sparsity in the electricity consumption data into explicit and computable compression guidance information. This not only eliminates the interference of invalid data on compression efficiency, allowing subsequent processing to focus only on valid non-zero data, but also replaces the storage and transmission of the original electricity consumption data with a lightweight marker bitmap. While maintaining the integrity of the data's logical structure, it provides a precise adaptive basis for achieving extremely high compression ratios, thus effectively solving the compression efficiency bottleneck problem caused by general algorithms ignoring domain data characteristics.

[0134] This embodiment provides a method for compressing electricity consumption data, which can be used in the aforementioned electricity consumption data compression system. Figure 4 This is a flowchart of an electricity data compression method according to an embodiment of the present invention, such as... Figure 4 As shown, the process includes the following steps:

[0135] Step S401: Obtain raw electricity consumption data. For details, please refer to [link to relevant documentation]. Figure 3 Step S301 of the illustrated embodiment will not be described again here.

[0136] Step S402: Generate an electricity consumption data matrix based on the original electricity consumption data. For details, please refer to [link to relevant documentation]. Figure 3 Step S302 of the illustrated embodiment will not be described again here.

[0137] Step S403: Determine if the data in the electricity consumption data matrix is ​​valid, and generate matrix markers. For details, please refer to [link to relevant documentation]. Figure 3 Step S303 of the illustrated embodiment will not be described again here.

[0138] Step S404: Filter valid data based on matrix marking, compress the valid data using a compression algorithm, and generate compressed data blocks by combining matrix marking.

[0139] Specifically, step S404 includes:

[0140] Step S4041: Construct the metadata area based on row deletion flags and column enable flags.

[0141] Specifically, the metadata area is 15 bytes, including:

[0142] Row deletion flag: Represented by DeleteFlag, 12 bytes, used to identify logically deleted rows. If the row data is all 0s or contains abnormal data, the DeleteFlag is marked on the row of electricity consumption data during data cleaning (there are a fixed 96 electricity consumption records per day, 96 bits for row deletion flag, each bit indicates whether a row is deleted, 96 bits = 12 bytes. The original electricity consumption data uses 1 byte DeleteFlag to identify each row of data, totaling 96 bytes, which saves seven-eighths of the space compared to the previous solution).

[0143] The column enable flag, consisting of 18 bits, indicates whether the 18 electricity data columns E, E1-E8, RE, and RE1-RE8 are enabled. If there is no data in the column, it is not enabled (for example, for a four-rate meter, if there is no data in the electricity data columns E2, E3, and E4 during peak hours, these three columns are not enabled).

[0144] Compression algorithm identifier: 6 bits, used to identify the type of compression algorithm used.

[0145] The two key markers used to describe global sparsity—row deletion marker (indicating invalid rows) and column enable marker (indicating valid columns)—are encapsulated in a predetermined format to form the “header” or “directory” of the compressed data block.

[0146] Step S4042: For each valid column indicated by the column enable flag, determine whether to generate a column data description flag for the corresponding column based on the distribution characteristics of the valid column data, and construct a column description area based on the generated column data description flag.

[0147] Specifically, for each column marked as valid, the distribution of its internal data (primarily the number of zero values) is further analyzed. Based on the analysis results (e.g., whether there are enough zero values), an adaptive decision is made as to whether to create a column data description label for that column. Finally, all the description labels generated by the decisions are assembled.

[0148] Furthermore, the description area includes:

[0149] Column data description enabled flag: 1 byte / column, used to identify whether column data description is enabled for this column.

[0150] Column data description flag (enabled on demand): 12 bytes / column. The column description is only available when enabled. It describes whether the data in each row of the column is 0 (96 rows of data, 96 bits in total, each bit describes the data in one row).

[0151] It should be noted that column data description is only enabled when the number of zero values ​​in the column exceeds 12. This is because if column data description is not enabled, all electricity data in the column will need to be compressed and stored. Each zero value occupies one byte. When the number of zero values ​​in the column is greater than 12, that is, the storage space for zero values ​​is greater than 12 bytes, enabling the column description flag will save space.

[0152] Step S4043: Based on the row deletion flag, column enable flag, and column data description flag, filter out all valid data that needs to be compressed from the electricity consumption data matrix.

[0153] Specifically, the original electricity consumption data matrix is ​​filtered by comprehensively utilizing the generated row deletion flags, column enable flags, and column data description flags. All rows marked as invalid, disabled columns, and positions within columns described as zero values ​​are precisely skipped, ultimately extracting a clean set containing only the necessary valid non-zero data.

[0154] Step S4044: The selected valid data is compressed and encoded using a compression algorithm, and the encoding result is used to construct a compressed data area.

[0155] In an optional implementation, step S4044 includes:

[0156] Step c1 involves taking the selected valid data as input, calling a compression algorithm for encoding processing, and generating the corresponding compressed data stream.

[0157] For the selected clean data set, a compression algorithm (such as Varint algorithm) is called to encode it and generate the corresponding compressed data stream.

[0158] Step c2: Write the compressed data stream sequentially into the storage space reserved for compressed data to construct the compressed data area.

[0159] Specifically, the resulting compact binary stream is stored in storage space reserved for the effective content, forming the main body of the compressed data block, which uses the selected compression algorithm to store non-zero electricity consumption data values.

[0160] It should be noted that the compression algorithm is selected according to the following rules:

[0161] Varint algorithm: Suitable for compressing small integer values. The highest bit of each byte is used as a continuation bit, and the lower 7 bits are used to store the data. It has extremely high compression efficiency for values ​​of 1-2 bytes. (The compressed non-zero power consumption data is calculated on average as 1.5 bytes).

[0162] Simple8b algorithm (alternative): suitable for scenarios with larger values ​​or higher compression ratio requirements, packing multiple integers into a 64-bit (8-byte) word.

[0163] Compression ratio analysis:

[0164] Assuming the Varint algorithm is used to compress electricity consumption data, the average value of the compressed non-zero electricity consumption data is 1.5 bytes for calculation:

[0165] 1. Typical scenario (four expense ratios, accounting for 99% of cases):

[0166] Compression ratio calculation formula = [96×1.5 + (13 + 96×25%×1.5)×4 + 15 + 12] / 7008

[0167] = (144+52+144+15+12) / 7008

[0168] = 367 / 7008≈5.24%.

[0169] Compression ratio: approximately 19:1.

[0170] In the above formula: 96 represents 96 records per day, 1.5 represents the number of bytes after compression, and the total space occupied by the total electricity consumption E is 96 × 1.5; 13 represents the column description + column description enable flag; 96 × 25% × 1.5 represents the space of each column after compression, which is only one-quarter of the rate data with data retained; there are a total of 4 rates, i.e., 4 columns, so the total space of each column needs to be multiplied by 4; 15 represents the fixed size of the metadata area, 12 represents the total fixed size of the row deletion flag; 7008 represents the total daily electricity consumption space under the previous compression scheme.

[0171] 2. Future Scenarios (Eight Fee Rates):

[0172] The calculation formula is:

[0173] (96×1.5 + (13 + 96×12.5%×1.5)×8 + 15 + 12) / 7008

[0174] = (144 + 104 + 144 + 15 + 12) / 7008

[0175] = 419 / 7008≈5.98%.

[0176] Compression ratio: approximately 17:1.

[0177] In the above formula, 96×12.5%×1.5 is one-eighth of the rate data that retains only the data, multiplied by 1.5 bytes, which is the compressed space for each column.

[0178] 3. Worst-case scenario (eight-fee rate + reverse):

[0179] Calculation formula:

[0180] (96×1.5×2 + (13 + 96×12.5%×1.5)×16 + 15 + 12) / 7008

[0181] = (288 + 208 + 288 + 15 + 12) / 7008

[0182] = 811 / 7008≈11.57%.

[0183] Compression ratio: approximately 8.6:1.

[0184] In the above formula: 96×1.5×2 is the total space of total electricity consumption E and reverse total electricity consumption RE.

[0185] The number of meters using both the forward and reverse rates of the eight-rate system is extremely small, less than 1%. Therefore, it has little impact on the overall data volume.

[0186] Step S4045: Summarize the metadata area, column description area, and compressed data area to obtain the compressed data block.

[0187] Specifically, the three logical partitions (metadata area, column description area, and compressed data area) are assembled together in a predetermined order to form a complete, self-contained compressed data block, which serves as the final output of the entire compression process.

[0188] This embodiment provides a method for compressing electricity consumption data. By systematically transforming the extracted row / column markers into a hierarchical, adaptive compressed data structure, a closed loop from data feature analysis to efficient storage is achieved. The metadata area provides global oversight, the column description area provides detailed descriptions, and the compressed data area focuses on storing effective information. The collaboration of these three elements ensures that the compression process can fully utilize the row and column markers of the data to achieve an extremely high compression ratio while maintaining complete structural information for lossless restoration, thus achieving an optimal balance between compression efficiency and data integrity.

[0189] As one or more specific application embodiments of the present invention, the electricity data compression method provided by the present invention will be further described in detail, and the specific process is as follows:

[0190] 1) Collect raw electricity consumption data: The smart meter collects raw electricity consumption data every 15 minutes, including total electricity consumption (E), electricity consumption at each tariff rate (E1-E8), total reverse electricity consumption (RE), and electricity consumption at each reverse tariff rate (RE1-RE8).

[0191] It should be noted that four-rate meters only include E and E1-E4, eight-rate meters include E and E1-E8, and eight-rate reverse meters include E, E1-E8, RE, and RE1-RE8.

[0192] 2) Data concentrator stores data: An edge data concentrator receives raw electricity consumption data from multiple smart meters and stores the raw electricity consumption data in the raw data cache.

[0193] 3) The compression processing module processes the raw electricity consumption data, including: analyzing the raw electricity consumption data and generating the compressed block metadata area content; analyzing each column of data and marking columns with all zeros as disabled; counting the number of zero values ​​in data columns to determine whether to enable the column description flag; after all the above operations are completed, generating the compressed block column description area content; compressing and encoding non-zero electricity consumption data using the Varint (or Simple8b algorithm); and assembling the metadata area, the column description area of ​​enabled columns, and the compressed electricity consumption data into a complete compressed data block.

[0194] 4) Network transmission equipment: All data compressed by the edge data concentrators are transmitted to the cloud platform via the wireless communication module.

[0195] 5) Cloud platform decompresses data: The cloud platform parses the data packets, reads the metadata area information, decodes the compressed data area according to the algorithm identifier, and fills the decompressed non-zero values ​​into the correct position of an all-zero matrix according to the metadata and column description information, thus reconstructing the original 96-row × 18-column electricity consumption data matrix.

[0196] 6) Electricity consumption data big data platform storage for subsequent applications: The electricity consumption data big data platform stores the decompressed complete data (i.e., the reconstructed electricity consumption data matrix) for subsequent advanced applications such as electricity bill calculation, electricity consumption behavior analysis, load forecasting, and big data dashboard display.

[0197] The compressed data block structure includes a metadata area, a column description area, and a compressed data area.

[0198] 1. The metadata area is 15 bytes, including:

[0199] Row deletion flag: Represented by DeleteFlag, 12 bytes, used to identify logically deleted rows. If the row data is all 0s or contains abnormal data, the DeleteFlag is marked on the row of electricity consumption data during data cleaning (there are a fixed 96 electricity consumption records per day, 96 bits for row deletion flag, each bit indicates whether a row is deleted, 96 bits = 12 bytes. The original electricity consumption data uses 1 byte DeleteFlag to identify each row of data, totaling 96 bytes, which saves seven-eighths of the space compared to the previous solution).

[0200] The column enable flag, consisting of 18 bits, indicates whether the 18 electricity data columns E, E1-E8, RE, and RE1-RE8 are enabled. If there is no data in the column, it is not enabled (for example, for a four-rate meter, if there is no data in the electricity data columns E2, E3, and E4 during peak hours, these three columns are not enabled).

[0201] Compression algorithm identifier: 6 bits, used to identify the type of compression algorithm used.

[0202] 2. Column description area:

[0203] Column data description enabled flag: 1 byte / column, used to identify whether column data description is enabled for this column.

[0204] Column data description flag (enabled on demand): 12 bytes / column. The column description is only available when enabled. It is used to describe whether the data in each row of the column is 0 (96 rows of data, 96 bits in total, each bit describes the data in one row).

[0205] 3. Compressed Data Area: Stores non-zero electricity consumption data values ​​using the selected compression algorithm.

[0206] The electricity consumption data compression method provided in this embodiment utilizes the characteristics of sparse rows and columns and continuous distribution of zero values ​​in electricity consumption data to process multiple electricity consumption data in a centralized manner. The zero values ​​are not stored, and the other values ​​are compressed, stored and sent after marking their positions in the matrix rows and columns. They can be restored according to the markings later, which improves compression efficiency and real-time performance and solves the problem of reduced compression efficiency and real-time performance in traditional electricity consumption data compression methods.

[0207] This embodiment also provides an electrical data compression device for implementing the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, hardware implementations, or a combination of software and hardware, are also possible and contemplated.

[0208] This embodiment provides a power consumption data compression device, such as... Figure 5 As shown, it includes:

[0209] The electricity data acquisition module 501 is used to acquire raw electricity data.

[0210] The electricity consumption data matrix generation module 502 is used to generate an electricity consumption data matrix based on the original electricity consumption data.

[0211] The matrix marker generation module 503 is used to determine whether the data in the electricity consumption data matrix is ​​valid and to generate matrix markers.

[0212] The compressed data block generation module 504 is used to filter valid data based on matrix markings, compress the valid data using a compression algorithm, and generate compressed data blocks in combination with matrix markings.

[0213] In some alternative implementations, the electricity consumption data matrix generation module 502 includes:

[0214] The raw electricity consumption data recording unit is used to acquire raw electricity consumption data at multiple time points within a preset time period, with preset time intervals as the time points; the raw electricity consumption data includes multiple time points and multiple electricity consumption types.

[0215] The electricity consumption data matrix generation unit is used to generate an electricity consumption data matrix with each time point as a row and electricity consumption type as a column.

[0216] In some alternative implementations, the matrix marker generation module 503 includes:

[0217] The row deletion marker generation unit is used to determine if a row of data in the electricity consumption data matrix has been logically deleted or is invalid, and to generate a row deletion marker.

[0218] The column enable tag generation unit is used to determine columns in the electricity data matrix where all data are preset invalid values ​​and generate column enable tags.

[0219] In some optional implementations, the row deletion marker generation unit includes:

[0220] The first judgment subunit is used to traverse each row of the electricity consumption data matrix and determine whether the current row of data meets the preset invalidity condition.

[0221] The first marker subunit is used to set the value of the row deletion marker at the corresponding position to a first value indicating logical deletion or data abnormality if a preset invalidity condition is met; otherwise, it is set to a second value indicating valid data.

[0222] In some optional implementations, the column enable tag generation unit includes:

[0223] The second judgment subunit is used to traverse each column of the electricity consumption data matrix and determine whether all data elements in the current column are preset invalid values.

[0224] The second marker subunit is used to set the value at the corresponding position in the column enable marker to a third value indicating that the column is disabled if all of them are preset invalid values; otherwise, it is set to a fourth value indicating that the column is enabled.

[0225] In some alternative implementations, the compressed data block generation module 504 includes:

[0226] Metadata area building units are used to build metadata areas based on row deletion flags and column enable flags.

[0227] The column description area construction unit is used to determine whether to generate a column data description flag for each valid column indicated by the column enable flag, based on the distribution characteristics of the valid column data, and to construct the column description area based on the generated column data description flag.

[0228] The filtering unit is used to filter out all valid data that needs to be compressed from the electricity consumption data matrix based on row deletion flags, column enable flags, and column data description flags.

[0229] The compressed data area construction unit is used to compress and encode the selected valid data using a compression algorithm, and to construct the encoding result into a compressed data area.

[0230] The summary unit is used to summarize the metadata area, column description area, and compressed data area to obtain a compressed data block.

[0231] In some optional implementations, the compressed data area construction unit includes:

[0232] The encoding subunit is used to take the selected valid data as input, call the compression algorithm for encoding processing, and generate the corresponding compressed data stream.

[0233] The write sub-unit is used to sequentially write the compressed data stream into the storage space reserved for compressed data, thus constructing the compressed data area.

[0234] The power consumption data compression device provided in this embodiment of the invention can execute the power consumption data compression method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects for executing the method. Further functional descriptions of the various modules and units described above are the same as in the corresponding embodiments described above, and will not be repeated here.

[0235] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention.

[0236] The following is a detailed reference. Figure 6 This diagram illustrates a suitable structural design for implementing an electronic device according to embodiments of the present invention. The electronic device may include a processor (e.g., a central processing unit, graphics processor, etc.) 601, which can perform various appropriate actions and processes based on a program stored in read-only memory (ROM) 602 or a program loaded from memory 608 into random access memory (RAM) 603. RAM 603 also stores various programs and data required for the operation of the electronic device. The processor 601, ROM 602, and RAM 603 are interconnected via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0237] Typically, the following devices can be connected to I / O interface 605: input devices 606 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 607 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; memory devices 608 including, for example, magnetic tapes, hard disks, etc.; and communication devices 609. Communication device 609 allows electronic devices to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 6 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown, and more or fewer devices may be implemented or have instead.

[0238] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 609, or installed from a memory 608, or installed from a ROM 602. When the computer program is executed by the processor 601, it performs the functions defined in the power consumption data compression method of the embodiments of the present invention.

[0239] Figure 6The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of the present invention.

[0240] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code. When the software or computer code is accessed and executed by the computer, processor, or hardware, the electrical data compression method shown in the above embodiments is implemented.

[0241] A portion of this invention can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to the invention through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled program, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0242] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A method for compressing electrical data, characterized in that, The method includes: Obtain raw electricity consumption data; An electricity consumption data matrix is ​​generated based on the original electricity consumption data; Based on the original electricity consumption data, an electricity consumption data matrix is ​​generated, including: Generate an electricity consumption data matrix with each time point as a row and electricity consumption type as a column; Determine whether the data in the electricity consumption data matrix is ​​valid and generate a matrix marker; determining whether the data in the electricity consumption data matrix is ​​valid and generating a matrix marker includes: Determine which rows of data in the electricity consumption data matrix have been logically deleted or are invalid, and generate a row deletion marker; Determine which rows of data in the electricity consumption data matrix have been logically deleted or are invalid, and generate row deletion markers, including: If the preset invalidity condition is met, the value at the corresponding position of the row deletion marker is set to the first value indicating logical deletion or data abnormality; otherwise, it is set to the second value indicating valid data. If a column in the electricity consumption data matrix contains all preset invalid values, a column enable flag is generated. If a column in the electricity consumption data matrix contains only preset invalid values, a column enable flag is generated, including: If all values ​​are preset invalid values, the value at the corresponding position in the column enable flag is set to the third value indicating disabling; otherwise, it is set to the fourth value indicating enabling. Based on the matrix markers, valid data is filtered, and a compression algorithm is used to compress the valid data, and compressed data blocks are generated by combining the matrix markers. Based on the matrix markers, valid data is filtered, and a compression algorithm is used to compress the valid data. A compressed data block is then generated by combining the matrix markers, including: Based on the row deletion flag and column enable flag, construct the metadata area; For each valid column indicated by the column enable flag, based on the distribution characteristics of the valid column data, determine whether to generate a column data description flag for the corresponding column, and construct a column description area based on the generated column data description flag; Based on the row deletion flag, column enable flag, and column data description flag, all valid data that needs to be compressed are filtered out from the electricity consumption data matrix; The selected valid data is compressed and encoded using a compression algorithm, and the encoding result is used to construct a compressed data area. The compressed data block is obtained by summarizing the metadata area, column description area, and compressed data area.

2. The method for compressing electricity consumption data according to claim 1, characterized in that, Based on the original electricity consumption data, an electricity consumption data matrix is ​​generated, including: The system acquires raw electricity consumption data at multiple time points within a preset time period, using a preset time interval as the time point; the raw electricity consumption data includes multiple time points and multiple electricity consumption types.

3. The method for compressing electricity consumption data according to claim 1, characterized in that, Determine which rows of data in the electricity consumption data matrix have been logically deleted or are invalid, and generate row deletion markers, including: Traverse each row of the electricity consumption data matrix and determine whether the current row of data meets the preset invalidity condition.

4. The method for compressing electricity consumption data according to claim 1, characterized in that, If a column in the electricity consumption data matrix contains only preset invalid values, a column enable flag is generated, including: Traverse each column of the electricity consumption data matrix and determine whether all data elements in the current column are preset invalid values.

5. The method for compressing electricity consumption data according to claim 1, characterized in that, The step of using a compression algorithm to compress and encode the selected valid data, and constructing the encoding result into a compressed data area, includes: The selected valid data is used as input, and a compression algorithm is called to perform encoding processing to generate the corresponding compressed data stream. The compressed data stream is sequentially written into the storage space reserved for compressed data to construct the compressed data area.

6. A power data transmission system, characterized in that, The system includes: A cloud platform and a compression subsystem applied to the electricity data compression method according to any one of claims 1 to 5; the compression subsystem includes terminal equipment and edge equipment; The terminal device is used to acquire raw electricity consumption data; The edge device is configured to generate an electricity consumption data matrix based on the original electricity consumption data; determine whether the data in the electricity consumption data matrix is ​​valid and generate a matrix marker; filter valid data based on the matrix marker and compress the valid data using a compression algorithm, and generate compressed data blocks in combination with the matrix marker; determining whether the data in the electricity consumption data matrix is ​​valid and generating a matrix marker includes: determining rows in the electricity consumption data matrix that have been logically deleted or are invalid, and generating a row deletion marker; determining columns in the electricity consumption data matrix where all data are preset invalid values, and generating a column enable marker; Based on the original electricity consumption data, an electricity consumption data matrix is ​​generated, including: Generate an electricity consumption data matrix with each time point as a row and electricity consumption type as a column; Determine which rows of data in the electricity consumption data matrix have been logically deleted or are invalid, and generate row deletion markers, including: If the preset invalidity condition is met, the value at the corresponding position of the row deletion marker is set to the first value indicating logical deletion or data abnormality; otherwise, it is set to the second value indicating valid data. If a column in the electricity consumption data matrix contains only preset invalid values, a column enable flag is generated, including: If all values ​​are preset invalid values, the value at the corresponding position in the column enable flag is set to the third value indicating disabling; otherwise, it is set to the fourth value indicating enabling. Based on the matrix markers, valid data is filtered, and a compression algorithm is used to compress the valid data. A compressed data block is then generated by combining the matrix markers, including: Based on the row deletion flag and column enable flag, construct the metadata area; For each valid column indicated by the column enable flag, based on the distribution characteristics of the valid column data, determine whether to generate a column data description flag for the corresponding column, and construct a column description area based on the generated column data description flag; Based on the row deletion flag, column enable flag, and column data description flag, all valid data that needs to be compressed are filtered out from the electricity consumption data matrix; The selected valid data is compressed and encoded using a compression algorithm, and the encoding result is used to construct a compressed data area. By summing the metadata area, column description area, and compressed data area, a compressed data block is obtained; The cloud platform is connected to the edge device. The cloud platform is used to receive compressed data blocks output by the edge device, decompress the compressed data blocks, and restore the electricity consumption data matrix.

7. A power data compression device, characterized in that, The device includes: The electricity consumption data acquisition and electricity consumption data matrix generation module is used to acquire raw electricity consumption data. The electricity consumption data matrix generation module is used to generate an electricity consumption data matrix based on the original electricity consumption data. Based on the original electricity consumption data, an electricity consumption data matrix is ​​generated, including: Generate an electricity consumption data matrix with each time point as a row and electricity consumption type as a column; A matrix marker generation module is used to determine whether the data in the electricity consumption data matrix is ​​valid and to generate matrix markers. The matrix label generation module includes: The row deletion marker generation unit is used to determine the row data in the electricity data matrix that has been logically deleted or is invalid, and to generate a row deletion marker. The column enable tag generation unit is used to determine columns in the electricity consumption data matrix where all data are preset invalid values, and generate column enable tags. The compressed data block generation module is used to filter valid data based on the matrix markers, compress the valid data using a compression algorithm, and generate compressed data blocks in combination with the matrix markers. The compressed data block generation module includes: Metadata area building units are used to construct metadata areas based on row deletion flags and column enable flags; The column description area construction unit is used to determine whether to generate a column data description flag for each valid column indicated by the column enable flag, based on the distribution characteristics of the valid column data, and to construct the column description area based on the generated column data description flag. The filtering unit is used to filter out all valid data that needs to be compressed from the electricity consumption data matrix based on row deletion flags, column enable flags, and column data description flags. The compressed data area construction unit is used to compress and encode the selected valid data using a compression algorithm, and to construct the encoding result into a compressed data area. The summary unit is used to summarize the metadata area, column description area, and compressed data area to obtain a compressed data block.

8. An electronic device, characterized in that, include: A memory and a processor are communicatively connected, the memory storing computer instructions, and the processor executing the computer instructions to perform the power data compression method according to any one of claims 1 to 5.

Images