A power distribution district data compression reconstruction method and system based on compressed sensing

Abstract

The application belongs to the technical field of power grid data processing, and particularly relates to a power distribution area data compression reconstruction method and system based on compressed sensing. The method comprises the following steps: analyzing the collection message of the power distribution area into a table, taking the data item name after hierarchical connection as the column name, the data item content as the cell data, and one collection message as one row; classifying the columns of the table according to the data type, compressing the column data of the table according to the data type of the column using the corresponding coding type to form a key-value pair; compressing the key-value pair into a transmission message and uploading the transmission message; decompressing the transmission message to obtain the key-value pair; decompressing the key-value pair using the corresponding coding type to obtain a reconstructed table; and analyzing the column name of the reconstructed table to reconstruct the content of each row of the reconstructed table into one reconstructed message. The application can establish the repeated content and the reuse structure among the collection messages, combine the same data, classify and compress, and improve the overall data compression efficiency.

Description

Technical Field

[0001] This invention belongs to the field of power grid data processing technology, specifically relating to a method and system for compressing and reconstructing distribution substation data based on compressed sensing. Background Technology

[0002] Data compression in distribution substations is crucial, as it forms the foundation of high-frequency power data acquisition. Without an efficient compression algorithm, the storage, transmission, and application of massive amounts of power data are impossible. Currently, acquired data is presented as hexadecimal protocol messages at the terminal layer, as JSON format strings at the edge, network, and platform layers, and stored in a database at the business layer. Due to the complex data types, high real-time requirements, and large concurrency of the acquired message content, management departments need to invest significant hardware, time, and manpower in maintaining the network and database to transmit and store incremental data.

[0003] A common solution is to compress the transmitted and stored data. A common method is the LZ77 compression algorithm used by Zip, which is an effective compression algorithm for text files. However, simply compressing a single data packet as text does not achieve the desired compression ratio.

[0004] Traditional text compression reduces storage space by statistically analyzing repeated characters and removing redundant characters or bits from bytes in the data to be compressed, relying on the statistical characteristics of the data. A typical text compression method, such as Huffman coding, first encodes the text (e.g., ASCII), then scans character frequencies to generate a character encoding tree, allocating encoding lengths according to tree depth. Its advantages are simplicity and practicality; however, its disadvantages include limited effectiveness for very short files, as storing the tree information requires 1024 bytes; and when character source probabilities are relatively similar, Huffman coding tends to produce fixed-length codes, resulting in low efficiency.

[0005] Compression within the meter data packets yields unsatisfactory results because: individual packets are short, less than 1KB, resulting in high encoding overhead; numerous data items are used for parsing, the packet structure is nested and layered, and the data types are diverse, leading to similar source distribution after encoding and resulting in the worst encoding performance. Under these circumstances, many non-text compression techniques cannot be implemented due to structural limitations. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention provides a method and system for compressing and reconstructing distribution radio area data based on compressed sensing. This method can identify duplicate content and reuse structures among acquired messages, merge similar data, classify and compress them, and improve the overall data compression efficiency.

[0007] To address the shortcomings of existing technologies, the technical solution provided by this invention is as follows:

[0008] A method for compressing and reconstructing distribution radio area data based on compressed sensing, comprising:

[0009] The collected messages from the distribution radio area are parsed into a table, with the data item name as the column name and the data item content as the cell data after hierarchical connection, and one collected message is parsed into one row;

[0010] The columns of the table are categorized according to data type, and the data in that column is compressed using the corresponding encoding type to form key-value pairs.

[0011] Compress the key-value pairs into a transmission message and then upload it;

[0012] Decompress the transmitted message to obtain key-value pairs;

[0013] Decompress the key-value pairs using the appropriate encoding type to obtain the reconstructed table;

[0014] After resolving the column names of the reconstructed table, each row of the reconstructed table is reconstructed into a reconstructed message.

[0015] Preferably, the column names derived from hierarchical connection of data items include:

[0016] Use the JSON flattened data item name - layer 1. layer 2. ... layer n as the column name;

[0017] The classification of table columns according to data type includes,

[0018] Scan the data in the table by column and divide the columns into text columns, timestamp columns, integer columns, floating-point number columns that do not conform to sparsity, and floating-point number columns that conform to sparsity.

[0019] Preferably, the step of compressing the column data of the table using the corresponding encoding type according to the data type of the column to form key-value pairs includes,

[0020] Compress the data in that column of the table according to its data type and use the corresponding encoding type to obtain the corresponding encoding block, forming key-value pairs {[key is column name: data type]:[value is encoding block]}, with a one-to-one correspondence between data type and encoding type.

[0021] Preferably, the step of compressing the column data of the table using the corresponding encoding type according to the data type of the column to form key-value pairs includes,

[0022] The text column is compressed using Huffman encoding to generate encoded blocks, resulting in key-value pairs with the column name: Str as the key.

[0023] The timestamp column is compressed using line encoding to generate encoded blocks, resulting in key-value pairs with the column name: Time as the key.

[0024] The integer column is compressed using incremental encoding to generate encoded blocks, resulting in key-value pairs with the column name as the key: Integer.

[0025] Compress floating-point sequences that do not conform to sparsity using arithmetic encoding; generate encoded blocks, generating key-value pairs with the column name: Float as the key;

[0026] Subsampling is achieved for a sparse floating-point sequence using Fourier transform and a random Gaussian matrix. The subsampling is used as a coding block to generate key-value pairs with the column name CS as the key.

[0027] Preferably, the step of decompressing the transmitted message to obtain key-value pairs includes,

[0028] The transmitted message is decompressed to generate multiple key-value pairs: {[key is column name: data type]:[value is encoded block]}.

[0029] The process of decompressing key-value pairs using the corresponding encoding type to obtain the reconstructed table includes,

[0030] Use a delimiter to split column name: data type into column name and data type. Decompress the encoding block according to the encoding type corresponding to the data type to get the column name and the corresponding column array. After all the encoding blocks are decompressed, the reconstructed table is obtained.

[0031] The process of parsing the column names of the reconstructed table and reconstructing each row of the table into a single reconstructed message includes:

[0032] The column names of the table are split into layers 1, 2, ..., n. The data of a row in the table is reconstructed into a reconstruction message in JSON format.

[0033] Preferably, the step of decompressing the encoded block according to the encoding type corresponding to the data type includes,

[0034] Huffman encoding is used to decompress the encoded block of data type Str;

[0035] The encoded block of data type Time is decompressed using piecewise linear encoding;

[0036] Arithmetic encoding is used to decompress coded blocks of data type Float;

[0037] Incremental encoding is used to decompress encoded blocks of data type Integer;

[0038] The Bayesian compressed sensing reconstruction algorithm is used to decompress the coded block of data type CS.

[0039] A data compression and reconstruction system for distribution radio areas based on compressed sensing, including an edge IoT agent and an IoT management platform;

[0040] The edge IoT agent includes a data storage unit, a data compression unit, and a data reporting unit;

[0041] The IoT management platform includes a data receiving unit, a decompression unit, and a data reconstruction unit;

[0042] The data storage unit is used to parse the collected messages of the distribution radio area into a table, with the data item name as the column name, the data item content as the cell data, and one collected message as a row;

[0043] The data compression unit is used to classify the columns of the table according to the data type, and compress the data of that column of the table using the corresponding encoding type according to the data type of the column to form key-value pairs;

[0044] The data reporting unit is used to compress key-value pairs into a transmission message and then upload it to the data receiving unit.

[0045] The data receiving unit is used to receive transmitted messages;

[0046] The decompression unit is used to decompress the transmitted message to obtain key-value pairs; and to decompress the key-value pairs using the corresponding encoding type to obtain a reconstructed table.

[0047] The data reconstruction unit is used to parse the column names of the reconstruction table and reconstruct each row of the reconstruction table into a reconstruction message.

[0048] Preferably, the data compression unit is used to scan the data of the table by column and divide the columns of the table into text columns, timestamp columns, integer columns, floating-point number columns that do not conform to sparsity, and floating-point number columns that conform to sparsity according to data type.

[0049] Preferably, the data compression unit is used for,

[0050] Compress the data in that column of the table according to its data type and use the corresponding encoding type to obtain the corresponding encoding block, forming key-value pairs {[key is column name: data type]:[value is encoding block]}, with a one-to-one correspondence between data type and encoding type.

[0051] Preferably, the decompression unit is used for,

[0052] The transmitted message is decompressed to generate multiple key-value pairs: {[key is column name: data type]:[value is encoded block]}.

[0053] Use a delimiter to split column name: data type into column name and data type. Decompress the encoding block according to the encoding type corresponding to the data type to obtain the column name and the corresponding column array. After all encoding blocks are decompressed, the reconstructed table is obtained.

[0054] The beneficial effects of this invention are:

[0055] This invention provides a data compression and reconstruction method for distribution radio areas based on compression sensing. By caching messages from multiple devices simultaneously and parsing and flattening them into tables, the source distribution of the compressed objects is changed through caching and columnar storage. Data types are marked by columns and classified, compressed, and encoded according to type. By reusing structures and adopting corresponding compression types for data types, coding redundancy is reduced, and the overall data compression efficiency is improved. The overall compression rate is better than using a single compression method globally. Attached Figure Description

[0056] Figure 1 The architecture diagram of the data compression and reconstruction system for distribution radio stations based on compression sensing provided by the present invention;

[0057] Figure 2 The architecture diagram of the edge IoT agent provided by this invention;

[0058] Figure 3 This is an architecture diagram of the IoT management platform provided by the present invention. Detailed Implementation

[0059] The present invention will be further described below with reference to the embodiments. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and should not be used to limit the scope of protection of the present invention.

[0060] This invention provides a method for compressing and reconstructing distribution radio area data based on compressed sensing, including:

[0061] Step 1: Collect the electricity meter data of the distribution transformer area and package the meter data into a collection message;

[0062] Below is an example of the data collection messages from two electricity meters at a certain point in time:

[0063] Data collection message of meter 1

[0064]

[0065] Data collection message from meter 2

[0066]

[0067]

[0068] It can be seen that the sampled packets consume a large amount of storage and bandwidth. Based on the end devices, there is a lot of duplicate content among the packets generated per unit time, which cannot be used during intra-frame compression.

[0069] Step 2: Parse the data collection messages from the distribution transformer area. For a single time point, the data collection messages from multiple meters are flattened and merged into a single table. The table uses JSON flattened data item names - layer 1. layer 2. ... layer n as column names, and the data item content as cell data. Each data collection message, representing the data from one meter, is listed as a row. See Table 1 for the corresponding tables for the data collection messages.

[0070] Table 1 shows the tables corresponding to the collected messages.

[0071]

[0072] This application exposes duplicate content in the data collection messages by concatenating data from different meters at the same time into a table, such as message upload time and edge device number in Table 1. Clearly duplicated content, such as proper nouns and message upload times, can be effectively compressed using traditional compression methods after being flattened into a table.

[0073] Step 3: Scan the table data column by column, and divide the table columns into text columns, timestamp columns, integer columns, non-sparse floating-point number columns, and sparse floating-point number columns according to data type. Generate arrays with fixed sequences and fixed data types. Compress the data of the column according to the corresponding encoding type to obtain the corresponding encoding block, forming key-value pairs of {[key is column name: data type]:[value is encoding block]}, with a one-to-one correspondence between data type and encoding type.

[0074] Specifically, text columns of data type String (text) are compressed using Huffman encoding to generate encoded blocks. The column name is then appended with the separator Str, resulting in key-value pairs with the key column name:Str. The data type corresponding to Str is text, and the corresponding encoding type is Huffman encoding. For example, the meter number column in Table 1 is compressed using Huffman encoding, generating key-value pairs with the key param.devSn:Str.

[0075] Timestamp columns of data type "Time" are compressed using piecewise linear encoding to generate encoded blocks. The column name is then appended with "Time" as a separator, resulting in key-value pairs with the key being column name:Time. The data type corresponding to "Time" is timestamp, and the corresponding encoding type is piecewise linear encoding. For example, the message upload timestamps in Table 1 are compressed using piecewise linear encoding, generating key-value pairs with the key "Timestamp:Time".

[0076] For non-sparse floating-point numbers of data type Float (not conforming to sparsity rules), arithmetic encoding is used to compress the data, generating encoded blocks. Float is appended to the column names as delimiters, resulting in key-value pairs with the key being column name:Float. The data type corresponding to Float is non-sparse floating-point numbers, and the corresponding encoding type is arithmetic encoding. For example, the meter current data column in Table 1 is compressed using arithmetic encoding, generating key-value pairs with the key param.ActivePower.dataValue:Float.

[0077] For integer columns of data type Integer, incremental encoding is used for compression to generate encoded blocks. Integer is appended to the column name using a delimiter, resulting in key-value pairs with the key being column name: Integer. The data type corresponding to Integer is integer, and the encoding type is incremental encoding. For example, the message upload number column in Table 1 is compressed using incremental encoding, generating key-value pairs with the key being mid: Integer.

[0078] For sparse floating-point numbers (CS) of data type, compressed sensing is used for compression. This involves subsampling through Fourier transform and random Gaussian matrices, using the subsampled data as encoding blocks, and appending CS to the column names with a delimiter to generate key-value pairs with the key as column name:CS. The data type corresponding to CS is sparse floating-point numbers, and the corresponding encoding type is compressed sensing. For example, the meter voltage data column in Table 1 is compressed using compressed sensing, generating key-value pairs with the key param.ThreePhaseVoltage.dataValue:CS. Electricity data has discrete characteristics, and compression using incremental encoding or text encoding methods is not ideal. Electricity data exhibits sparsity, and the compression results using compressed sensing theory are better.

[0079] Step 4: Compress the column name: data type key-value pairs into plain text to obtain the transmission message, and immediately transmit and upload it to the IoT management platform via the MQTT protocol after compression;

[0080] Step 5: The IoT management platform receives the compressed package, decompresses the transmitted message, and generates multiple key-value pairs of {[key is column name: data type]:[value is encoded block]}. The column name: data type is split into column name and data type using a delimiter. The encoded block is decompressed according to the encoding type corresponding to the data type to obtain the column name and the corresponding column array. After all encoded blocks are decompressed, the reconstructed table is obtained.

[0081] Specifically, Huffman coding is used to decompress the coded block of data type Str; piecewise linear coding is used to decompress the coded block of data type Time; arithmetic coding is used to decompress the coded block of data type Float; incremental coding is used to decompress the coded block of data type Integer; and compressed sensing is used to decompress the coded block of data type CS, that is, the original data is reconstructed through the Bayesian compressed sensing reconstruction algorithm.

[0082] Step 6: Split the column names of the table by level, that is, split the column names into the form of level 1, level 2, ..., level n, and then reconstruct the data of a row in the table into a reconstruction message in JSON format.

[0083] If the column name in column 1 is divided into layer 1, layer 2, and data item A, and the corresponding data is 1; and the column name in column 2 is divided into layer 1, layer 2, and data item B, and the corresponding data is 2, then the reconstructed message format is {layer 1:{layer 2:{data item A:1, data item B:2}}}.

[0084] This invention also provides a data compression and reconstruction system for distribution radio stations based on compressed sensing, see [link to relevant documentation]. Figure 1 This includes edge IoT agents and IoT management platforms;

[0085] Edge IoT agents, as edge layers, see [link / reference] Figure 1 and Figure 2 It includes a data acquisition unit, a data storage unit, a data compression unit, and a data reporting unit.

[0086] The IoT management platform, as a platform layer, see [link / reference]. Figure 3 It includes a data receiving unit, a decompression unit, a data reconstruction unit, a network monitoring unit, a data dumping unit, and a data forwarding unit.

[0087] The acquisition unit is used to collect meter data from the terminal layer of the distribution radio area and package the meter data into acquisition messages to send to the data storage unit.

[0088] The data storage unit is used to cache the collected messages, parse the collected messages from the distribution radio area into a table, and parse each collected message into a row by using the data item name as the column name and the data item content as the cell data after hierarchical connection.

[0089] The data compression unit is used to classify the columns of a table according to their data types, and compress the data in that column of the table using the corresponding encoding type to form key-value pairs.

[0090] The data reporting unit is used to compress key-value pairs into transmission messages and then transmit them to the IoT management platform via network protocols.

[0091] The data receiving unit is used to receive transmitted messages.

[0092] The decompression unit is used to decompress the transmitted message to obtain key-value pairs; decompress the key-value pairs using the corresponding encoding type to obtain column arrays and then obtain the reconstructed table; and obtain the compression ratio of each column by calculating the ratio of the value of the key-value pair (encoding block encoding length) to (the sum of the encoding lengths of the column arrays).

[0093] The data reconstruction unit is used to parse the column names of the reconstruction table and reconstruct each row of the reconstruction table into a reconstruction message. Specifically, the column names of the table are split into layers 1, 2, ..., n, and the data of one row in the table is reconstructed into a reconstruction message in JSON format.

[0094] The network monitoring unit is used to record the size of the transmitted messages received by the data receiving unit, calculate and record the overall compression ratio (transmitted message / reconstructed message), calculate and record the decompression speed (reconstructed message size / decompression time), and record the compression ratio in each column. The network monitoring unit simultaneously monitors the overall compression ratio and decompression speed, and simultaneously counts the data traffic of transmitted messages and the data traffic of reconstructed messages, and then obtains the historical average network traffic and cumulative network traffic of transmitted messages and reconstructed messages, which facilitates the monitoring of the overall operation of the data compression and reconstruction system of the distribution radio area based on compression sensing.

[0095] The data dump unit stores the transmission messages, as well as the recently decompressed reconstruction tables and reconstruction messages, for verifying whether the reconstruction results meet expectations. Storing the transmission messages can reduce the hardware resources and computation time required for data migration and backup, and reduce the difficulty of operation and maintenance.

[0096] The data forwarding unit is used to forward reconstruction messages to external distribution radio area application systems.

[0097] The compression unit is specifically used to scan the data in a table by column, and divide the table columns into text columns, timestamp columns, integer columns, floating-point number columns that do not conform to sparsity, and floating-point number columns that conform to sparsity according to data type.

[0098] The data compression unit is used to compress the data in a column of a table according to its data type using the corresponding encoding type, resulting in corresponding encoding blocks and forming key-value pairs of {[key: column name: data type]:[value: encoding block]}, where the data type and encoding type correspond one-to-one. Specifically, text columns are compressed using Huffman encoding to generate encoding blocks, producing key-value pairs with the key being column name: Str; timestamp columns are compressed using line encoding to generate encoding blocks, producing key-value pairs with the key being column name: Time; integer columns are compressed using incremental encoding to generate encoding blocks, producing key-value pairs with the key being column name: Integer; floating-point columns that do not conform to sparsity are compressed using arithmetic encoding to generate encoding blocks, producing key-value pairs with the key being column name: Float; and floating-point columns that conform to sparsity are subsampled using Fourier transform and a random Gaussian matrix, with the subsampled data used as encoding blocks, producing key-value pairs with the key being column name: CS.

[0099] The decompression unit is used to decompress the transmitted message, generating multiple key-value pairs of {[key: column name: data type]:[value: encoded block]}. It then uses a delimiter to split the column name: data type into column name and data type, and decompresses the encoded blocks according to the encoding type corresponding to the data type, obtaining the column names and their corresponding column arrays. After all encoded blocks are decompressed, a reconstructed table is obtained. Specifically, Huffman coding is used to decompress encoded blocks of data type Str; piecewise linear coding is used to decompress encoded blocks of data type Time; arithmetic coding is used to decompress encoded blocks of data type Float; incremental coding is used to decompress encoded blocks of data type Integer; and Bayesian compressed sensing reconstruction algorithm is used to decompress encoded blocks of data type CS.

[0100] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0101] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0102] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0103] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0104] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims. All of these forms are within the protection scope of the present invention.

[0105] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of this invention is defined by the appended claims and their equivalents.

Claims

1. A method for compressing and reconstructing distribution radio area data based on compressed sensing, characterized in that, include, The collected messages from the distribution radio area are parsed into a table, with the data item name as the column name and the data item content as the cell data after hierarchical connection, and one collected message is parsed into one row; The columns of the table are categorized according to data type, and the data in that column is compressed using the corresponding encoding type to form key-value pairs. Compress the key-value pairs into a transmission message and then upload it; Decompress the transmitted message to obtain key-value pairs; Decompress the key-value pairs using the appropriate encoding type to obtain the reconstructed table; After resolving the column names of the reconstructed table, each row of the reconstructed table is reconstructed into a reconstructed message; The column name, which is the data item name after hierarchical connection, includes: Use the JSON flattened data item name - layer 1. layer 2. ... layer n as the column name; The classification of table columns according to data type includes, Scan the data in the table by column and divide the columns of the table into text columns, timestamp columns, integer columns, floating-point number columns that do not conform to sparsity, and floating-point number columns that conform to sparsity according to data type; The step of compressing the data in that column of the table according to its data type using the corresponding encoding type to form key-value pairs includes: Compress the data in that column of the table according to its data type and use the corresponding encoding type to obtain the corresponding encoding block, forming key-value pairs {[key is column name: data type]:[value is encoding block]}, with a one-to-one correspondence between data type and encoding type.

2. The method for data compression and reconstruction of distribution radio stations based on compressed sensing according to claim 1, characterized in that, The step of compressing the data in that column of the table according to its data type using the corresponding encoding type to form key-value pairs includes: The text column is compressed using Huffman encoding to generate encoded blocks, resulting in key-value pairs with the column name: Str as the key. The timestamp column is compressed using line encoding to generate encoded blocks, resulting in key-value pairs with the column name: Time as the key. The integer column is compressed using incremental encoding to generate encoded blocks, resulting in key-value pairs with the column name as the key: Integer. Compress floating-point sequences that do not conform to sparsity using arithmetic encoding; generate encoded blocks, generating key-value pairs with the column name: Float as the key; Subsampling is achieved for a sparse floating-point sequence using Fourier transform and a random Gaussian matrix. The subsampling is used as a coding block to generate key-value pairs with the column name CS as the key.

3. The method for data compression and reconstruction of distribution radio stations based on compressed sensing according to claim 1, characterized in that, The key-value pairs obtained by decompressing the transmitted message include, The transmitted message is decompressed to generate multiple key-value pairs: {[key is column name: data type]:[value is encoded block]}. The process of decompressing key-value pairs using the corresponding encoding type to obtain the reconstructed table includes, Use a delimiter to split column name: data type into column name and data type. Decompress the encoding block according to the encoding type corresponding to the data type to get the column name and the corresponding column array. After all the encoding blocks are decompressed, the reconstructed table is obtained. The process of parsing the column names of the reconstructed table and reconstructing each row of the table into a single reconstructed message includes: The column names of the table are split into layers 1, 2, ..., n. The data of a row in the table is reconstructed into a reconstruction message in JSON format.

4. The method for data compression and reconstruction of distribution radio stations based on compressed sensing according to claim 3, characterized in that, The step of decompressing the encoded block according to the encoding type corresponding to the data type includes, Huffman encoding is used to decompress the encoded block of data type Str; The encoded block of data type Time is decompressed using piecewise linear encoding; Arithmetic encoding is used to decompress coded blocks of data type Float; Incremental encoding is used to decompress encoded blocks of data type Integer; The Bayesian compressed sensing reconstruction algorithm is used to decompress the coded block of data type CS.

5. A data compression and reconstruction system for distribution radio stations based on compressed sensing, characterized in that, This includes edge IoT agents and IoT management platforms; The edge IoT agent includes a data storage unit, a data compression unit, and a data reporting unit; The IoT management platform includes a data receiving unit, a decompression unit, and a data reconstruction unit; The data storage unit is used to parse the collected messages of the distribution radio area into a table, with the data item name as the column name, the data item content as the cell data, and one collected message as a row; The data compression unit is used to classify the columns of the table according to the data type, and compress the data of that column of the table using the corresponding encoding type according to the data type of the column to form key-value pairs; The data reporting unit is used to compress key-value pairs into a transmission message and then upload it to the data receiving unit. The data receiving unit is used to receive transmitted messages; The decompression unit is used to decompress the transmitted message to obtain key-value pairs; and to decompress the key-value pairs using the corresponding encoding type to obtain a reconstructed table. The data reconstruction unit is used to parse the column names of the reconstruction table and reconstruct each row of the reconstruction table into a reconstruction message. The column name, which is the data item name after hierarchical connection, includes: Use the JSON flattened data item name - layer 1. layer 2. ... layer n as the column name; The classification of table columns according to data type includes, Scan the data in the table by column and divide the columns of the table into text columns, timestamp columns, integer columns, floating-point number columns that do not conform to sparsity, and floating-point number columns that conform to sparsity according to data type; The step of compressing the data in that column of the table according to its data type using the corresponding encoding type to form key-value pairs includes: Compress the data in that column of the table according to its data type and use the corresponding encoding type to obtain the corresponding encoding block, forming key-value pairs {[key is column name: data type]:[value is encoding block]}, with a one-to-one correspondence between data type and encoding type.

6. A data compression and reconstruction system for distribution radio stations based on compressed sensing according to claim 5, characterized in that, The decompression unit is used for, The transmitted message is decompressed to generate multiple key-value pairs: {[key is column name: data type]:[value is encoded block]}. Use a delimiter to split the column name: data type into column name and data type. Decompress the encoding block according to the encoding type corresponding to the data type to obtain the column name and the corresponding column array. After all the encoding blocks are decompressed, the reconstructed table is obtained.

Images