A power message compression method and system

Abstract

This invention provides a method and system for compressing power messages, relating to the field of power system control technology. The method includes: acquiring power messages via a first FPGA; timestamping the acquired power messages; caching the timestamped messages in a first shared memory; extracting messages from the first shared memory; parsing the messages according to their type and classifying and storing them in a second shared memory; extracting messages from the second shared memory by control block using a second FPGA; compressing the compressed packages corresponding to the control blocks; storing the compressed packages in a third shared memory; packaging and secondary compression according to a set time period; generating a total compressed package for that time period; and storing it on an SSD for dual-disk storage. This invention effectively reduces the storage space requirements for historical messages, improves the storage time of historical messages, and solves the problem of frequent failures in storing historical messages for post-event analysis of power systems.

Description

Technical Field

[0001] This invention belongs to the field of power system control technology, and more specifically, relates to a power message compression method and system. Background Technology

[0002] With the technological development of intelligent substations, new energy, power distribution, and power plants, the application scope of power system services is becoming increasingly broad, and network problems are becoming more diverse. To more effectively trace and locate the root cause of problems, storing service messages for post-fault reproduction is becoming increasingly important. More and more power service scenarios are using IEC 61850's GOOSE (Generic Object Oriented Substation Event) and SV (Sampled Value) messages for real-time service transmission. SV / GOOSE messages are characterized by large message volume and high message format consistency. When transmitting across routers and other devices, routable messages such as GOOSE and SV are used. In direct cable sampling scenarios, analog sample values ​​need to be recorded. To more effectively record service message and sample value information and facilitate post-fault analysis, it is necessary to store and record a large amount of SV / GOOSE message and sample value information. Conventional storage methods consume a large amount of storage disk space, and frequent disk write operations will severely reduce the lifespan of the disk, leading to frequent disk malfunctions, loss of records, and often untraceable failures. Summary of the Invention

[0003] To address the shortcomings of existing technologies, this invention provides a power message compression method and system. The method involves: acquiring power messages using a first FPGA (Field Programmable Gate Array); timestamping the acquired power messages; caching the timestamped messages in a first shared memory; classifying and storing messages according to protocol type in the first shared memory; extracting messages from the first shared memory; parsing the messages according to their type and classifying and storing them in a second shared memory; extracting SV messages, ZC messages, and GOOSE messages from the second shared memory according to control blocks using a second FPGA; compressing these to obtain ZC (state variables), GOOSE, and multiple SV compressed packages; placing the ZC, GOOSE, and multiple SV compressed packages into a third shared memory; packaging and secondary compression according to a set time period to generate a total compressed package for that time period; and storing the total compressed package on a dual-disk SSD (Solid State Drive). This invention addresses the core challenge of large data volumes through FPGA hardware-accelerated differential compression, solves the difficulty of retrieving massive amounts of data through multi-level indexing from time and port to control blocks, and addresses disk lifespan issues by using variable-frequency small input / output (Small Data Read / Write) for large-block sequential writes. This forms a complete high-performance solution encompassing acquisition, compression, indexing, storage, and retrieval, significantly reducing storage costs and extending hardware lifespan while substantially improving the efficiency and accuracy of fault location. It addresses the problem of reduced disk lifespan and data loss caused by frequent disk storage of large amounts of data, reducing the amount of stored data through high compression ratios, decreasing disk write cycles, extending disk lifespan, and ensuring the validity of stored data.

[0004] The present invention adopts the following technical solution.

[0005] A first aspect of the present invention provides a method for compressing power line messages, comprising: The first FPGA collects power messages, timestamps the collected power messages, caches the timestamped messages in the first shared memory, and stores the messages according to the protocol type in the first shared memory. Extract messages from the first shared memory, parse the messages according to their type, and then classify and store them in the second shared memory; The second FPGA extracts SV messages, ZC messages, and GOOSE messages from the second shared memory according to control blocks and compresses them to obtain ZC, GOOSE, and multiple SV compressed packages for the corresponding control blocks. Place ZC, GOOSE and multiple SV compressed packages into the third shared memory, package and compress them again according to the set time period, and generate a total message compressed package for that time period. The total message compressed package is stored on an SSD for dual-disk storage.

[0006] Preferably, the compression of SV messages, ZC messages, and GOOSE messages extracted from the second shared memory by control block via the second FPGA includes: The second FPGA extracts SV messages, ZC messages, and GOOSE messages from the second shared memory according to control blocks, and performs length alignment to obtain a control block message sequence that has completed length alignment preprocessing. The control block message sequence that has undergone length alignment preprocessing is deduplicated and compressed to obtain the difference sequence; The difference sequence is compressed using a path-by-path compression algorithm to obtain the ZC, GOOSE and multiple SV compressed packages of the corresponding control block.

[0007] Preferably, the obtained control block message sequence after length alignment preprocessing includes: Select any field combination rule setting control block; The second FPGA matches the fields of each message according to the control block, and classifies all messages with the same field combination as the control block into the original message sequence of the corresponding control block, thus obtaining multiple control block message sequences. The adjacent messages in each control block message sequence are length aligned to obtain a control block message sequence that has undergone length alignment preprocessing.

[0008] Preferably, the field combination rule includes a single field using only the multicast MAC address or only the APPID, a combined field of multicast MAC address and VLAN ID and a combined field of APPID and VLAN ID, or a complete field combining multicast MAC address, VLAN ID and APPID.

[0009] Preferably, if the lengths of consecutive frames of adjacent messages in the control block message sequence are equal, byte-by-byte subtraction is performed directly; If the length of the preceding frame of an adjacent message in the control block message sequence is greater than that of the following frame, only the portion of the preceding frame and the following frame of equal length is truncated for subtraction calculation. The portion of the preceding frame that is longer than the following frame is not included in the calculation, resulting in a difference frame. If the length of the later frame of an adjacent message in the control block message sequence is greater than that of the previous frame, the insufficient fields in the previous frame are padded with zeros to make its length equal to that of the later frame, and then the difference frame is calculated.

[0010] Preferably, the deduplication compression employs a universal adjacent frame difference mode, including: Take the first frame of the control block message sequence that has completed length alignment preprocessing; calculate the inter-frame difference between the next frame and the previous frame in sequence, and so on, arranging all the inter-frame differences in sequence to form a difference sequence.

[0011] Preferably, the deduplication compression method for SV control block messages using the SV dedicated cycle deduplication mode includes: taking all sampling points of the first cycle as the reference cycle, subtracting each sampling point of the second cycle from the sampling point of the corresponding sequence number of the first cycle to obtain a set of inter-cycle difference sequence, and processing subsequent cycles accordingly.

[0012] Preferably, the total message compressed packet is indexed by timestamp and input port.

[0013] Preferably, the CPU locates and reads the corresponding total message compressed package from the dual-disk storage according to the time period and port requested by the user. The CPU sends the total message compressed package to the second FPGA for decompression to obtain all ZC, GOOSE and multiple SV compressed packages within the time period. The CPU retrieves the specified SV compressed package according to the preset keywords and continues to decompress it to obtain the original message. The CPU sends the decompressed original message to the corresponding receiving device for use.

[0014] A second aspect of the present invention provides a power message compression system, which operates the power message compression method described in the first aspect, comprising: The first FPGA module is used to collect power messages through the first FPGA, timestamp the collected power messages, and cache the timestamped messages in the first shared memory. The first shared memory module is used to classify and store packets according to protocol type through the first shared memory; The CPU module is used to extract packets from the first shared memory, parse the packets according to their type, and then classify and store them in the second shared memory. The second FPGA module is used to extract SV messages, ZC messages and GOOSE messages from the second shared memory according to the control block through the second FPGA and compress them to obtain the ZC, GOOSE and multiple SV compressed packages of the corresponding control block; The third shared memory module is used to put ZC, GOOSE and multiple SV compressed packages into the third shared memory, package and compress them again according to a set time period, and generate a total message compressed package for that time period. The dual-disk storage module is used to store the total compressed packet on an SSD hard drive for dual-disk storage.

[0015] Compared with the prior art, the beneficial effects of the present invention include at least the following: This invention uses a first FPGA for line-speed packet acquisition and high-precision timestamping, and utilizes a first shared memory to achieve high-speed, low-latency raw data buffering and preliminary classification according to the protocol. This solves the problems of the raw packet "stream data" being difficult to be processed stably and orderly by the subsequent CPU, as well as the insufficient time synchronization accuracy. It avoids packet loss caused by untimely CPU processing or system scheduling delays, laying a precise time-stamping foundation for subsequent accurate analysis and fault backtracking, improving the real-time performance, completeness, and time-stamping accuracy of data acquisition; and reducing the real-time interrupt load on the CPU and the risk of packet loss. This invention involves the CPU reading packets from shared memory, performing deep protocol parsing, and classifying the packets into different control blocks with fine granularity based on key network layer fields (combinations of multicast MAC, VLAN ID, APPID, etc.), storing them in a second shared memory. This solves the problem of the original packet stream being "chaotic and disordered," making it unsuitable for efficient compression. It organizes the chaotic packet stream into independent, clean data streams with the same characteristics and changing patterns, creating the necessary conditions for the next step of highly targeted "inter-frame differential" compression, improving the orderliness and structure of the packet data; and providing crucial data preparation for subsequent compression algorithms to achieve high compression ratios. This invention extracts a clean message stream from a second shared memory by control block using a second FPGA. Leveraging the message's high format consistency and minimal changes between adjacent frames, it employs an inter-frame differential algorithm (with optional cycle deduplication for SV messages) to convert continuous messages into a "first frame + difference sequence." Then, it uses general algorithms such as Huffman / LZ77 for further compression. Each compressed packet is indexed by a key field in the control block. This is the core step in solving the problem of "large message volume and huge storage space consumption." Utilizing the parallel computing capabilities of the FPGA, lossless, extremely high compression ratio processing is performed before data is written to disk, fundamentally reducing the total amount of data requiring physical storage and greatly improving compression efficiency and data compression ratio (due to extremely low data redundancy after differential processing). This significantly reduces the amount of raw data that needs to be stored and transmitted subsequently, directly alleviating storage space pressure. This invention aggregates the compressed packages of each control block according to a preset time period, performs secondary packaging and compression to form a total compressed package, and adds advanced indexes such as time period and acquisition port to it. This solves the problems of low storage efficiency of small files in massive control block compressed packages and inability to quickly locate data within the target time range during post-event retrieval. By repackaging, the overall compression rate is improved. By adding time-port level indexes, a first-level directory for fast retrieval is established, improving the overall compression rate and file management efficiency of the storage system, and initially establishing fast retrieval capabilities. This invention uses the CPU to write the total compressed packet with a time index into a larger data block format to an SSD, and employs a dual-disk backup strategy. This solves the problems of "frequent small data write operations leading to severely reduced disk lifespan" and data loss due to single-point failures. It transforms massive, high-frequency small message write operations into low-frequency, large-block compressed packet write operations, and utilizes the characteristics of SSDs and dual-disk redundancy to ensure data security, greatly improving the reliability and data security of the storage system; it also significantly reduces disk write frequency and wear, extending disk lifespan. This invention supports rapid location of the total compressed packet based on time and port indexes, and then precise extraction of the target control block's compressed packet from the total packet based on control block keywords. The FPGA performs inverse differential operations to accurately and on-demand recover the original message, solving the pain point of "difficulty in quickly tracing faults using recorded messages." Users do not need to decompress all massive amounts of data; they can quickly locate and recover the original message of the fault time, specific device, or line through multi-level indexes from time and port to specific data streams, much like searching a database. This greatly improves fault analysis efficiency, significantly enhancing the efficiency and accuracy of post-fault analysis and tracing; and reduces the time and manpower costs of fault troubleshooting. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the power message compression method provided according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the SV control block message deduplication calculation method provided in accordance with an embodiment of the present invention; Figure 3 This is a schematic diagram of the GOOSE control block message deduplication calculation method provided in accordance with an embodiment of the present invention; Figure 4 This is a schematic diagram of a complete deduplication and compression embodiment provided according to the present invention. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of this invention. The described embodiments are merely some embodiments of this invention, and not all embodiments. Based on the spirit of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the protection scope of this invention.

[0018] like Figure 1 As shown, Embodiment 1 of the present invention provides a method for compressing power transmission messages, comprising the following steps: Step 1: Collect power messages through the first FPGA, timestamp the collected power messages, and cache the timestamped messages in the first shared memory.

[0019] In a preferred but non-limiting embodiment of the present invention, step 1 includes: Step 1.1: Acquire power messages via the first FPGA, including SV messages, GOOSE messages, and ZC messages. The original acquired messages can be recorded in pcap format.

[0020] Step 1.2: Timestamp each power message by recording the collection time of each power message from Step 1.1.

[0021] Step 1.3: Store the timestamped message directly into the first shared memory.

[0022] Step 1.4: The first shared memory classifies and stores messages according to protocol type, namely SV, ZC, and GOOSE messages. Sampled values ​​are multiple inputs, which are queued in a fixed order within the FPGA to form an internal storage structure, and then placed into a category for subsequent processing.

[0023] Step 2: The CPU extracts the message from the first shared memory, and after parsing the power message according to the message type, it is classified and stored in the second shared memory.

[0024] Step 3: Extract SV messages, ZC messages, and GOOSE messages from the second shared memory according to control blocks using the second FPGA, compress them, and obtain the ZC, GOOSE, and multiple SV compressed packages for the corresponding control blocks.

[0025] In a preferred but non-limiting embodiment of the present invention, step 3 includes: Step 3.1: Extract SV messages, ZC messages, and GOOSE messages from the second shared memory according to control blocks using the second FPGA, and perform length alignment to obtain a control block message sequence that has completed length alignment preprocessing.

[0026] More preferably, step 3.1 includes: Step 3.1.1, select any field combination rule setting control block. Field combination rules include a single field using only the multicast MAC (Media Access Control Address) address or only the APPID (Application Identification) field, a combination field of multicast MAC address and VLAN ID (VLAN Identifier) ​​combined with APPID (Application Identification) and VLAN ID, or a complete field combining multicast MAC address, VLAN ID, and APPID.

[0027] Step 3.1.2: The second FPGA matches the fields of each packet according to the control block, and includes all packets with the same field combination as the control block into the original packet sequence of the corresponding control block, resulting in multiple control block packet sequences. For example, but not limited to, all SV packets matching multicast MAC-A and VLAN 100 are included in the sequence of control block A.

[0028] Step 3.1.3: Perform length alignment on adjacent messages in each control block message sequence to obtain a control block message sequence that has completed length alignment preprocessing.

[0029] If the frame lengths of adjacent messages in the control block message sequence are equal, perform byte-by-byte subtraction directly; If the length of the preceding frame of an adjacent message in the control block message sequence is greater than that of the following frame, only the portion of the preceding frame and the following frame of equal length is truncated for subtraction calculation. The portion of the preceding frame that is longer than the following frame is not included in the calculation, resulting in a difference frame. If the length of the later frame of an adjacent message in a control block message sequence is greater than that of the earlier frame, the insufficient fields in the earlier frame are padded with zeros to make its length equal to that of the later frame, and then the difference frame is calculated. The length of the calculated difference frame is recorded based on the length of the later frame.

[0030] Step 3.2: Deduplicatize and compress the control block message sequence that has completed length alignment preprocessing to obtain the difference sequence.

[0031] More preferably, step 3.2 includes: Step 3.2.1, the general adjacent frame difference mode for deduplication and compression includes: taking the first frame of the control block message sequence that has completed length alignment preprocessing; calculating the inter-frame difference between the next frame and the previous frame in sequence, and so on, arranging all inter-frame differences in sequence to form a difference sequence, such as... Figure 3 As shown.

[0032] The collected packets are classified according to GOOSE, SV, and ZC control blocks. For each control block, the first frame (Frame1) is taken as the reference frame. The difference between the second frame (Frame2) and the first frame (Frame2) is calculated byte-wise to obtain ΔFrame1 = Frame2 - Frame1. The difference between the third frame (Frame3) and the second frame (Frame3) is calculated byte-wise to obtain ΔFrame2 = Frame3 - Frame2, and so on. The packets of this control block are arranged according to Frame1, ΔFrame1, ΔFrame2, ... to obtain the sequence Queue1. This sequence Queue1 is compressed to obtain the compressed packet Compress1 for this control block. The other control blocks are compressed in the same way to obtain Compress2, Compress3, ... The compressed packets of all control blocks are packaged and compressed to obtain the compressed packet All-Compress for all packets. One control block corresponds to one compressed packet, and multiple control blocks may exist simultaneously.

[0033] Step 3.2.2, Deduplication and Compression: For SV control block messages, a dedicated SV cycle deduplication mode can also be used, including: using all sampling points of the first cycle as the reference cycle; subtracting the corresponding sampling point of the first cycle from each sampling point of the second cycle to obtain a sequence of inter-cycle differences; and processing subsequent cycles accordingly, such as... Figure 2 As shown.

[0034] Compression of an SV control block message can also use a cycle deduplication method, with N samples per cycle, where N defaults to 80. The first sampled SV of the first cycle is marked as SV1_1, the second sampled SV as SV1_2, and so on, with the last sampled SV of the first cycle being 1_80. The first cycle message of each control block is used as a reference frame. The 80 samples of the second cycle are labeled SV2_1, SV2_2, ..., SV2_80. The samples of the first cycle, SV1_1, SV1_2, ..., SV1_80, are saved. The second cycle is processed by subtracting the corresponding sequence number of the message from the first cycle in sequence: △S2_1 = SV2_1 - SV1_1, △S2_2 = SV2_2 - SV1_2, ..., △S2_80 = SV2_80 - ​​SV1_80. The third cycle and subsequent sampled messages are processed in the same way to obtain a new sequence SV1_1, SV1_2, ..., SV1_80, △S2_1, △S2_2, ..., △S2_80, ... The messages in this sequence are compressed to obtain Compress1. The SV message processing method for other control blocks is the same; the decompression process is the reverse of the compression process.

[0035] In step 1, the acquisition message of FPGA1 is the acquisition message of all control blocks within a certain period of time. The duration of the time period can be adjusted according to the actual application requirements, for example, the time range is 1 second to 60 seconds.

[0036] Step 3.3: Compress the difference sequence using a path-by-path compression algorithm to obtain SV, ZC, and GOOSE message compressed packets. The path-by-path compression algorithm can be various common compression algorithms such as Huffman coding, LZ77 (Lempel Ziv 77 dictionary algorithm), and RLE (run length encoding), or a combination of Huffman and RLE compression algorithms can be used to obtain ZC, GOOSE, and multiple SV compressed packets for the corresponding control blocks.

[0037] When compressing compressed files, Compress adds keywords identified by the control block as an index, allowing for quick retrieval of historical messages containing specific keywords when using stored messages. Each compressed file uses the keywords of the control block as its index and filename.

[0038] Step 4: After compressing the ZC, GOOSE, and multiple SV compressed packages obtained in Step 3, place them into the third shared memory. Then, package and compress the above compressed packages according to the set time period to generate a total message compressed package for that time period.

[0039] In a preferred but not limiting embodiment of the present invention, step 4 includes: Step 4.1, the compressed ZC, GOOSE and multiple SV archives obtained in step 3 are placed into the third shared memory.

[0040] When performing All-Compress compression of the total message package, timestamps and input ports are added as indexes to the compressed messages. This allows for quick retrieval of historical messages from a specified time and port when using stored messages. The All-Compress package can use the time and the collection port number as the filename. All-Compress is a collection of compressed packages containing all compressed packages for a specific time period and port.

[0041] Step 4.2: Pack the compressed packets in the third shared memory into packages and perform secondary compression according to the set time periods to generate a total compressed packet for that time period.

[0042] Step 5: The CPU is responsible for compressing the total message packets corresponding to multiple time periods in Step 4 and storing them on the SSD hard drive for dual-disk storage.

[0043] Step 6: When it is necessary to read historical messages, the CPU locates and reads the corresponding total message compressed package from the SSD according to the time period and port requested by the user. The CPU sends the total message compressed package to the second FPGA for decompression to obtain all ZC, GOOSE and multiple SV compressed packages within the time period. The CPU retrieves the specified SV compressed package according to the preset keywords and continues to decompress it to obtain the original message. The CPU sends the decompressed original message to the corresponding receiving device for use.

[0044] During decompression, All-Compress decompression is performed first, and then the corresponding Compress is decompressed as needed. The decompressed Frama1, △Frame1, △Frame2, ... sequences are used to recover the original message. △Frame1 is added to Frama1 to recover Frame2, △Frame2 is added to Frama2 to recover Frame3, and so on, recovering all △Frames one by one. Finally, the Frama1, Frama2, Frama3, ... sequences of the control block are obtained, and the original message is saved according to pcap (packet capture) format.

[0045] This method is also applicable to compressing and storing other types of messages with uniform format and length. For example, it can be used to compress and store other messages with the same characteristics, such as multiple analog sample values ​​stored in a fixed format.

[0046] Embodiment 2 of the present invention provides a power message compression system, which operates the power message compression method described in Embodiment 1, comprising: a first FPGA module, used to collect power messages through the first FPGA, timestamp the collected power messages, and cache the timestamped messages in a first shared memory; a first shared memory module, used to classify and store messages according to protocol type through the first shared memory; a CPU module, used to extract messages from the first shared memory, parse the messages according to message type, and classify and store them in a second shared memory; a second FPGA module, used to extract SV messages, ZC messages, and GOOSE messages from the second shared memory according to control blocks through the second FPGA and compress them to obtain ZC, GOOSE, and multiple SV compressed packages corresponding to the control blocks; a third shared memory module, used to put the ZC, GOOSE, and multiple SV compressed packages into the third shared memory, package and compress them again according to a set time period to generate a total message compressed package for that time period; and a dual-disk storage module, used to store the total message compressed package on an SSD hard drive for dual-disk storage.

[0047] Embodiment 3 of the present invention provides that the original message of a control block is an SV message. The key search terms of the original message are: DMAC is 01-0C-CD-04-00-01, VLANID is 2, and APPID is 0x4001.

[0048] The original message contains three frames: SV1, SV2, and SV3. First, the control block message is deduplicated, keeping SV1 unchanged. Then, ΔSV21 = SV2 - SV1 and ΔSV32 = SV3 - SV2 are calculated. This forms the deduplicated control block sequence SV1, ΔSV21, and ΔSV32. As shown in the figure, the deduplicated ΔSV21 and ΔSV32 frames only have a few fields with non-zero differences; the differences in most other fields are zero.

[0049] Sequences SV1, △SV21, and △SV32 are compressed in real time using compression algorithms such as LZ77, Huffman, LRE, LZ77, LZW (Lempel-Ziv-Welch encoding), LZO (Lempel-Ziv-Oberhumer encoding), and DEFLATE to obtain a compressed file of the control block. The compressed file of the control block is named 01-0C-CD-04-00-01_2_4001.

[0050] Multiple control block messages collected within the same time period are compressed separately to form multiple compressed files. These compressed files for the same time period are then packaged and compressed together to form a single compressed file for that time period. For example, if the sampling time is 2025-07-16 10:32:03 and the sampling port is P1, and within 1 second after that time, messages with the keywords 01-0C-CD-04-00-01_2_4001, 01-0C-CD-04-00-01_2_4002, and 01-0C-CD-01-00-01_2_1001 are collected and compressed separately to form three control block compressed packages, then these control block compressed packages are packaged and compressed together to form a single compressed file named 2025-07-16_10:32:03_P1.

[0051] When you need to retrieve the original packet with the keywords DMAC: 01-0C-CD-04-00-01, VLANID: 2, APPID: 0x4001 at 10:32:03 on 2025-07-16, first decompress 2025-07-16_10:32:03_P1 to obtain the compressed package 01-0C-CD-04-00-01_2_4001. Decompressing the 01-0C-CD-04-00-01_2_4001 compressed package will then yield the desired packet. Similarly, GOOSE, SV, and analog sample values ​​can all be obtained using the same... Figure 1 The same process is used to process them.

[0052] Figure 4 This is a specific application example of the high-ratio compression method for power messages provided by the present invention, such as... Figure 4 As shown, this example illustrates a complete implementation block diagram for message acquisition, compression, and decompression. This implementation uses two FPGAs and one CPU. The sampled data includes SV, GOOSE, and ZC (direct analog sampling) information acquisition. All of these information can be compressed to a high ratio. The entire process is illustrated using SV as an example.

[0053] FPGA1 acquires SV1 control block messages, records the acquisition time (timestamping), and places the acquired SV control block messages into shared memory 1. Shared memory 1 stores messages categorized by protocol type: SV, ZC, and GOOSE. The CPU extracts messages from shared memory 1, categorizes GOOSE and SV messages by type, and stores them into shared memory 2. SV messages are then divided into control blocks and stored in shared memory 2. FPGA2 extracts SV messages from shared memory 2 by control block and compresses them separately. ZC and GOOSE messages are extracted and compressed. In this use case, the GOOSE message traffic is relatively small, so a high compression ratio method is not used. Compressed ZC, GOOSE, and multiple SV compressed packages are placed into shared memory 3. These compressed packages are then compressed together to obtain a compressed package for the current time period and placed into shared memory 4. The CPU is responsible for storing the entire compressed package on an SSD.

[0054] When it is necessary to find a specific SV control block message within a certain time period, the entire compressed package is extracted according to the time period. The CPU extracts it and puts it into FPGA2 for decompression. After decompression, multiple SV control block compressed packages and GOOSE and ZC compressed packages for that time period are obtained. The specified SV compressed package is retrieved by keyword and decompressed to obtain the original message. The CPU sends the decompressed message to the corresponding receiving device for use.

[0055] Compared with the prior art, the beneficial effects of the present invention include at least the following: The present invention uses a first FPGA for line-speed acquisition of messages and high-precision timestamping, and utilizes a first shared memory to achieve high-speed, low-latency raw data buffering and preliminary classification according to the protocol. This solves the problems that the raw message "stream data" is difficult to be processed stably and orderly by the subsequent CPU, as well as the problem of insufficient time synchronization accuracy. It avoids message loss caused by untimely CPU processing or system scheduling delays, lays a precise time-stamping foundation for subsequent accurate analysis and fault backtracking, and improves the real-time performance, integrity, and time-stamping accuracy of data acquisition; it also reduces the real-time interrupt load of the CPU and the risk of message loss. This invention involves the CPU reading packets from shared memory, performing deep protocol parsing, and classifying the packets into different control blocks with fine granularity based on key network layer fields (combinations of multicast MAC, VLAN ID, APPID, etc.), storing them in a second shared memory. This solves the problem of the original packet stream being "chaotic and disordered," making it unsuitable for efficient compression. It organizes the chaotic packet stream into independent, clean data streams with the same characteristics and changing patterns, creating the necessary conditions for the next step of highly targeted "inter-frame differential" compression, improving the orderliness and structure of the packet data; and providing crucial data preparation for subsequent compression algorithms to achieve high compression ratios. This invention extracts a clean message stream from a second shared memory by control block using a second FPGA. Leveraging the message's high format consistency and minimal changes between adjacent frames, it employs an inter-frame differential algorithm (with optional cycle deduplication for SV messages) to convert continuous messages into a "first frame + difference sequence." Then, it uses general algorithms such as Huffman / LZ77 for further compression. Each compressed packet is indexed by a key field in the control block. This is the core step in solving the problem of "large message volume and huge storage space consumption." Utilizing the parallel computing capabilities of the FPGA, lossless, extremely high compression ratio processing is performed before data is written to disk, fundamentally reducing the total amount of data requiring physical storage and greatly improving compression efficiency and data compression ratio (due to extremely low data redundancy after differential processing). This significantly reduces the amount of raw data that needs to be stored and transmitted subsequently, directly alleviating storage space pressure. This invention aggregates the compressed packages of each control block according to a preset time period, performs secondary packaging and compression to form a total compressed package, and adds advanced indexes such as time period and acquisition port to it. This solves the problems of low storage efficiency of small files in massive control block compressed packages and inability to quickly locate data within the target time range during post-event retrieval. By repackaging, the overall compression rate is improved. By adding time-port level indexes, a first-level directory for fast retrieval is established, improving the overall compression rate and file management efficiency of the storage system, and initially establishing fast retrieval capabilities. This invention uses the CPU to write the total compressed packet with a time index into a larger data block format to an SSD, and employs a dual-disk backup strategy. This solves the problems of "frequent small data write operations leading to severely reduced disk lifespan" and data loss due to single-point failures. It transforms massive, high-frequency small message write operations into low-frequency, large-block compressed packet write operations, and utilizes the characteristics of SSDs and dual-disk redundancy to ensure data security, greatly improving the reliability and data security of the storage system; it also significantly reduces disk write frequency and wear, extending disk lifespan. This invention supports rapid location of the total compressed packet based on time and port indexes, and then precise extraction of the target control block's compressed packet from the total packet based on control block keywords. The FPGA performs inverse differential operations to accurately and on-demand recover the original message, solving the pain point of "difficulty in quickly tracing faults using recorded messages." Users do not need to decompress all massive amounts of data; they can quickly locate and recover the original message of the fault time, specific device, or line through multi-level indexes from time and port to specific data streams, much like searching a database. This greatly improves fault analysis efficiency, significantly enhancing the efficiency and accuracy of post-fault analysis and tracing; and reduces the time and manpower costs of fault troubleshooting.

[0056] This disclosure can be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of this disclosure.

[0057] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the protection scope of the claims of the present invention.

Claims

1. A method for compressing power transmission messages, characterized in that, include: The first FPGA collects power messages, timestamps the collected power messages, caches the timestamped messages in the first shared memory, and stores the messages according to the protocol type in the first shared memory. Extract messages from the first shared memory, parse the messages according to their type, and then classify and store them in the second shared memory; The second FPGA extracts SV messages, ZC messages, and GOOSE messages from the second shared memory according to control blocks and compresses them to obtain ZC, GOOSE, and multiple SV compressed packages for the corresponding control blocks. Place ZC, GOOSE and multiple SV compressed packages into the third shared memory, package and compress them again according to the set time period, and generate a total message compressed package for that time period. The total message compressed package is stored on an SSD for dual-disk storage.

2. The method for compressing power transmission messages according to claim 1, characterized in that: The compression of SV, ZC, and GOOSE messages extracted from the second shared memory by control block using the second FPGA includes: The second FPGA extracts SV messages, ZC messages, and GOOSE messages from the second shared memory according to control blocks, and performs length alignment to obtain a control block message sequence that has completed length alignment preprocessing. The control block message sequence that has undergone length alignment preprocessing is deduplicated and compressed to obtain the difference sequence; The difference sequence is compressed using a path-by-path compression algorithm to obtain the ZC, GOOSE and multiple SV compressed packages of the corresponding control block.

3. The method for compressing power transmission messages according to claim 2, characterized in that: The obtained control block message sequence after length alignment preprocessing includes: Select any field combination rule setting control block; The second FPGA matches the fields of each message according to the control block, and classifies all messages with the same field combination as the control block into the original message sequence of the corresponding control block, thus obtaining multiple control block message sequences. The adjacent messages in each control block message sequence are length aligned to obtain a control block message sequence that has undergone length alignment preprocessing.

4. The method for compressing power transmission messages according to claim 3, characterized in that: Field combination rules include a single field using only the multicast MAC address or only the APPID, a combined field of multicast MAC address and VLAN ID with a combined field of APPID and VLAN ID, or a complete field combining multicast MAC address, VLAN ID, and APPID.

5. The method for compressing power transmission messages according to claim 3, characterized in that: If the frame lengths of adjacent messages in the control block message sequence are equal, perform byte-by-byte subtraction directly; If the length of the preceding frame of an adjacent message in the control block message sequence is greater than that of the following frame, only the portion of the preceding frame and the following frame of equal length is truncated for subtraction calculation. The portion of the preceding frame that is longer than the following frame is not included in the calculation, resulting in a difference frame. If the length of the later frame of an adjacent message in the control block message sequence is greater than that of the previous frame, the insufficient fields in the previous frame are padded with zeros to make its length equal to that of the later frame, and then the difference frame is calculated.

6. The method for compressing power transmission messages according to claim 2, characterized in that: Deduplication and compression employ a common adjacent frame difference mode, including: Take the first frame of the control block message sequence that has completed length alignment preprocessing; calculate the inter-frame difference between the next frame and the previous frame in sequence, and so on, arranging all the inter-frame differences in sequence to form a difference sequence.

7. The method for compressing power transmission messages according to claim 6, characterized in that: Deduplication and compression for SV control block messages employs a dedicated SV cycle deduplication mode, including: All sampling points of the first cycle are used as reference cycles. The difference between each sampling point of the second cycle and the corresponding sampling point of the first cycle is calculated to obtain a set of inter-cycle difference sequences. Subsequent cycles are processed in the same way.

8. The method for compressing power transmission messages according to claim 1, characterized in that: The total message compressed package is indexed by timestamp and input port.

9. The method for compressing power transmission messages according to claim 1, characterized in that: Based on the user's requested time period and port, the CPU locates and reads the corresponding total message compressed package from the dual-disk storage. The CPU sends the total message compressed package to the second FPGA for decompression, obtaining all ZC, GOOSE and multiple SV compressed packages within that time period. The CPU then retrieves the specified SV compressed package according to preset keywords and continues to decompress it to obtain the original message. The CPU then sends the decompressed original message to the corresponding receiving device for use.

10. A power message compression system, operating the power message compression method according to any one of claims 1-9, characterized in that: The first FPGA module is used to collect power messages through the first FPGA, timestamp the collected power messages, and cache the timestamped messages in the first shared memory. The first shared memory module is used to classify and store packets according to protocol type through the first shared memory; The CPU module is used to extract packets from the first shared memory, parse the packets according to their type, and then classify and store them in the second shared memory. The second FPGA module is used to extract SV messages, ZC messages and GOOSE messages from the second shared memory according to the control block through the second FPGA and compress them to obtain the ZC, GOOSE and multiple SV compressed packages of the corresponding control block; The third shared memory module is used to put ZC, GOOSE and multiple SV compressed packages into the third shared memory, package and compress them again according to a set time period, and generate a total message compressed package for that time period. The dual-disk storage module is used to store the total compressed packet on an SSD hard drive for dual-disk storage.

Images