A data compression transmission method, device, equipment and readable storage medium

Abstract

This invention discloses a data compression and transmission method, apparatus, device, and readable storage medium, applied in the field of data compression and transmission. The method includes: constructing corresponding character sets based on the data characteristics of the data to be transmitted; evaluating the character sets; determining the appropriate compression algorithm based on the evaluation results and compressing the data to obtain compressed data; and transmitting the compressed data according to a preset single-packet data utilization rate. This invention addresses the limitations of BeiDou short message communication by constructing character sets based on data characteristics and automatically selecting an appropriate compression algorithm based on data evaluation results, then transmitting the compressed data according to a preset single-packet data utilization rate. Compared to traditional fixed character sets, this method increases data frequency; significantly improves data transmission efficiency while ensuring data compression reliability; and also guarantees single-packet data utilization, thus improving communication efficiency.

Description

Technical Field

[0001] This invention relates to the field of data compression and transmission, and particularly to a data compression and transmission method, apparatus, device, and computer-readable storage medium. Background Technology

[0002] The short message communication resources of the BeiDou satellite system are limited. The satellite's communication bandwidth and processing capacity are fixed, and resource constraints arise when a large number of users simultaneously request short message communication services. Therefore, data compression is generally necessary. For example, binary encoding can be used to compress all data to be transmitted; this method is suitable for data that changes little in a short period. Alternatively, compression protocols and corresponding transmission protocols can be used to compress PVT (position, velocity, and time) information; this method is suitable for data transmission with continuous correlation. For data with low or no correlation, or data that changes significantly in a short period, existing compression methods are not effective.

[0003] Therefore, how to provide an efficient compression method for data with low correlation, high frequency of change, and large data volume is a technical problem that urgently needs to be solved. Summary of the Invention

[0004] In view of this, the purpose of the present invention is to provide a data transmission method, apparatus, device and computer-readable storage medium, which solves the problem in the prior art that there is no efficient compression and data transmission method for data with low correlation, high change frequency and large data volume.

[0005] To address the aforementioned technical problems, this invention provides a data compression and transmission method, comprising: constructing corresponding character sets according to the data characteristics of the data to be transmitted; evaluating the character sets; determining a corresponding compression algorithm based on the evaluation results and compressing the data to obtain compressed data; and transmitting the compressed data according to a preset single-packet data utilization rate.

[0006] Optionally, based on the data characteristics of the data to be transmitted, corresponding character sets are constructed, including: constructing a first character set by segmenting and optimizing the fault data through analysis of the frequency distribution and time series characteristics of the fault data; constructing a second character set based on time information, wherein the time information includes the fault start time, end time and current system time; and constructing a third character set based on the wind farm number, wind turbine number and whether it is the first fault.

[0007] Optionally, the character set is evaluated, and a corresponding compression algorithm is determined and compressed based on the evaluation results to obtain compressed data. This includes: calculating the length, repetition density, and data entropy of the repeating substrings of the character set; compressing a fourth character set whose repeating substring length is greater than a first threshold, repetition density is greater than a second threshold, and data entropy is not greater than a third threshold using a pre-filled dictionary and probabilistic interval coding to obtain first compressed data; compressing a fifth character set whose repeating substring length is not greater than the first threshold, repetition density is not greater than the second threshold, and data entropy is greater than the third threshold using a header compression mechanism and prime base quantization to obtain second compressed data; and compressing a sixth character set with uniform character frequency distribution using the Deflate compression method to obtain third compressed data.

[0008] Optionally, the first compressed data is obtained by compressing a fourth character set whose repeating substring length is greater than a first threshold, repeating density is greater than a second threshold, and data entropy is not greater than a third threshold using a pre-filled dictionary and probabilistic interval coding. This includes: constructing the pre-filled dictionary based on historical data; outputting a dictionary matching flag and index for characters in the fourth character set that are concentrated in the pre-filled dictionary; performing matching using the LZ77 algorithm for characters in the fourth character set that are not in the pre-filled dictionary, and outputting a sliding window flag, distance, and length if the match is successful; outputting an original value flag and original value if the match fails; using the dictionary matching flag, sliding window flag, and original value flag as the first data to be encoded; using the distance, length, and original value as the second, third, and fourth data to be encoded, respectively; and performing probabilistic interval coding on the first, second, third, and fourth data to be encoded, respectively, to obtain the first compressed data.

[0009] Optionally, the first data to be encoded, the second data to be encoded, the third data to be encoded, and the fourth data to be encoded are respectively subjected to probability interval encoding to obtain the first compressed data, including: taking the first data to be encoded, the second data to be encoded, the third data to be encoded, and the fourth data to be encoded as input data respectively; defining symbols for the input data and calculating the frequency of each symbol, and setting an initial symbol interval according to the frequency of each symbol; updating the symbol interval based on the initial symbol interval and each symbol until a final symbol interval is obtained; and taking the shortest binary number in the final symbol interval as the first compressed data.

[0010] Optionally, a fifth character set whose repeating substring length is no greater than the first threshold, whose repeating density is no greater than the second threshold, and whose data entropy is greater than the third threshold is compressed using a header compression mechanism and prime base quantization to obtain second compressed data. This includes: calculating the symbol frequencies of the fifth character set to generate a frequency dictionary and a frequency sequence; the frequency dictionary includes a symbol sequence and a quantization frequency difference sequence; calculating the total frequency based on the frequencies of all symbols in the fifth character set, and using prime numbers greater than or equal to the total frequency as the quantization base; quantizing the frequency sequence using the quantization base to obtain a quantized frequency sequence; constructing a state space based on the quantized frequency sequence; generating an encoding table based on the state space; converting the fifth character set into a compressed bitstream using the encoding table; and compressing the header information using dynamic bit-packing and fixed bit-packing methods to obtain a header compression result; the header information includes the frequency dictionary and the quantization base.

[0011] Optionally, transmitting the compressed data according to a preset single-packet data utilization rate includes: if the amount of compressed data is less than the minimum value of a target range, calculating a first difference between the amount of compressed data and the minimum value, and increasing the amount of data to be transmitted based on the first difference; the target range is determined based on the preset single-packet data utilization rate; if the amount of compressed data is greater than the maximum value of the target range, transmitting a portion of the compressed data, the amount of the portion of data being the maximum value of the target range; calculating a second difference between the amount of compressed data and the maximum value, and decreasing the amount of data to be transmitted based on the second difference; if the amount of compressed data is within the target range, transmitting the compressed data.

[0012] The present invention also provides a data compression and transmission device, comprising: a character set construction module, used to construct corresponding character sets according to the data characteristics of the data to be transmitted; a compression module, used to evaluate the character sets, determine the corresponding compression algorithm according to the evaluation results and compress the data to obtain compressed data; and a transmission module, used to transmit the compressed data according to a preset single packet data utilization rate.

[0013] The present invention also provides a data compression and transmission device, comprising: a memory for storing a computer program; and a processor for executing the computer program to implement the steps of the data compression and transmission method described above.

[0014] The present invention also provides a computer-readable storage medium storing computer-executable instructions, which, when loaded and executed by a processor, implement the steps of the data compression and transmission method described above.

[0015] As can be seen, this invention constructs corresponding character sets based on the characteristics of the data to be transmitted; evaluates the character sets, determines the appropriate compression algorithm based on the evaluation results, and compresses the data to obtain compressed data; and transmits the compressed data according to a preset single-packet data utilization rate. This invention addresses the limitations of BeiDou short message communication by constructing character sets based on data characteristics, automatically selecting an appropriate compression algorithm based on data evaluation results, and transmitting compressed data according to a preset single-packet data utilization rate. Compared to traditional fixed character sets, this increases data frequency; significantly improves data transmission efficiency while ensuring data compression reliability; and also guarantees single-packet data utilization, thus improving communication efficiency.

[0016] In addition, the present invention also provides a data compression and transmission apparatus, device and readable storage medium, which also have the above-mentioned beneficial effects. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0018] Figure 1 This is a flowchart of a data compression and transmission method provided in an embodiment of the present invention.

[0019] Figure 2 This is an example diagram illustrating a dynamic priority filtering mechanism provided in an embodiment of the present invention.

[0020] Figure 3 This is a flowchart illustrating a compression algorithm provided in an embodiment of the present invention.

[0021] Figure 4 A flowchart illustrating another compression algorithm provided in an embodiment of the present invention.

[0022] Figure 5 This is an example diagram of an adaptive data filtering method provided in an embodiment of the present invention.

[0023] Figure 6 This is a flowchart illustrating an overall data compression method provided in an embodiment of the present invention.

[0024] Figure 7 This is a schematic diagram of a data compression and transmission device provided in an embodiment of the present invention.

[0025] Figure 8This is a schematic diagram of a data compression and transmission device provided in an embodiment of the present invention. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0027] Please refer to Figure 1 , Figure 1 This is a flowchart of a data compression and transmission method provided in an embodiment of the present invention. The method may include steps S101-S103.

[0028] S101: Construct corresponding character sets according to the data characteristics of the data to be transmitted.

[0029] The execution subject in this embodiment is a terminal. This embodiment does not limit the type of terminal, as long as it can perform the data compression and transmission method. The data to be transmitted in this embodiment is large in volume, has low correlation, and changes frequently. For example, the data to be transmitted can be wind power data, where wind turbine data can include fault data, time data, and other data. This embodiment does not limit the construction of the character set. It is understood that a corresponding character set can be constructed based on the characteristics of the real-time data to be transmitted; therefore, the character set can be a dynamic character set.

[0030] Furthermore, the above-mentioned character sets are constructed according to the data characteristics of the data to be transmitted, which may specifically include steps 11-13.

[0031] Step 11: Optimize the segmentation of fault data by analyzing its frequency distribution and time series characteristics, and construct the first character set.

[0032] Specifically, fault data is typically a long field. Based on this fault data, a frequency adjustment scheme is formulated by analyzing the data frequency distribution and time series characteristics, identifying high-frequency and low-frequency patterns, performing segmentation optimization, and constructing a corresponding character set. For example, fault data over a period of time is used to statistically analyze the frequency of occurrence and formulate a suitable frequency adjustment method. The frequency distribution and time series characteristics of the data are analyzed to identify high-frequency and low-frequency patterns. Then, the fault data is segmented into several independent fields, each containing a different value. Since some fields in the fault data appear frequently, and some fields are always 0, this is based on the characteristics of the fault codes.

[0033] Step 12: Construct a second character set based on time information; the time information includes the fault start time, end time, and current system time.

[0034] Specifically, based on time information, including fault time (fault start time, fault end time) and current system time, a corresponding character set is constructed. Among them, the time information can be preprocessed, and the specific process can include: (1) Converting the time to a UNIX timestamp. All fault times (such as fault start time and fault end time) are converted to UNIX timestamps, which can ensure that all times are unified to a standard time representation, which is convenient for subsequent differential calculation. However, 32-bit UNIX can only represent from January 1, 1970 00:00:00UTC to January 19, 2038 03:14:07UTC. In order to extend the time limit that can be represented and reduce the value of the converted UNIX timestamp, the difference between the converted UNIX timestamp and January 1, 2023 00:00:00 is calculated, so that the UNIX timestamp value of January 1, 2023 00:00:00 is represented by 0. The improved usable time limit extends to January 19, 2091 03:14:07UTC. (2) Differential encoding. Using the earliest occurrence time of the initial fault as the base time, the end time of the initial fault and the start and end times of other faults are calculated using differential encoding, and the time difference is converted into binary time code for storage. Special bytes are used for transmission of times with large differences from the base time. For example, if fault 1 occurred on December 21, 2024 at 12:00:00 (62563200), and fault 2 occurred on December 21, 2024 at 12:00:40 (62563240), fault 2 is represented as +40 using fault 1 as the base time, and can be represented by bytes as 00101000. During data backlog, where the fault time difference is large, dynamic byte encoding is used to store the differential encoded values. A mode identifier code is introduced for processing. A mode identifier code of 1 indicates normal data compression mode, and 0 indicates data backlog mode. Simultaneously, dynamic bytes are introduced for joint encoding based on different data differences, as shown in Table 1.

[0035] Table 1 Joint coding under data backlog mode

[0036]

[0037] (3) Time update: During each data transmission, the base time will be updated according to the latest occurrence time of the current fault event. Suppose that after a fault data transmission, the latest fault start time is 12:03:20 on December 21, 2024, which is 1703155400, then the base time will be updated to 1703155400 next time, and the time difference between adjacent events will continue to be calculated.

[0038] It should be noted that the first character set, the second character set, and the third character set mentioned above are different character sets obtained based on different data characteristics.

[0039] Step 13: Construct a third character set based on the wind farm number, wind turbine number, and whether it is the first fault.

[0040] Specifically, a byte stream is constructed using bits as the unit. Since the number of wind turbine numbers is no more than 127, and whether it is the first fault is represented by a Boolean character, the 1-bit binary number indicating whether it is the first fault is combined with the 7-bit binary number of the wind turbine according to certain rules to form an 8-bit binary encoded string.

[0041] Furthermore, before constructing the corresponding character sets according to the data characteristics of the data to be transmitted, the above may include the following steps: Step 1: Assign weights to each data in the database according to the evaluation indicators, and calculate the fault weighted total score of each data using a weighted method; the evaluation indicators include at least one of fault type, fault time, fault level, and first-release marker; Step 2: Determine the transmission priority of each data according to the fault weighted total score, and determine the transmission order of each data according to the transmission priority; Step 3: Determine the preset initial size according to the historical compression ratio; Step 4: Determine the data to be transmitted according to the transmission order and the preset initial size.

[0042] For example, you can refer to Figure 2 , Figure 2 This diagram illustrates a dynamic priority filtering mechanism provided in an embodiment of the present invention. For wind turbine data in the wind turbine database, a weighted method is used to assign weights to indicators such as fault type, fault occurrence time, and fault level. A weighted total fault score is calculated and the data is sorted, with higher weighted scores indicating higher priority. Data is selected for transmission in descending order of priority. The size of the selected data volume is determined based on historical compression ratios. Specifically, the system makes a preliminary estimate based on the designed high-efficiency compression method, pre-setting the initial data volume to be compressed. This initial volume can be estimated based on historical compression ratios.

[0043] S102: Evaluate the character set, determine the appropriate compression algorithm based on the evaluation results, and compress the data to obtain the compressed data.

[0044] This embodiment evaluates data for each of the aforementioned character sets. The evaluation criteria are not specifically limited; for example, they could be character frequency distribution or substring repetition. A corresponding compression algorithm is selected based on the data evaluation structure. For instance, Deflate compression is used for evenly distributed or high-frequency characters, while other efficient and adaptive compression methods are used for unevenly distributed characters.

[0045] Furthermore, the above-mentioned data evaluation of the character set, the determination of the corresponding compression algorithm based on the evaluation results, and the compression to obtain compressed data can specifically include steps 21-24.

[0046] Step 21: Calculate the length of repeating substrings, repeating density, and data entropy of the character set.

[0047] Specifically, assume the data sequence is Each symbol The frequency is represented as follows: .

[0048] in, Symbols Number of times it appears Symbols The frequency of occurrence, i.e., the frequency distribution of characters; n represents the total number of symbols.

[0049] Define substring Indicates from position arrive The substring, for any substring The number of times it appears is defined as The length of the substring is The total length of all repeating substrings and repeat density for: .

[0050] Data entropy is an important indicator of the randomness of data. Based on character frequency distribution... The Shannon entropy formula can represent data entropy. as follows: .

[0051] If the frequency of certain symbols If the entropy is significantly higher than other symbols, it indicates that the data has high redundancy and is suitable for entropy coding; if the data entropy is significantly higher than other symbols, it indicates that the data has high redundancy and is suitable for entropy coding. A lower entropy indicates that there is a lot of redundant information in the data, making it suitable for entropy coding. If the value is close to the maximum, it indicates that the data is nearly uniformly distributed, and the compression effect may be poor; if the repetition density is high... A high repetition density indicates that there are many repeating patterns in the data, which is suitable for dictionary encoding. If the repetition density is low, it indicates that there are few repeating patterns in the data, which may require other compression methods.

[0052] Step 22: Compress the fourth character set, whose repeating substring length is greater than the first threshold, repeating density is greater than the second threshold, and data entropy is not greater than the third threshold, using a pre-filled dictionary and probabilistic interval coding to obtain the first compressed data.

[0053] It should be noted that steps 22 and 23 in this embodiment are for non-uniformly distributed data. This embodiment does not limit the first, second, and third thresholds. Specifically, for a large number of repeating substrings... (Repeating substrings are relatively long), medium to low entropy High repetition density The data is compressed using a pre-filled dictionary and probabilistic interval coding. To address the inefficiency in the initial stage of the LZ77 (Lempel-Ziv 1977, a lossless compression algorithm) algorithm, a pre-filled dictionary is designed to capture high-frequency patterns using prior knowledge, reducing early redundant character output, improving initial compression efficiency, avoiding inefficiency when fields are empty, and accelerating compression convergence. The ability to capture short repeating patterns is enhanced, improving compression efficiency for locally repeating data, reducing early redundant character output, and increasing the compression ratio. Simultaneously, to address the issue of residual symbol redundancy in the LZ77 algorithm output, an encoding strategy (i.e., probabilistic interval coding) is added to eliminate statistical redundancy.

[0054] Furthermore, the above-mentioned compression of the fourth character set, where the length of the repeating substring is greater than the first threshold, the repetition density is greater than the second threshold, and the data entropy is not greater than the third threshold, using a pre-filled dictionary and probabilistic interval coding to obtain the first compressed data, can specifically include the following steps: Step 221: Construct a pre-filled dictionary based on historical data; Step 222: For characters in the fourth character set that are concentrated in the pre-filled dictionary, output the dictionary matching flag and index; Step 223: For characters in the fourth character set that are not in the pre-filled dictionary, use the LZ77 algorithm for matching. If the match is successful, output the sliding window flag, distance, and length; if the match fails, output the original value flag and the original value; Step 224: Use the dictionary matching flag, sliding window flag, and original value flag as the first data to be encoded; Step 225: Use the distance, length, and original value as the second, third, and fourth data to be encoded, respectively; Step 226: Perform probabilistic interval coding on the first, second, third, and fourth data to be encoded, respectively, to obtain the first compressed data.

[0055] Specifically, you can refer to Figure 3 , Figure 3 This is a flowchart illustrating a compression algorithm provided in an embodiment of the present invention. Figure 3 The data input in this context refers to the data (i.e., symbols) in the fourth character set. (1) Construct a pre-filled dictionary based on historical data: Statistically analyze the frequency of historical data, determine the capacity of the pre-filled dictionary, arrange the symbols in reverse order according to their frequency, and assign shorter binary indices to high-frequency symbols. (2) Read the symbols in the fourth character set. If the symbol is in the pre-filled dictionary, output [dictionary matching flag] [index], i.e. Figure 3The [mark bit 1][index] in the middle, and at the same time add the processed symbols to the sliding window. (3) LZ77 sliding window compression: If the pre-filled dictionary fails to match, find the longest repeated field of the symbol in the sliding window. If the match is successful in the sliding window, output [sliding window mark bit][distance][length], that is Figure 3 The [mark bit 2][distance][length] values ​​are used to output the [original value mark][original value] if a match fails. Figure 3 [Marker 3][Distance][Original Value]. When matching the data stream (referring to the fourth character set) in the LZ77 sliding window, the processed symbols are added to the sliding window, and the earliest symbol is eliminated when it exceeds the window size. Furthermore, the pre-filled dictionary can also be updated. The specific condition is: the frequency of the current value is statistically analyzed and the pre-filled dictionary is dynamically updated. If it exceeds the fourth threshold, it is added to the pre-filled dictionary. (4) Probability interval encoding. The above [dictionary matching flag], [sliding window flag] and [original value flag] [original value] are constructed as unified data to be encoded. At the same time, [index], [distance] and [length] are constructed as unified data to be encoded. Probability interval encoding is performed on the above four sets of data to be encoded, and four sets of compressed data are output.

[0056] Furthermore, the above-mentioned probabilistic interval encoding of the first data to be encoded, the second data to be encoded, the third data to be encoded, and the fourth data to be encoded to obtain the first compressed data may specifically include the following steps: Step 2261: Take the first data to be encoded, the second data to be encoded, the third data to be encoded, and the fourth data to be encoded as input data respectively; Step 2262: Define symbols for the input data and calculate the frequency of each symbol, and set an initial symbol interval according to the frequency of each symbol; Step 2263: Update the symbol interval based on the initial symbol interval and each symbol until the final symbol interval is obtained; Step 2264: Take the shortest binary number in the final symbol interval as the first compressed data.

[0057] Specifically, you can refer to Figure 3 The probability interval encoding part in the code. (1) Define symbols: Define various symbol types such as [dictionary matching flag] (i.e., A), [original value flag] [original value] (i.e., B+original value) and [sliding window flag]. (2) Set initial symbol frequency and initial symbol interval: Obtain each symbol frequency as the initial frequency, and map the symbols to the interval [0, 1) to divide the initial interval. (3) Read the current field and encode it according to the current interval. (4) Read the data stream (referring to the input data): For each input symbol, further divide the current interval (assuming it is [L, H]) into sub-intervals according to its probability, and update the current interval to the sub-interval corresponding to the symbol. The rules for updating the sub-intervals are as follows: .

[0058] in, and They are symbols and The probability of; Indicates the new lowest, Indicates new highest; This indicates the previous highest value. This indicates the lowest value in the previous interval. (5) Repeat the above process until all symbols have been processed. Finally, select the shortest binary fraction from the final encoding interval as the encoding result.

[0059] To better understand steps 2261-2264, refer to the following example: Divide the total interval [0, 1) according to the probability of the symbols appearing. For example, the probability of symbol A is 50%, corresponding to the interval [0, 0.5); the probability of symbol B is 30%, corresponding to the interval [0.5, 0.8); the probability of symbol C is 20%, corresponding to the interval [0.8, 1). Initially, the encoding range is the entire interval: lowest point = 0, highest point = 1. Encoding "BAC": the first symbol B: B is in the segment [0.5, 0.8) of the original interval; new interval: lowest point = 0.5, highest point = 0.8. The second symbol A: Within the current interval [0.5, 0.8), the value is repositioned according to the proportion of A (originally 0~50%): New lowest = 0.5 + (0.8 - 0.5) × 0 = 0.5, New highest = 0.5 + (0.8 - 0.5) × 0.5 = 0.65; At this point, the interval is narrowed to 0.5~0.65. The third symbol C: Within the current interval (0.5~0.65), the value is repositioned according to the proportion of C (originally 80%~100%): New lowest = 0.5 + (0.65 - 0.5) × 0.8 = 0.5 + 0.12 = 0.62, New highest = 0.5 + (0.65 - 0.5) × 1 = 0.65; The final interval is 0.62~0.65. Output the shortest code: Find the shortest binary decimal within the final range (0.62~0.65). For example, 0.625 (binary 0.101) is exactly within the range, so the final code result is 101.

[0060] Step 23: Compress the fifth character set, whose repeating substring length is no greater than the first threshold, repeating density is no greater than the second threshold, and data entropy is greater than the third threshold, using the header compression mechanism and prime number base quantization to obtain the second compressed data.

[0061] For fewer repeating substrings (i.e., short repeating substrings) and high data entropy Low repetition density The data is compressed using a header compression mechanism and a prime base quantization method. To facilitate binary arithmetic, the traditional ANS (Asymmetric Numeral Systems) algorithm quantizes symbols to powers of 2 (e.g., 1 / 2, 1 / 4, 1 / 8). However, for small data, the actual probability distribution may deviate significantly from powers of 2, leading to substantial quantization errors. Furthermore, its state partitioning is based on uniform probability, meaning multiple symbols may map to the same state interval, requiring additional collision handling during encoding and reducing compression efficiency. Therefore, an improved ANS algorithm for small data is designed, employing prime base quantization for more refined and flexible probability interval partitioning, bringing the probability closer to the actual frequency, reducing quantization errors, avoiding state overlap, improving compression efficiency, and fully leveraging the potential of ANS. For small data, the symbol table stored by the ANS compression algorithm consumes a large amount of memory; therefore, a header compression method is designed to reduce data overhead.

[0062] Furthermore, the above-mentioned compression mechanism and prime base quantization are used to compress the fifth character set whose repeating substring length is no greater than the first threshold, whose repeating density is no greater than the second threshold, and whose data entropy is greater than the third threshold, to obtain the second compressed data. Specifically, this can include the following steps: Step 231: Calculate the symbol frequency of the fifth character set to generate a frequency dictionary and a frequency sequence; the frequency dictionary includes the symbol sequence and the quantization frequency difference sequence; Step 232: Calculate the total frequency based on the frequency of all symbols in the fifth character set, and use prime numbers greater than or equal to the total frequency as the quantization base; Step 233: Quantize the frequency sequence using the quantization base to obtain a quantized frequency sequence, construct a state space based on the quantized frequency sequence, generate an encoding table based on the state space, and use the encoding table to convert the fifth character set into a compressed bitstream; Step 234: Compress the header information using dynamic bit-packing and fixed bit-packing methods to obtain the header compression result; the header information includes the frequency dictionary and the quantization base.

[0063] Specifically, you can refer to Figure 4 , Figure 4 A flowchart illustrating another compression algorithm provided in an embodiment of the present invention. Figure 4 The left side shows the mainstream data compression process, which includes the construction of the frequency dictionary, the selection of the quantization basis, the generation of the state space and encoding table, and the compression of data in the fifth character set; Figure 4 The right side shows the header information compression process. (1) Generate frequency dictionary and frequency sequence: Statistically count the symbol frequencies of the input data stream (referring to the fifth character set) to generate a frequency dictionary. Sort the symbols in the frequency dictionary from low to high frequency and convert them into a difference sequence. The conversion formula is as follows: (2) Quantization basis selection: A predefined list of prime number candidates is used for quantization probability. The actual frequency of the symbol is calculated. and actual probability Find the total frequency of all symbols. And select a frequency greater than or equal to the total frequency. prime numbers as the quantization basis The actual probability and actual frequency are quantized based on the quantization basis, and the quantization formula is as follows: .

[0064] in, Character frequency; The quantized character frequency; Character probability; This represents the quantized character probability.

[0065] (3) Data encoding: Define the global state space of the state table and the frequency subspace of each symbol as follows: Define the initial state as (Assuming rules with the decompression end), l is the scaling factor. When reading the current character, to ensure that the current state is always within the character's state subspace, the current state needs to be scaled and the bitstream output. The calculation of the output bitstream and the scaled state is as follows: .

[0066] Calculate the state based on the state transition rule. and the corresponding number of bits in the output and Bit stream And complete the corresponding state transition table (i.e., encoding table) based on the previous state. Update new status The rules are as follows: .

[0067] in, Indicates cumulative frequency .

[0068] During encoding, the corresponding new state and output bit stream are looked up in the encoding table based on the current state and input symbol. Finally, a string of bits is output as the main body of data compression.

[0069] (4) Header compression: The header information mainly includes the frequency dictionary and the quantization basis. The frequency dictionary includes the symbol sequence and the quantization frequency difference sequence. The quantization basis is represented in fixed bytes. The symbol sequence is compressed using dynamic bit-packing (a technique to reduce storage space by compressing data bits), and the number of bytes required to store each symbol in the symbol sequence is calculated. If all symbol storage requires the same number of bytes, that is... If the number of bytes required for storing all symbols is different, then the identifier code 1 is used; if the number of bytes required for storing all symbols is different, then the identifier code 0 is used, and the number of bytes required for storing each symbol, i.e., the bitstream, needs to be stored. By default, each symbol can be represented by a maximum of 4 bytes, but 2 bits are used for representation. The final storage length is A bitstream of bits. The frequency difference sequence is compressed using a fixed bit-packing method, storing it with a fixed number of bits. The maximum number of bits required to store the frequency difference sequence is calculated. All elements in the frequency difference sequence are in the maximum number of bits. Stored, the final result is of length. The bitstream.

[0070] Step 24: Use the Deflate compression method to compress the sixth character set, which has a uniform character frequency distribution, to obtain the third compressed data.

[0071] Steps 22 and 23 are both for data with uneven character frequency distribution. If the character frequency distribution is uniform or the characters are high-frequency, the Deflate (DEFLATE Compression Method) compression method can be used. Deflate is a widely used data compression algorithm, which will not be elaborated here. It should be noted that the fourth, fifth, and sixth character sets mentioned above are different character sets obtained after evaluating the first, second, and third character sets. Appropriate and efficient compression algorithms are used for the fourth and fifth character sets respectively. Whether the character frequency is uniform can be determined based on the entropy ratio, and the specific entropy ratio formula is: , n is the size of the character set.

[0072] S103: Transmit compressed data according to the preset single packet data utilization rate.

[0073] Since the communication capacity of BeiDou short messages is N, to improve communication efficiency, an adaptive filtering and compression caching mechanism is used to ensure that the utilization rate of a single packet data reaches M (e.g., 96%) or higher. Therefore, this embodiment dynamically adjusts the compressed data by preset single packet data rate to achieve high-efficiency communication.

[0074] Furthermore, the above-mentioned transmission of compressed data based on a preset single-packet data utilization rate may specifically include the following steps: Step 31: If the amount of compressed data is less than the minimum value of the target range, a first difference is calculated based on the amount of compressed data and the minimum value, and the amount of data to be transmitted is increased based on the first difference; the target range is determined based on the preset single-packet data utilization rate; Step 32: If the amount of compressed data is greater than the maximum value of the target range, a portion of the compressed data is transmitted, and the amount of the portion of data is the maximum value of the target range; a second difference is calculated based on the amount of compressed data and the maximum value, and the amount of data to be transmitted is reduced based on the second difference; Step 33: If the amount of compressed data is within the target range, the compressed data is transmitted.

[0075] To better understand steps 31-33, please refer to... Figure 5 , Figure 5 This is an example diagram illustrating an adaptive data filtering method provided in an embodiment of the present invention. The initial data is compressed using an efficient compression algorithm, and the size of the compressed data is calculated in real time. By compressing the current data and calculating the amount of compressed data, the system can evaluate the actual compression effect, providing a basis for the next step of screening and adjustment. Specifically, (1) Determine the compatibility between the compressed data size and the target range: Determine the size of the compressed data. Is it within the target range bit? ~ Within the bit, if If the amount of data to be compressed is insufficient, more data needs to be added. In this case, the system will continue to select higher-priority data from the remaining uncompressed data to supplement it, increasing the data volume to the target range. This indicates that the amount of data is too large and some data needs to be reduced. (2) Dynamically adjust the amount of data: Calculate the difference between the current compressed data and the target range, evaluate the amount of data that needs to be added or reduced, and if data needs to be added, select high-priority data from the priority list and append it to the data to be compressed. If data needs to be reduced, remove low-priority data first. (3) Continuous iterative optimization: Repeat the above steps until the size of the compressed data meets the target range requirements.

[0076] Furthermore, a character set caching mechanism is provided: the constructed character set is cached to reduce repetitive data processing operations and greatly improve the system's processing efficiency, especially when processing large amounts of data, it can significantly reduce computing time and resource consumption.

[0077] The data compression and transmission method provided in this invention constructs corresponding character sets based on the characteristics of the data to be transmitted; evaluates the character sets, determines the appropriate compression algorithm based on the evaluation results, and compresses the data to obtain compressed data; and transmits the compressed data according to a preset single-packet data utilization rate. This invention addresses the limitations of BeiDou short message communication by constructing character sets based on data characteristics and automatically selecting an appropriate compression algorithm based on data evaluation results, then compressing the data according to a preset single-packet data utilization rate before transmission. Compared to traditional fixed character sets, this method increases data frequency; significantly improves data transmission efficiency while ensuring data compression reliability; and also guarantees single-packet data utilization, thus improving communication efficiency. To address the specific characteristics of the data, a dynamic character set is constructed based on mechanisms such as packet optimization, segmentation optimization, and differential encoding. Compared to traditional fixed character sets, this increases the data frequency, significantly improving data compression efficiency. For data with numerous repeating substrings, low to medium entropy, and high repetition density, a compression algorithm using pre-filled dictionaries and probability interval encoding is proposed. This strategy improves both compression speed and efficiency. For data with fewer repeating substrings, high entropy, and low repetition density, a compression algorithm using a header compression mechanism and prime base quantization is proposed. Prime base quantization allows for more precise and flexible division of probability intervals, bringing the probability closer to the actual frequency and reducing quantization errors. To improve compression efficiency, a header compression mechanism is designed for the data header, reducing header transmission resources and greatly saving data storage space. Addressing the issue of low frequency and small capacity in BeiDou short message communication, making it difficult to transmit large amounts of wind farm data, a dynamic priority filtering mechanism is designed to prioritize wind turbine data, ensuring priority transmission of key information under limited communication resources, thus improving the efficiency and accuracy of fault diagnosis. To address the limited single-transmission capacity and unknown compression ratio of BeiDou short message transmission, an adaptive data filtering mechanism is proposed. Based on adaptive filtering adjustment and caching mechanisms, the utilization rate of a single packet data reaches M (e.g., 96%) or higher, improving communication efficiency. Furthermore, this invention designs a pre-filled dictionary to capture high-frequency patterns using prior knowledge, reducing early redundant character output, improving compression efficiency in the initial stage, avoiding inefficiency when fields are empty, and accelerating compression convergence. It enhances the ability to capture short repeating patterns, improving compression efficiency for locally repeating data, reducing early redundant character output, and increasing the compression ratio. Simultaneously, addressing the issue of symbol redundancy in the LZ77 algorithm output, an encoding strategy is added to eliminate statistical redundancy. Moreover, based on an improved ANS algorithm for small data, prime-based quantization is used to more finely and flexibly divide probability intervals, making the probability closer to the actual frequency, reducing quantization errors, avoiding state overlap, improving compression efficiency, and fully leveraging the potential of ANS. For small data, the symbol table stored by the ANS compression algorithm occupies a large amount of memory; therefore, a header compression method is provided to reduce data overhead.

[0078] To address the limitations of BeiDou short message communication, a data compression and transmission method based on wind turbine data was developed, which significantly improves data transmission efficiency while ensuring data compression reliability. For a clearer understanding of this invention, please refer to the following details. Figure 6 , Figure 6 The flowchart of an overall data compression method provided in this embodiment of the invention includes: Construction of a dynamic character set: Based on data characteristics, a dynamic SCS (Super-Character Set) is constructed using a combination of mechanisms such as packet optimization, segmentation optimization, and differential coding. Data compression mechanism: A compression algorithm using a pre-filled dictionary and probability interval coding is proposed for data with a large number of repeating substrings, low to medium entropy, and high repetition density; a header compression mechanism and prime base quantization compression algorithm are proposed for data with fewer repeating substrings, high entropy, and low repetition density; and the Deflate compression method is used for data with uniform character frequency distribution, high entropy, lack of obvious repetition patterns, or high-frequency characters. Data evaluation and preprocessing mechanism: Due to the limited single-transmission capacity of BeiDou short message transmission, an adaptive data filtering mechanism is proposed when the compression ratio is unknown. Based on adaptive filtering adjustment and compression caching mechanisms, the utilization rate of a single packet data reaches M (e.g., 96%) or higher.

[0079] The data compression and transmission apparatus provided in the embodiments of the present invention will be described below. The data compression and transmission method described below can be referred to in correspondence with each other.

[0080] Please refer to the details. Figure 7 , Figure 7 The present invention provides a schematic diagram of a data compression and transmission device, which may include: a character set construction module 100, used to construct corresponding character sets according to the data characteristics of the data to be transmitted; a compression module 200, used to evaluate the character sets, determine the corresponding compression algorithm according to the evaluation results, and compress the data to obtain compressed data; and a transmission module 300, used to transmit the compressed data according to a preset single packet data utilization rate.

[0081] Based on the above embodiments, the character set construction module 100 may include: a first construction unit, used to segment and optimize the fault data by analyzing the frequency distribution and time series characteristics of the fault data, and construct a first character set; a second construction unit, used to construct a second character set based on time information, the time information including the fault start time, end time and current system time; and a third construction unit, used to construct a third character set based on the wind farm number, wind turbine number and whether it is the first fault.

[0082] Based on the above embodiments, the compression module 200 may include: a calculation unit for calculating the length of repeating substrings, repeating density, and data entropy of the character set; a first compression unit for compressing a fourth character set whose repeating substring length is greater than a first threshold, repeating density is greater than a second threshold, and data entropy is not greater than a third threshold using a pre-filled dictionary and probability interval coding to obtain first compressed data; a second compression unit for compressing a fifth character set whose repeating substring length is not greater than the first threshold, repeating density is not greater than the second threshold, and data entropy is greater than the third threshold using a header compression mechanism and prime base quantization to obtain second compressed data; and a third compression unit for compressing a sixth character set with uniform character frequency distribution using the Deflate compression method to obtain third compressed data.

[0083] Based on the above embodiments, the first compression unit may include: a dictionary construction subunit, used to construct the pre-filled dictionary based on historical data; a first output subunit, used to output a dictionary matching flag and index for characters in the fourth character set that are concentrated in the pre-filled dictionary; a second output subunit, used to perform matching using the LZ77 algorithm for characters in the fourth character set that are not in the pre-filled dictionary, and output a sliding window flag, distance, and length if the matching is successful; and output an original value flag and original value if the matching fails; a first encoded data determination subunit, used to use the dictionary matching flag, sliding window flag, and original value flag as first data to be encoded; a second encoded data determination subunit, used to use the distance, length, and original value as second, third, and fourth data to be encoded, respectively; and a probability interval encoding subunit, used to perform probability interval encoding on the first data to be encoded, the second data to be encoded, the third data to be encoded, and the fourth data to be encoded, respectively, to obtain the first compressed data.

[0084] Based on the above embodiments, the probability interval encoding subunit includes: an input data determination subunit, used to take the first data to be encoded, the second data to be encoded, the third data to be encoded, and the fourth data to be encoded as input data respectively; an initial symbol interval determination subunit, used to define symbols for the input data and calculate the frequency of each symbol, and set an initial symbol interval according to the frequency of each symbol; a symbol interval update subunit, used to update the symbol interval based on the initial symbol interval and each symbol until a final symbol interval is obtained; and a first compression subunit, used to take the shortest binary number in the final symbol interval as the first compressed data.

[0085] Based on the above embodiments, the second compression unit includes: a statistics subunit, used to count the symbol frequencies of the fifth character set and generate a frequency dictionary and a frequency sequence; the frequency dictionary includes a symbol sequence and a quantization frequency difference sequence; a quantization basis determination subunit, used to calculate the total frequency based on the frequencies of all symbols in the fifth character set, and use prime numbers greater than or equal to the total frequency as the quantization basis; a second compression subunit, used to quantize the frequency sequence using the quantization basis to obtain a quantized frequency sequence, construct a state space based on the quantized frequency sequence, generate an encoding table based on the state space, and convert the fifth character set into a compressed bitstream using the encoding table; and a third compression subunit, used to compress the header information using dynamic bit-packing and fixed bit-packing methods to obtain a header compression result; the header information includes the frequency dictionary and the quantization basis.

[0086] Based on any of the above embodiments, the transmission module 300 may include: a data volume increasing unit, configured to calculate a first difference based on the compressed data volume and the minimum value if the data volume of the compressed data is less than the minimum value of the target range, and increase the data volume of the data to be transmitted based on the first difference; the target range is determined based on the preset single packet data utilization rate; a data volume decreasing unit, configured to transmit a portion of the compressed data if the data volume of the compressed data is greater than the maximum value of the target range, the data volume of the portion of data being the maximum value of the target range; and calculate a second difference based on the data volume of the compressed data and the maximum value, and decrease the data volume of the data to be transmitted based on the second difference; and a transmission unit, configured to transmit the compressed data if the data volume of the compressed data is within the target range.

[0087] It should be noted that the order of the modules and units in the above-mentioned data compression and transmission device can be changed without affecting the logic.

[0088] The data compression and transmission apparatus provided in this embodiment of the invention comprises a character set construction module 100, used to construct corresponding character sets according to the data characteristics of the data to be transmitted; a compression module 200, used to evaluate the character sets, determine the corresponding compression algorithm based on the evaluation results, and compress the data to obtain compressed data; and a transmission module 300, used to transmit the compressed data according to a preset single-packet data utilization rate. This apparatus addresses the limitations of BeiDou short message communication by constructing character sets based on data characteristics, automatically selecting an appropriate compression algorithm based on data evaluation results, and transmitting the compressed data according to a preset single-packet data utilization rate. Compared to traditional fixed character sets, it increases data frequency; significantly improves data transmission efficiency while ensuring data compression reliability; and also guarantees single-packet data utilization, thus improving communication efficiency. To address the specific characteristics of the data, a dynamic character set is constructed based on mechanisms such as packet optimization, segmentation optimization, and differential encoding. Compared to traditional fixed character sets, this increases the data frequency, significantly improving data compression efficiency. For data with numerous repeating substrings, low to medium entropy, and high repetition density, a compression algorithm using pre-filled dictionaries and probability interval encoding is proposed. This strategy improves both compression speed and efficiency. For data with fewer repeating substrings, high entropy, and low repetition density, a compression algorithm using a header compression mechanism and prime base quantization is proposed. Prime base quantization allows for more precise and flexible division of probability intervals, bringing the probability closer to the actual frequency and reducing quantization errors. To improve compression efficiency, a header compression mechanism is designed for the data header, reducing header transmission resources and greatly saving data storage space. Addressing the issue of low frequency and small capacity in BeiDou short message communication, making it difficult to transmit large amounts of wind farm data, a dynamic priority filtering mechanism is designed to prioritize wind turbine data, ensuring priority transmission of key information under limited communication resources, thus improving the efficiency and accuracy of fault diagnosis. To address the limited single-transmission capacity and unknown compression ratio of BeiDou short message transmission, an adaptive data filtering mechanism is proposed. Based on adaptive filtering adjustment and caching mechanisms, the utilization rate of a single packet data reaches M (e.g., 96%) or higher, improving communication efficiency. Furthermore, this invention designs a pre-filled dictionary to capture high-frequency patterns using prior knowledge, reducing early redundant character output, improving compression efficiency in the initial stage, avoiding inefficiency when fields are empty, and accelerating compression convergence. It enhances the ability to capture short repeating patterns, improving compression efficiency for locally repeating data, reducing early redundant character output, and increasing the compression ratio. Simultaneously, addressing the issue of symbol redundancy in the LZ77 algorithm output, an encoding strategy is added to eliminate statistical redundancy. Moreover, based on an improved ANS algorithm for small data, prime-based quantization is used to more finely and flexibly divide probability intervals, making the probability closer to the actual frequency, reducing quantization errors, avoiding state overlap, improving compression efficiency, and fully leveraging the potential of ANS. For small data, the symbol table stored by the ANS compression algorithm occupies a large amount of memory; therefore, a header compression method is provided to reduce data overhead.

[0089] The data compression and transmission device provided in the embodiments of the present invention will be described below. The data compression and transmission device described below can be referred to in correspondence with the data compression and transmission method described above.

[0090] Please refer to Figure 8 , Figure 8 The present invention provides a schematic diagram of the structure of a data compression and transmission device, which may include: a memory 10 for storing computer programs; and a processor 20 for executing the computer programs to implement the above-described data compression and transmission method.

[0091] The memory 10, processor 20, and communication interface 31 all communicate with each other through the communication bus 32.

[0092] In this embodiment of the invention, the memory 10 is used to store one or more programs. The programs may include program code, which includes computer operation instructions. In this embodiment of the invention, the memory 10 may store programs for implementing the following functions: constructing corresponding character sets according to the data characteristics of the data to be transmitted; evaluating the character sets, determining the corresponding compression algorithm based on the evaluation results, and compressing the data to obtain compressed data; and transmitting the compressed data according to a preset single-packet data utilization rate.

[0093] In one possible implementation, the memory 10 may include a program storage area and a data storage area, wherein the program storage area may store the operating system and applications required for at least one function; and the data storage area may store data created during use.

[0094] Furthermore, memory 10 may include read-only memory and random access memory, providing instructions and data to the processor. A portion of the memory may also include NVRAM. The memory stores operating systems and operating instructions, executable modules, or data structures, or subsets thereof, or extended sets thereof, wherein the operating instructions may include various operating instructions for implementing various operations. The operating system may include various system programs for implementing various basic tasks and handling hardware-based tasks.

[0095] Processor 20 can be a central processing unit (CPU), an application-specific integrated circuit, a digital signal processor, a field-programmable gate array, or other programmable logic device. Processor 20 can be a microprocessor or any conventional processor. Processor 20 can call programs stored in memory 10.

[0096] Communication interface 31 can be an interface for the communication module, used to connect with other devices or systems.

[0097] Of course, it should be noted that, Figure 8 The structure shown does not constitute a limitation on the data compression and transmission device in the embodiments of the present invention. In practical applications, the data compression and transmission device may include devices with higher compression ratios than... Figure 8 More or fewer components as shown, or combinations of certain components.

[0098] The readable storage medium provided in the embodiments of the present invention is described below. The readable storage medium described below and the data compression and transmission method described above can be referred to in correspondence.

[0099] The present invention also provides a readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the above-described data compression and transmission method.

[0100] The computer-readable storage medium may include various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0101] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0102] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0103] Finally, it should be noted that in this document, relationships such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0104] The above provides a detailed description of a data compression and transmission method, apparatus, device, and computer-readable storage medium provided by the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A data compression and transmission method, characterized in that, include: Based on the characteristics of the data to be transmitted, construct the corresponding character sets respectively; The character set is evaluated, and a corresponding compression algorithm is determined and compressed based on the evaluation results to obtain compressed data. The compressed data is transmitted according to a preset single-packet data utilization rate; The character set is evaluated, and a corresponding compression algorithm is determined and compressed based on the evaluation results to obtain compressed data, including: Calculate the length, repetition density, and data entropy of the repeating substrings in the character set; The fourth character set, whose repeating substring length is greater than the first threshold, whose repeating density is greater than the second threshold, and whose data entropy is not greater than the third threshold, is compressed using a pre-filled dictionary and probabilistic interval coding to obtain the first compressed data. The fifth character set, whose repeating substring length is no greater than the first threshold, whose repeating density is no greater than the second threshold, and whose data entropy is greater than the third threshold, is compressed using a header compression mechanism and prime number base quantization to obtain the second compressed data. The sixth character set, with a uniform character frequency distribution, is compressed using the Deflate compression method to obtain the third compressed data; The compressed data is transmitted according to a preset single-packet data utilization rate, including: If the amount of compressed data is less than the minimum value of the target range, a first difference is calculated based on the amount of compressed data and the minimum value, and the amount of data to be transmitted is increased based on the first difference; the target range is determined based on the preset single packet data utilization rate. If the amount of compressed data is greater than the maximum value of the target range, then a portion of the compressed data will be transmitted, and the amount of the portion of data will be the maximum value of the target range; and a second difference will be calculated based on the amount of compressed data and the maximum value, and the amount of data to be transmitted will be reduced based on the second difference. If the amount of compressed data is within the target range, then the compressed data will be transmitted.

2. The data compression and transmission method according to claim 1, characterized in that, Based on the characteristics of the data to be transmitted, corresponding character sets are constructed, including: The fault data is segmented and optimized by analyzing the frequency distribution and time series characteristics of the fault data, and a first character set is constructed. A second character set is constructed based on time information; the time information includes the fault start time, end time, and current system time. A third character set is constructed based on the wind farm number, the wind turbine number, and whether it is the first fault.

3. The data compression and transmission method according to claim 1, characterized in that, The fourth character set, whose repeating substring length is greater than a first threshold, repeating density is greater than a second threshold, and data entropy is not greater than a third threshold, is compressed using a pre-padded dictionary and probabilistic interval coding to obtain the first compressed data, including: The pre-filled dictionary is constructed based on historical data; For the fourth character that is concentrated in the pre-filled dictionary, output the dictionary matching flag and index; For characters in the fourth character set that are not in the pre-filled dictionary, the LZ77 algorithm is used for matching. If the match is successful, the sliding window flag, distance, and length are output; if the match fails, the original value flag and the original value are output. The dictionary matching flag, the sliding window flag, and the original value flag are used as the first data to be encoded. The distance, the length, and the original value are respectively used as the second data to be encoded, the third data to be encoded, and the fourth data to be encoded. The first data to be encoded, the second data to be encoded, the third data to be encoded, and the fourth data to be encoded are respectively subjected to probability interval encoding to obtain the first compressed data.

4. The data compression and transmission method according to claim 3, characterized in that, The first compressed data is obtained by performing probability interval encoding on the first data to be encoded, the second data to be encoded, the third data to be encoded, and the fourth data to be encoded, respectively, including: The first data to be encoded, the second data to be encoded, the third data to be encoded, and the fourth data to be encoded are respectively used as input data; Define symbols for the input data and calculate the frequency of each symbol, and set an initial symbol interval based on the frequency of each symbol; Based on the initial symbol interval and each symbol, the symbol interval is updated until the final symbol interval is obtained; The shortest binary number within the final symbol interval is used as the first compressed data.

5. The data compression and transmission method according to claim 1, characterized in that, The fifth character set, whose repeating substring length is no greater than the first threshold, whose repeating density is no greater than the second threshold, and whose data entropy is greater than the third threshold, is compressed using a header compression mechanism and prime number base quantization to obtain the second compressed data, including: The symbol frequencies of the fifth character set are statistically analyzed to generate a frequency dictionary and a frequency sequence; the frequency dictionary includes the symbol sequence and the quantization frequency difference sequence. The total frequency is calculated based on the frequency of all symbols in the fifth character set, and prime numbers greater than or equal to the total frequency are used as the quantization basis. The frequency sequence is quantized using the quantization basis to obtain a quantized frequency sequence, a state space is constructed based on the quantized frequency sequence, an encoding table is generated based on the state space, and the fifth character set is converted into a compressed bit stream using the encoding table. The header information is compressed using dynamic bit-packing and fixed bit-packing methods to obtain the header compression result; the header information includes the frequency dictionary and quantization basis.

6. A data compression and transmission device, characterized in that, include: The character set construction module is used to construct the corresponding character set according to the characteristics of the data to be transmitted; The compression module is used to evaluate the character set, determine the corresponding compression algorithm based on the evaluation result, and perform compression to obtain compressed data. The transmission module is used to transmit the compressed data according to a preset single-packet data utilization rate; Compression module, including: A calculation unit is used to calculate the length of repeating substrings, the repeating density, and the data entropy of the character set; The first compression unit is used to compress the fourth character set whose repeating substring length is greater than the first threshold, repeating density is greater than the second threshold, and data entropy is not greater than the third threshold using a pre-filled dictionary and probability interval coding, to obtain the first compressed data. The second compression unit is used to compress the fifth character set whose repeating substring length is not greater than the first threshold, whose repeating density is not greater than the second threshold, and whose data entropy is greater than the third threshold using a header compression mechanism and prime number base quantization, to obtain the second compressed data. The third compression unit is used to compress the sixth character set, which has a uniform character frequency distribution, using the Deflate compression method to obtain the third compressed data; The transmission module includes: The data volume increase unit is used to calculate a first difference based on the compressed data volume and the minimum value of the target range if the data volume of the compressed data is less than the minimum value of the target range, and to increase the data volume of the data to be transmitted based on the first difference; the target range is determined based on the preset single packet data utilization rate. A data volume reduction unit is configured to transmit a portion of the compressed data if the data volume of the compressed data is greater than the maximum value of the target range, wherein the data volume of the portion of the compressed data is the maximum value of the target range; and to calculate a second difference based on the data volume of the compressed data and the maximum value, and to reduce the data volume of the data to be transmitted based on the second difference. A transmission unit is configured to transmit the compressed data if the amount of compressed data is within the target range.

7. A data compression and transmission device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of the data compression and transmission method as described in any one of claims 1 to 5 when executing the computer program.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when loaded and executed by a processor, implement the steps of the data compression and transmission method as described in any one of claims 1 to 5.

Images