Coding and decoding method for data compression transmission

Abstract

The invention discloses a coding and decoding method for data compression transmission, which comprises the following steps of: inputting data u with the length of B * q into a sparsifier at a sending end, and grouping to obtain a total number m of character string types; constructing a sparse code table with the size of # imgabs0 # according to the occurrence frequency of the character string type; sparse mapping is carried out on data u according to a code table to obtain a sequence v with the length of # imgabs1 #, the sequence v is interleaved and divided into L groups according to k bits in each group, and a sequence with the length of Lk is obtained.

Description

Technical Field

[0001] The invention belongs to the technical field of digital communication, and in particular relates to a coding and decoding method for data compression transmission. Background Art

[0002] In some communication business scenarios, a large amount of data is often generated, which usually occupies a large storage space and bandwidth, bringing challenges to transmission and storage. Therefore, data compression technology came into being, aiming to reduce the amount of data while retaining the quality of the data as much as possible. By adopting efficient compression algorithms, storage requirements and bandwidth consumption can be significantly reduced, thereby improving the efficiency of data transmission. This is particularly important in many applications, such as video streaming, audio transmission, and big data storage.

[0003] Existing data compression schemes such as Huffman coding and LZW coding can achieve lossless data compression by building dictionaries and using variable-length coding to represent characters with higher frequencies using shorter codewords. However, compression methods such as Huffman coding are actually very sensitive to bit errors. This is because they use variable-length coding. A small amount of bit errors may cause error propagation, thereby affecting the transmission quality of the entire data.

[0004] In addition, data may be affected by noise and errors may occur during transmission, so channel coding technology is introduced to correct the errors that occur to ensure the reliability of data transmission. The block Markov superposition transmission (BMST) method is a type of convolutional long code that can be constructed from short codes. It has a simple coding algorithm and can be decoded using a sliding window iterative decoding algorithm. The lower bound of its decoding performance can be determined by the basic code performance and the size of the coding memory (Sun Yat-sen University, A block Markov superposition coding method [P]: CN103152060A). By designing the construction of the connection matrix, a BMST codeword with a more flexible code rate can be obtained, which can realize the role of source coding, channel coding or source-channel joint coding. Summary of the invention

[0005] The main purpose of the present invention is to overcome the shortcomings and deficiencies of the prior art, and to provide a coding and decoding method for data compression transmission for communication business scenarios. While tolerating a certain error, compared with the existing variable-length compression transmission scheme, it can avoid error propagation and make the data more readable.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] The present invention provides a coding and decoding method for data compression transmission, comprising the following steps:

[0008] (1) At the transmitting end, let the data to be transmitted be u, whose length is B*q, and encode the data u into a transmission codeword of length N. in Representing N-dimensional binary space, the encoding method includes the following steps:

[0009] (1.1) Input the data u of length B*q into the sparsifier and map it into a data of length sequence The specific steps are as follows:

[0010] (1.1.1) Divide the data u to be transmitted with a length of B*q into B groups of equal length u=(u (0) ,u (1) ,…,u (B-1) ), each group size is q characters, and frequency statistics are performed on each group to obtain the total number of types m and frequency distribution results;

[0011] (1.1.2) Use the statistical frequency distribution results to construct a sparse code table, each group u (i) Using binary codewords Represents, where 0≤i≤B-1;

[0012] (1.1.3) The binary code word p (i) Arrange in order into a mapping sequence

[0013] (1.2) Interleave the sequence v and divide it into L groups with k bits per group, and obtain a sequence of length Lk v , and statistically obtain its sparsity Where W H ( v )yes v The Hamming weight of

[0014] (1.3) Sequence v Perform single-path or hierarchical post-multipath encoding to obtain a transmission codeword sequence c of length N;

[0015] (2) At the receiving end, for a received sequence y of length N, the estimated length of data is B*q The decoding method comprises the following steps:

[0016] (2.1) For the received sequence y of length N, single-path or multi-path decoding is performed to obtain a sequence of length Lk

[0017] (2.2) Sequence After the deinterleaver in the corresponding step (1.3), the length is obtained sequence

[0018] (2.3) Divide the sequence into B groups of equal length The length of each group is Input into the de-sparsifier, and demap according to the sparsification code table constructed in step (1.2) to obtain an estimate of the original data with a length of B*q

[0019] As a preferred technical solution, in step (1.1.2), the specific implementation method is:

[0020] Determine that the length of the binary codeword is according to the total number of types m counted in step (1.1.1). Reorder the binary codewords in ascending order of Hamming weight, and finally make them correspond one by one with the character group types sorted in descending order of occurrence frequency.

[0021] As a preferred technical solution, step (1.1.2) is specifically:

[0022] (1.1.2.1) Generate m binary codewords p of length j without repetition to construct a sparsification code table, where j = 0, 1,..., m - 1 is the type serial number;

[0023] (1.1.2.2) Map according to the frequency of occurrence of the character groups, so that the character groups with higher frequency correspond to the binary codewords with smaller Hamming weight, and the character groups with lower frequency correspond to the binary codewords with larger Hamming weight.

[0024] As a preferred technical solution, in step (1.3), perform single-path or hierarchical multi-path coding on the sequence v , specifically:

[0025] Adopt single-path coding or perform multi-path coding after data hierarchicalization. Each path uses the grouped Markov superposition transmission coding method to implement channel coding or joint source-channel coding.

[0026] As a preferred technical solution, in step (2.1), adopt the single-path or multi-path channel decoding or joint source-channel decoding algorithm corresponding to step (1.3).

[0027] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0028] In the encoding and decoding method for data compression transmission of the invention, at the sending end, first, input the data u with a length of B*q into a sparsifier. Divide it into B groups according to every q characters as a group for statistics, and obtain the total number m of string types; allocate binary codewords from high to low according to the occurrence frequency of string types, and the codeword length is and sort them in ascending order of Hamming weight. Finally, construct a code table with a size of m× ; perform sparse mapping on the data u according to the code table to obtain a sequence v with a length of . Interleave it and divide it into L groups according to k bits per group (padding with zeros if necessary) to obtain a sequence v with a length of Lk. Divide the sequence v into one or multiple paths for encoding (channel encoding / joint source-channel encoding) to obtain a codeword sequence c with a length of N. At the receiving end, decode the received sequence y with a length of N to obtain a sequence with a length of Lk. De-interleave the sequence and remove the padded zero bits to obtain a sequence with a length of . After inputting it into a de-sparsifier, obtain . Since the present invention uses fixed-length codewords for mapping the original data, there will be no error propagation phenomenon caused by bit errors as in the case of variable-length codewords, and the error can be limited to a single packet. The present invention reduces the requirement for transmission reliability. Under the condition of allowing certain data errors, compared with the variable-length compression transmission coding scheme, it obtains an improvement in data readability. In addition, the coding scheme of the present invention is simple in structure and can combine channel coding or joint source-channel coding technology according to actual needs to achieve reliable and efficient transmission of data. Description of the Drawings

[0029] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0030] Figure 1 is a transmission schematic diagram of the encoding and decoding method for data compression transmission of the present invention;

[0031] Figure 2 is the original Rubik's Cube image in Embodiment 1 of the present invention;

[0032] Figure 3It is the restored image transmitted when the Rubik's Cube image is encoded by Sparsifier - cascaded BMST (Sparsifier - BMST) at a signal - to - noise ratio (SNR) of 7 dB in Embodiment 1 of the present invention;

[0033] Figure 4 It is the restored image transmitted when the Rubik's Cube image is encoded by Huffman - cascaded BMST (Huffman - BMST) at SNR = 7 dB in Embodiment 1 of the present invention;

[0034] Figure 5 It is the original fireworks image in Embodiment 2 of the present invention;

[0035] Figure 6 It is the restored image transmitted when the fireworks image is encoded by Sparsifier - BMST at SNR = 7 dB in Embodiment 2 of the present invention;

[0036] Figure 7 It is the restored image transmitted when the fireworks image is encoded by Huffman - BMST at SNR = 7 dB in Embodiment 2 of the present invention. Detailed implementation manners

[0037] To enable those skilled in the art of this technology to better understand the solutions of this application, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of this application. Obviously, the described embodiments are only a part of the embodiments of this application, rather than all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative efforts fall within the scope of protection of this application.

[0038] When "embodiment" is mentioned in this application, it means that the specific features, structures or characteristics described in combination with the embodiment can be included in at least one embodiment of this application. The phrase appears at various positions in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment mutually exclusive with other embodiments. Those skilled in the art explicitly and implicitly understand that the embodiments described in this application can be combined with other embodiments.

[0039] Embodiment 1

[0040] As Figure 1 shown, Embodiment 1 of the present invention provides an encoding and decoding method for data compression and transmission, including the following steps:

[0041] (1) At the sending end, assume that the data u with a length of B * q to be transmitted is encoded into a transmission codeword with a length of N where Denote an N - dimensional binary space, and its encoding method includes the following steps:

[0042] (1.1) Input the data u with length B * q into a sparsifier and map it to a sequence with length as follows: The specific steps are as follows:

[0043] (1.1.1) Divide the data u to be transmitted with length B * q into B groups of equal length u=(u (0) , u (1) , …, u (B-1) ), with each group having a size of q characters. Conduct frequency statistics on it to obtain the total number of types m and the frequency distribution result.

[0044] (1.1.2) Use the statistically obtained frequency distribution result to construct a codebook. Each group u (i) is represented by a binary codeword , where 0 ≤ i ≤ B - 1.

[0045] (1.1.2.1) Generate m non - repeating binary codewords p with length j to construct a codebook, where j = 0, 1, …, m - 1 is the type serial number.

[0046] (1.1.2.2) Map according to the frequency of the character groups from high to low, so that the character groups with higher frequencies correspond to binary codewords with smaller Hamming weights, and the character groups with lower frequencies correspond to binary codewords with larger Hamming weights.

[0047] (1.1.3) Arrange the codewords p (i) , 0 ≤ i ≤ B - 1, in sequence to form a mapping sequence

[0048] (1.2) Interleave the sequence v and divide it into L groups of k bits each (padding with zeros if necessary) to obtain a sequence with length Lk v (padding with zeros if necessary), and statistically obtain its sparsity where W H ( v ) is v the Hamming weight of

[0049] (1.3) Perform single - path or hierarchical multi - path encoding on the sequence v in an appropriate way (realize channel coding / joint source - channel coding in the form of block - Markov superposition transmission coding) to obtain a transmission codeword c with length N.

[0050] (2) At the receiving end, for the received sequence y with length N, estimate the data with length B * q Its decoding method includes the following steps:

[0051] (2.1) Perform single-path or multi-path decoding on the received sequence y of length N to obtain a sequence of length Lk

[0052] (2.2) The sequence Passes through the deinterleaver in the corresponding step (1.3) (and removes the zero-padding bits) to obtain a sequence of length

[0053] (2.3) Divide the sequence into B groups of equal length Each group has a length of Put into the de-sparsifier, and perform demapping according to the code table constructed in step (1.2) to obtain an estimate of the original data of length B*q

[0054] The data of this Embodiment 1 is Figure 2 The pixel sequence u of the Rubik's Cube image shown, with a length of 18,874,368 characters. Statistically grouped by 96 characters per group, there are 105,755 types in total. The codeword length of each type is 17 bits, and a sparsification code table of size 105,755×17 is constructed. After sparsification mapping, a sequence v of length 3,342,336 is obtained. The sequence v is zero-padded and interleaved to obtain a sequence of length 3,344,000 v , and its sparsity θ = 0.284 is statistically obtained. For v Perform BMST encoding, generating sub-codewords of length n = 2000 for every k = 2000 bits, with a memory length of m = 2, and finally obtaining a codeword sequence c of length 3,348,000. The codeword sequence c is transmitted through an additive white Gaussian noise (AWGN) channel after binary phase-shift keying (BPSK) modulation. At the receiving end, a sliding window iterative decoding algorithm is used, with a decoding delay d = 9, to obtain a sequence estimate of length 3,348,000 After deinterleaving and removing the redundant zero-padding bits, a sequence of length 3,342,336 is finally obtained After de-sparsification mapping, an estimate of the original data is obtained Restore the image. Embodiment 1 uses the recovery performance of Huffman coding cascaded with BMST coding for image data encoding and transmission as a comparison object, where after Huffman coding, a sequence v of length 2,513,626 is obtained. The sequence v is zero-padded and interleaved to obtain a sequence of length 2,514,688v , sequence v is BMST-encoded. Each k = 1504 bits generates a sub-codeword of length n = 2000, with a memory length of m = 2, and finally a codeword sequence c of length 3348000 is obtained. From Figure 3 and Figure 4 in comparison, at SNR = 7 dB, the image restored by the Sparsifier-cascaded BMST (Sparsifier-BMST) scheme is more readable. The peak signal-to-noise ratio (PSNR) results after simulation are shown in Table 1. It can be seen that in the region of relatively low signal-to-noise ratio, the proposed scheme has higher image quality compared to the Huffman-cascaded BMST (Huffman-BMST) scheme. Through the encoding method for data compression and transmission in this Embodiment 1, binary codewords with different Hamming weights are allocated according to the occurrence frequency of character groups, and BMST encoding is used to achieve joint source-channel coding to improve transmission efficiency and combat transmission noise. Its method steps are simple and easy to implement. Compared with the traditional method of Huffman coding compression cascaded channel coding, it can avoid error propagation and ensure the quality of data.

[0055] Table 1 Performance comparison between Sparsifier-BMST scheme and Huffman-BMST scheme at different SNRs

[0056] SNR 6dB 7dB 8dB 9dB Sparsifier-BMST 26.41dB 36.71dB 47.90dB 61.61dB Huffman-BMST 11.1dB 11.52dB 10.98dB 13.68dB

[0057] Embodiment 2

[0058] An encoding method for data compression and transmission provided in this Embodiment 2, the data is Figure 5 the pixel sequence u of the displayed fireworks image, with a length of 88510445 characters. It is statistically analyzed in groups of 96 characters, and there are a total of 626390 types. The codeword length of each type is 20 bits, and a sparsification code table of size 626390×20 is constructed. After sparsification mapping, a sequence v of length 18439680 is obtained. The sequence v is padded with zeros and interleaved to obtain a sequence v of length 18439985, and its sparsity θ = 0.31827 is statistically obtained. For v is BMST-encoded. Each k = 2005 generates a sub-codeword of length n = 2005, with a memory length of m = 2, and finally a codeword sequence c of length 18443995 is obtained. The codeword sequence c is transmitted through an AWGN channel after BPSK modulation. At the receiving end, a sliding window iterative decoding algorithm is adopted, with a decoding delay d = 10, to obtain a sequence estimate of length 18439985 After deinterleaving and removing the redundant zero-padding bits, a sequence of length 18439680 is obtained The estimation of the original data is obtained after de-sparsification mapping Restore the image. In this Embodiment 3, the recovery performance of encoding and transmitting image data using Huffman coding cascaded with BMST coding is used as the comparison object. After Huffman coding, a sequence v with a length of 15165098 is obtained. After padding zeros and interleaving the sequence v, a sequence with a length of 15165853 is obtained v For the sequence v perform BMST coding. Each k = 1649 generates sub-codewords with a length of n = 2005, and the memory length is m = 2, obtaining a codeword sequence c with a length of 18443995. From Figure 6 and Figure 7 In comparison, the restored image by the Sparsifier - BMST scheme has better readability at SNR = 7dB. The PSNR results after simulation are shown in Table 3. It can be seen that in the region with relatively low signal - to - noise ratio, the proposed scheme has higher image quality compared to the Huffman - BMST scheme. Through the encoding scheme for data compression and transmission in this Embodiment 2, binary codewords with different Hamming weights are assigned according to the occurrence frequency of character groups, and BMST coding is used to achieve joint source - channel coding to improve transmission efficiency and combat transmission noise. Its method steps are simple and easy to implement. Compared with the traditional method of Huffman coding compression cascaded with channel coding, it can avoid error propagation and ensure the quality of data

[0059] Table 3 Performance comparison between Sparsifier - BMST scheme and Huffman - BMST scheme at different SNR

[0060] SNR 6dB 7dB 8dB 9dB Sparsifier-BMST 17.51dB 40.86dB 52.95dB 63.61dB Huffman-BMST 15.70dB 15.66dB 16.24dB 17.16dB

[0061] Embodiment 3

[0062] A coding and decoding method for data compression and transmission in this Embodiment 3 considers a data sequence u with a length of 1000000, which consists of four symbols: A, B, C, and D, with corresponding probabilities {0.1081, 0.3244, 0.5405, 0.0270}, and the source entropy is 1.5067. Statistically, taking 1 character as a group, there are a total of four types, and the codeword length for each type is 2 bits. A sparsification code table with a size of 4×2 is constructed as follows:

[0063] Symbol Probability Codeword A 0.1081 10 B 0.3244 01 C 0.5405 00 D 0.0270 11

[0064] After sparsification mapping according to the code table, a sequence v with a length of 2000000 is obtained. Single - path coding or multi - path coding after grading can be adopted. (1) When single - path coding is adopted, the bit sparsity of the sequence v is statistically obtained as θ = 0.2432, and after interleaving, a 2000000 - bit sequence is obtainedv Perform BMST encoding on v , generating sub-codewords of length n = 1700 for every k = 1000 bits, with a memory length of m = 8. Finally, a codeword sequence c of length 1713600 is obtained, and the code rate is 1.7136. At the decoding end, a sliding window iterative decoding algorithm is used, with a decoding delay d = 16, to obtain a sequence estimate of length 2000000 After de-interleaving, a sequence of length 2000000 is finally obtained After sparse de-mapping, an estimate of the original data is obtained No errors occurred. (2) When multi-channel encoding is used after grading, the sequence v is divided into two channels according to the number of bits of the codeword to obtain sequences v1 and v2 with lengths of 1000000 each. The sparsity of v1 is statistically obtained as θ1 = 0.8649, and the sparsity of v2 is θ2 = 0.6486. After interleaving, two sequences of 1000000 bits each are obtained v 1 and v 2. Perform BMST encoding on v 1 and v 2 respectively. For the first channel, sub-codewords of length n = 646 are generated for every k = 1000 bits, with a memory length of m = 8, to obtain a codeword sequence c1 of length 651168; for the second channel, sub-codewords of length n = 998 are generated for every k = 1000 bits, with a memory length of m = 8, to obtain a codeword sequence c2 of length 1005984. Finally, the two channels are combined to obtain a codeword sequence c of length 1657152, and the code rate is 1.6572. Assume that the codeword sequence c is transmitted through a channel without noise interference. At the decoding end, the received sequence y of length 1657152 is divided into two sequences y1 and y2 with lengths of 651168 and 1005984 respectively. Combining conditional probabilities, the sliding window iterative decoding algorithm is used for the two channels respectively, with a decoding delay d = 16, to obtain sequence estimates of length 1000000 and After de-interleaving and combining, a sequence of length 2000000 is obtained After sparse de-mapping, an estimate of the original data is obtained No errors occurred.

[0065] Through the encoding method for data compression transmission in this Embodiment 3, binary codewords with different Hamming weights are allocated according to the appearance frequency of character groups, and can be implemented by single-channel or multi-channel BMST encoding after grading. Its method steps are simple, easy to implement, and approach the source entropy.

[0066] It should be noted that, for the foregoing method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations. However, those skilled in the art should know that the present invention is not limited by the described action sequence, because according to the present invention, some steps can be performed in other sequences or simultaneously.

[0067] Those of ordinary skill in the art can understand that all or part of the processes in the above method embodiments can be completed by instructing relevant hardware through a computer program. The program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the above method embodiments. Among them, any reference to a memory, storage, database, or other medium used in the various embodiments provided in the present application can include non-volatile and / or volatile memories. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0068] The technical features of the above embodiments can be combined arbitrarily. For the sake of concise description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered to be within the scope described in this specification.

Claims

1. A coding and decoding method for data compression transmission, characterized in that: The steps include: (1) At the transmitting end, let the data to be transmitted be u, whose length is B*q, and encode the data u into a transmission codeword of length N. in Representing N-dimensional binary space, the encoding method includes the following steps: (1.1) Input the data u of length B*q into the sparsifier and map it into a data of length sequence The specific steps are as follows: (1.1.1) Divide the data u to be transmitted with a length of B*q into B groups of equal length u=(u (0) ,u (1) ,…,u (B-1) ), each group size is q characters, and frequency statistics are performed on each group to obtain the total number of types m and frequency distribution results; (1.1.2) Use the statistical frequency distribution results to construct a sparse code table, each group u (i) Using binary codewords Represents, where 0≤i≤B-1; (1.1.3) The binary code word p (i) Arrange in order into a mapping sequence (1.2) Interleave the sequence v and divide it into L groups with k bits per group, and obtain a sequence of length Lk v , and statistically obtain its sparsity Where W H ( v )yes v The Hamming weight of (1.3) Sequence v Perform single-path or hierarchical post-multipath encoding to obtain a transmission codeword sequence c of length N; (2) At the receiving end, for a received sequence y of length N, the estimated length of data is B*q The decoding method comprises the following steps: (2.1) For the received sequence y of length N, single-path or multi-path decoding is performed to obtain a sequence of length Lk (2.2) Sequence After the deinterleaver in the corresponding step (1.3), the length is obtained sequence (2.3) Change the sequence Divided into equal length groups B The length of each group is Will Input the desparser, demap according to the sparse code table constructed in step (1.2), and obtain an estimate of the original data of length B*q 2. The encoding and decoding method for data compression transmission according to claim 1, characterized in that: In step (1.1.2), the specific implementation method is: According to the total number of types m counted in step (1.1.1), the length of the binary codeword is determined as The binary code words are reordered according to the Hamming weight from small to large, and finally correspond one to one with the character group types after being sorted from large to small according to the frequency of occurrence.

3. The encoding and decoding method for data compression transmission according to claim 2, characterized in that: Step (1.1.2) is as follows: (1.1.2.1) Generate m numbers of length The non-repeating binary code word p j Construct a sparse code table, where j = 0, 1, ..., m-1 is the type number; (1.1.2.2) Character groups are mapped according to their frequencies, so that character groups with higher frequencies correspond to binary codewords with smaller Hamming weights, and character groups with lower frequencies correspond to binary codewords with larger Hamming weights.

4. The encoding and decoding method for data compression transmission according to claim 1, characterized in that: In step (1.3), the sequence v is encoded in a single-pass or hierarchical manner, specifically: Single-channel coding is adopted or multi-channel coding is adopted after data is classified, and each channel uses group Markov superposition transmission coding to realize channel coding or joint source channel coding.

5. The encoding and decoding method for data compression transmission according to claim 1, characterized in that: In step (2.1), the single-channel or multi-channel channel decoding or joint source channel decoding algorithm corresponding to step (1.3) is adopted.

Images