Data processing method, system, storage medium and device based on ansi encoding

Abstract

The application provides an ANS-based data processing method, system, storage medium and device, the method comprising: sequentially extracting symbols from a symbol sequence, and performing ANS encoding on the extracted current symbol to obtain a current encoding value, and determining whether the current encoding value is greater than a first threshold; in response to the current encoding value being greater than the first threshold, determining that the current encoding value is a large number, and making the large number spit out according to a preset bit number, and transmitting the data of the preset bit number to a code stream; determining whether the data obtained after the current encoding value performs spitting out once is less than or equal to a second threshold; in response to the data being less than or equal to the second threshold, determining that the data is a small number, and stopping spitting out, and transmitting the small number to the code stream. The application changes a part of information of the large number into a small number by spitting out, solves the problem that the current mainstream computer cannot process a large number, and limits the application scene of ANS encoding, and is also helpful to realizing streaming decoding during subsequent ANS decoding.

Description

Technical Field

[0001] This invention relates to the field of encoding technology, and in particular to a data processing method, system, storage medium, and device based on ANS encoding. Background Technology

[0002] Entropy coding is a lossless data compression scheme. The core idea of ​​entropy coding is to use fewer bits to represent frequently occurring symbols and more bits to represent rarely occurring elements.

[0003] Huffman coding and arithmetic coding are two of the most common entropy coding methods, which will be briefly described below. The algorithm of Huffman coding is simple, based on the probabilistic ordering of the symbol set to allocate the code length. Huffman coding always uses integer bits to represent a symbol, and it encodes each symbol individually. Therefore, Huffman coding cannot guarantee optimal compression. Huffman coding produces the best results when the probabilities of all symbols are negative powers of 2. In Huffman coding, each occurrence of a symbol is always encoded with the same codeword. The advantage of Huffman coding is its fast encoding speed.

[0004] According to information theory, the ideal codeword length of a single symbol is determined solely by the probability of the symbol's occurrence: code-length(x) = -log p(x) If the probability of a symbol occurring is 0.4, then the ideal codeword length is 1.32(-log). 0.4 Unfortunately, Huffman coding can only assign codewords of integer length. This is a major drawback of the Huffman coding algorithm.

[0005] Arithmetic coding addresses this weakness of Huffman coding; it's another entropy coding technique. It encodes input data as a range of real numbers between 0 and 1. As the number of bits required to encode the input increases, this range becomes smaller. Unlike Huffman coding, arithmetic coding uses near-precise probabilities, thus achieving compression rates close to the theoretical limit. However, the algorithm for arithmetic coding is more complex, and its coding efficiency is very low (roughly 1 / 10 that of Huffman coding), limiting its application in real-time data compression applications.

[0006] Therefore, in recent years, many scholars have been searching for a new algorithm that achieves a compression ratio close to that of arithmetic coding and a coding efficiency close to that of Huffman coding. In 2009, Jarek Duda proposed Asymmetric Numeral Systems (ANS). According to Jarek Duda, Asymmetric Numeral Systems achieve a compression ratio comparable to arithmetic coding while having a processing speed similar to Huffman coding. The mathematical principles of ANS are introduced below:

[0007] Suppose we need to convert a set S of two symbols... i The sequence of symbols ∈A = {0, 1} is encoded into a number num. First, considering the standard binary number system, this sequence of symbols can be encoded as num = ∑S i ·2 i In this case, each symbol ("0" or "1") in the sequence occupies 1 bit. The bit width of the number num is n. Obviously, the binary number system ignores the frequency of occurrence of the two symbols (0,1), that is, the binary number system is applicable to the case that conforms to a uniform distribution of (0,1).

[0008] According to Shannon's information theory, assuming the probability distribution of each symbol in a sequence of symbols is {P1, P2, P3, ..., P...} n}, then the average amount of information per symbol in the symbol set is:

[0009]

[0010] The amount of information contained in symbol S and the probability P of that symbol S Related log(1 / P) s The information content of the two symbols S1 and S2 combined is:

[0011] Assuming natural numbers Therefore, the information content of x combined with S2 is:

[0012]

[0013] Based on the mathematical relationships in the above formula, Jarek Duda further hypothesizes that if the number x and the symbol s are combined and encoded into a natural number x′, then the formula for calculating x′ would be as follows:

[0014]

[0015] The above is the basic mathematical principle of ANS. ANS encodes a sequence of symbols into natural numbers, which is similar to the principle of the AC (arithmetic coding) algorithm.

[0016] In fact, formula (3) cannot be directly used for ANS encoding. To facilitate the description of the ANS encoding and decoding process, two expressions are introduced as follows:

[0017]

[0018] s is the current symbol, x is the current number, x' is the encoded number, C(~) represents the encoding algorithm, and D(~) represents the decoding algorithm. (x,s) represents the decoded result.

[0019] Formula (3) describes the encoding principle of natural numbers and (probability) symbols, which belongs to the local level. The encoding principle of natural numbers and symbol sequences is as follows: Suppose that the natural number x is encoded with the symbol sequence [S1, S2, S3, ... S... n The symbols are synthesized and encoded, with the probabilities of each symbol being [P1, P2, P3, ..., Pn].

[0020] x and S1 are combined to generate the natural number x1:

[0021] x2 and S2 are combined to generate the natural number x2:

[0022] x2 and S3 are combined to generate the natural number x3:

[0023] x n-1 With S n Synthetic encoding generates the natural number x3:

[0024] From a macroscopic perspective, the symbol sequence encoded using the ANS algorithm ultimately generates the number y as follows:

[0025]

[0026] Clearly, the natural number calculated by formula (5) is an extremely large number. Due to the limitations of computer data processing bit width, both hardware and software struggle to handle such a large number. It can be said that it is almost impossible to achieve such a "whole package" ANS encoding and decoding.

[0027] ANS will convert the symbol sequence into a very large number, so it is impossible to construct an infinitely long list. The most feasible solution is to construct a fixed-length list in a finite field and extend the list to infinity, that is, the symbol distribution function is a periodic function S(x+n×T)=S(x).

[0028] Assume that symbol s appears f times within one period. s Therefore, it is clear that the number of cycles in which the x-th s will appear is... And this is the mth occurrence within that period, where m = mod(x, f). s ).

[0029] Suppose the period length of S(x) is T. For a specific number y, the sign at position y is obviously the same as the sign at position y′ = mod(y,T) in the first period, i.e., S(y) = S(mod(y,T)). Let S(y′) be the m-th occurrence in this period, and let y be the x-th occurrence, then:

[0030]

[0031] Symbol sequences encoded using ANS (Answer Sequence of Symbols) result in extremely large natural numbers, which current mainstream computers cannot process, thus limiting the application scenarios of ANS. To solve this problem, the encoding mode of ANS needs to be improved. Summary of the Invention

[0032] In view of this, the purpose of this invention is to propose a data processing method, system, storage medium and device based on ANS encoding, so as to solve the problem that the application scenarios of ANS encoding are limited because mainstream computers cannot process large numbers.

[0033] To achieve the above objectives, the present invention provides a data processing method based on ANS encoding, comprising the following steps:

[0034] The symbols are extracted sequentially from the symbol sequence, and the extracted current symbol is encoded using ANS to obtain the current encoded value. The current encoded value is then determined to be greater than the first threshold.

[0035] In response to the current encoded value being greater than the first threshold, the current encoded value is determined to be a large number, and the large number is output according to a preset number of bits, and the preset number of bits of data is transmitted to the bit stream;

[0036] Determine whether the data obtained after each output of the current encoded value is less than or equal to the second threshold.

[0037] In response to the data being less than or equal to the second threshold, the data is determined to be a decimal, the output stops, and the decimal is transmitted to the bitstream.

[0038] In some embodiments, the second threshold is the probability of the next symbol after the current symbol.

[0039] In some embodiments, causing a large number to be output according to a preset number of bits, and transmitting the preset number of bits to the bitstream includes:

[0040] This allows the large number to output one bit of data at a time, and outputs that one bit of data to the bitstream.

[0041] In some embodiments, the first threshold is greater than the period value, which is the reciprocal of the sum of the probabilities of the symbol sequences.

[0042] In some embodiments, the first threshold is 2. 32 Or 2 64 .

[0043] In some embodiments, the method further includes:

[0044] In response to the current encoded value not being a large or small number, the current encoded value is used as the synthesized encoded object for the next symbol.

[0045] In some embodiments, using the current encoded value as the synthetic encoded object for the next symbol includes:

[0046] Use the probability of the next symbol to divide the current encoded value to perform synthetic encoding.

[0047] Another aspect of the present invention provides a data processing system based on ANS encoding, comprising:

[0048] The first judgment module is configured to extract symbols sequentially from the symbol sequence, perform ANS encoding on the extracted current symbol to obtain the current encoding value, and determine whether the current encoding value is greater than the first threshold.

[0049] The output module is configured to respond to the current encoded value being greater than a first threshold, determine that the current encoded value is a large number, output the large number according to a preset number of bits, and transmit the preset number of bits of data to the bit stream;

[0050] The second judgment module is configured to determine whether the data obtained after each output of the current encoded value is less than or equal to a second threshold; and

[0051] The transmission module is configured to, in response to data being less than or equal to a second threshold, determine that the data is a decimal, stop outputting numbers, and transmit the decimal to the bitstream.

[0052] In another aspect, the present invention provides a computer-readable storage medium storing computer program instructions that, when executed by a processor, implement the above-described method.

[0053] In another aspect, the present invention provides a computer device including a memory and a processor, wherein the memory stores a computer program that, when executed by the processor, performs the above-described method.

[0054] The present invention has at least the following beneficial technical effects:

[0055] The data processing method based on ANS encoding of the present invention extracts symbols sequentially from a symbol sequence, performs ANS encoding on the extracted current symbol to obtain the current encoded value, and determines whether the current encoded value is greater than a first threshold. If the current encoded value is greater than the first threshold, the current encoded value is determined to be a large number, and the large number is processed by outputting a preset number of bits. The preset number of bits is then transmitted to the bitstream. It is determined whether the data obtained after each output of the current encoded value is less than or equal to a second threshold. If the data is less than or equal to the second threshold, the data is determined to be a small number, the output stops, and the small number is transmitted to the bitstream. This method "outputs" a portion of the information of the large number and transforms it into a small number, solving the problem that the current mainstream computers cannot process large numbers, which limits the application scenarios of ANS encoding. At the same time, it helps to achieve streaming decoding during subsequent ANS decoding. Attached Figure Description

[0056] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other embodiments can be obtained based on these drawings without creative effort.

[0057] Figure 1 This is a schematic diagram of a data processing method based on ANS encoding according to an embodiment of the present invention;

[0058] Figure 2 This is a schematic diagram of a data processing system based on ANS encoding according to an embodiment of the present invention;

[0059] Figure 3 This is a schematic diagram of a computer-readable storage medium for implementing an ANS-encoded data processing method according to an embodiment of the present invention;

[0060] Figure 4 This is a schematic diagram of the hardware structure of a computer device for executing an ANS-based data processing method according to an embodiment of the present invention. Detailed Implementation

[0061] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be further described in detail below with reference to specific examples and the accompanying drawings.

[0062] It should be noted that all uses of "first" and "second" in the embodiments of the present invention are for the purpose of distinguishing two different entities or different parameters with the same name. Therefore, "first" and "second" are merely for convenience of expression and should not be construed as limiting the embodiments of the present invention. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, such as other steps or units inherent in a process, method, system, product, or device that includes a series of steps or units.

[0063] To achieve the above objectives, the first aspect of the present invention provides an embodiment of a data processing method based on ANS encoding. Figure 1 The diagram shown is a schematic representation of an embodiment of the data processing method based on ANS encoding provided by the present invention. Figure 1 As shown, the embodiments of the present invention include the following steps:

[0064] Step S10: Extract symbols sequentially from the symbol sequence, perform ANS encoding on the extracted current symbol to obtain the current encoding value, and determine whether the current encoding value is greater than the first threshold.

[0065] Step S20: In response to the current encoded value being greater than the first threshold, determine that the current encoded value is a large number, and make the large number output according to the preset number of bits, and transmit the preset number of bits to the bit stream;

[0066] Step S30: Determine whether the data obtained after each output of the current encoded value is less than or equal to the second threshold;

[0067] Step S40: In response to the data being less than or equal to the second threshold, determine that the data is a decimal, stop outputting numbers, and transmit the decimal to the bitstream.

[0068] The data processing method based on ANS encoding in this invention extracts symbols sequentially from a symbol sequence, performs ANS encoding on the extracted current symbol to obtain the current encoded value, and determines whether the current encoded value is greater than a first threshold. If the current encoded value is greater than the first threshold, the current encoded value is determined to be a large number, and the large number is processed by outputting a preset number of bits. The preset number of bits is then transmitted to the bitstream. It is determined whether the data obtained after each output of the current encoded value is less than or equal to a second threshold. If the data is less than or equal to the second threshold, the data is determined to be a small number, the output stops, and the small number is transmitted to the bitstream. This method "outputs" a portion of the information of the large number and transforms it into a small number, solving the problem that current mainstream computers cannot process large numbers, which limits the application scenarios of ANS encoding. It also helps to achieve streaming decoding during subsequent ANS decoding.

[0069] In some embodiments, the second threshold is the probability of the next symbol after the current symbol.

[0070] In some embodiments, causing a large number to output data according to a preset number of bits and transmitting the preset number of bits to the bitstream includes: causing the large number to output one bit of data at a time and outputting one bit of data to the bitstream.

[0071] In this embodiment, large numbers are output one bit at a time, or multiple bits of data can be output at a time.

[0072] In some embodiments, the first threshold is greater than the period value, which is the reciprocal of the sum of the probabilities of the symbol sequences.

[0073] In some embodiments, the first threshold is 2. 32 Or 2 64 .

[0074] In some embodiments, the method further includes: in response to the current encoded value not being a large number or a small number, using the current encoded value as a synthetic encoded object for the next symbol.

[0075] In some embodiments, using the current encoded value as the synthetic encoding object for the next symbol includes: dividing the current encoded value by the probability of the next symbol to perform synthetic encoding.

[0076] The following are specific embodiments of the data processing method based on ANS encoding of the present invention:

[0077] 1) Method for determining large numbers: A common method is the threshold method. In this embodiment, a fixed threshold is used to determine large numbers: a threshold Thresh (i.e., the first threshold, which is much larger than the period T, T=2) is set. n In the encoding process, if the current encoded number x is greater than this threshold, x will be considered a large number and needs to be output. In this embodiment, the recommended first threshold is 2. 32 Or 2 64 .

[0078] 2) Decimal determination method: In this embodiment, the "adaptive" threshold method is used to determine the decimal: The threshold (i.e. the second threshold) is related to the probability of the next symbol after the current encoded symbol. When the number y is less than the threshold, the number y will be determined as a decimal and the output of numbers will stop.

[0079] 3) Implementation of the "Spit Out" function:

[0080] ① "Discharge" timing and conditions: During the encoding process, once the value of the encoded number exceeds the first threshold mentioned above, the "discharge" function will be activated.

[0081] ② "Spit out" position: "Spit out" data from the least significant part of the large number into the bit stream according to the preset number of bits.

[0082] ③ The function and length of "spitting out": The portion of a large number to be spat out is divided into two parts for spat out:

[0083] Suppose the large number is X, and the probability of the next symbol after the current symbol is fs (the second threshold):

[0084]

[0085] Here, >> represents a right shift operation based on bits.

[0086] The smaller numbers generated after the large numbers are "spit out" will continue to be used as the initial input for the next stage of coding.

[0087] 4) Implementation of the "swallowing" function on the decoding end:

[0088] ① Timing of "swallowing": According to the decoding formula of ANS (Asymmetric Numeral Systems), a number X can be decoded into a symbol S by ANS. old And a new number X′. In this embodiment, when X′ <f s At this time, data needs to be read ("swallowed") from the bitstream before the next stage of decoding is performed.

[0089] ② Similar to the "output" function of encoding, decoding also requires "ingesting" bit by bit and updating the number X. The pseudocode is as follows:

[0090]

[0091] The pseudocode for block coding is as follows:

[0092]

[0093]

[0094] The pseudocode used in the decoding process in this embodiment is as follows:

[0095]

[0096]

[0097] A second aspect of the present invention also provides a data processing system based on ANS encoding. Figure 2 The diagram shown is a schematic representation of an embodiment of the data processing system based on ANS encoding provided by the present invention. Figure 2As shown, a data processing system based on ANS encoding includes: a first judgment module 10, configured to extract symbols sequentially from a symbol sequence, perform ANS encoding on the extracted current symbol to obtain a current encoded value, and determine whether the current encoded value is greater than a first threshold; a number output module 20, configured to, in response to the current encoded value being greater than the first threshold, determine that the current encoded value is a large number, output the large number according to a preset number of bits, and transmit the preset number of bits to the bitstream; a second judgment module 30, configured to, determine whether the data obtained after each output of the current encoded value is less than or equal to a second threshold; and a transmission module 40, configured to, in response to the data being less than or equal to the second threshold, determine that the data is a small number, stop outputting, and transmit the small number to the bitstream.

[0098] The data processing system based on ANS encoding in this invention extracts symbols sequentially from a symbol sequence, performs ANS encoding on the extracted current symbol to obtain the current encoded value, and determines whether the current encoded value is greater than a first threshold. If the current encoded value is greater than the first threshold, it is determined that the current encoded value is a large number, and the large number is processed by outputting a preset number of bits. The preset number of bits is then transmitted to the bitstream. It is determined whether the data obtained after each output of the current encoded value is less than or equal to a second threshold. If the data is less than or equal to the second threshold, the data is determined to be a small number, the output stops, and the small number is transmitted to the bitstream. This process "outputs" a portion of the information of the large number and transforms it into a small number, solving the problem that current mainstream computers cannot process large numbers, which limits the application scenarios of ANS encoding. It also helps to achieve streaming decoding during subsequent ANS decoding.

[0099] In some embodiments, the second threshold is the probability of the next symbol after the current symbol.

[0100] In some embodiments, the output module 20 includes a bit output module configured to output one bit of data at a time for large numbers and output one bit of data to the bit stream.

[0101] In this embodiment, large numbers are output one bit at a time, or multiple bits of data can be output at a time.

[0102] In some embodiments, the first threshold is greater than the period value, which is the reciprocal of the sum of the probabilities of the symbol sequences.

[0103] In some embodiments, the first threshold is 2. 32 Or 2 64 .

[0104] In some embodiments, the system further includes a synthetic coding module configured to use the current coding value as the synthetic coding object for the next symbol in response to the current coding value not being a large number or a small number.

[0105] In some embodiments, the synthetic coding module is further configured to perform synthetic coding by dividing the current coding value by the probability of the next symbol.

[0106] A third aspect of the present invention also provides a computer-readable storage medium. Figure 3 A schematic diagram of a computer-readable storage medium implementing an ANS-encoded data processing method according to an embodiment of the present invention is shown. Figure 3 As shown, the computer-readable storage medium 3 stores computer program instructions 31. When executed by the processor, the computer program instructions 31 perform the following steps:

[0107] The symbols are extracted sequentially from the symbol sequence, and the extracted current symbol is encoded using ANS to obtain the current encoded value. The current encoded value is then determined to be greater than the first threshold.

[0108] In response to the current encoded value being greater than the first threshold, the current encoded value is determined to be a large number, and the large number is output according to a preset number of bits, and the preset number of bits of data is transmitted to the bit stream;

[0109] Determine whether the data obtained after each output of the current encoded value is less than or equal to the second threshold.

[0110] In response to the data being less than or equal to the second threshold, the data is determined to be a decimal, the output stops, and the decimal is transmitted to the bitstream.

[0111] In some embodiments, the second threshold is the probability of the next symbol after the current symbol.

[0112] In some embodiments, causing a large number to output data according to a preset number of bits and transmitting the preset number of bits to the bitstream includes: causing the large number to output one bit of data at a time and outputting one bit of data to the bitstream.

[0113] In some embodiments, the first threshold is greater than the period value, which is the reciprocal of the sum of the probabilities of the symbol sequences.

[0114] In some embodiments, the first threshold is 2. 32 Or 2 64 .

[0115] In some embodiments, the step further includes: in response to the current encoded value not being a large number or a small number, using the current encoded value as the synthetic encoded object for the next symbol.

[0116] In some embodiments, using the current encoded value as the synthetic encoding object for the next symbol includes: dividing the current encoded value by the probability of the next symbol to perform synthetic encoding.

[0117] It should be understood that, where there is no conflict, all the embodiments, features and advantages described above for the data processing method based on ANS encoding according to the present invention are equally applicable to the data processing system and storage medium based on ANS encoding according to the present invention.

[0118] A fourth aspect of the present invention also provides a computer device, including as follows: Figure 4 The memory 402 and processor 401 shown are provided. The memory 402 stores a computer program that, when executed by the processor 401, implements the method of any of the above embodiments.

[0119] like Figure 4 The diagram shown is a hardware structure schematic of an embodiment of a computer device for executing an ANS-based data processing method provided by the present invention. Figure 4 Taking the computer device shown as an example, this computer device includes a processor 401 and a memory 402, and may also include an input device 403 and an output device 404. The processor 401, memory 402, input device 403, and output device 404 can be connected via a bus or other means. Figure 4 Taking a bus connection as an example, input device 403 can receive input digital or character information, and generate key signal inputs related to user settings and function control of the ANSI-encoded data processing system. Output device 404 may include display devices such as a display screen.

[0120] Memory 402, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as the program instructions / modules corresponding to the ANS-encoded data processing method in this embodiment. Memory 402 may include a program storage area and a data storage area, wherein the program storage area may store the operating system and application programs required for at least one function; the data storage area may store data created by the use of the ANS-encoded data processing method, etc. In addition, memory 402 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, memory 402 may optionally include memory remotely located relative to processor 401, and these remote memories can be connected to the local module via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0121] The processor 401 executes various functional applications and data processing of the server by running non-volatile software programs, instructions and modules stored in the memory 402, that is, it implements the data processing method based on ANS encoding in the above method embodiment.

[0122] Finally, it should be noted that the computer-readable storage medium (e.g., memory) described herein can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. By way of example and not limitation, non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM), which may act as external cache memory. The storage devices disclosed herein are intended to include, but are not limited to, these and other suitable types of memory.

[0123] Those skilled in the art will also understand that the various exemplary logic blocks, modules, circuits, and algorithm steps described in conjunction with the disclosure herein can be implemented as electronic hardware, computer software, or a combination of both. To clearly illustrate this interchangeability between hardware and software, the functionality of various illustrative components, blocks, modules, circuits, and steps has been generally described. Whether this functionality is implemented as software or as hardware depends on the specific application and the design constraints imposed on the system as a whole. Those skilled in the art can implement the functionality in various ways for each specific application, but such implementation decisions should not be construed as departing from the scope of the embodiments disclosed herein.

[0124] The various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure herein can be implemented or performed using the following components designed to perform the functions herein: general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic, discrete hardware components, or any combination of these components. A general-purpose processor may be a microprocessor, but alternatively, the processor may be any conventional processor, controller, microcontroller, or state machine.

[0125] The above are exemplary embodiments disclosed in this invention. However, it should be noted that various changes and modifications can be made without departing from the scope of the embodiments of this invention as defined by the claims. The functions, steps, and / or actions of the methods according to the disclosed embodiments described herein do not need to be performed in any particular order. Furthermore, although the elements disclosed in the embodiments of this invention may be described or claimed individually, they may be understood as multiple unless explicitly limited to a singular number.

[0126] It should be understood that, as used herein, the singular form "a" is intended to include the plural form as well, unless the context clearly supports an exception. It should also be understood that, as used herein, "and / or" refers to any and all possible combinations of one or more of the associatedly listed items. The embodiment numbers disclosed above are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0127] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of the invention (including the claims) is limited to these examples. Within the framework of the invention, technical features of the above embodiments or different embodiments can be combined, and many other variations of different aspects of the invention exist, which are not provided in the details for the sake of brevity. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the invention should be included within the protection scope of the invention.

Claims

1. A data processing method based on ANS encoding, characterized in that, Includes the following steps: Symbols are extracted sequentially from the symbol sequence, and the extracted current symbol is encoded using ANS to obtain the current encoded value. It is then determined whether the current encoded value is greater than a first threshold. In response to the current encoded value being greater than the first threshold, the current encoded value is determined to be a large number, and the large number is output according to a preset number of bits, and the preset number of bits is transmitted to the bitstream; Determine whether the data obtained after each output of the current encoded value is less than or equal to the second threshold; In response to the data being less than or equal to the second threshold, the data is determined to be a decimal, and the output of numbers is stopped, and the decimal is transmitted to the bitstream.

2. The method according to claim 1, characterized in that, The second threshold is the probability of the next symbol after the current symbol.

3. The method according to claim 1, characterized in that, The process of outputting the large number according to a preset number of bits and transmitting the preset number of bits to the bitstream includes: The large number outputs one bit of data at a time, and the one bit of data is output to the bitstream.

4. The method according to claim 1, characterized in that, The first threshold is greater than the period value, where the period value is the reciprocal of the sum of the probabilities of the symbol sequence.

5. The method according to claim 4, characterized in that, The first threshold is 2 32 Or 2 64 .

6. The method according to claim 1, characterized in that, Also includes: In response to the current encoded value not belonging to the large number or the small number, the current encoded value is used as the synthesized encoded object for the next symbol.

7. The method according to claim 6, characterized in that, The synthetic encoding object that uses the current encoded value as the next symbol includes: The current encoded value is divided by the probability of the next symbol to perform synthetic encoding.

8. A data processing system based on ANS encoding, characterized in that, include: The first judgment module is configured to extract symbols sequentially from the symbol sequence, perform ANS encoding on the extracted current symbol to obtain the current encoding value, and determine whether the current encoding value is greater than a first threshold. The output module is configured to respond to the current encoded value being greater than the first threshold, determine that the current encoded value is a large number, output the large number according to a preset number of bits, and transmit the preset number of bits to the bitstream; The second judgment module is configured to determine whether the data obtained after each output of the current encoded value is less than or equal to the second threshold. as well as The transmission module is configured to, in response to the data being less than or equal to the second threshold, determine that the data is a decimal, stop transmitting numbers, and transmit the decimal to the bitstream.

9. A computer-readable storage medium, characterized in that, The system stores computer program instructions that, when executed by a processor, implement the method as described in any one of claims 1-7.

10. A computer device, comprising a memory and a processor, characterized in that, The memory stores a computer program, which, when executed by the processor, performs the method as described in any one of claims 1-7.

Images