A method, system, device, and storage medium for calculating a digital effective bit width

Abstract

The application provides a method, system, device and storage medium for calculating the effective bit width of a number, which comprises the following steps: converting the numbers with the same effective bit width in a preset standard bit width into corresponding characteristic numbers to obtain all characteristic numbers in the preset standard bit width; determining a standard value so that the standard value and the results obtained by processing the all characteristic numbers respectively satisfy uniform distribution, and obtaining a corresponding mapping list according to the standard value; converting a number to be calculated into a corresponding characteristic number, and processing the characteristic number and the standard value to obtain a result to be processed; and determining the effective bit width of the number to be calculated from the mapping list according to the result to be processed. Compared with the current bit-by-bit comparison method, the application can greatly reduce the number of calculations and is easy to implement in hardware; not only can the effective bit width of a number be quickly obtained, but also the solving time is fixed.

Description

Technical Field

[0001] This invention relates to the field of data compression, and more specifically, to a method, system, apparatus, and storage medium for calculating the effective bit width of a digital data. Background Technology

[0002] With the rapid development of cutting-edge technologies such as big data, artificial intelligence, and blockchain, data is growing explosively, putting immense pressure on existing storage devices. As cloud computing replaces traditional computing architectures, the structure of data storage is also changing, with computing and storage resources further converging in top data centers, further straining server storage. Faced with this ever-increasing volume of data, data compression has become one of the effective methods to alleviate the burden on server storage and reduce storage costs.

[0003] Data compression can reduce data volume to decrease storage space and improve transmission, storage, and processing efficiency without losing useful information. Lossless data compression is generally achieved through two methods: one is a dictionary-based compression algorithm, including the LZ series algorithms, which can perform duplicate data search; the other is an entropy coding algorithm based on statistical models, such as Huffman codes, arithmetic codes, and asymmetric number systems. Most current mainstream data compression standards adopt a hybrid approach: using the LZ77 algorithm to search for duplicate data in the original data, and then using an entropy coding scheme to encode and compress the symbols generated by the LZ77 algorithm. As mentioned above, entropy coding is based on statistical models. The entropy coding scheme used in current data compression standards typically does not directly statistically analyze the symbols themselves, but rather classifies the symbols and then calculates the total frequency of symbols within each class.

[0004] Data compression standards typically use the effective bit width of a symbol as a classification criterion. Therefore, when performing frequency statistics, entropy coding schemes need to first calculate the effective bit width of each symbol, then determine the classification number, and finally perform class frequency statistics. In computer systems, to accurately represent and transmit data, a fixed standard bit width is used to store data; 32 bits is a commonly used standard bit width. As mentioned earlier, the entropy coding scheme used in data compression standards classifies symbols based on their effective bit width. The effective bit width of a number refers to the position of the highest bit "1" in a binary digit. For example, the decimal number 531 has the binary form '0b 0000 0010 0001 0011', and its effective bit width is 10. Currently, to calculate the effective bit width of a number, a bit-by-bit comparison algorithm is usually used.

[0005] Starting from the most significant bit of the standard bit width of a number, each bit is compared sequentially from highest to lowest. The bit containing the first "1" indicates the effective bit width of the number. Because the bit-by-bit comparison scheme uses a cyclical approach to calculate the bit width, the number of calculations varies for a given number: for the 32-bit standard bit width number 12345 ('0b 00000000 0000 0000 0011 0000 0011 1001'), the number of calculations is 18; for 98765 ('0b 0000 000000000 0001 1000 0001 1100 1101'), the number of calculations is 15. In hardware implementations, the time consumption is typically required to be consistent for the same functional module. Therefore, the bit-by-bit comparison scheme cannot be used in hardware implementations. Summary of the Invention

[0006] In view of this, the purpose of this invention is to provide a method, system, computer device, and computer-readable storage medium for calculating the effective bit width of a number. If implemented in software, this invention can significantly reduce the number of calculations compared to the current bit-by-bit comparison method. This invention only uses the simplest calculation operations (bit OR, shift, multiplication, and table lookup), so it is easy to implement in hardware. It can not only quickly obtain the effective bit width of a number, but also fix the solution time. In addition, applying this invention to data compression-related algorithms can improve the data compression rate.

[0007] Based on the above objectives, one aspect of the present invention provides a method for calculating the effective bit width of a number, comprising the following steps: converting numbers with the same effective bit width in a preset standard bit width into corresponding feature numbers to obtain all feature numbers in the preset standard bit width; determining a standard value such that the results of processing the standard value and all feature numbers respectively satisfy a uniform distribution, and obtaining a corresponding mapping list based on the standard value; converting the number to be calculated into a corresponding feature number, and processing the feature number and the standard value to obtain a processing result; and determining the effective bit width of the number to be calculated from the mapping list based on the processing result.

[0008] In some implementations, converting numbers with the same effective bit width in the preset standard bit width into corresponding feature numbers includes: setting all bits below the most significant bit to the first number and performing format conversion.

[0009] In some implementations, determining a standard value such that the result of processing the standard value with all the feature numbers satisfies a uniform distribution includes: multiplying the standard value with all the feature numbers and shifting the shifted number of bits based on the preset standard bit width.

[0010] In some implementations, obtaining the corresponding mapping list based on the standard value includes: sorting all feature numbers in ascending order of effective bit width, processing the standard value with each of the feature numbers in order, and forming a mapping list from the processed results.

[0011] In another aspect, this invention provides a system for calculating the effective bit width of a number, comprising: a conversion module configured to convert numbers with the same effective bit width in a preset standard bit width into corresponding feature numbers, so as to obtain all feature numbers in the preset standard bit width; a determination module configured to determine a standard value such that the results of processing the standard value and all feature numbers respectively satisfy a uniform distribution, and to obtain a corresponding mapping list based on the standard value; a processing module configured to convert the number to be calculated into a corresponding feature number, and to process the feature number and the standard value to obtain a processing result; and a mapping module configured to determine the effective bit width of the number to be calculated from the mapping list based on the processing result.

[0012] In some implementations, the conversion module is configured to: set all bits below the most significant bit to a first number and perform format conversion.

[0013] In some implementations, the determining module is configured to: multiply the standard value by all the feature numbers and shift them, wherein the number of shifts is determined based on the preset standard bit width.

[0014] In some implementations, the determining module is configured to: sort all feature numbers in ascending order of effective bit width, process the standard value with each of the feature numbers in order, and form a mapping list from the processed results.

[0015] In another aspect of the present invention, a computer device is provided, comprising: at least one processor; and a memory storing computer instructions executable on the processor, the instructions, when executed by the processor, implementing the steps of the method described above.

[0016] In another aspect, the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method steps.

[0017] This invention has the following beneficial technical effects: By converting numbers with the same effective bit width in a preset standard bit width into corresponding feature numbers, determining the standard value that satisfies the uniform distribution of the result after processing the feature numbers, and determining the effective bit width of the number according to the mapping list corresponding to the standard value, the number of calculations can be greatly reduced compared with the current bit-by-bit comparison method; this invention only uses the simplest calculation operations (bit OR, shift, multiplication, table lookup), so it is easy to implement in hardware. It can not only quickly obtain the effective bit width of the number, but also fix the solution time. In addition, applying this invention to data compression related algorithms can improve the data compression rate. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other embodiments can be obtained based on these drawings without creative effort.

[0019] Figure 1 A schematic diagram illustrating an embodiment of the method for calculating the effective bit width of a digital number provided by the present invention;

[0020] Figure 2 A schematic diagram of an embodiment of the system for calculating the effective bit width of a number provided by the present invention;

[0021] Figure 3 A schematic diagram of the hardware structure of an embodiment of a computer device for calculating the effective bit width of numbers provided by the present invention;

[0022] Figure 4 This is a schematic diagram of an embodiment of a computer storage medium for calculating the effective bit width of numbers provided by the present invention. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be further described in detail below with reference to specific examples and the accompanying drawings.

[0024] It should be noted that all uses of "first" and "second" in the embodiments of the present invention are for the purpose of distinguishing two entities or parameters with the same name but different names. It is clear that "first" and "second" are only for the convenience of expression and should not be construed as limiting the embodiments of the present invention. Subsequent embodiments will not explain this in detail.

[0025] In a first aspect, an embodiment of a method for calculating the effective bit width of a number is proposed. Figure 1 The diagram shown is an embodiment of the method for calculating the effective bit width of a digital number provided by the present invention. Figure 1 As shown, the embodiments of the present invention include the following steps:

[0026] S1. Convert the numbers with the same effective bit width in the preset standard bit width into the corresponding feature numbers to obtain all the feature numbers in the preset standard bit width;

[0027] S2. Determine a standard value such that the results of processing the standard value and all the feature numbers satisfy a uniform distribution, and obtain a corresponding mapping list based on the standard value;

[0028] S3. Convert the number to be calculated into a corresponding feature number, and process the feature number and the standard value to obtain the processing result; and

[0029] S4. Determine the effective bit width of the number to be calculated from the mapping list based on the result to be processed.

[0030] This invention is applicable to effective bit widths of standard bit widths such as 8 bits, 16 bits, and 32 bits. The implementation steps are explained using a 32-bit number as an example.

[0031] Numbers with the same valid bit width in the preset standard bit width are converted into corresponding feature numbers to obtain all feature numbers in the preset standard bit width. A number with a standard bit width of 32 bits will be transformed into 32 possible numbers, which are the feature numbers in this application.

[0032] According to binary digit theory, for a specific effective bit width n, the corresponding digital value range is 2^n. n ……2 n +1 To convert numbers within the above range to the same number, the simplest method is to convert them to the maximum or minimum value within the range. Since the minimum value (min) and maximum value (max) of each interval have the following relationship: max = 2 * min - 1, it can be seen that there is a simple operational relationship between the maximum and minimum values. Therefore, converting numbers within the above range to the maximum value is essentially equivalent to converting them to the minimum value. Furthermore, the conversion algorithm should be the same for different values ​​of n, meaning the conversion algorithm should be universal.

[0033] In some implementations, converting numbers with the same effective bit width in a preset standard bit width into corresponding feature numbers includes: setting all bits below the most significant bit to a first number and performing a format conversion. The first number can be 0 or 1. In this embodiment of the invention, numbers with the same effective bit width are converted to the maximum value; therefore, in this embodiment of the invention, the first number is 1. All 32-bit numbers are converted into the following 32 numbers: {0x1, 0x3, 0x7, 0xf, 0x1f, 0x3f, 0x7f, 0xff, 0x1ff, 0x3ff, 0x7ff, 0xfff, 0x1fff, 0x3fff, 0x7fff, 0xffff, 0x1ffff, 0x3ffff, 0x7ffff, 0xfffff, 0x1fffff, 0x3fffff, 0x7fffff, 0x7ffffff, 0x1fffffff, 0x3ffffff, 0x7ffffff, 0xfffffff, 0x1fffffff, 0x7fffffff, 0xfffffff}.

[0034] Determine a standard value such that the results of processing all the feature numbers according to the standard value satisfy a uniform distribution, and obtain a corresponding mapping list based on the standard value.

[0035] In some implementations, determining a standard value such that the result of processing the standard value with all the feature numbers satisfies a uniform distribution includes: multiplying the standard value with all the feature numbers and shifting the shifted number of bits based on the preset standard bit width.

[0036] Ideally, a constant should be found that satisfies the following conditions: the result of adding the constant to 32 characteristic numbers and then shifting is exactly equal to its effective bit width, or the result of multiplying the constant to 32 characteristic numbers and then shifting is exactly equal to its effective bit width, i.e., i = (x[i] + c) >> 27 or i = (x[i] * c) >> 27, where i represents the effective bit width of the number, x[i] represents the characteristic number of the number, c represents the constant to be found, and >> represents a signed right shift. However, no such constant has been found. Therefore, this embodiment of the invention finds a constant that satisfies the following conditions: the result of multiplying the constant to 32 characteristic numbers and then shifting is uniformly distributed, as shown in the following numbers: 131148599, 131148659, 131148699, 131150519, 131151447, 2015605587, 2015605963, 20156 06067, 2015697367, 2015697371, 2279248243, 2279249713, 2279250105, 2279250199, 2279250289, 4164560471, 4164560587, 4164561703, 4164568423, 4164576343. This embodiment of the invention uses the constant c = 4164560587 as an example for illustration.

[0037] In some implementations, obtaining the corresponding mapping list based on the standard value includes: sorting all feature numbers in ascending order of effective bit width, processing the standard value with each of the feature numbers in order, and forming a mapping list from the processed results.

[0038] Assuming the mapping list is Map

[32] , the calculation process of the mapping list is as follows:

[0039] Map

[32] = {0}

[0040] For(i=0;i<32;i++){

[0041] Y=(unit32_t)(4164560587*x[i]);

[0042] Map[Y] = i;

[0043] }

[0044] Using the above code, the mapping list Map

[32] of 4164560587 is {31, 4, 5, 14, 6, 15, 25, 18, 7, 12, 16, 26, 28, 22, 19, 8, 30, 3, 13, 24, 17, 11, 27, 21, 29, 2, 23, 10, 20, 1, 9, 0}.

[0045] The number to be calculated is converted into a corresponding feature number, and the feature number and the standard value are processed to obtain the result.

[0046] The effective bit width of the number to be calculated is determined from the mapping list based on the processing result.

[0047] The embodiments of the present invention will use the number 123456 as an example to describe the above steps in detail:

[0048] The 123456 ('0b 0000 0000 0000 0001 1110 0010 0100 0000') is converted using the algorithm described above, resulting in a feature value of 131071 ('0b 0000 0000 0000 0001 11111111 1111 1111'). The feature value 131071 is multiplied by 4164560587 and then shifted right by 27 bits, as shown in the following calculation: Y = (131071 * 4164560587) >> 27 = 10. The effective bit width of 123456 is calculated using the mapping list: Valid_bits = Map

[10] = 16, so the effective bit width of 123456 is 16.

[0049] The entire calculation scheme can be described using the following pseudocode:

[0050] x=n|1

[0051] x|=x>>1

[0052] x|=x>>2

[0053] x|=x>>4

[0054] x|=x>>8

[0055] x|=x>>16

[0056] y = (c * x) >> 27

[0057] valid_bits(n) = Map[y]

[0058] This invention calculates the effective bit width of a 32-bit number using only 7 lines of code, making it a "unique and simple" solution. When implemented in software, this invention significantly reduces the number of calculations compared to current bit-by-bit comparison methods. Utilizing only the simplest computational operations, it is easy to implement in hardware, providing a fast and fixed solution time for the effective bit width. In data compression algorithms, the effective bit width is typically used as the basis for classifying symbols, leading to frequent calls to the effective bit width calculation interface. Applying this invention to data compression algorithms can improve data compression speed.

[0059] It should be particularly noted that the steps in each embodiment of the above method for calculating the effective bit width of a number can be interleaved, substituted, added, or deleted. Therefore, these reasonable permutations and combinations of the method for calculating the effective bit width of a number should also fall within the protection scope of this invention, and the protection scope of this invention should not be limited to the embodiments.

[0060] Based on the above objectives, a second aspect of the present invention proposes a system for calculating the effective bit width of a number. For example... Figure 2 As shown, system 200 includes the following modules: a conversion module, configured to convert numbers with the same effective bit width in a preset standard bit width into corresponding feature numbers, so as to obtain all feature numbers in the preset standard bit width; a determination module, configured to determine a standard value so that the results of processing the standard value and all feature numbers respectively satisfy a uniform distribution, and obtain a corresponding mapping list according to the standard value; a processing module, configured to convert the number to be calculated into the corresponding feature number, and process the feature number and the standard value to obtain the result to be processed; and a mapping module, configured to determine the effective bit width of the number to be calculated from the mapping list according to the result to be processed.

[0061] In some implementations, the conversion module is configured to: set all bits below the most significant bit to a first number and perform format conversion.

[0062] In some implementations, the determining module is configured to: multiply the standard value by all the feature numbers and shift them, wherein the number of shifts is determined based on the preset standard bit width.

[0063] In some implementations, the determining module is configured to: sort all feature numbers in ascending order of effective bit width, process the standard value with each of the feature numbers in order, and form a mapping list from the processed results.

[0064] Based on the above objectives, a third aspect of the present invention provides a computer device, comprising: at least one processor; and a memory storing computer instructions executable by the processor to perform the following steps: S1, converting numbers with the same effective bit width in a preset standard bit width into corresponding feature numbers to obtain all feature numbers in the preset standard bit width; S2, determining a standard value such that the results of processing the standard value and all feature numbers respectively satisfy a uniform distribution, and obtaining a corresponding mapping list based on the standard value; S3, converting the number to be calculated into a corresponding feature number, and processing the feature number and the standard value to obtain a processing result; and S4, determining the effective bit width of the number to be calculated from the mapping list based on the processing result.

[0065] In some implementations, converting numbers with the same effective bit width in the preset standard bit width into corresponding feature numbers includes: setting all bits below the most significant bit to the first number and performing format conversion.

[0066] In some implementations, determining a standard value such that the result of processing the standard value with all the feature numbers satisfies a uniform distribution includes: multiplying the standard value with all the feature numbers and shifting the shifted number of bits based on the preset standard bit width.

[0067] In some implementations, obtaining the corresponding mapping list based on the standard value includes: sorting all feature numbers in ascending order of effective bit width, processing the standard value with each of the feature numbers in order, and forming a mapping list from the processed results.

[0068] like Figure 3 The diagram shown is a hardware structure schematic of an embodiment of the computer device for calculating the effective bit width of numbers provided by the present invention.

[0069] For example Figure 3 Taking the device shown as an example, the device includes a processor 301 and a memory 302.

[0070] Processor 301 and memory 302 can be connected via a bus or other means. Figure 3 Taking the example of a connection between China and Israel via a bus.

[0071] Memory 302, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as the program instructions / modules corresponding to the method for calculating the effective bit width of a number in this embodiment. Processor 301 executes various functional applications and data processing of the server by running the non-volatile software programs, instructions, and modules stored in memory 302, thereby implementing the method for calculating the effective bit width of a number.

[0072] Memory 302 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created using a method for calculating the effective bit width of a number. Furthermore, memory 302 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, memory 302 may optionally include memory remotely located relative to processor 301, and these remote memories may be connected to the local module via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0073] One or more computer instructions 303 corresponding to the method of calculating the effective bit width of a number are stored in memory 302. When executed by processor 301, the method of calculating the effective bit width of a number in any of the above method embodiments is executed.

[0074] Any embodiment of the computer device that performs the above-described method for calculating the effective bit width of a number can achieve the same or similar effects as any of the aforementioned method embodiments.

[0075] The present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, performs a method for calculating the effective bit width of a number.

[0076] like Figure 4 The diagram shown is a schematic representation of an embodiment of the computer storage medium for calculating the effective bit width of numbers provided by the present invention. Figure 4 Taking the computer storage medium shown as an example, the computer-readable storage medium 401 stores a computer program 402 that, when executed by a processor, performs the above method.

[0077] Finally, it should be noted that those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program for calculating the effective bit width of a number can be stored in a computer-readable storage medium. When executed, the program can include the processes of the embodiments of the above methods. The storage medium for the program can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc. The above computer program embodiments can achieve the same or similar effects as any of the corresponding foregoing method embodiments.

[0078] The above are exemplary embodiments disclosed in this invention. However, it should be noted that various changes and modifications can be made without departing from the scope of the embodiments of this invention as defined by the claims. The functions, steps, and / or actions of the methods according to the disclosed embodiments described herein do not need to be performed in any particular order. Furthermore, although the elements disclosed in the embodiments of this invention may be described or claimed individually, they may be understood as multiple unless explicitly limited to a singular number.

[0079] It should be understood that, as used herein, the singular form “a” is intended to include the plural form as well, unless the context clearly supports an exception. It should also be understood that, as used herein, “and / or” refers to any and all possible combinations of one or more of the associated listed items.

[0080] The embodiment numbers disclosed in the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0081] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0082] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of the invention (including the claims) is limited to these examples. Within the framework of the invention, technical features of the above embodiments or different embodiments can be combined, and many other variations of different aspects of the invention exist, which are not provided in the details for the sake of brevity. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the invention should be included within the protection scope of the invention.

Claims

1. A method for calculating the effective bit width of a number, characterized in that, Includes the following steps: Convert the numbers with the same effective bit width in the preset standard bit width into the corresponding feature numbers to obtain all the feature numbers in the preset standard bit width. The conversion of the numbers with the same effective bit width in the preset standard bit width into the corresponding feature numbers includes: setting all bits below the most significant bit to the first number and performing format conversion. Determining a standard value such that the result of processing the standard value with all the feature numbers satisfies a uniform distribution includes: multiplying the standard value with all the feature numbers and shifting the shift by a number of bits based on the preset standard bit width; and obtaining a corresponding mapping list based on the standard value includes: sorting all the feature numbers in ascending order of effective bit width, processing the standard value with all the feature numbers in order, and forming a mapping list from the processed results. The number to be calculated is converted into a corresponding feature number, and the feature number and the standard value are processed to obtain the result; and The effective bit width of the number to be calculated is determined from the mapping list based on the processing result. The standard value is a constant determined based on the following conditions: the result of multiplying the constant by all the feature numbers and then shifting the result equals its effective bit width.

2. A system for calculating the effective bit width of a digital number, characterized in that, include: The conversion module is configured to convert numbers with the same effective bit width in the preset standard bit width into corresponding feature numbers, so as to obtain all feature numbers in the preset standard bit width; The determination module is configured to determine a standard value such that the results of processing the standard value and all the feature numbers satisfy a uniform distribution, and to obtain a corresponding mapping list based on the standard value; The processing module is configured to convert the number to be calculated into a corresponding feature number, and process the feature number and the standard value to obtain the processing result; as well as A mapping module is configured to determine the effective bit width of the number to be calculated from the mapping list based on the result to be processed. The conversion module is configured to: set all bits below the most significant bit to the first digit and perform format conversion; The determining module is configured to: multiply the standard value by all the feature numbers and shift them, wherein the number of shifts is determined based on the preset standard bit width; and sort all the feature numbers in ascending order of effective bit width, process the standard value by all the feature numbers in order, and form a mapping list from the processed results; wherein the standard value is a constant determined based on the following condition, wherein the result of multiplying the constant by all the feature numbers and then shifting them is equal to its effective bit width.

3. A computer device, characterized in that, include: At least one processor; as well as A memory storing computer instructions executable on the processor, which, when executed by the processor, implement the steps of the method of claim 1.

4. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method of claim 1.

Images