Data compression and decompression method and related equipment
By fuzzing and integer processing on non-integer data, the problem of unstable data compression in the prior art is solved, and better compression effect and efficiency are achieved.
Patent Information
- Application Number
- CN202411144233.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-20
- Publication Date
- 2024-11-29
AI Technical Summary
The existing data compression algorithms are unstable when processing non-integer data, which may cause the storage space of compressed data to occupy more than the storage space of pre-compressed data.
By fuzzing the non-integer data, multiplying it by 10 to the N power and rounding it, converting it into an integer and compressing it to ensure that the compressed data is an integer, thereby stabilizing the data compression effect.
It effectively avoids fluctuations in the compression effect of non-integer data, ensures that the storage space of compressed data is always less than or equal to the pre-compressed data, significantly improving the stability and efficiency of data compression.
Smart Images

Figure CN119051666A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the technical field of data storage, and in particular to a data compression and decompression method and related equipment. Background Art
[0002] In the process of storing data sequences, the use of data compression algorithms can effectively reduce the storage space occupied when storing data sequences.
[0003] It is found in the application that when the data values included in the data sequence are integers, the data compression algorithm provided by the relevant technology can obtain a relatively stable data compression effect, that is, the storage space occupied by the compressed data is always much smaller than the storage space occupied by the data before compression; and when the data values included in the data sequence are non-integer, the data compression effect of the data compression algorithm provided by the relevant technology fluctuates greatly, that is, the difference between the storage space occupied by the compressed data and the storage space occupied by the data before compression fluctuates. In special scenarios, the storage space occupied by the compressed data may even be larger than the storage space occupied by the data before compression.
[0004] That is to say, the data compression effect of the data compression scheme provided by the related art is unstable. Summary of the invention
[0005] The purpose of the present disclosure is to provide a data compression and decompression method and related equipment, so as to solve the technical problem that the data compression effect of the data compression scheme provided by the related art is unstable.
[0006] In a first aspect, the present application provides a data compression method, the method comprising:
[0007] In the case where there are non-integer original data in the target sequence, each original data in the target sequence is fuzzy processed to obtain a fuzzy sequence, wherein the non-integer original data is original data including decimal places in the target sequence, the fuzzy sequence includes fuzzy data corresponding to the original data in the target sequence one by one, the fuzzy data is the integer value of the product of the corresponding original data and the fuzzy coefficient, the fuzzy coefficient is 10 to the Nth power, and N is a positive integer less than or equal to a preset coefficient threshold;
[0008] The fuzzy sequence is compressed to obtain target compressed data.
[0009] In a second aspect, the present application also provides a data decompression method, the method comprising:
[0010] Decompress the target compressed data to obtain a fuzzy sequence;
[0011] In the case where the target compressed data corresponds to original data including decimal places, inverse fuzzy processing is performed on each fuzzy data in the fuzzy sequence to obtain a first sequence, wherein the first sequence includes decompressed data corresponding one-to-one to the fuzzy data in the fuzzy sequence, and the decompressed data is the ratio of the corresponding fuzzy data to the fuzzy coefficient, and the fuzzy coefficient is 10 to the power of N, where N is a positive integer less than or equal to a preset coefficient threshold.
[0012] In a third aspect, the present application further provides a data compression device, the device comprising:
[0013] A fuzzy processing module, configured to perform fuzzy processing on each original data in the target sequence respectively to obtain a fuzzy sequence when there is non-integer original data in the target sequence, wherein the non-integer original data is original data including decimal places in the target sequence, the fuzzy sequence includes fuzzy data corresponding to the original data in the target sequence one by one, the fuzzy data is the integer value of the product of the corresponding original data and a fuzzy coefficient, the fuzzy coefficient is 10 to the power of N, and N is a positive integer less than or equal to a preset coefficient threshold;
[0014] The data compression module is used to compress the fuzzy sequence to obtain target compressed data.
[0015] In a fourth aspect, the present application further provides a data decompression device, the device comprising:
[0016] A data decompression module is used to decompress the target compressed data to obtain a fuzzy sequence;
[0017] The inverse fuzzy module is used to perform inverse fuzzy processing on each fuzzy data in the fuzzy sequence when the target compressed data corresponds to the original data including decimal places, so as to obtain a first sequence, wherein the first sequence includes decompressed data corresponding to the fuzzy data in the fuzzy sequence one by one, and the decompressed data is the ratio of the corresponding fuzzy data to the fuzzy coefficient, and the fuzzy coefficient is 10 to the power of N, where N is a positive integer less than or equal to a preset coefficient threshold.
[0018] In a fifth aspect, the present application provides an electronic device, comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the method described in the first aspect, or implements the steps of the method described in the second aspect.
[0019] In a sixth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the method described in the first aspect are implemented, or the steps of the method described in the second aspect are implemented.
[0020] In a seventh aspect, the present application provides a computer program product, comprising computer instructions, which, when executed by a processor, implement the steps of the method described in the first aspect, or implement the steps of the method described in the second aspect.
[0021] In the present application, before compressing the original data, the original data is multiplied and rounded to achieve integer processing of the original data, and then the integerized original data is compressed. This can avoid the fluctuation problem of the data compression effect of the original data in the non-integer case, so that the final target compressed data can maintain a good data compression effect. BRIEF DESCRIPTION OF THE DRAWINGS
[0022] Figure 1 is a flow chart of a data compression method provided by an embodiment of the present disclosure;
[0023] Figure 2 is a schematic diagram of a data structure of a sequence before and after compression provided by an embodiment of the present disclosure;
[0024] Figure 3 It is a flowchart of a data decompression method provided by an embodiment of the present disclosure;
[0025] Figure 4 is a structural schematic diagram of a data compression device provided by an embodiment of the present disclosure;
[0026] Figure 5 is a structural schematic diagram of a data decompression device provided by an embodiment of the present disclosure;
[0027] Figure 6 It is a schematic diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION
[0028] The following will be combined with the drawings in the embodiments of the present disclosure to clearly and completely describe the technical solutions in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, rather than all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.
[0029] The present disclosure provides a data compression method. Figure 1 As shown, the data compression method comprises:
[0030] Step 101: When there are non-integer original data in the target sequence, fuzzy processing is performed on each original data in the target sequence to obtain a fuzzy sequence.
[0031] Among them, the non-integer original data is the original data including decimal places in the target sequence, the fuzzy sequence includes fuzzy data corresponding one to one with the original data in the target sequence, the fuzzy data is the integer value of the product of the corresponding original data and the fuzzy coefficient, and the fuzzy coefficient is 10 to the power of N, where N is a positive integer less than or equal to a preset coefficient threshold.
[0032] The target sequence is formed by arranging a plurality of sequence elements in order, and each of the sequence elements at least includes an attribute representing a numerical value (such as an integer value and / or a decimal value).
[0033] In an example, when a sequence element may only include attributes representing numerical values, the target sequence may be understood as a pure numerical sequence, for example, a sequence [3, 2, 4, 5, 1].
[0034] In another example, in addition to the attribute representing the numerical value, the sequence element may also include at least one other attribute (such as a time attribute, a spatial attribute, a gender attribute, etc.), for example: a time series data sequence [(time 1, value 1), (time 2, value 2), (time 3, value 3)].
[0035] In the present application, the original data may be understood as the attribute data representing the numerical value in the sequence elements included in the target sequence, and the non-integer original data may be understood as the original data including decimal places.
[0036] For example, if N is set to 1, and the target sequence includes multiple original data of 0.11, 0.2 and 0.3 respectively, then the products of the multiple original data and the fuzzy coefficients are 1.1, 2 and 3 respectively. After rounding the products of the original data and the fuzzy coefficients, the three fuzzy data corresponding to the obtained fuzzy sequence are 1, 2 and 3 respectively.
[0037] Referring to the above example, in view of the situation that the product of the original data and the fuzzy coefficient may still include decimals in the application, rounding is performed to ensure that the fuzzy data in the obtained fuzzy sequence are all integers that do not include decimals, thereby ensuring the stability of the subsequent data compression effect.
[0038] It should be noted that the setting of the above-mentioned preset coefficient threshold is to match the situation in which business scenarios usually only focus on values within a limited number of decimal places (such as seven digits). In addition, setting too large a fuzzy coefficient may increase the risk of the numerical length of the multiplied data exceeding the bounds (the numerical length exceeding the bounds will cause memory overflow and cause program errors). In order to adapt to the actual application needs of business scenarios and reduce the risk of the numerical length of the multiplied data exceeding the bounds, the above-mentioned preset coefficient threshold is set accordingly.
[0039] In the application, the above preset coefficient threshold can be set and adjusted according to actual needs. For example, the preset coefficient threshold can be set to 7.
[0040] Step 102: compress the fuzzy sequence to obtain target compressed data.
[0041] In the present application, before compressing the original data, the original data is multiplied and rounded to achieve integer processing of the original data, and then the integerized original data is compressed. This can avoid the fluctuation problem of the data compression effect of the original data in the non-integer case, so that the final target compressed data can maintain a good data compression effect.
[0042] Exemplarily, the data compression operation on the fuzzy sequence may be completed by performing an exclusive OR (XOR) operation on the fuzzy sequence.
[0043] For example, if the XOR operation is used for data compression, and the original data are floating point values 0.1, 0.2, and 0.3, each original data storage needs to occupy 64 bits;
[0044] If the method described in the present application is not used for data compression, the XOR value of 0.1 and 0.2 is 0x0070000000000000, which takes up 16 bits after compression, and the XOR value of 0.2 and 0.3 is 0x001aaaaaaaaaaaa9, which takes up 66 bits after compression. Among them, the bit occupancy of the XOR value of 0.2 and 0.3 is higher than the bit occupancy of the original data.
[0045] When the method described in the present application is used for data compression, the floating-point values 0.1, 0.2 and 0.3 can be converted into integer values 1, 2 and 3. The XOR value of 1 and 2 is 0x7FF0000000000000, which requires 24 bits after compression; the XOR value of 2 and 3 is 0x0008000000000000, which requires 14 bits after compression. This compression method will not cause the bit occupancy of the compressed XOR value to be higher than the bit occupancy of the original data, that is, the bit occupancy of the XOR value formed after compression will always be less than the bit occupancy of the original data, and the total bit occupancy of the compressed XOR value is lower.
[0046] In the application, in order to distinguish the compressed data that has been compressed after being blurred and the compressed data that has been directly compressed without being blurred, different storage structures can be used to store the two separately. For example, table A is used to store the compressed data that has been compressed after being blurred, and table B is used to store the compressed data that has been directly compressed without being blurred. When taking data from table A, the compressed data is first decompressed, and then the corresponding reverse blur processing (that is, multiplication processing) is performed to restore the real original data; and when taking data from table B, the compressed data is directly decompressed to obtain the real original data.
[0047] In one embodiment, when there are non-integer original data in the target sequence, each original data in the target sequence is fuzzy processed to obtain a fuzzy sequence, including:
[0048] Comparing the ratio of the number of non-integer original data in the target sequence with a preset ratio threshold to obtain a comparison result;
[0049] When the comparison result indicates that the quantity proportion is greater than or equal to a preset ratio threshold, each original data in the target sequence is fuzzy processed to obtain a fuzzy sequence, wherein the target compressed data includes a fuzzy identifier, and the fuzzy identifier is used to indicate that the original data corresponding to the target compressed data has been fuzzy processed.
[0050] If the proportion of original data including decimals in the target sequence is too large, fuzzy processing will be performed to ensure the data compression effect; if the proportion is too small, fuzzy processing will not be performed. Although fuzzy processing can ensure reliable data compression effect, the operation itself will bring certain resource overhead and cause loss of data storage efficiency. Compared with the benefits brought by data compression effect, the efficiency loss brought by applying fuzzy processing in this case may be more unacceptable. For example: in the process of storing large quantities of data in the warehouse, if fuzzy processing is performed on each sequence of data, the data storage efficiency will be seriously reduced.
[0051] Based on this, in this embodiment, by calculating the proportion of non-integer original data in the target sequence, and only when the calculated proportion is greater than or equal to a preset ratio threshold, each original data in the target sequence is fuzzy processed separately, so that the storage space reduction effect of the above-mentioned multiplication, rounding and compression scheme can be fully utilized to offset the additional computing overhead caused by the multiplication and rounding operations.
[0052] The ratio of the number of non-integer original data in the target sequence is: the ratio of the number of non-integer original data in the target sequence to the number of all original data in the target sequence;
[0053] For example, if the number of non-integer raw data in the target sequence is 50 and the number of all raw data in the target sequence is 100, then the number of non-integer raw data in the target sequence accounts for 50%.
[0054] In applications, the above-mentioned preset ratio threshold value can be set and adjusted according to actual needs. For example, the preset ratio threshold value can be set to 50%.
[0055] Among them, the setting of the fuzzy mark is to distinguish the compressed data corresponding to the blurred original data and the compressed data corresponding to the unfuzzified original data when the compressed data corresponding to the blurred original data and the unfuzzified original data are stored in the same data storage structure, so that when the data is decompressed, the compressed data carrying the fuzzy mark is decompressed and then the corresponding division processing is performed to restore the real original data; when the compressed data does not carry the fuzzy mark, the above-mentioned division processing will not be performed.
[0056] In one embodiment, after comparing the ratio of the number of non-integer original data in the target sequence with a preset ratio threshold and obtaining the comparison result, the method further includes:
[0057] When the comparison result indicates that the quantity proportion is less than the preset ratio threshold, data compression is performed on the target sequence to obtain standard compressed data, wherein the standard compressed data includes a non-ambiguous identifier, and the non-ambiguous identifier is used to indicate that the original data corresponding to the standard compressed data has not been fuzzy processed.
[0058] In this embodiment, the data structures of the target data and the standard compressed data are unified by setting a non-ambiguous identifier corresponding to the fuzzy identifier.
[0059] Exemplarily, the fuzzy flag may be "1" and the non-fuzzy flag may be "0".
[0060] In one example, a one-bit leading mark bit may be added to the header of the compressed data to store the aforementioned ambiguous identifier and the non-ambiguous identifier.
[0061] In one embodiment, before performing fuzzy processing on each original data in the target sequence to obtain the fuzzy sequence, the method further includes:
[0062] In the target sequence, at least one first original data is determined, where the first original data is original data including decimal places in the target sequence;
[0063] Among at least one first original data, determining the first original data including the longest decimal place as the second original data;
[0064] When the length of decimal places included in the second original data is greater than or equal to the preset coefficient threshold, assigning a value to N according to the preset coefficient threshold;
[0065] When the length of the decimal places included in the second original data is less than the preset coefficient threshold, assigning a value to N according to the length of the decimal places included in the second original data;
[0066] The target compression data includes a multiple identifier, and the multiple identifier is used to represent the N value.
[0067] In this embodiment, based on the length of the longest decimal place corresponding to the original data included in the target sequence, a value N is assigned, so as to dynamically select an appropriate multiplication size based on the decimal place lengths in different target sequences, thereby avoiding the adverse effects of excessive or insufficient multiplication factors on subsequent compression processing.
[0068] Exemplarily, if the preset coefficient threshold is set to 7, and at least one first original data in the target sequence is 0.1 (decimal length is 1), 0.12 (decimal length is 2), and 0.333 (decimal length is 3), then 0.333 can be understood as the second original data, and the N value is 3 at this time.
[0069] Among them, the setting of the multiple flag is to cooperate with the above-mentioned dynamic selection of the multiplication size setting, and to indicate the reverse fuzzy processing (that is, the multiplication processing) during the decompression process to ensure that the decompression obtains the real original data.
[0070] In one example, a P-bit multiple mark bit can be added to the header of the compressed data to store the aforementioned multiple identifier, where the P value is the number of bits occupied by the binary representation of the preset coefficient threshold. For example, when the preset coefficient threshold is 7, the P value is 3; when the preset coefficient threshold is 3, the P value is 2.
[0071] For ease of understanding, the following examples are provided:
[0072] like Figure 2 As shown, the target sequence is a time series data sequence (i.e. Figure 2 The data stream (Data Stream) in ) includes three sequence elements, each of which includes a time attribute and a value attribute. Among them, the time attribute of the first sequence element is: June 1 2024 10:01:02, and the value attribute is 0.1 (which can be understood as the aforementioned original data); the time attribute of the second sequence element is: June 1 2024 10:02:02, and the value attribute is 0.2 (which can be understood as the aforementioned original data); the time attribute of the first sequence element is: June 12024 10:03:02, and the value attribute is 0.3 (which can be understood as the aforementioned original data).
[0073] As mentioned above, according to the target sequence, the N value is 1, and the numerical attributes of the three sequence elements are multiplied by 10^1 (which can be understood as the aforementioned fuzzy coefficient), and the new numerical attributes are 1, 2, and 3 (which can be understood as the aforementioned fuzzy data). The numerical part is compressed by XOR operation, and the time attribute is compressed by delta-delta operation to obtain the target compressed data (Compressed data), wherein the first element after the header can be understood as a fuzzy identifier '1' (occupying 1 bit of storage space), the second element is used to represent the multiple identifier 1 (occupying 3 bits of storage space), the third element is used to represent the compression result of the time attribute of the first sequence element (occupying 16 bits of storage space), the fourth element is used to represent the compression result of the numerical attribute of the first sequence element after multiplication (occupying 9 bits of storage space), the fifth element is used to represent the compression result of the time attribute of the second sequence element (occupying 16 bits of storage space), the sixth element is used to represent the compression result of the numerical attribute of the second sequence element after multiplication (occupying 24 bits of storage space), the seventh element is used to represent the compression result of the time attribute of the third sequence element (occupying 1 bit of storage space), and the eighth element is used to represent the compression result of the numerical attribute of the third sequence element after multiplication (occupying 14 bits of storage space).
[0074] For the case where the sequence elements in the sequence to be compressed include numerical attributes with decimal places, a relatively stable data compression effect can be obtained by uniformly multiplying the numerical attributes and then compressing them. This can also significantly improve the compression ratio before and after data compression (i.e., the ratio of the storage space occupied by the compressed data to the storage space occupied by the data before compression), thereby greatly reducing the data storage space occupied.
[0075] Exemplarily, when the target sequence is a time series data sequence, the data compression algorithm used may be the Gorilla algorithm.
[0076] In some implementations, when there are non-integer original data in the target sequence, each original data in the target sequence may be fuzzy processed to obtain a fuzzy sequence, and the fuzzy sequence may be compressed to obtain target compressed data;
[0077] and compressing the target sequence to obtain standard compressed data;
[0078] By comparing the storage space occupied by the target compressed data and the storage space occupied by the standard compressed data, the compressed data occupying the smaller storage space is selected for storage.
[0079] The present disclosure provides a data decompression method. Figure 3 As shown, the data decompression method includes:
[0080] Step 301: When the target compressed data corresponds to original data including decimal places, decompress the target compressed data to obtain a fuzzy sequence.
[0081] The target compressed data is data formed after the sequence is subjected to data compression processing, and the data decompression operation is a reverse operation of the data compression operation used for the sequence corresponding to the target compressed data.
[0082] Step 302: Perform inverse fuzzy processing on each fuzzy data in the fuzzy sequence to obtain a first sequence.
[0083] Among them, the first sequence includes decompressed data corresponding one-to-one to the fuzzy data in the fuzzy sequence, and the decompressed data is the ratio of the corresponding fuzzy data to the fuzzy coefficient, and the fuzzy coefficient is 10 to the power of N, where N is a positive integer less than or equal to a preset coefficient threshold.
[0084] In the present application, when the target compressed data corresponds to original data including decimal places, after the target compressed data is decompressed, the fuzzy data obtained by decompression is subjected to reverse fuzzy processing, that is, multiplication processing, to restore the original data and obtain the true original data. Among them, the data compression method of the target compressed data can avoid the fluctuation problem of the data compression effect of the original data in the case of non-integer, so that the final target compressed data can maintain a better data compression effect.
[0085] In one example, if it is set to distinguish between compressed data that has been subjected to blur processing and then compressed and compressed data that has been directly compressed without blur processing, and different storage structures are used to store the two separately, then when the target compressed data is compressed data taken out from the storage structure corresponding to the compressed data that has been subjected to blur processing and then compressed, it can be considered that the target compressed data corresponds to non-integer original data.
[0086] In one embodiment, when the target compressed data corresponds to non-integer original data, decompressing the target compressed data to obtain the fuzzy sequence includes:
[0087] In the case where the target compressed data includes a fuzzy identifier, decompressing the target compressed data to obtain a fuzzy sequence, wherein the fuzzy identifier is used to indicate that the original data corresponding to the compressed data has been fuzzy processed;
[0088] The method further comprises:
[0089] In the case where the target compressed data includes a non-ambiguous identifier, the target compressed data is decompressed to obtain a second sequence, wherein the non-ambiguous identifier is used to indicate that the original data corresponding to the target compressed data has not been fuzzy processed.
[0090] In this embodiment, by setting fuzzy identifiers and non-fuzzy identifiers, it is convenient to distinguish between compressed data that has been fuzzy processed and then compressed and compressed data that has not been fuzzy processed and then compressed, so that the compressed data that has been fuzzy processed and then compressed and the compressed data that has not been fuzzy processed and then compressed can be stored in the same data structure, which can further facilitate the storage of compressed data.
[0091] In one embodiment, before decompressing the target compressed data to obtain the fuzzy sequence, the method includes:
[0092] Parsing the multiple identifier included in the target compressed data to determine the N value;
[0093] The fuzzy coefficient is calculated according to the determined N value.
[0094] The multiple identifier is determined according to the decimal length of the original data with the longest decimal place among the multiple original data corresponding to the target compressed data, and the multiple identifier is less than or equal to a preset coefficient threshold.
[0095] In this embodiment, the multiplication multiple of the target compressed data is dynamically determined based on the length of the decimal places of the original data including the longest decimal place in the corresponding original data. By setting the multiplication multiple identifier, the multiplication multiple corresponding to each target compressed data is recorded to ensure accurate decompression and restoration of the original data on the basis of supporting the above-mentioned dynamic determination of the multiplication multiple scheme.
[0096] See also Figure 4 , Figure 4 is a data compression device provided by an embodiment of the present disclosure, such as Figure 4 As shown, the data compression device 400 includes:
[0097] A fuzzy processing module 401 is used to perform fuzzy processing on each original data in the target sequence to obtain a fuzzy sequence when there is non-integer original data in the target sequence, wherein the non-integer original data is original data including decimal places in the target sequence, the fuzzy sequence includes fuzzy data corresponding to the original data in the target sequence one by one, the fuzzy data is the integer value of the product of the corresponding original data and the fuzzy coefficient, the fuzzy coefficient is 10 to the Nth power, and N is a positive integer less than or equal to a preset coefficient threshold;
[0098] The data compression module 402 is used to perform data compression on the fuzzy sequence to obtain target compressed data.
[0099] In one embodiment, the fuzzy processing module 401 includes:
[0100] A comparison unit, used to compare the ratio of the number of non-integer original data in the target sequence with a preset ratio threshold to obtain a comparison result;
[0101] A fuzzy processing unit is used to perform fuzzy processing on each original data in the target sequence to obtain a fuzzy sequence when the comparison result indicates that the quantity proportion is greater than or equal to a preset ratio threshold, wherein the target compressed data includes a fuzzy identifier, and the fuzzy identifier is used to indicate that the original data corresponding to the target compressed data has been fuzzy processed.
[0102] In one embodiment, the data compression module 402 includes:
[0103] A fuzzy compression unit, used for compressing the fuzzy sequence to obtain target compressed data;
[0104] A standard compression unit is used to compress the target sequence to obtain standard compressed data when the comparison result indicates that the quantity proportion is less than the preset ratio threshold, wherein the standard compressed data includes a non-ambiguous identifier, and the non-ambiguous identifier is used to indicate that the original data corresponding to the standard compressed data has not been fuzzy processed.
[0105] In one embodiment, the data compression device 400 further includes a dynamic value assignment module, and the dynamic value assignment module is used to:
[0106] In the target sequence, at least one first original data is determined, where the first original data is original data including decimal places in the target sequence;
[0107] Among at least one first original data, determining the first original data including the longest decimal place as the second original data;
[0108] When the length of decimal places included in the second original data is greater than or equal to the preset coefficient threshold, assigning a value to N according to the preset coefficient threshold;
[0109] When the length of the decimal places included in the second original data is less than the preset coefficient threshold, assigning a value to N according to the length of the decimal places included in the second original data;
[0110] The target compression data includes a multiple identifier, and the multiple identifier is used to represent the N value.
[0111] The data compression device 400 provided in the embodiment of the present disclosure can implement each process in the above-mentioned data compression method embodiment, and will not be described again here to avoid repetition.
[0112] See also Figure 5 , Figure 5 A data decompression device is provided in an embodiment of the present disclosure. Figure 5 As shown, the data decompression device 500 includes:
[0113] A data decompression module 501 is used to decompress the target compressed data to obtain a fuzzy sequence;
[0114] The reverse fuzzification module 502 is used to perform reverse fuzzification processing on each fuzzy data in the fuzzy sequence when the target compressed data corresponds to the original data including decimal places, so as to obtain a first sequence, wherein the first sequence includes decompressed data corresponding to the fuzzy data in the fuzzy sequence one by one, and the decompressed data is the ratio of the corresponding fuzzy data to the fuzzy coefficient, and the fuzzy coefficient is 10 to the power of N, where N is a positive integer less than or equal to a preset coefficient threshold.
[0115] In one embodiment, the data decompression module 501 includes:
[0116] A first decompression unit is used to decompress the target compressed data to obtain a fuzzy sequence when the target compressed data includes a fuzzy identifier, wherein the fuzzy identifier is used to indicate that the original data corresponding to the compressed data has been fuzzy processed;
[0117] The second decompression unit is used to decompress the target compressed data to obtain a second sequence when the target compressed data includes a non-ambiguous identifier, wherein the non-ambiguous identifier is used to indicate that the original data corresponding to the target compressed data has not been fuzzy processed.
[0118] In one embodiment, the data decompression device 500 further includes a coefficient parsing module, and the coefficient parsing module is used to:
[0119] Parsing the multiple identifier included in the target compressed data to determine the N value;
[0120] The fuzzy coefficient is calculated according to the determined N value.
[0121] The data decompression device 500 provided in the embodiment of the present disclosure can implement each process in the above-mentioned data decompression method embodiment, and will not be described again here to avoid repetition.
[0122] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device and a readable storage medium.
[0123] Figure 6 A schematic block diagram of an example electronic device 600 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.
[0124] like Figure 6As shown, the device 600 includes a computing unit 601, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 602 or a computer program loaded from a storage unit 608 to a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the device 600 can also be stored. The computing unit 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.
[0125] A number of components in the device 600 are connected to the I / O interface 605, including: an input unit 606, such as a keyboard, a mouse, etc.; an output unit 607, such as various types of displays, speakers, etc.; a storage unit 608, such as a disk, an optical disk, etc.; and a communication unit 609, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 609 allows the device 600 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.
[0126] The computing unit 601 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSP), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 601 performs the various methods and processes described above, such as data compression / data decompression methods. For example, in some embodiments, the data compression / data decompression method may be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 608. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 600 via the ROM 602 and / or the communication unit 609. When the computer program is loaded into the RAM 603 and executed by the computing unit 601, one or more steps of the data compression / data decompression method described above may be performed. Alternatively, in other embodiments, the computing unit 601 may be configured to perform the data compression / data decompression method in any other appropriate manner (eg, by means of firmware).
[0127] Various embodiments of the systems and techniques described above herein may be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.
[0128] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.
[0129] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0130] As used herein, the term "machine-readable medium" refers to any computer program product, apparatus, and / or device (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.
[0131] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).
[0132] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.
[0133] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.
[0134] The present application also provides a computer program product, including computer instructions, which, when executed by a processor, implement the above Figure 1 or Figure 3The various processes of the method embodiment shown can achieve the same technical effect, and will not be described again here to avoid repetition.
[0135] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.
[0136] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.
Claims
1. A data compression method, characterized in that: The method comprises: In the case where there are non-integer original data in the target sequence, each original data in the target sequence is fuzzy processed to obtain a fuzzy sequence, wherein the non-integer original data is original data including decimal places in the target sequence, the fuzzy sequence includes fuzzy data corresponding to the original data in the target sequence one by one, the fuzzy data is the integer value of the product of the corresponding original data and the fuzzy coefficient, the fuzzy coefficient is 10 to the Nth power, and N is a positive integer less than or equal to a preset coefficient threshold; The fuzzy sequence is compressed to obtain target compressed data.
2. The method according to claim 1, characterized in that In the case that there are non-integer original data in the target sequence, each original data in the target sequence is fuzzy processed to obtain a fuzzy sequence, including: Comparing the ratio of the number of non-integer original data in the target sequence with a preset ratio threshold to obtain a comparison result; When the comparison result indicates that the quantity proportion is greater than or equal to a preset ratio threshold, each original data in the target sequence is fuzzy processed to obtain a fuzzy sequence; wherein the target compressed data includes a fuzzy identifier, and the fuzzy identifier is used to indicate that the original data corresponding to the target compressed data has been fuzzy processed.
3. The method according to claim 2, characterized in that After comparing the ratio of the number of non-integer original data in the target sequence with a preset ratio threshold and obtaining a comparison result, the method further includes: When the comparison result indicates that the quantity proportion is less than the preset ratio threshold, data compression is performed on the target sequence to obtain standard compressed data, wherein the standard compressed data includes a non-ambiguous identifier, and the non-ambiguous identifier is used to indicate that the original data corresponding to the standard compressed data has not been fuzzy processed.
4. The method according to any one of claims 1 to 3, characterized in that Before performing fuzzy processing on each original data in the target sequence to obtain the fuzzy sequence, the method further includes: In the target sequence, at least one first original data is determined, where the first original data is original data including decimal places in the target sequence; In at least one first original data, determining the first original data having the longest decimal place as the second original data; When the length of decimal places included in the second original data is greater than or equal to the preset coefficient threshold, assigning a value to N according to the preset coefficient threshold; When the length of the decimal places included in the second original data is less than the preset coefficient threshold, assigning a value to N according to the length of the decimal places included in the second original data; The target compression data includes a multiple identifier, and the multiple identifier is used to represent the N value.
5. A data decompression method, characterized in that: The method comprises: Decompress the target compressed data to obtain a fuzzy sequence; In the case where the target compressed data corresponds to original data including decimal places, inverse fuzzy processing is performed on each fuzzy data in the fuzzy sequence to obtain a first sequence, wherein the first sequence includes decompressed data corresponding one-to-one to the fuzzy data in the fuzzy sequence, and the decompressed data is the ratio of the corresponding fuzzy data to the fuzzy coefficient, and the fuzzy coefficient is 10 to the power of N, where N is a positive integer less than or equal to a preset coefficient threshold.
6. The method according to claim 5, characterized in that In the case where the target compressed data corresponds to non-integer original data, decompressing the target compressed data to obtain a fuzzy sequence includes: In the case where the target compressed data includes a fuzzy identifier, decompressing the target compressed data to obtain a fuzzy sequence, wherein the fuzzy identifier is used to indicate that the original data corresponding to the compressed data has been fuzzy processed; The method further comprises: In the case where the target compressed data includes a non-ambiguous identifier, the target compressed data is decompressed to obtain a second sequence, wherein the non-ambiguous identifier is used to indicate that the original data corresponding to the target compressed data has not been fuzzy processed.
7. The method according to any one of claims 5 to 6, characterized in that Before decompressing the target compressed data to obtain the fuzzy sequence, the method includes: Parsing the multiple identifier included in the target compressed data to determine the N value; The fuzzy coefficient is calculated according to the determined N value.
8. A data compression device, characterized in that: The device comprises: A fuzzy processing module, configured to perform fuzzy processing on each original data in the target sequence respectively to obtain a fuzzy sequence when there is non-integer original data in the target sequence, wherein the non-integer original data is original data including decimal places in the target sequence, the fuzzy sequence includes fuzzy data corresponding to the original data in the target sequence one by one, the fuzzy data is the integer value of the product of the corresponding original data and a fuzzy coefficient, the fuzzy coefficient is 10 to the power of N, and N is a positive integer less than or equal to a preset coefficient threshold; The data compression module is used to compress the fuzzy sequence to obtain target compressed data.
9. A data decompression device, characterized in that: The device comprises: A data decompression module is used to decompress the target compressed data to obtain a fuzzy sequence; The inverse fuzzy module is used to perform inverse fuzzy processing on each fuzzy data in the fuzzy sequence when the target compressed data corresponds to the original data including decimal places, so as to obtain a first sequence, wherein the first sequence includes decompressed data corresponding to the fuzzy data in the fuzzy sequence one by one, and the decompressed data is the ratio of the corresponding fuzzy data to the fuzzy coefficient, and the fuzzy coefficient is 10 to the power of N, where N is a positive integer less than or equal to a preset coefficient threshold.
10. An electronic device, characterized in that: The method comprises a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the method as claimed in any one of claims 1 to 4, or implements the steps of the method as claimed in any one of claims 5 to 7.
11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the computer program implements the steps of the method as claimed in any one of claims 1 to 4, or implements the steps of the method as claimed in any one of claims 5 to 7.
