Data processing method and data processing apparatus, electronic device

Abstract

The present application provides a kind of data processing method and data processing device, electronic equipment and non-transient computer readable storage medium, the method includes in response to the decompression instruction of data, the check of received compressed data is executed;In the case where check passes, according to the preset continuous data length, the compressed data is divided into multiple data segments, wherein the compressed data is compressed according to the preset continuous data length;The decompression operation of the multiple data segments in the preset address space is respectively executed, and the decompressed data is obtained;Whether the decompressed data is consistent with the data amount of the original data corresponding to the compressed data is compared;In the case where data amount is consistent, the decompression task of the compressed data is ended.According to embodiment, by dividing compressed data into multiple data segments and performing double check on decompressed data, not only the accuracy of data decoding is improved, but also the speed and efficiency of decompression are improved.

Description

Technical Field

[0001] This invention relates to the field of data transmission, and more specifically, to a data processing method and apparatus, an electronic device, and a non-transitory computer-readable storage medium. Background Technology

[0002] Dynamic data compression and decompression technologies have wide applications in many fields, including image processing, audio processing, video processing, and power systems. These technologies not only improve the efficiency of data storage and transmission but also drive technological advancements in related fields.

[0003] The advantage of dynamic data compression lies in its ability to dynamically select the optimal compression strategy based on the characteristics of the data and changes in the compression progress, thus typically providing better compression results with less storage space. However, to perform real-time analysis and decision-making during the compression process, dynamic data compression and decompression require more computing resources.

[0004] Run-length encoding is characterized by its simplicity, ease of implementation, and high compression and decompression efficiency. Furthermore, as a lossless compression algorithm, it does not lose any information from the original data during compression. However, run-length encoding also has some limitations. For example, its compression performance may not be ideal when the data contains a large amount of non-repeating data. Additionally, run-length encoding may not be effective in certain situations, such as when the length of repeating data blocks exceeds one byte.

[0005] Run-length encoding compression algorithms compress a series of data into {consecutive data values ​​[7:0], data [7:0]}, where the consecutive data values ​​[7:0] can be any value from 1 to 255. When decompressing the run-length encoded data, the 64 bytes of compressed data are divided into 32 groups (16 bits per group). Since the consecutive data values ​​[7:0] are random numbers within the range of 0 to 255, and there is a strict order between each group of data, if the consecutive value of the preceding group is 100 and the consecutive value of the following group is 50, the final output will only be completed after the first group is decompressed. Traditional compression algorithms have uncontrollable hardware decompression time and cannot efficiently perform parallel hardware decompression. Summary of the Invention

[0006] The present invention aims to provide a data processing method, a data processing device, an electronic device, and a non-transitory computer-readable storage medium to solve the problem of high hardware resource consumption during data compression and decompression.

[0007] According to one aspect of the present invention, a data processing method is proposed, comprising: In response to the data decompression command, perform verification on the received compressed data; If the verification passes, the compressed data is divided into multiple data segments according to a preset continuous data length, wherein the compressed data is obtained by compressing according to the preset continuous data length; Decompression operations are performed on the multiple data segments in a preset address space to obtain decompressed data; Compare whether the decompressed data has the same data volume as the original data corresponding to the compressed data; If the amount of data is consistent, the decompression task of the compressed data is terminated.

[0008] According to some embodiments, the preset continuous data value is no greater than 15, and the corresponding continuous occupied space is no greater than 12 bits.

[0009] According to some embodiments, before performing verification on the received compressed data in response to a data decompression command, the method further includes: The original data is compressed to obtain compressed data; Calculate the verification data corresponding to the compressed data.

[0010] According to some embodiments, the verification data corresponding to the compressed data is calculated, including: The compressed data is divided into multiple groups; Calculate the parity value within each group, the location fingerprint of the compressed data, and the global parity value for each group. Using the intra-group parity value, the location fingerprint, and the global parity value, the verification data corresponding to the compressed data is calculated.

[0011] According to some embodiments, before performing a compression operation on the original data to obtain compressed data, the method further includes: The amount of data in the original data is calculated.

[0012] According to some embodiments, the compressed data is divided into multiple data segments based on a preset continuous data length, including: Based on the preset continuous data length, the compressed data is divided into data segments of data length / continuous occupied space to ensure real-time decompression of the compressed data.

[0013] According to one aspect of the present invention, a data processing apparatus is provided, comprising: The verification unit is used to perform verification on the received compressed data in response to the data decompression command; The data segmentation unit is used to divide the compressed data into multiple data segments according to a preset continuous data length, provided that the verification passes. The compressed data is obtained by compressing the data according to the preset continuous data length. The decompression unit is used to perform decompression operations on the multiple data segments in a preset address space to obtain decompressed data; The comparison unit is used to compare whether the decompressed data is consistent with the data volume of the original data corresponding to the compressed data; The decompression task termination unit is used to terminate the decompression task of the compressed data when the data volume is consistent.

[0014] According to one aspect of the present invention, an electronic device is provided, comprising: a processor; and a memory storing a computer program that, when executed by the processor, causes the processor to perform a data processing method as described in any of the preceding embodiments.

[0015] According to one aspect of the present invention, a non-transitory computer-readable storage medium is provided, on which computer-readable instructions are stored, which, when executed by a processor, cause the processor to perform a data processing method as described in any of the preceding embodiments.

[0016] According to embodiments of the present invention, by dividing the compressed data into multiple data segments, the problem of long waiting times between decompression units due to excessively long consecutive data values ​​is solved, thereby improving the speed and efficiency of decompression. Simultaneously, by performing dual verification on the decompressed data, the accuracy of data decoding is improved.

[0017] It should be understood that the above general description and the following detailed description are merely exemplary and do not limit the invention. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. The above and other objectives, features, and advantages of the present invention will become more apparent from the detailed description of exemplary embodiments with reference to the accompanying drawings.

[0019] Figure 1 A flowchart of a data processing method according to an exemplary embodiment of the present invention is shown.

[0020] Figure 2 A flowchart illustrating a method for adding verification to compressed data according to an exemplary embodiment of the present invention is shown.

[0021] Figure 3 A flowchart illustrating a method for performing segmented decompression on run-length encoded data according to an exemplary embodiment of the present invention is shown.

[0022] Figure 4 A flowchart illustrating a dynamic compression and decompression hardware collaborative workflow according to an exemplary embodiment of the present invention is shown.

[0023] Figure 5a A schematic diagram of a hardware implementation of a segmented decompression operation according to the present invention is shown.

[0024] Figure 5b A schematic diagram of the structure of an rld_grp submodule according to an example embodiment of the present invention is shown.

[0025] Figure 5c A schematic diagram of a decoding architecture corresponding to run-length encoding according to an exemplary embodiment of the present invention is shown.

[0026] Figure 6 A block diagram of a data processing apparatus according to an exemplary embodiment of the present invention is shown.

[0027] Figure 7 An electronic device according to an exemplary embodiment of the present invention is shown. Detailed Implementation

[0028] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the embodiments set forth herein; rather, they are provided so that the invention will be thorough and complete, and the concept of the exemplary embodiments will be fully conveyed to those skilled in the art. The same drawings in the figures show the same or similar parts, and therefore repeated descriptions of them will be omitted.

[0029] The described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a full understanding of embodiments of this disclosure. However, those skilled in the art will recognize that the technical solutions of this disclosure can be practiced without one or more of these specific details, or other methods, components, materials, apparatus, or operations may be employed. In these cases, well-known structures, methods, apparatuses, implementations, materials, or operations will not be shown or described in detail.

[0030] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0031] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this invention are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0032] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0033] Figure 1 A flowchart of a data processing method according to an exemplary embodiment of the present invention is shown, such as... Figure 1 The data processing method shown includes steps 101, 103, 105, 101, and 109. The following will use... Figure 1 Taking an example, a data processing method according to an exemplary embodiment of the present invention will be described in detail. Figure 1 The method shown is applied to run-length encoding.

[0034] In step S101, in response to the data decompression command, the received compressed data is verified.

[0035] Traditional hardware compression and decompression module collaborative workflows lack effective mechanisms for handling data transmission errors, data errors, and fault tolerance, which may lead to data loss or system crashes. According to an embodiment of the present invention, before step S101, Figure 1 The data processing method shown further includes: performing a compression operation on the original data to obtain compressed data, and calculating the verification data corresponding to the compressed data.

[0036] In some embodiments, when calculating the verification data corresponding to the compressed data, firstly, the original data is divided into multiple groups according to the preset continuous space occupied by the continuous data; then, the verification data corresponding to each group is calculated respectively.

[0037] According to an embodiment of the present invention, when the verification is successful, step S103 is executed; when an error occurs during verification, a decompression interruption command is reported.

[0038] Run-length encoding compression algorithms compress a series of data into {continuous data values ​​[7:0], data [7:0]}, which is equivalent to {8 bits long, 8 bits of original data}. The continuous data values ​​[7:0] can be any value from 1 to 255. When decompressing run-length encoded compressed data, the 64 bytes of compressed data are divided into 32 groups (16 bits per group) for decompression. Since the continuous data values ​​[7:0] are random numbers within the range of 0 to 255, and there is a strict order relationship between each group of data, if the continuous value of the preceding group is 100 and the continuous value of the following group is 50, the final output result will not be completed until the first group is decompressed. Traditional compression algorithms have uncontrollable hardware decompression time and cannot efficiently perform parallel hardware decompression.

[0039] According to embodiments of the present invention, the consecutive data values ​​are limited to no more than 15, ensuring that the length of consecutive data is no more than 4 bits. This reduces the time required for decompression, improving the efficiency and performance of hardware decompression and limiting the hardware resources occupied by consecutive data values. For example, the size of the consecutive data values ​​can be 15, 8, or 4.

[0040] When the consecutive data value is 15, a series of data is compressed into {4 bits long, 8 bits of original data}. The maximum length of 4 bits is 15. This makes the decompression time of each consecutive set controllable, avoiding the problem of long waiting times between decompression units due to excessively long consecutive data values, thus improving decompression efficiency. Because the consecutive data value is limited to 15, the compressed data changes from the original 16 bits {consecutive data values ​​[7:0], data [7:0]} to 12 bits {consecutive data values ​​[3:0], data [7:0]}. The compressed data is then aligned to 512 bits. Originally, 512 bits contained 32 compressed data (32 x 16), but after the improvement, 512 bits contain 42 compressed data (42 x 12), reducing the compressed data volume by approximately 15% compared to the original RLE compression. Assuming the uncompressed data source is discontinuous, the traditional run-length encoding algorithm doubles the data size after compression, while the optimized algorithm only increases the data size by 0.5 times, reducing the negative effect of increased data size caused by the discontinuity of uncompressed data. Furthermore, the original 512 bits contained 32 compressed data points, while the improved algorithm contains 42 compressed data points, reducing the compressed data size by approximately 15% compared to the original run-length encoding compression. In fact, the 42 compressed data points only occupy 504 bits, with the remaining 8 bits padded with 0s, resulting in a loss of approximately 1% compression ratio.

[0041] In step S103, if the verification passes, the compressed data is divided into multiple data segments according to a preset continuous data length, wherein the compressed data is obtained by compressing according to the preset continuous data length.

[0042] According to an embodiment of the present invention, by limiting the length of continuous data values, the continuous length of compressed data is limited at the algorithm level. This can minimize the problem of low efficiency caused by the mutual dependence of decompression data order. Therefore, in step S103, the compressed data is divided into multiple data segments according to the preset continuous data length to solve the problem of long waiting time between decompression units due to excessively long continuous data values, thereby improving the decompression speed and efficiency.

[0043] To ensure real-time decompression of compressed data, according to an embodiment of the present invention, the compressed data is divided into data segments of data length / continuous occupied space according to a preset continuous data length, so as to ensure real-time decompression of the compressed data, maximize the efficiency of the compression algorithm and the performance of hardware decompression, and effectively shorten the evaluation cycle of the compression algorithm.

[0044] In other embodiments, the number of data segments can be configured, and the specific value can be determined based on hardware resources and decompression efficiency.

[0045] In step S105, decompression operations are performed on the multiple data segments in a preset address space to obtain decompressed data.

[0046] Traditional dynamic decompression hardware requires configuring the starting address of compressed data storage before it can operate. Since configuring the storage address introduces additional latency, this latency can lead to performance degradation or failure to meet real-time requirements in applications requiring real-time processing. According to an embodiment of the present invention, in step S105, decompression operations are performed on multiple data segments separately within a preset address space, thereby eliminating the need for configuring the storage address and meeting real-time requirements.

[0047] In a specific embodiment, by fixing the operation space corresponding to compression and decompression operations respectively, the process of configuring the storage addresses corresponding to compression and decompression operations is reduced, thereby meeting the real-time data transmission requirements of application scenarios that require real-time processing.

[0048] For example, by fixing the address space for storing compressed data, during compression, the first two records in the fixed storage space store the amount of compressed data and the amount of decompressed data, which are used to automatically configure direct memory access during the decompression process; when the decompression operation retrieves data from the fixed storage space, it can also directly obtain the amount of compressed data to be read and the amount of decompressed data for automatic verification.

[0049] In step S107, the data volume of the decompressed data is compared with that of the original data corresponding to the compressed data.

[0050] To improve the accuracy of data compression and identify whether there are erroneous data in the decompressed data, according to an embodiment of the present invention, before step S103, it is also necessary to count the amount of original data. The original data is the uncompressed data corresponding to the compressed data obtained in step S101.

[0051] In step S109, if the data volume is consistent, the decompression task of the compressed data is terminated.

[0052] According to an embodiment of the present invention, if the amount of data in the decompressed data is the same as the amount of data in the original data corresponding to the compressed data, a decompression completion command needs to be reported; if the amount of data in the decompressed data is different from the amount of data in the original data corresponding to the compressed data, a decompression interruption command needs to be reported.

[0053] according to Figure 1 The illustrated embodiment addresses the issue of long waiting times between decompression units due to excessively long consecutive data values ​​by dividing the compressed data into multiple data segments, thereby improving decompression speed and efficiency. Simultaneously, performing dual verification on the decompressed data improves the accuracy of data decoding.

[0054] Figure 2 A flowchart illustrating a method for adding verification to compressed data according to an exemplary embodiment of the present invention is shown, such as... Figure 2 The method shown includes steps S201, S203 and S205.

[0055] like Figure 2 As shown, in step S201, the compressed data is divided into multiple groups; In step S203, the parity value within each group, the location fingerprint of the compressed data, and the global parity value are calculated for each group. In step S205, the parity value within the group, the location fingerprint, and the global parity value are used to calculate the verification data corresponding to the compressed data.

[0056] Assume the compressed data group is N*512 bits, which includes N compressed data packets, each of which is 512 bits. Each compressed data packet contains multiple compressed data, and each compressed data is 12 bits long. First, the 12 bits of compressed data are divided into 4 parts, each of which is 4 bits, and the parity value within each part is calculated. For example, an XOR operation is performed on each group of 4 bits to obtain a 3-bit group parity value (p0, p1, p2). Then, the global parity value is calculated, that is, the parity values ​​within the 3 groups are XORed again to obtain a 1-bit global parity value (p). Then, the highest bit of each group is extracted and XORed to generate 3-bit position information (s), as shown in formula (1).

[0057] s[3:0] = { (v[3]⊕v[7]⊕v

[11] ), (v[2]⊕v[6]⊕v

[10] ), (v[1]⊕v[5]⊕v[9]), (v[0]⊕v[4]⊕v[8])} (1) Where v[i] is the value of each element in each group.

[0058] Next, the verification data is calculated. Specifically, the parity values ​​within the group, the location fingerprint, and the global parity values ​​calculated as before are concatenated to obtain an 8-bit final verification value with the structure [p(1), s(3), p0(1), p1(1) p2(1)] (a total of 8 bits).

[0059] For example, assuming the grouped data is 0b1101_1010_1111 (decimal 1887, 12 bits), first calculate the parity value within the group; Group 0 (bits 3~0): v[3:0] = 1111 → p0 = 1⊕1⊕1⊕1 = 0 Group 1 (bits 7-4): v[7:4] = 1010 → p1 = 1⊕0⊕1⊕0 = 0 Group 2 (bits 11~8): v[11:8] = 1101 → p2 = 1⊕1⊕0⊕1 = 1 Next, calculate the global parity value; p = p0 ⊕ p1 ⊕ p2 = 0 ⊕ 0 ⊕ 1 = 1 Next, the location fingerprint is calculated; s = 0011 Finally, perform the verification merge operation.

[0060] Final checksum: c = {1, 0011, 0, 0, 1} → binary concatenation is 1 001 0 0 1 = 0b1001001 (decimal 73).

[0061] It should be noted that this invention adds 8 bits of mixed parity and position checksum to the compressed data volume (number of data) after the data compression process is completed. After receiving the compressed data, the mixed parity and position checksum of the data volume are calculated. If the data volume and checksum are equal, it is normal. If either is not equal, an interruption is reported and the data is retransmitted.

[0062] Figure 3 A flowchart illustrating a method for performing segmented decompression on run-length encoded data according to an exemplary embodiment of the present invention is shown, such as... Figure 3As shown, the compressed data is first divided into multiple segments. Then, run-length encoding is performed on each segment for decompression, and the length of each segment is determined. If the length of the data is not equal to zero, the segmented data is decompressed to obtain the corresponding continuous data, and all the data of each decompressed segment is concatenated together in turn; otherwise, the current decompressed data is invalid, and data interruption reporting is performed.

[0063] This embodiment limits the continuous length of compressed data, thereby avoiding the problem of low efficiency caused by the interdependence of decompression data order, and improves the speed and efficiency of decompression by adopting a parallel decompression method.

[0064] Figure 4 A flowchart illustrating a dynamic compression and decompression hardware collaborative workflow according to an exemplary embodiment of the present invention is shown, such as... Figure 3 As shown, when compressing data, firstly, the original data to be compressed is stored in the compression model; then, the amount of input data is counted; after that, the input data is compressed, and 1 byte of verification data as described above is added to each compressed part of the data, and the amount of compressed data is counted, and the data compression is completed.

[0065] During decompression, the decompressed data is first obtained from the storage space corresponding to the decompression module, and the data check value is calculated. If the check matches, the data is decompressed; otherwise, an interrupt is reported. Then, the decompressed data volume is checked to see if it matches the original data volume. If the match fails, an interrupt is reported; if the match succeeds, the decompressed data is decompressed.

[0066] Figure 5a A schematic diagram of a hardware implementation of a segmented decompression operation according to the present invention is shown, as follows: Figure 5a The decompression module RLD shown consists of four main submodules: rld_dmux, rld_grp, rld_mux, and rld_combo. RLD has two operating modes: decompression mode and bypass mode.

[0067] To control RLD decompression latency (to address the issue of sequential waiting of decompressed data and balance decompression time), during the RLE compression stage, the number of consecutive values ​​is limited to 15, and the compressed data is 12 bits (count_len[3:0], data[7:0]). This algorithm limits the continuous length of the compressed data, thereby minimizing the problem of inefficiency caused by the interdependence of decompressed data order and improving the speed and efficiency of decompression.

[0068] The `rld_dmux` submodule receives the input compressed data and communicates with each `rld_grp`. The RLD contains N `rld_grp`s, and `rld_demux` interacts with them in the order of `rld_grp` 0 to `N-1`. The value of N is configurable, depending on hardware resources and decompression efficiency. If the input data length is 512 bits and contains 42 compressed data points, real-time decompression can be guaranteed when N=42.

[0069] The rld_grp submodule includes N parallel paths, receives N DMA (Direct Memory Access) 512-bit transmissions and decompresses them in parallel. Based on the flag bits [511:504], it performs mixed parsing, decompression, bypassing and concatenation operations on each data.

[0070] The rld_mux submodule receives decompressed data from rld_grp 0~N and interacts with rle_combo in sequence.

[0071] The rld_combo submodule concatenates the decompressed data output by rld_grp 0~N into 64 bytes (dn_data[511:0]) in sequence, generating the dn_valid signal.

[0072] Figure 5b The diagram illustrates the structure of an rld_grp submodule according to an exemplary embodiment of the present invention. The decompression module RLD contains N rld_grp submodules, which can receive N DMA 512-bit transmissions and perform functions such as mixed parsing, decompression, bypassing, and splicing of data.

[0073] like Figure 5b As shown, the rld_grp submodule mainly consists of a state machine and a FIFO. The state machine is responsible for parsing the data according to the input data format. It can parse one compressed data segment (12 bits {count_len[3:0], data[7:0]}) per clock cycle and concatenate the parsed data into a 512-bit array. The FIFO is responsible for buffering the data parsed by the state machine, buffering it once each time a 512-bit array is formed. However, if the final decompressed data is less than 512 bits, the data is also written into the FIFO, and the rld_grp_last_loc signal is used to record the number of valid data segments.

[0074] Figure 5c A schematic diagram of a decoding architecture corresponding to run-length encoding according to an exemplary embodiment of the present invention is shown, such as... Figure 5cAs shown, each clock cycle can parse a 12-bit compressed data circuit ({count_len[3:0], data[7:0]}), which can be regarded as a small lookup table. This architecture can complete the decompression work that would take tens of cycles in traditional architectures in one cycle; the output result conforms to the run-length encoding algorithm. The circuit area is more compact, significantly reducing design complexity and saving chip area cost.

[0075] This embodiment adopts a configurable parallel decompression hardware architecture, in which each segment of data is decompressed independently, making it suitable for different hardware platforms. Since each segment of data is processed in parallel, the decompression latency is controllable, improving decompression efficiency and making it suitable for real-time application scenarios.

[0076] The foregoing mainly describes the embodiments of the present invention from a methodological perspective. Those skilled in the art should readily recognize that, in conjunction with the operations or steps of the various examples described in connection with the embodiments disclosed herein, the present invention can be implemented in hardware or a combination of hardware and computer software. Those skilled in the art can implement the described functions in different ways for each specific operation or method, and such implementations should not be considered beyond the scope of the present invention.

[0077] The following describes embodiments of the apparatus of the present invention. For details not described in the embodiments of the apparatus of the present invention, please refer to the embodiments of the method of the present invention.

[0078] Figure 6 A block diagram of a data processing apparatus according to an exemplary embodiment of the present invention is shown, such as Figure 6 The data processing device shown includes a verification unit 601, a data segmentation unit 603, a decompression unit 605, a comparison unit 607, and a decompression task termination unit 609. The verification unit 601 verifies the received compressed data in response to a decompression command. The data segmentation unit 603, if the verification passes, segments the compressed data into multiple data segments according to a preset continuous data length, wherein the compressed data is obtained by compressing the data according to the preset continuous data length. The decompression unit 605 performs decompression operations on the multiple data segments in a preset address space to obtain decompressed data. The comparison unit 607 compares whether the decompressed data has the same data volume as the original data corresponding to the compressed data. The decompression task termination unit 609 terminates the decompression task if the data volume is consistent.

[0079] Figure 7 An electronic device according to an exemplary embodiment of the present invention is shown below. Figure 7 To describe an electronic device 200 according to this embodiment of the present invention. Figure 7The electronic device 200 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0080] like Figure 7 As shown, the electronic device 200 is presented in the form of a general-purpose computing device. The components of the electronic device 200 may include, but are not limited to: at least one processing unit 210, at least one storage unit 220, a bus 230 connecting different system components (including storage unit 220 and processing unit 210), a display unit 240, etc.

[0081] The storage unit stores program code, which can be executed by the processing unit 210, causing the processing unit 210 to perform the methods described in this specification according to various exemplary embodiments of the present invention. For example, the processing unit 210 can perform, for example... Figure 1 The method shown.

[0082] Storage unit 220 may include readable media in the form of volatile storage units, such as random access memory (RAM) 2201 and / or cache memory 2202, and may further include read-only memory (ROM) 2203.

[0083] Storage unit 220 may also include a program / utility 2204 having a set (at least one) program module 2205, such program module 2205 including but not limited to: operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0084] Bus 230 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0085] Electronic device 200 can also communicate with one or more external devices 300 (e.g., keyboard, pointing device, Bluetooth device, etc.), and with one or more devices that enable a user to interact with electronic device 200, and / or with any device that enables electronic device 200 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 250. Furthermore, electronic device 200 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 260. Network adapter 260 can communicate with other modules of electronic device 200 via bus 230. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 200, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0086] Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. The technical solutions of the embodiments of the present invention can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, portable hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, or network device, etc.) to execute the methods described above according to the embodiments of the present invention.

[0087] Software products may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections with one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0088] Computer-readable storage media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable storage medium may also be any readable medium other than a readable storage medium that can transmit, propagate, or transfer a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0089] Program code for performing the operations of this invention can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0090] The aforementioned computer-readable medium carries one or more programs, which, when executed by a device, cause the computer-readable medium to perform the aforementioned functions.

[0091] Those skilled in the art will understand that the above modules can be distributed in the device as described in the embodiments, or they can be modified accordingly and placed in one or more devices that are unique to this embodiment. The modules in the above embodiments can be combined into one module, or they can be further divided into multiple sub-modules.

[0092] According to an embodiment of the present invention, a computer program is provided, comprising a computer program or instructions, which, when executed by a processor, can perform the methods described above.

[0093] The embodiments of the present invention have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the embodiments above are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, any changes or modifications made by those skilled in the art based on the ideas of the present invention, its specific implementation methods, and its application scope, are all within the scope of protection of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A data processing method, characterized in that, include: In response to the data decompression command, perform verification on the received compressed data; If the verification passes, the compressed data is divided into multiple data segments according to a preset continuous data length, wherein the compressed data is obtained by compressing according to the preset continuous data length; Decompression operations are performed on the multiple data segments in a preset address space to obtain decompressed data; Compare whether the decompressed data has the same data volume as the original data corresponding to the compressed data; If the amount of data is consistent, the decompression task of the compressed data is terminated.

2. The data processing method according to claim 1, characterized in that, The preset continuous data value is no greater than 15, and the corresponding continuous occupied space is no greater than 12 bits.

3. The data processing method according to claim 2, characterized in that, Before performing verification on the received compressed data in response to the data decompression command, the process also includes: The original data is compressed to obtain compressed data; Calculate the verification data corresponding to the compressed data.

4. The data processing method according to claim 3, characterized in that, Calculating the verification data corresponding to the compressed data includes: The compressed data is divided into multiple groups; Calculate the parity value within each group, the location fingerprint of the compressed data, and the global parity value for each group. Using the intra-group parity value, the location fingerprint, and the global parity value, the verification data corresponding to the compressed data is calculated.

5. The data processing method according to claim 4, characterized in that, The location fingerprint is calculated by extracting the most significant bit of each group; Perform an XOR operation on the highest bit of all groups to obtain the location fingerprint.

6. The data processing method according to claim 3, characterized in that, Before performing compression on the original data to obtain compressed data, the process also includes: The amount of data in the original data is calculated.

7. The data processing method according to claim 2, characterized in that, Based on a preset continuous data length, the compressed data is divided into multiple data segments, including: Based on the preset continuous data length, the compressed data is divided into data segments of data length / continuous occupied space to ensure real-time decompression of the compressed data.

8. A data processing apparatus, characterized in that, include: The verification unit is used to perform verification on the received compressed data in response to the data decompression command; The data segmentation unit is used to divide the compressed data into multiple data segments according to a preset continuous data length, provided that the verification passes. The compressed data is obtained by compressing the data according to the preset continuous data length. The decompression unit is used to perform decompression operations on the multiple data segments in a preset address space to obtain decompressed data; The comparison unit is used to compare whether the decompressed data is consistent with the data volume of the original data corresponding to the compressed data; The decompression task termination unit is used to terminate the decompression task of the compressed data when the data volume is consistent.

9. An electronic device, comprising: processor; as well as A memory storing a computer program that, when executed by the processor, causes the processor to perform the data processing method as described in any one of claims 1-7.

10. A non-transitory computer-readable storage medium having stored thereon computer-readable instructions that, when executed by a processor, cause the processor to perform the data processing method as described in any one of claims 1-7.

Images