Data decompression method, controller and storage medium

Abstract

The invention relates to the technical field of data processing, in particular to a data decompression method, a controller and a storage medium, and the method comprises the following steps: after determining that downloaded data are at least two compressed data packets, distributing a decompression cache, the size of the decompression cache being at least the data length of the maximum decompression data obtained by decompressing the compressed data packets; and decompressing the compressed data packets to obtain decompressed data, storing the decompressed data to a decompression cache, reading the decompressed data from the decompression cache, and storing the decompressed data to external storage until the last compressed data packet. According to the method, the decompression cache is used for storing the decompressed data obtained by decompressing the single compressed data packet, the decompressed data is read from the decompression cache and stored in the external storage, and by decompressing the compressed data packet in a subpackage mode, the occupation of a decompression memory can be reduced, the decompressed data is stored in the external storage device in time, and the user experience is improved. And the data decompression efficiency and the data writing efficiency are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to a data decompression method, a controller and a storage medium. Background Art

[0002] When the existing car system updates the software, it usually downloads the entire compressed software package from OTA, stores it in the memory, decompresses the compressed software package at one time, and stores it in a certain address range in the memory. This method requires two large continuous memory cache data, which takes more time and requires a larger memory space for decompression. OTA upgrades with large data volumes will cause users to need more time to decompress the compressed software package after the download is completed, and in the case of poor network speed, there is a greater chance of upgrade timeout or failure, resulting in a poor user experience. Summary of the invention

[0003] In view of this, an embodiment of the present invention provides a data decompression method, a controller and a storage medium, which can reduce the occupancy of decompression memory, store the decompressed data to an external storage device in a timely manner, and improve data decompression efficiency and data writing efficiency.

[0004] To solve the above technical problems, the embodiments of the present invention provide the following technical solutions:

[0005] In a first aspect, an embodiment of the present invention provides a data decompression method, the method comprising:

[0006] After determining that the downloaded data is at least two compressed data packets, allocating a decompression buffer, wherein the size of the decompression buffer is at least the data length of the maximum decompressed data obtained by decompressing the compressed data packets;

[0007] The compressed data packet is decompressed to obtain decompressed data, and the decompressed data is stored in the decompression buffer, the decompressed data is read from the decompression buffer, and the decompressed data is stored in an external storage until the last compressed data packet.

[0008] In some embodiments, decompressing the compressed data packet to obtain decompressed data, and storing the decompressed data in the decompression buffer, includes:

[0009] Decompressing the compressed data packet to obtain decompressed data;

[0010] Determining the remaining storage space of the decompression cache based on the write start address of the decompression cache;

[0011] If the data length of the decompressed data is greater than the capacity of the remaining storage space, resetting the write start address of the decompression cache to the head address of the decompression cache;

[0012] Storing the decompressed data in the decompression buffer starting from the write start address;

[0013] The write start address is updated to the current write start address plus the data length of the decompressed data.

[0014] In some embodiments, after determining that the downloaded data is at least two compressed data packets and before allocating the decompression buffer, the method further includes:

[0015] Receiving a download request, parsing the download request to obtain a first data length, where the first data length is the sum of the data lengths of the at least two compressed data packets;

[0016] The method further comprises:

[0017] If the compressed data packet is the first compressed data packet, parsing the file header of the first compressed data packet to obtain a second data length, where the second data length is the sum of the data lengths of the original data before compression of the at least two compressed data packets;

[0018] Configure target decompression parameters, where the target decompression parameters include the first data length and the second data length.

[0019] In some embodiments, the method further comprises:

[0020] Record current decompression parameters, wherein the current decompression parameters include the sum of input lengths and the sum of output lengths, the input length is the data length of the decompressed compressed data packet, and the output length is the data length of the decompressed data obtained by decompression.

[0021] In some embodiments, the method further comprises:

[0022] If the sum of the input lengths is equal to the first data length, and the sum of the output lengths is equal to the second data length, then it is determined that the decompression is successful;

[0023] If the sum of the input lengths is not equal to the first data length, and / or the sum of the output lengths is not equal to the second data length, it is determined that the decompression has failed.

[0024] In some embodiments, if the compressed data packet is the first compressed data packet, the input length is the data length of the first compressed data packet minus the length of the file header;

[0025] If the compressed data packet is not the first compressed data packet, the input length is the data length of the compressed data packet.

[0026] In some embodiments, the write start address, the current decompression parameters and the target decompression parameters are stored in a decompression structure, and the decompression structure is a data structure.

[0027] In some embodiments, the decompression structure further includes a dictionary, and the storing of the decompressed data to the decompression cache starting from the write start address includes:

[0028] If the first data identical to the decompressed data does not exist in the dictionary, storing the decompressed data in the decompression buffer starting from the write start address;

[0029] If the dictionary contains first data identical to the decompressed data, the storage address of the first data is recorded.

[0030] In a second aspect, an embodiment of the present invention provides a controller, including:

[0031] A processor and a memory communicatively connected to the processor;

[0032] The memory stores computer program instructions executable by the processor, and when the computer program instructions are called by the processor, the processor executes any one of the data decompression methods proposed in the first aspect, or executes the steps in any one of the implementations of any one of the data decompression methods proposed in the first aspect.

[0033] In a third aspect, an embodiment of the present invention provides a computer-readable storage medium, on which is stored computer program instructions executable by a processor, wherein when the computer program instructions are called by the processor, the processor executes any one of the data decompression methods proposed in the first aspect, or executes the steps in any one of the implementations of any one of the data decompression methods proposed in the first aspect.

[0034] Beneficial effects of the embodiments of the present invention: Different from the prior art, the data decompression method provided by the embodiments of the present invention includes: after determining that the downloaded data is at least two compressed data packets, allocating a decompression cache, the size of the decompression cache is at least the data length of the maximum decompressed data obtained by decompressing the compressed data packets; decompressing the compressed data packets to obtain decompressed data, and storing the decompressed data in the decompression cache, reading the decompressed data from the decompression cache, and storing the decompressed data in external storage until the last compressed data packet.

[0035] In the data decompression method provided by the embodiment of the present invention, the decompression cache is used to store the decompressed data obtained by decompressing a single compressed data packet, and the decompressed data is read from the decompression cache and stored in the external storage. By decompressing the compressed data packet in a packetized manner, the occupancy of the decompression memory can be reduced, the decompressed data can be stored in the external storage device in a timely manner, and the data decompression efficiency and data writing efficiency can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art description. Obviously, the drawings described below only illustrate certain embodiments of the present invention and should not be regarded as limiting the scope of protection. For those of ordinary skill in the art, other related drawings can also be obtained based on these drawings without creative work.

[0037] Figure 1 is a schematic diagram of an application scenario of a data decompression method provided by some embodiments of the present invention;

[0038] Figure 2 is a schematic diagram of the structure of a controller provided by some embodiments of the present invention;

[0039] Figure 3 is a flowchart of a data decompression method provided by some embodiments of the present invention;

[0040] Figure 4 yes Figure 3 A schematic diagram of a sub-process of step S200 in the data decompression method shown in the embodiment;

[0041] Figure 5 is a schematic diagram of a structure of a decompression structure in some embodiments of the present invention;

[0042] Figure 6 yes Figure 4 A schematic diagram of a sub-flow chart of step S240 in the data decompression method shown in the embodiment. DETAILED DESCRIPTION

[0043] In order to make the purpose and advantages of the embodiments of the present invention easier to understand, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. The detailed description of the embodiments of the present invention in the drawings below does not limit the scope of protection claimed by the present invention, but only represents the selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0044] It should be noted that, if no conflict is constituted, the various technical features involved in the embodiments of the present invention described below can be combined with each other, and are all within the scope of protection of the present invention. In addition, although the functional modules are divided in the device or structural schematic diagram, and the logical order is shown in the flow chart, in some cases, the steps shown or described can be performed in a different order from the module division in the device, or in a different order from the flow chart. In addition, the "first", "second", "third" and other similar expressions used herein do not limit the data and execution order, but are only for the purpose of convenience of explanation and to distinguish between the same or similar items with basically the same functions and effects, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features.

[0045] Unless otherwise defined, the technical terms and scientific terms used in this specification have the same meanings as those generally understood by those of ordinary skill in the art of the present invention. The terms used in this specification are only for the purpose of describing specific embodiments and are not intended to limit the present invention. It should be understood that the term "and / or" used in this specification includes any and all combinations of one or more of the related listed items.

[0046] See also Figure 1 , Figure 1 The following is a schematic diagram showing an application scenario of a data decompression method in some embodiments of the present invention.

[0047] like Figure 1 As shown, the application scenario includes: a compressed data packet 101, a compressed data packet 102, a controller 100 and an external storage 200.

[0048] In some embodiments, the data downloaded and decompressed by the controller 100 includes at least two compressed data packets (e.g., Figure 1 As shown in the compressed data packet 101 and the compressed data packet 102), in the previous download and decompression method, after all the compressed data packets are downloaded, they are decompressed at one time, and then the decompressed data obtained by decompression is written to the external storage 200.

[0049] However, in this way, the controller 100 needs to allocate two large-capacity memories to store the downloaded compressed data packets and the decompressed data, respectively, which results in the downloading and decompression occupying a large memory space. In addition, in some special cases, due to the limitation of the internal cache size of the controller 100, it is difficult to decompress all the compressed data packets at one time, resulting in decompression failure. In some embodiments, even if all the compressed data packets can be decompressed at one time, it still takes a long time to wait for the decompression to complete, which greatly reduces the data decompression efficiency and data writing efficiency, and is not applicable to some occasions with strict decompression time requirements.

[0050] Based on this, an embodiment of the present invention provides a data decompression method. After determining that the downloaded and decompressed data is a plurality of compressed data packets, the controller 100 allocates a decompression buffer of a suitable size for this decompression ( Figure 1 The decompression buffer is used to store the decompressed data obtained by decompression, and the size of the decompression buffer is at least the data length of the maximum decompressed data obtained by decompressing a single compressed data packet to avoid memory overflow when storing the decompressed data in the decompression buffer during the decompression process.

[0051] After allocating the decompression buffer, the controller 100 decompresses the downloaded compressed data packet (for example, compressed data packet 101), obtains the corresponding decompressed data, then stores the decompressed data in the decompression buffer, reads the decompressed data from the decompression buffer, and writes the decompressed data to the external storage 200. Then the controller 100 downloads and decompresses the next compressed data packet (for example, compressed data packet 102), stores the decompressed data obtained by decompression in the decompression buffer, then reads the decompressed data from the decompression buffer, and writes the decompressed data to the external storage 200. The download and decompression process is performed for each compressed data packet until the download and decompression process of the last compressed data packet is completed, and the complete decompressed data is obtained.

[0052] It is easy to understand that the external storage 200 can be a variety of devices obtained by combining any one or more of the memories such as flash memory, hard disk, optical disk, register, magnetic surface storage, removable disk, CD-ROM, random access memory (RAM), read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, etc., or it can also be any other form of storage medium known in the technical field. The embodiment of the present invention is described in detail with the external storage 200 as a hard disk, but it does not impose any limitation on the scope of protection of the present invention.

[0053] It should be understood that Figure 1 In the application scenario shown, only a situation in which the controller 100 downloads and decompresses a compressed data packet and writes the decompressed data obtained by decompression to the external storage 200 is schematically shown, which does not impose any restrictions on the number, size, and other conditions of the compressed data packets in other application scenarios, and the structure, type, and number of the controller 100 and the external storage 200. For example, in some other application scenarios, a larger number (for example, 5) of compressed data packets may be included, and the controller 100 may also write the decompressed data obtained by decompression to multiple external storages 200 respectively.

[0054] To facilitate understanding of the data decompression method provided by the embodiment of the present invention, the controller provided by the embodiment of the present invention is firstly introduced in detail.

[0055] See also Figure 2 , Figure 2 The schematic diagram shows the structure of the controller 100 provided in some embodiments of the present invention.

[0056] The controller 100 includes at least one processor 110 and a memory 120 ( Figure 2 In the example of a bus system connection and a processor, the various components of the controller 100 are coupled together through the bus system 130, and the bus system 130 is used to realize the connection and communication between these components. It is easy to understand that the bus system 130 includes not only a data bus, but also a power bus, a control bus, and a status signal bus. However, for the sake of clarity and brevity, Figure 2 In the figure, various buses are labeled as bus system 130. Those skilled in the art can understand that Figure 2 The structure shown in the embodiment is only for illustration and does not impose any limitation on the structure of the controller 100. For example, the controller 100 may also include Figure 2 The structure shown in FIG. 1 may have more or fewer components, or may have the same Figure 2 Different configurations of the structure are shown.

[0057] Specifically, the processor 110 is used to provide computing and control capabilities to control the controller 100 to perform corresponding tasks. For example, the controller 100 is controlled to perform any one of the data decompression methods provided in the embodiments of the present invention, or to perform any one of the steps in any possible implementation of any one of the data decompression methods provided in the embodiments of the present invention. Those skilled in the art will appreciate that the processor 110 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.

[0058] The memory 120, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer executable programs, instructions and modules, for example, programs, instructions and modules corresponding to the data decompression method in the embodiment of the present invention. In some embodiments, the memory 120 may include a program storage area and a data storage area, and the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the processor 110, etc. The processor 110 executes various functional applications and data processing of the controller 100 by running the non-transitory software programs, instructions and modules stored in the memory 120 to implement any data decompression method provided in the embodiment of the present invention, or execute any step in any possible implementation of any data decompression method provided in the embodiment of the present invention. Among them, the memory 120 may include a high-speed random access memory, and may also include a non-transitory memory. For example, at least one disk storage device, a flash memory device or other non-transitory solid-state storage device. In some embodiments, the memory 120 may also include a memory remotely arranged relative to the processor 110, and these remotely arranged memories may be connected to the processor 110 through a communication network. It can be understood that examples of the above-mentioned communication network include but are not limited to the Internet, corporate intranet, local area network, mobile communication network and combinations thereof.

[0059] It is easy to understand that the implementation and execution subject of any data decompression method provided in the embodiments of the present invention can be any suitable type of controller with certain computing and control capabilities. In some possible implementations, any data decompression method provided in the embodiments of the present invention can be implemented by a processor calling a computer program instruction stored in a memory.

[0060] It can be understood from the above that the data decompression method provided by the embodiment of the present invention can be implemented by various suitable types of controllers with certain computing and control capabilities, for example, it can be implemented and executed by the above controller. The data decompression method provided by the embodiment of the present invention will be described in detail below in conjunction with the exemplary application and implementation of the controller provided by the embodiment of the present invention.

[0061] See also Figure 3 , Figure 3 A schematic flow chart of a data decompression method provided by some embodiments of the present invention is shown.

[0062] Those skilled in the art will appreciate that the data decompression method provided in the embodiment of the present invention may be applied to the above controller (eg, controller 100). Specifically, the data decompression method is executed by one or at least two processors of the controller.

[0063] like Figure 3As shown, the data decompression method includes but is not limited to the following steps S100-S200:

[0064] S100: after determining that the downloaded data is at least two compressed data packets, allocating a decompression buffer, the size of which is at least the data length of the maximum decompressed data obtained by decompressing the compressed data packets.

[0065] It is understandable that a compressed file is identified as belonging to a sequence of compressed data packets through a specific naming rule, such as file.part1.zip, file.part2.zip, etc. Therefore, it is possible to determine whether the downloaded data is multiple compressed data packets by parsing the file name or extension. In some embodiments, the file header of the first compressed data packet contains data compression information (e.g., the total number of files and the current fragment sequence number), and the number of compressed data packets can be determined by reading the file header information.

[0066] Specifically, before downloading data, the controller first identifies the type of the downloaded data. If the file name or extension of the parsed data is determined to be at least two compressed data packets, the controller allocates the decompression buffer required for downloading and decompressing at least two compressed data packets according to the data length of the maximum decompressed data obtained by decompressing a single compressed data packet. That is, the size of the decompression buffer is at least the data length of the maximum decompressed data obtained by decompressing a single compressed data packet. The decompression buffer is used as a temporary storage space to store the decompressed data obtained by decompression for subsequent storage processing.

[0067] For example, in some embodiments, the downloaded data is three compressed data packets. After the three compressed data packets are decompressed, the corresponding decompressed data lengths are 2k, 3k and 4k. Then the size of the allocated decompression cache is the data length of the maximum decompressed data, 4k.

[0068] By allocating the above decompression cache of an appropriate size, it is ensured that the decompressed data obtained by decompressing a single compressed data packet can be completely stored in the decompression cache, avoiding the situation of insufficient cache, and ensuring that the data is efficiently stored and processed during the decompression process.

[0069] In some embodiments, the controller responds to the received download request and parses the download request to obtain corresponding decompression parameters.

[0070] In some embodiments, after determining that the downloaded data is at least two compressed data packets and before allocating the decompression buffer, the data decompression method further includes but is not limited to the following steps S101:

[0071] S101: receiving a download request, parsing the download request to obtain a first data length, where the first data length is the sum of data lengths of at least two compressed data packets.

[0072] Specifically, the controller receives a download request sent by a server, system or client through a suitable communication method (such as a wired communication network, a wireless communication network or other methods), such as a user clicking a download link or a download request sent through an API interface.

[0073] After receiving the download request, the controller parses the download request and extracts key information in the download request, such as the type of the downloaded file, the file name, the file path, the file size, etc. In the embodiment of the present invention, the controller reads the request header information, parameters or metadata of the download request to obtain the total length of the compressed data packets to be downloaded, that is, obtains the first data length. Obviously, the first data length is the sum of the data lengths of at least two compressed data packets, that is, the sum of the sizes of all compressed data packets to be downloaded.

[0074] In some embodiments, after determining that the downloaded data is at least two compressed data packets, the first compressed data packet of the at least two compressed data packets includes a file header, the file header of the first compressed data packet is parsed to obtain corresponding decompression parameters, and then the corresponding compressed data packet is downloaded and decompressed.

[0075] In some embodiments, the data decompression method further includes but is not limited to the following steps S102-S103:

[0076] S102: If the compressed data packet is the first compressed data packet, parse the file header of the first compressed data packet to obtain a second data length, where the second data length is the sum of the data lengths of original data before compression of at least two compressed data packets.

[0077] Specifically, if the compressed data packet currently processed by the controller is the first compressed data packet, the controller parses the file header of the first compressed data packet and extracts key information in the file header, such as the size of the original data, the type of compression algorithm, and the checksum (used to verify data integrity).

[0078] In an embodiment of the present invention, the controller reads the information, parameters or metadata of the file header of the first compressed data packet, obtains the sum of the data lengths of the original data before compression of at least two compressed data packets, that is, obtains the second data length. Obviously, the second data length is the sum of the data lengths of the original data before compression of at least two compressed data packets, that is, the sum of the sizes of the decompressed data obtained after decompression of all compressed data packets.

[0079] S103: Configure target decompression parameters, where the target decompression parameters include a first data length and a second data length.

[0080] Among them, the target decompression parameters are key parameters used to guide data decompression to ensure that the decompression operation is performed as expected and to avoid decompression errors or data loss.

[0081] Specifically, after obtaining the first data length and the second data length, the target decompression parameters are configured according to the obtained first data length and the second data length, wherein the target decompression parameters include the first data length and the second data length. The first data length refers to the sum of the data lengths of at least two data packets obtained when receiving and parsing the download request, and is used to guide the cache allocation, decompression operation sequence, and final storage location of the decompressed data during the decompression process, etc., to ensure that all compressed data packets are correctly downloaded and processed. The second data length refers to the sum of the data lengths of the original data before compression of at least two data packets, that is, the total size of the original data corresponding to all compressed data packets. The second data length is used to verify the integrity and accuracy of the decompressed data, and ensure that the decompressed data matches the original data to avoid data loss or damage.

[0082] In some embodiments, configuring the target decompression parameters includes: setting the first data length and the second data length and setting related decompression environments, such as file storage location, decompression order, etc.

[0083] S200: decompressing the compressed data packet to obtain decompressed data, storing the decompressed data in a decompression buffer, reading the decompressed data from the decompression buffer, and storing the decompressed data in an external storage until the last compressed data packet.

[0084] Specifically, after determining that the downloaded data is at least two compressed data packets, the controller downloads one compressed data packet at a time, and uses a decompression algorithm to decompress the downloaded compressed data packet, thereby obtaining the corresponding decompressed data. The decompressed data obtained by decompression is then temporarily stored in a pre-allocated decompression buffer. Next, the stored decompressed data is read from the decompression buffer, and then the read decompressed data is written to a specified location of an external storage (such as a hard disk), that is, the read decompressed data is stored in the hard disk.

[0085] For each compressed data packet, the controller performs the above downloading and decompression operations until the last compressed data packet is downloaded and decompressed, and stores the decompressed data obtained by decompressing the last compressed data packet to the hard disk.

[0086] In some embodiments, in the process of storing decompressed data in the decompression cache, it is necessary to first determine the remaining storage space of the decompression cache based on the write start address of the decompression cache, and then compare the size of the decompressed data with the remaining storage space to determine the write start address to start storing the decompressed data, and then start storing the decompressed data in the decompression cache at the write start address.

[0087] See also Figure 4 , Figure 4 A sub-flow diagram of step S200 in the data decompression method provided by some embodiments of the present invention is shown.

[0088] like Figure 4 As shown, in some embodiments, decompressing the compressed data packet to obtain decompressed data and storing the decompressed data in the decompression buffer specifically includes but is not limited to the following steps S210-S250:

[0089] S210: Decompress the compressed data packet to obtain decompressed data.

[0090] Specifically, the controller downloads one compressed data packet at a time, and decompresses the downloaded compressed data packet using a decompression algorithm, thereby acquiring corresponding decompressed data.

[0091] S220: Determine the remaining storage space of the decompression cache based on the write start address of the decompression cache.

[0092] S230: If the data length of the decompressed data is greater than the capacity of the remaining storage space, the write start address of the decompression cache is reset to the head address of the decompression cache.

[0093] S240: Storing the decompressed data into the decompression buffer starting from the write start address.

[0094] The write start address refers to the location where the decompressed data starts to be stored in the decompression buffer. Each time the decompressed data is stored in the decompression buffer, it starts from the write start address.

[0095] Specifically, after decompressing the compressed data packet to obtain the decompressed data, the remaining storage space of the decompression cache can be determined according to the current write start address of the decompression cache. Among them, the remaining storage space is equal to the size of the decompression cache (the size of the decompression cache has been determined when it was initially allocated) minus the occupied cache space corresponding to the write start address. For example, in some embodiments, the size of the decompression cache is 4k, and the current write start address of the decompression cache is located at the 2.5k storage capacity of the decompression cache, indicating that 2.5k of the decompression cache is occupied, then the remaining storage space of the decompression cache = the size of the decompression cache 4k-the occupied cache space corresponding to the write start address 2.5k, that is, the remaining storage space = 4k-2.5k = 1.5k.

[0096] After determining the remaining storage space of the decompression cache, the data length of the decompressed data is compared with the remaining storage space. If the data length of the decompressed data is greater than the capacity of the remaining storage space, it means that the capacity of the remaining storage space cannot accommodate the decompressed data, and the decompressed data cannot be stored in the decompression cache starting from the current write start address of the decompression cache. To this end, the current write start address of the decompression cache is reset to the head address of the decompression cache (that is, the starting position of the decompression cache), and new decompressed data is stored starting from the head address of the decompression cache, overwriting the previously stored decompressed data.

[0097] After the write start address of the decompression cache is reset to the head address of the decompression cache, the decompressed data is stored in the decompression cache byte by byte or block by block starting from the reset write start address (ie, the head address of the decompression cache).

[0098] In some embodiments, if the data length of the decompressed data is less than or equal to the capacity of the remaining storage space, it means that the capacity of the remaining storage space can accommodate the decompressed data, and there is no need to reset the current write start address of the decompression cache. The decompressed data can be stored in the decompression cache starting from the current write start address of the decompression cache.

[0099] S250: Update the write start address to the current write start address plus the data length of the decompressed data.

[0100] Specifically, after storing the decompressed data in the decompression buffer, the write start address of the decompression buffer is updated to point to the location where the next decompressed data starts to be stored. The write start address of the decompression buffer is updated to the current write start address of the decompression buffer plus the data length of the decompressed data.

[0101] For example, in some embodiments, the data length of the decompressed data is 3.0k. After storing 3.0k of decompressed data in the decompressed cache starting from the head address of the decompressed cache, the write start address of the decompressed cache is updated to the head address of the decompressed cache (i.e., the current write start address) plus the data length of the decompressed data, that is, the write start address of the decompressed cache should be updated to be located at 0k+3.0k=3.0k storage capacity of the decompressed cache.

[0102] In some embodiments, during the process of decompressing at least two compressed data packets, current decompression parameters for decompressing each compressed data packet are recorded, and the current decompression parameters are compared with target decompression parameters to determine whether the at least two compressed data packets are decompressed successfully.

[0103] In some embodiments, the data decompression method further includes but is not limited to the following steps S201:

[0104] S201: Record current decompression parameters, wherein the current decompression parameters include the sum of input lengths and the sum of output lengths, the input length is the data length of the decompressed compressed data packet, and the output length is the data length of the decompressed data obtained by decompression.

[0105] Among them, for each compressed data packet, the input length is the data length of the compressed data packet that has been decompressed. If the compressed data packet currently being decompressed is the first compressed data packet, the input length is the data length of the first compressed data packet minus the length of the file header. If the compressed data packet currently being decompressed is not the first compressed data packet, the input length is the data length of the compressed data packet. The output length is the data length of the decompressed data obtained by decompressing the compressed data packet.

[0106] Specifically, in the process of decompressing a compressed data packet, the sum of the input lengths is calculated according to the data length of the compressed data packet currently being decompressed, that is, the sum of the sizes of the compressed data packets that have been decompressed. In some embodiments, if the compressed data packet currently being decompressed is the first compressed data packet, the sum of the input lengths is the input length of the first compressed data packet, that is, the sum of the input lengths is equal to the data length of the first compressed data packet minus the length of the file header. If the compressed data packet currently being decompressed is not the first compressed data packet, the sum of the current input lengths is equal to the sum of the input length of the compressed data packet currently being decompressed (that is, the data length of the compressed data packet currently being decompressed) and the sum of the input lengths recorded last time.

[0107] After decompressing the compressed data packet to obtain the corresponding decompressed data, the sum of the output lengths is calculated according to the data length of the decompressed data currently decompressed, that is, the sum of the sizes of the compressed data obtained by decompression. In some embodiments, the sum of the output lengths is equal to the sum of the output length of the decompressed data currently decompressed (that is, the data length of the decompressed data currently decompressed) and the sum of the output lengths recorded last time.

[0108] After the sum of the input lengths and the sum of the output lengths are calculated, the sum of the input lengths and the sum of the output lengths are recorded as current decompression parameters for subsequent determination of whether at least two compressed data packets are successfully decompressed.

[0109] In some embodiments, during the process of decompressing the compressed data packets, the recorded current decompression parameters are compared with the target decompression parameters to determine whether the at least two compressed data packets are successfully decompressed.

[0110] In some embodiments, the data decompression method further includes but is not limited to the following steps S300-S400:

[0111] S300: If the sum of the input lengths is equal to the first data length, and the sum of the output lengths is equal to the second data length, it is determined that the decompression is successful.

[0112] S400: If the sum of the input lengths is not equal to the first data length, and / or the sum of the output lengths is not equal to the second data length, it is determined that the decompression fails.

[0113] Specifically, the sum of the input lengths in the current decompression parameters is compared with the first data length in the target decompression parameters, and the sum of the output lengths in the current decompression parameters is compared with the second data length in the target decompression parameters. If the sum of the input lengths is equal to the first data length, and the sum of the output lengths is equal to the second data length, it is determined that at least two compressed data packets are decompressed successfully.

[0114] If the sum of the input lengths is not equal to the first data length, and / or the sum of the output lengths is not equal to the second data length, it is determined that decompression of at least two compressed data packets has failed.

[0115] In some embodiments, the write start address of the decompression cache, the recorded current decompression parameters, and the configured target decompression parameters are stored in a decompression structure, which is a data structure specifically used to store and manage various parameters and address information related to the decompression process. By using the decompression structure to centrally store important information required in the decompression process, it is convenient for the controller to manage and control the data decompression operation, reduce scattered variable management, and improve decompression efficiency and accuracy.

[0116] The write start address refers to the location where the decompressed data starts to be stored in the decompression cache. During the decompression process, as the decompressed data is continuously stored in the cache, the write start address is continuously updated. By storing the write start address in the decompression structure, the controller can quickly access and update it when needed.

[0117] The current decompression parameters include the sum of the input length and the sum of the output length, which are used to track the progress of the decompression process. By storing the current decompression parameters in the decompression structure, the controller can obtain and update the current decompression parameters at any time during the decompression process to ensure the correctness of the decompression operation.

[0118] The target decompression parameters include a first data length and a second data length, and the first data length and the second data length are used to guide the configuration and execution of the decompression process. By storing the target decompression parameters in the decompression structure, it is ensured that the controller can access and obtain the target decompression parameters at any time before and during the decompression process, and perform corresponding configuration and adjustment.

[0119] See also Figure 5, Figure 5 A structure of a decompression structure in some embodiments of the present invention is schematically shown.

[0120] In some embodiments, the decompression structure also includes a dictionary, wherein the dictionary is used to store decompressed data obtained by decompressing the compressed data packet. After decompressing the current compressed data packet to obtain the decompressed data, the dictionary is searched to determine whether there is first data identical to the decompressed data, so as to determine whether to store the decompressed data in the decompression buffer.

[0121] In some embodiments, the dictionary may be a hash table, a mapping table or other data structures, and the dictionary is used to store identifiers or hash values ​​of decompressed data that have been decompressed, so as to facilitate fast search and verification of whether the decompressed data already exists.

[0122] See also Figure 6 , Figure 6 A sub-flow diagram of step S240 in the data decompression method provided by some embodiments of the present invention is shown.

[0123] like Figure 6 As shown, in some embodiments, storing the decompressed data to the decompression cache starting from the write start address specifically includes but is not limited to the following steps S2401-S2402:

[0124] S2401: If the first data identical to the decompressed data does not exist in the dictionary, the decompressed data is stored in the decompression buffer starting from the write start address.

[0125] S2402: If the dictionary contains first data identical to the decompressed data, the storage address of the first data is recorded.

[0126] Specifically, after decompressing the current compressed data packet to obtain the corresponding current decompressed data, the current decompressed data is compared with the decompressed data stored in the dictionary one by one to determine whether there is first data identical to the current decompressed data in the dictionary. It should be understood that the comparison method can be to directly compare the contents between the decompressed data, or to compare the hash values ​​between the decompressed data. For example, when the data length of the decompressed data is long, the hash values ​​between the decompressed data can be compared to determine whether there is first data identical to the current decompressed data in the dictionary, thereby improving the comparison speed.

[0127] In some embodiments, if the comparison determines that there is no first data identical to the current decompressed data in the dictionary, it means that the current decompressed data corresponding to the decompression of the current compressed data packet is different from the decompressed data obtained by the previous decompression, and the current decompressed data is stored in the decompression cache starting from the write start address of the decompression cache, and then the current decompressed data in the decompression cache is read and stored in the hard disk. In addition, the controller adds the current decompressed data entry or its identifier (e.g., hash value) to the dictionary for subsequent search and inspection.

[0128] If the comparison determines that there is the first data identical to the current decompressed data in the dictionary, it means that the current decompressed data obtained by decompressing the current compressed data packet is the same as one of the decompressed data obtained by the previous decompression. At this time, it is not necessary to store the current decompressed data in the decompression cache, and only the storage address of the first data in the dictionary needs to be recorded. Then, the storage address of the first data is read from the dictionary to obtain the current decompressed data, and the current decompressed data is stored in the hard disk.

[0129] By using a dictionary in the decompression structure, you can effectively avoid repeated storage and processing of the same decompressed data, which is especially effective when decompressing large files or processing duplicate data, and can greatly improve the decompression efficiency. By using a dictionary to quickly find and verify the existence of decompressed data, you can improve the efficiency of data processing, avoid unnecessary repeated calculations and storage operations, and use memory resources more efficiently.

[0130] To summarize, the data decompression method provided by the embodiment of the present invention includes: after determining that the downloaded data is at least two compressed data packets, allocating a decompression cache, the size of the decompression cache being at least the data length of the maximum decompressed data obtained by decompressing the compressed data packet; decompressing the compressed data packet to obtain the decompressed data, and storing the decompressed data in the decompression cache, reading the decompressed data from the decompression cache, and storing the decompressed data in external storage until the last compressed data packet.

[0131] In the data decompression method provided by the embodiment of the present invention, the decompression cache is used to store the decompressed data obtained by decompressing a single compressed data packet, and the decompressed data is read from the decompression cache and stored in the external storage. By decompressing the compressed data packet in a packetized manner, the occupancy of the decompression memory can be reduced, the decompressed data can be stored in the external storage device in a timely manner, and the data decompression efficiency and data writing efficiency can be improved.

[0132] An embodiment of the present invention provides a computer-readable storage medium, on which computer program instructions executable by a processor are stored. When the computer program instructions are called by the processor, the processor executes any one of the data decompression methods provided by the embodiment of the present invention, or executes steps in any one of the possible implementation methods of any one of the data decompression methods provided by the embodiment of the present invention.

[0133] In some embodiments, the storage medium may be a flash memory, a hard disk, an optical disk, a register, a magnetic surface storage, a removable disk, a CD-ROM, a random access memory (RAM), a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM or other memory, or any other form of storage medium known in the technical field, or various devices including one or any combination of the above storage media.

[0134] In some embodiments, computer program instructions may be in the form of a program, software, software module, script or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine or other unit suitable for use in a computing environment.

[0135] As an example, computer program instructions may, but do not necessarily, correspond to a file in a file system, may be stored as part of a file that stores other programs or data, for example, in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (for example, files storing one or more modules, subroutines, or code portions).

[0136] As an example, the computer program instructions may be deployed to be executed on a computing device (including devices such as smart terminals and servers), or on multiple computing devices located at one location, or on multiple computing devices distributed at multiple locations and interconnected by a communication network. It is easy to understand that all or part of the steps of the method described in the embodiments of the present invention may be directly implemented using electronic hardware or processor executable computer program instructions, or a combination of the two.

[0137] The technicians can understand that the embodiments provided by the present invention are only illustrative, and the writing order of each step in the method of the embodiment does not mean a strict execution order and does not constitute any limitation on the implementation process. The order can be adjusted, merged and deleted according to actual needs. The modules or sub-modules, units or sub-units in the device or system of the embodiment can be merged, divided and deleted according to actual needs. For example, the division of units is only a logical function division, and there can be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed.

[0138] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a general hardware platform, and of course can also be implemented by hardware. Those skilled in the art can understand that all or part of the processes in the above embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above methods. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0139] It should be noted that the above embodiments are intended to illustrate the technical concept and features of the present invention, and their purpose is to enable people familiar with this technology to understand the content of the present invention and implement it accordingly, and cannot be used to limit the scope of protection of the present invention. Ordinary technicians in this field can understand that all or part of the processes of the above embodiments can be implemented according to the technical solutions recorded in the embodiments of the present invention, or some of the technical features can be equivalently replaced. It can be understood that these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention, and should be regarded as equal changes and modifications based on the embodiments of the present invention, and should all fall within the scope of the claims of the present invention.

Claims

1. A data decompression method, characterized in that: The method comprises: After determining that the downloaded data is at least two compressed data packets, allocating a decompression buffer, wherein the size of the decompression buffer is at least the data length of the maximum decompressed data obtained by decompressing the compressed data packets; The compressed data packet is decompressed to obtain decompressed data, and the decompressed data is stored in the decompression buffer, the decompressed data is read from the decompression buffer, and the decompressed data is stored in an external storage until the last compressed data packet.

2. The method according to claim 1, characterized in that The decompressing the compressed data packet to obtain decompressed data, and storing the decompressed data in the decompression buffer, comprises: Decompressing the compressed data packet to obtain decompressed data; Determining the remaining storage space of the decompression cache based on the write start address of the decompression cache; If the data length of the decompressed data is greater than the capacity of the remaining storage space, resetting the write start address of the decompression cache to the head address of the decompression cache; Storing the decompressed data in the decompression buffer starting from the write start address; The write start address is updated to the current write start address plus the data length of the decompressed data.

3. The method according to claim 2, characterized in that After determining that the downloaded data is at least two compressed data packets and before allocating the decompression buffer, the method further includes: Receiving a download request, parsing the download request to obtain a first data length, where the first data length is the sum of the data lengths of the at least two compressed data packets; The method further comprises: If the compressed data packet is the first compressed data packet, parsing the file header of the first compressed data packet to obtain a second data length, where the second data length is the sum of the data lengths of the original data before compression of the at least two compressed data packets; Configure target decompression parameters, where the target decompression parameters include the first data length and the second data length.

4. The method according to claim 3, characterized in that The method further comprises: Record current decompression parameters, wherein the current decompression parameters include the sum of input lengths and the sum of output lengths, the input length is the data length of the decompressed compressed data packet, and the output length is the data length of the decompressed data obtained by decompression.

5. The method according to claim 4, characterized in that The method further comprises: If the sum of the input lengths is equal to the first data length, and the sum of the output lengths is equal to the second data length, then it is determined that the decompression is successful; If the sum of the input lengths is not equal to the first data length, and / or the sum of the output lengths is not equal to the second data length, it is determined that the decompression has failed.

6. The method according to claim 4 or 5, characterized in that: If the compressed data packet is the first compressed data packet, the input length is the data length of the first compressed data packet minus the length of the file header; If the compressed data packet is not the first compressed data packet, the input length is the data length of the compressed data packet.

7. The method according to claim 4, characterized in that The write start address, the current decompression parameter and the target decompression parameter are stored in a decompression structure, which is a data structure.

8. The method according to claim 7, characterized in that The decompression structure also includes a dictionary, and the storing the decompressed data to the decompression cache starting from the write start address includes: if there is no first data identical to the decompressed data in the dictionary, storing the decompressed data to the decompression cache starting from the write start address; If the dictionary contains first data identical to the decompressed data, the storage address of the first data is recorded.

9. A controller, characterized in that: include: A processor and a memory communicatively connected to the processor; The memory stores computer program instructions executable by the processor, and when the computer program instructions are called by the processor, the processor executes the data decompression method according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program instructions executable by a processor, and when the computer program instructions are called by the processor, the processor executes the data decompression method according to any one of claims 1 to 8.

Images