Compressed data processing method and device, product, equipment and medium

Abstract

The invention discloses a compressed data processing method and device, a product, equipment and a medium, and the method comprises the steps: generating a streaming downloading task set in response to a downloading request for a compressed data packet; the compressed data packet is obtained by compressing target data, the compressed data packet comprises M compressed data blocks, the streaming downloading task set comprises M streaming downloading tasks, and the compressed data blocks are in one-to-one correspondence with the streaming downloading tasks; creating N threads for executing the streaming downloading tasks, and distributing the streaming downloading tasks needing to be executed for each thread based on the streaming downloading task set; calling N threads to execute respective streaming downloading tasks so as to download compressed data blocks corresponding to the streaming downloading tasks from the data server, and performing real-time streaming decompression processing on the downloaded compressed data blocks; and obtaining target data after the M compressed data blocks are subjected to streaming decompression processing. By adopting the method and the device, the efficiency of downloading and decompressing the compressed data packet can be improved.

Description

Technical Field

[0001] The present application relates to the technical field of data processing, and in particular to a method, apparatus, product, equipment and medium for processing compressed data. Background Art

[0002] Data compression is the process of encoding original data using less storage space. It aims to reduce the amount of data to reduce storage space and improve data transmission efficiency without losing useful information.

[0003] In existing applications, the terminal can download compressed data packets from the data server. However, in some cases, even the compressed data packets obtained after compressing the original data can be very large, which makes it very time-consuming for the terminal to download the compressed data packets. Moreover, after the terminal downloads the entire compressed data packet from the data server, it also needs to decompress the downloaded compressed data packet, which is another very time-consuming process, resulting in extremely low efficiency of the terminal in downloading and decompressing the compressed data packet. Summary of the Invention

[0004] The present application provides a method, apparatus, product, device and medium for processing compressed data, which can improve the efficiency of downloading and decompressing compressed data packets.

[0005] On one hand, the present application provides a method for processing compressed data, the method comprising:

[0006] In response to a download request for a compressed data packet, a streaming download task set is generated; the compressed data packet is stored in a data server, the compressed data packet is obtained by compressing the target data, the compressed data packet includes M compressed data blocks, and the streaming download task set includes M streaming download tasks, each compressed data block corresponds to a streaming download task, where M is a positive integer;

[0007] Create N threads for executing streaming download tasks, and assign each thread a streaming download task to be executed based on the streaming download task set; N is a positive integer;

[0008] N threads are called to execute their respective streaming download tasks, so as to download the compressed data blocks corresponding to each streaming download task from the data server, and perform real-time streaming decompression processing on the downloaded compressed data blocks; wherein, the target data is obtained after the M compressed data blocks are stream-decompressed.

[0009] On one hand, the present application provides a device for processing compressed data, the device comprising:

[0010] A generation module is configured to generate a streaming download task set in response to a download request for a compressed data packet; the compressed data packet is stored in a data server, the compressed data packet is obtained by compressing the target data, the compressed data packet includes M compressed data blocks, and the streaming download task set includes M streaming download tasks, each compressed data block corresponds to a streaming download task, where M is a positive integer;

[0011] An allocation module is used to create N threads for executing streaming download tasks, and allocate the streaming download tasks to be executed to each thread based on the streaming download task set; N is a positive integer;

[0012] The calling module is used to call N threads to execute their respective streaming download tasks, so as to download the compressed data blocks corresponding to each streaming download task from the data server respectively, and perform real-time streaming decompression processing on the downloaded compressed data blocks; wherein, the target data is obtained after the M compressed data blocks are stream-decompressed.

[0013] Optionally, the generation module generates a streaming download task set in response to a download request for a compressed data packet, including:

[0014] In response to a download request for a compressed data packet, obtaining a download address of the compressed data packet;

[0015] Based on the download address, the description information of each compressed data block is read from the data server; each compressed data block has corresponding description information;

[0016] Based on the description information of each compressed data block, a streaming download task corresponding to each compressed data block is generated.

[0017] Optionally, the description information of the M compressed data blocks is located at the end of the compressed data packet; and the generation module reads the description information of each compressed data block from the data server based on the download address, including:

[0018] Establishing a data connection with a data server;

[0019] Based on the data connection, a memory block of the first size is used to read the compressed data in the compressed data packet from the data server one or more times in a back-to-front direction, and the size of the compressed data obtained by any data reading is the first size;

[0020] If the compressed data obtained by reading data using the memory block contains M description information, then the data reading of the remaining compressed data in the compressed data packet is stopped;

[0021] The description information of the M compressed data blocks includes M description information obtained by reading data.

[0022] Optionally, the description information of the M compressed data blocks exists in the compressed data packet in the form of a linked list;

[0023] The generation module reads the description information of each compressed data block from the data server based on the download address, including:

[0024] Based on the download address, M description information of the M compressed data blocks are read in sequence from the linked list, and in the process of reading the description information, reading of the compressed data block to which the read description information belongs is skipped.

[0025] Optionally, the description information of any compressed data block includes at least one of the following:

[0026] The offset information of any compressed data block in the compressed data packet; the data size of any compressed data block; the data size of any compressed data block after decompression; the disk location of the data after decompression of any compressed data block; the data identifier of the data after decompression of any compressed data block;

[0027] The offset information of any compressed data block in the compressed data packet is used to indicate the data position of any compressed data block in the compressed data packet.

[0028] Optionally, after generating the streaming download task set, the processing device is further configured to:

[0029] Get the data size of each compressed data block in the compressed data packet;

[0030] Sorting the streaming download tasks corresponding to each compressed data block in descending order of the data size of each compressed data block to obtain sorted streaming download tasks;

[0031] The sorted streaming download tasks are added to a task pool; wherein the task pool is used to store unprocessed streaming download tasks.

[0032] Optionally, any one of the N threads is a target thread; the allocation module allocates the streaming download task to be executed to each thread based on the streaming download task set, including:

[0033] When the target thread is in an idle state, the streaming download task that is arranged at the front is taken out from the remaining streaming download tasks in the task pool;

[0034] Assign the extracted streaming download task to the target thread.

[0035] Optionally, any one of the N threads is a target thread, and the streaming download task to be executed by the target thread includes the target streaming download task;

[0036] The calling module calls N threads to respectively execute respective streaming download tasks, so as to download the compressed data blocks corresponding to each streaming download task from the data server respectively, including:

[0037] The target thread is called to extract offset information of the corresponding compressed data block in the compressed data packet from the target streaming download task; the extracted offset information is used to indicate the data position of the compressed data block corresponding to the target streaming download task in the compressed data packet;

[0038] The target thread is called to download corresponding compressed data blocks from the compressed data packet at the data server based on the extracted offset information.

[0039] Optionally, the target data includes K file data, where K is a positive integer, and each file data in the K file data has its own file compression ratio;

[0040] The compressed data packet includes compressed file data obtained by compressing first file data using a first compression method, and compressed file data obtained by compressing second file data using a second compression method; the first file data is file data having a file compression ratio less than a preset ratio threshold, and the second file data is file data having a file compression ratio greater than or equal to the ratio threshold;

[0041] The compressed file data obtained by compressing the first file data using the first compression method does not support segmentation, and the compressed file data obtained by compressing the second file data using the second compression method supports segmentation.

[0042] Optionally, the M compressed data blocks include compressed file data of the first file data and a plurality of compressed file blocks obtained by segmenting the compressed file data of the second file data;

[0043] The compressed file data of a first file data is a compressed data block in the compressed data packet, and the compressed file block obtained by segmenting the compressed file data of a second file data is a compressed data block in the compressed data packet.

[0044] Optionally, any one of the K file data is target file data, and the target file data has an original file size before being compressed using the first compression method, and a compressed file size after being compressed using the first compression method;

[0045] The file compression ratio of the target file data is the ratio of the compressed file size to the original file size.

[0046] Optionally, any one of the N threads is a target thread, the streaming download task to be executed by the target thread includes a target streaming download task, and the compressed data block corresponding to the target streaming download task is a target compressed data block;

[0047] The calling module calls N threads to respectively execute respective streaming download tasks, so as to respectively download the compressed data blocks corresponding to the respective streaming download tasks from the data server, and performs real-time streaming decompression processing on the downloaded compressed data blocks, including:

[0048] Invoking a target thread to execute a target streaming download task based on a cache block of the second size, so as to download the target compressed data block multiple times from the data server, wherein each data download is used to obtain a local compressed data of the target compressed data block, and a size of the local compressed data is less than or equal to the second size; and

[0049] Perform real-time streaming decompression on the locally compressed data obtained from each data download, and write the locally decompressed data obtained from the decompression of the locally compressed data to disk in real time; the multiple locally decompressed data corresponding to the multiple locally compressed data obtained from multiple data downloads constitute the decompressed data blocks corresponding to the target compressed data blocks;

[0050] Among them, multiple data downloads include the i-th data download, i is a positive integer, and after the local decompressed data corresponding to the local compressed data obtained by the i-th data download is successfully written to the disk, the local compressed data obtained by the i-th data download is used to be deleted from the cache block, and the cache block after the data deletion is used to continue the i+1-th data download of the target compressed data block.

[0051] Optionally, the target data is game installation data. The M compressed data blocks are stream-decompressed to obtain M decompressed data blocks that are written to the disk. Each compressed data block corresponds to a decompressed data block. The M decompressed data blocks that are written to the disk constitute the installation data.

[0052] The processing device is also used for:

[0053] Install the game based on the installation data dropped to the disc and run the installed game.

[0054] In one aspect, the present application provides a computer device including a memory and a processor, wherein the memory stores a computer program. When the computer program is executed by the processor, the processor executes the method in one aspect of the present application.

[0055] In one aspect, the present application provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the processor executes the method in the above aspect.

[0056] According to one aspect of the present application, a computer program product is provided, comprising a computer program stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium and executes the computer program, causing the computer device to perform the method provided in various optional embodiments such as the above-mentioned aspect.

[0057] The present application can generate a streaming download task set in response to a download request for a compressed data packet; the compressed data packet is stored in a data server, the compressed data packet is obtained after compressing the target data, the compressed data packet contains M compressed data blocks, the streaming download task set contains M streaming download tasks, and each compressed data block corresponds to a streaming download task; and N threads can be created for executing streaming download tasks, and based on the streaming download task set, each thread is assigned the streaming download task to be executed; N and M are both positive integers; thus, N threads can be called to execute their respective streaming download tasks, so as to download the compressed data blocks corresponding to each streaming download task from the data server respectively, and perform real-time streaming decompression on the downloaded compressed data blocks; wherein, the target data is obtained after the M compressed data blocks are stream-decompressed. It can be seen that the method proposed in this application can realize the rapid download of compressed data packets through multiple threads, and can realize real-time decompression processing of downloaded compressed data blocks through streaming download and decompression. Therefore, this application combines the multi-threading and process download and decompression methods to simultaneously improve the efficiency of compression and decompression processing of compressed data packets. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the technical solutions in the present application or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0059] Figure 1 This is a schematic diagram of a data download network architecture provided by an embodiment of the present application;

[0060] Figure 2 This is a flow chart of a method for processing compressed data provided by an embodiment of the present application;

[0061] Figure 3 This is a schematic diagram of a scenario for executing a streaming download task set provided by an embodiment of the present application;

[0062] Figure 4This is a schematic diagram of a framework for downloading a compressed data packet provided by an embodiment of the present application;

[0063] Figure 5 This is a flow chart of a method for generating a streaming download task set provided by an embodiment of the present application;

[0064] Figure 6 This is a schematic diagram of a scenario for reading description information of compressed data blocks provided by an embodiment of the present application;

[0065] Figure 7 This is a schematic diagram of a scenario for determining the information location of description information provided by an embodiment of the present application;

[0066] Figure 8 This is a schematic diagram of a scenario for establishing a data connection provided by an embodiment of the present application;

[0067] Figure 9 This is a flow chart of a data downloading and decompression method provided in an embodiment of the present application;

[0068] Figure 10 This is a schematic diagram of a scenario of data streaming downloading and decompression provided by an embodiment of the present application;

[0069] Figure 11 This is a schematic diagram of a multi-threaded download and decompression scenario provided by an embodiment of the present application;

[0070] Figure 12 This is a comparative diagram of the time taken to download and decompress data provided by an embodiment of the present application;

[0071] Figure 13 This is a comparative diagram of the space occupied by data during downloading and decompression provided by an embodiment of the present application;

[0072] Figure 14 1 is a schematic structural diagram of a compressed data processing device provided in an embodiment of the present application;

[0073] Figure 15 It is a structural diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0074] The following will be combined with the accompanying drawings to clearly and completely describe the technical solutions in this application. Obviously, the embodiments described are only some of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making any creative efforts are within the scope of protection of this application.

[0075] This application relates to cloud technology, which refers to a hosting technology that unifies hardware, software, network and other resources within a wide area network or local area network to achieve data computing, storage, processing and sharing.

[0076] Cloud technology is a general term for network technology, information technology, integration technology, management platform technology, and application technology based on the cloud computing business model. It can form a resource pool for on-demand, flexible and convenient use. Cloud computing technology will become a key support. Backend services of technical network systems, such as video websites, image websites, and more portals, require extensive computing and storage resources. With the rapid development and application of the internet industry, every item will likely have its own unique identification mark, which will need to be transmitted to backend systems for logical processing. Different levels of data will be processed separately. All types of industry data require strong system support, which can only be achieved through cloud computing.

[0077] This application can implement storage and transmission of compressed data packets through the "cloud", for example, storage and processing of compressed data packets can be implemented through a cloud server.

[0078] First of all, it needs to be explained that all data collected by this application (such as compressed data packets, download addresses of compressed data packets, description information of compressed data blocks, and other related data) are collected with the consent and authorization of the object to which the data belongs (such as users, institutions or enterprises), and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of the relevant regions.

[0079] See Figure 1 , Figure 1 This is a schematic diagram of a data download network architecture provided by an embodiment of the present application. Figure 1 As shown, the network architecture may include a terminal device 100 and a data server 200. The data server 200 may store a number of data, and the terminal device 100 may be any terminal device that needs to download data from the data server 200. The terminal device 100 may establish a network connection with the data server 200 so that the terminal device 100 can exchange data with the data server 200 via the network connection.

[0080] like Figure 1The data server 200 shown can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (content distribution network), and big data and artificial intelligence platforms. The terminal device 100 can be any smart terminal such as a smartphone, tablet computer, laptop computer, desktop computer, smart TV, in-vehicle terminal, smart home appliance, in-vehicle terminal, smart home, etc.

[0081] The data server 200 may store a compressed data packet, which may be a data packet obtained by compressing the target data. The compressed data packet may include M compressed data blocks (compressed data block 1 to compressed data block M), where M is a positive integer. For example, the target data may be game installation data.

[0082] The terminal device 100 can request the data server 200 to download the compressed data packet, specifically including: the terminal device 100 can create multiple threads (such as thread 1 to thread 4 here, a total of 4 threads), and use the multiple threads to synchronously stream and download and decompress each compressed data block in the compressed data packet from the data server 200, so that each compressed data packet can be decompressed at the same time as the download of each compressed data block is completed. After streaming and decompressing each compressed data packet, the above-mentioned target data can be directly obtained. The terminal device can use the target data for further business processing (such as installing the game through the target data, and further performing functional testing on the installed game, etc.). This process can also be found in the specific description of the following embodiments.

[0083] By adopting the method of the present application, multi-threading and process download and decompression methods can be combined to achieve fast downloading and real-time decompression of compressed data packets, thereby reducing the downloading time and decompression time of compressed data packets.

[0084] See Figure 2 , Figure 2 This is a flow chart of a method for processing compressed data provided by an embodiment of the present application. The execution subject in the embodiment of the present application may be a terminal device, or may be other devices, which may be determined based on the actual application scenario. The following description will be made using the execution subject in the embodiment of the present application as an example of a terminal device. Figure 2 As shown, the method may include:

[0085] Step S101, in response to a download request for a compressed data packet, a streaming download task set is generated; the compressed data packet is stored in a data server, the compressed data packet is obtained after compressing the target data, the compressed data packet contains M compressed data blocks, the streaming download task set contains M streaming download tasks, each compressed data block corresponds to a streaming download task, and M is a positive integer.

[0086] Optionally, a user is supported to trigger a download operation for a compressed data packet on a terminal device, so that the terminal device can generate a download request for the compressed data packet in response to the download operation. The download request can be used to request downloading the compressed data packet. For example, the download operation can be triggered by a download control on a terminal interface of the terminal device, or by a voice command on the terminal device for downloading the compressed data packet, or by other triggering methods, which are not limited in this application.

[0087] The compressed data packet may be obtained by compressing the target data, i.e., the target data is the original data of the compressed data packet. Conversely, it can be understood that the target data can be obtained by decompressing the compressed data packet. Optionally, the download request may be a download request for the target data from the user's perspective, but in reality, what needs to be downloaded is the compressed data packet of the target data. Therefore, the download request may actually be a download request for the compressed data packet.

[0088] The type of target data may also be different for different application scenarios. For example, in a specific game application scenario, the target data may be the game's installation data (i.e., data used to install the game, such as a game installation package), and the compressed data packet may be the game's compressed installation package. In a broader software application scenario, the target data may be the software (app)'s installation data (i.e., data used to install the software, such as a software installation package), and the compressed data packet may be the software's compressed installation package. This application may be applied to any scenario where a compressed data packet needs to be downloaded and decompressed, i.e., the target data may be any data that needs to be compressed before downloading.

[0089] The compressed data packet may be stored in a data server, and the download request may be used to request the data server to download the compressed data packet. The data server may also be called a file server, which may be used to store data and provide download services for the stored data.

[0090] The compressed data packet may include M compressed data blocks, where M is a positive integer. It can be understood that the compressed data packet may be divided into the M compressed data blocks, and the specific value of M may be determined according to the actual application scenario.

[0091] In response to a download request for a compressed data packet, a terminal device may generate a streaming download task set, i.e., a task set for downloading the compressed data packet. The streaming download task set may include M streaming download tasks, with each compressed data block in the compressed data packet corresponding to a streaming download task in the streaming download task set. That is, each compressed data block may correspond one-to-one to each streaming download task, with one compressed data block corresponding to one streaming download task, and the streaming download task corresponding to a compressed data block being used to download the compressed data block.

[0092] The specific process of the terminal device responding to the above download request to generate the streaming download task set can also be found in the following Figure 5 Corresponding to the relevant description in the embodiment.

[0093] The present application can compress the target data using a mixed compression method to obtain the above-mentioned compressed data packet, so that M compressed data blocks of the compressed data packet can be obtained through the mixed compression method. This process can be performed by the data server and can be performed in advance by the data server, so as to further improve the download efficiency of the compressed data packet, as described below.

[0094] The mixed compression method may include a first compression method and a second compression method. The compression ratio of the first compression method is smaller than the compression ratio of the second compression method. The compression ratio may be the ratio of the data size of the compressed data to the data size of the original data before compression. It can be understood that the amount of data after the data is compressed using the first compression method is less than the amount of data after the data is compressed using the second compression method.

[0095] A compression method with a relatively low compression ratio, such as the first compression method, is a compression method that does not support splitting, that is, the compressed data does not support splitting. For example, the first compression method may be a zip (a file compression format) compression method. Zip compression is a file compression technology used to merge multiple files or folders into a compressed file (such as a compressed data package) to save storage space.

[0096] The above-mentioned second compression method can be a compression method that supports splitting, that is, the compressed data supports being split (such as being transmitted after splitting). For example, the second compression method can be a Stored (a file compression storage format) compression method. The Stored compression method can be a one-to-one compression method, that is, the data amount of the data before compression and the data amount of the compressed data can be the same.

[0097] The target data may include K file data (also referred to as files), where K is a positive integer. The specific value of K may be determined based on the actual application scenario. Each of the K file data may have its own file compression ratio. Any of the K file data may be the target file data. Since the method for calculating the file compression ratio of each file data is the same, the calculation of the file compression ratio of the target file data is used as an example for description.

[0098] Here, the file compression ratio of each file data may be the file compression ratio of each file data under the first compression method. Therefore, the target file data has a file size before compression using the first compression method (i.e., the data volume of the target file data), which can be referred to as the original file size. The target file data also has a file size after compression using the first compression method (i.e., the data volume of the target file data after compression using the first compression method), which can be referred to as the compressed file size.

[0099] Therefore, the data server can obtain the original file size and the compressed file size of the target file data, and can use the ratio of the compressed file size to the original file size as the file compression ratio of the target file data. Therefore, if the file compression ratio is large, it indicates that the target file data does not differ much before and after being compressed using the first compression method, that is, the compression effect is not significant. In this case, the target file data can be compressed without using the first compression method. Conversely, if the file compression ratio is small, it indicates that the target file data does not differ much before and after being compressed using the first compression method, that is, the compression effect is significant. In this case, the target file data can be compressed using the first compression method.

[0100] Using the above method, the data server can obtain the file compression ratio of each file data in the target data. File data with a file compression ratio less than a preset ratio threshold can be referred to as first file data, and file data with a file compression ratio greater than or equal to the ratio threshold can be referred to as second file data. The ratio threshold can be any preset threshold that meets actual needs and is used to evaluate the effectiveness of compressing file data using the first compression method.

[0101] Alternatively, optionally, the second file data may be file data having a file compression ratio greater than or equal to the ratio threshold, and a corresponding compressed file size (i.e., the data size after compression processing using the first compression method) greater than or equal to a set size threshold. Other file data among the above-mentioned multiple file data except for the second file data may be the first file data, i.e., the first file data may be file data having a file compression ratio less than the ratio threshold, or a corresponding compressed file size less than the set size threshold.

[0102] Alternatively, the second file data may be file data whose corresponding compressed file size is greater than or equal to the set size threshold, and the first file data may be file data whose corresponding compressed file size is smaller than the set size threshold.

[0103] Therefore, the present application can compress the first file data using a first compression method to obtain compressed file data of the first file data, and can compress the second file data using a second compression method to obtain compressed file data of the second file data. A first file data has a compressed file data after being compressed using the first compression method, and a second file data has a compressed file data after being compressed using the second compression method.

[0104] Therefore, the compressed data packet may include compressed file data obtained by compressing the first file data using the first compression method, and compressed file data obtained by compressing the second file data using the second compression method.

[0105] The compressed file data obtained by compressing the first file data using the first compression method does not support splitting, that is, the compressed file data corresponding to the first file data does not support splitting; while the compressed file data obtained by compressing the second file data using the second compression method supports splitting, that is, the compressed file data corresponding to the second file data supports splitting.

[0106] Since the compressed file data corresponding to the first file data does not support being split, the compressed file data corresponding to a first file data can be a compressed file block in the compressed data packet; and the compressed file data corresponding to the second file data supports being split, therefore, the present application can split the compressed file data corresponding to the second file data, and can obtain multiple compressed file blocks of the compressed file data corresponding to the second file data. The compressed file data corresponding to each second file data can be split to obtain multiple compressed file blocks of the compressed file data corresponding to each second file data. A compressed file block obtained by splitting the compressed file data of a second file data can be a compressed file block in the compressed data packet.

[0107] In other words, a compressed data block in the compressed data packet may be compressed file data corresponding to a first file data, or a compressed file block obtained by segmenting compressed file data corresponding to a second file data.

[0108] By adopting the above process of the present application, for the second file data whose compression effect is not obvious using the first compression method, a second divisible compression method can be used for compression, which allows the compressed file data corresponding to the second file data to be further divided into multiple compressed file blocks, and the multiple compressed file blocks can be downloaded using the above N threads (i.e., multi-threaded downloading).

[0109] By adopting the above-mentioned hybrid compression method of the present application, different compression methods can be adaptively adopted for compression processing of different file data that need to be compressed together according to actual business needs, thereby achieving different business effects. For example, if the first file data is file data with a file compression ratio less than a set ratio threshold, and the second file data is file data with a file compression ratio greater than or equal to the ratio threshold, the following business effects can be achieved: This method focuses on the different compression effects on different file data. For the second file data with an unobvious compression effect, the second compression method in which the compressed data can be split can be adopted for compression processing, so that the multiple compressed file blocks obtained by splitting the second file data can be naturally synchronized through multiple threads of the present application (such as the N threads mentioned above) for multi-threaded downloading.

[0110] For another example, if the first file data is file data with a file compression ratio less than the ratio threshold, or the corresponding compressed file size is less than the set size threshold, and the second file data is file data with a file compression ratio greater than or equal to the ratio threshold, and the corresponding compressed file size is greater than or equal to the set size threshold, then the following business effect can be achieved: This method comprehensively considers the file compression ratio of the file data and the size of the compressed file after compression. For the second file data with an unobvious compression effect and a large compressed file size after compression, a second compression method in which the compressed data can be split can be used for compression processing, so that the multiple compressed file blocks obtained by splitting the second file data can also be naturally downloaded synchronously through multiple threads of the present application.

[0111] For example, if the first file data is file data whose corresponding compressed file size is greater than or equal to the set size threshold, and the second file data is file data whose corresponding compressed file size is less than the set size threshold, the following business effect can be achieved: This method focuses on the compressed file size after the file data is compressed. For the second file data whose data size is still large after compression, a second compression method in which the compressed data can be split can be used for compression processing, so that the second file data can be split into multiple compressed file blocks, which can be downloaded synchronously by multiple threads of this application, that is, multi-threaded synchronous and extremely fast download can be achieved for one second file data.

[0112] In summary, the present application innovatively uses the above-mentioned hybrid compression method, so that for multiple file data that need to be compressed together, the compressed file data corresponding to some file data (such as the first file data) can subsequently be streamed downloaded and decompressed as a whole through a single assigned thread, while the multiple compressed data blocks of the compressed file data corresponding to some file data (such as the second file data) can subsequently be streamed downloaded and decompressed in blocks through multiple assigned threads, which can greatly improve the overall efficiency of downloading compressed data packets and reduce the overall time spent on downloading and decompressing compressed data packets.

[0113] In addition, it should be noted that the above-mentioned hybrid compression method of the present application also has very outstanding technical effects when applied in actual application scenarios. For example, in the application scenario of games, the game installation package (i.e., installation data) is usually obtained by compressing a large number of program data (one program data can be a file data) together, and the data size of the large number of program data varies. The effect of compression using the first compression method is also good or bad. For program data with very large data size or program data with very poor compression effect, it will directly and seriously affect the download and decompression time of the entire game installation package. Therefore, the above-mentioned hybrid compression method of the present application can achieve extremely fast download of the game installation package, allowing relevant game developers to download and test the latest version of the game installation package at extremely fast speed every day, and allowing relevant game players to also download and install the game installation package (which can also be the game update package) at extremely fast speed, thereby greatly improving the user experience of downloading and using the game installation package.

[0114] Optionally, when segmenting the compressed file data corresponding to the second file data, the segmentation can be performed based on a set target data size. That is, the data size of each compressed file block obtained by segmenting the compressed file data corresponding to the second file data can be the target data size. In particular, when the data size of the compressed file data corresponding to the second file data is not an integer multiple of the target data size, the data size of the last compressed file block obtained by segmenting the compressed file data corresponding to the second file data can also be smaller than the target data size. The target data size can be any set data size that meets actual needs and is used for data segmentation.

[0115] Step S102: Create N threads for executing streaming download tasks, and allocate a streaming download task to be executed to each thread based on the streaming download task set; N is a positive integer.

[0116] Optionally, the terminal device can create N threads for executing streaming download tasks, where N is a positive integer and the specific value of N can be determined based on the actual application scenario. In actual application scenarios, N can be a moderate value to ensure the performance of data downloading, such as N can be any value between 3 and 5. The present application can execute (i.e., process) the streaming download tasks in the above-generated streaming download task set through the created N threads, as described below.

[0117] The terminal device may allocate the streaming download tasks to be executed by each of the N threads using the streaming download task set. A thread may sequentially execute one or more assigned streaming download tasks. Optionally, the terminal device may allocate the streaming download tasks to be executed by the N threads in the following manner:

[0118] In order to ensure the efficiency of using N threads to execute the streaming decompression task set, the terminal device can also obtain the data size of each compressed data block in the compressed data packet after generating the streaming download task set. The data size of each compressed data block is the data volume of each compressed data block. For example, the data size of a compressed data block can be 20mb (20 megabytes). The data size of each compressed data block here can be obtained when creating the streaming download task set. For details on how to obtain it, please refer to the following Figure 6 Corresponding to the relevant description in the embodiment.

[0119] The terminal device can sort the streaming download tasks corresponding to each compressed data block in descending order of the data size of each compressed data block to obtain a sorted streaming download task, that is, the sorted streaming download task can include M streaming download tasks that are sorted in descending order of the data size of the corresponding compressed data blocks.

[0120] Then, the terminal device can add the sorted streaming download tasks to a task pool, which can be used to store unprocessed (understood as unassigned) streaming download tasks. The task pool can initially contain M sorted streaming download tasks. Subsequently, the terminal device can assign the streaming download tasks in the task pool to each thread.

[0121] Any one of the N threads can be the target thread. The following is a specific description using the process of allocating streaming download tasks to the target thread as an example. It can be understood that the principle of allocating streaming download tasks to each thread in the N threads can be the same.

[0122] When the target thread is idle (e.g., not executing a streaming download task), the terminal device can extract the first streaming download task from the remaining streaming download tasks in the task pool and assign the extracted streaming download task to the target thread. That is, the streaming download task corresponding to the compressed data block with the largest data size among the remaining unprocessed streaming download tasks can be assigned to the currently idle thread for execution. This means that the lost download task assigned to the corresponding thread can be removed from the task pool and will not be subsequently assigned to other threads for execution.

[0123] Optionally, the target thread may process the idle state after executing a streaming download task assigned, or when the streaming download task has not yet been executed initially. Therefore, when the target thread is in the idle state, the remaining streaming download tasks in the task pool may include M streaming download tasks, or include less than M streaming download tasks. For example, the sorted M streaming download tasks initially added in the task pool may include streaming download task r1, streaming download task r2, streaming download task r3, streaming download task r4, and streaming download task r5. When the target thread is in the idle state, the task pool may include the streaming download tasks r3, streaming download task r4, and streaming download task r5 that are sorted in sequence, indicating that streaming download task r1 and streaming download task r2 have been executed or are being executed by other threads. Then, the streaming download task r3 that is sorted at the front may be taken out as the streaming download task that is currently assigned to the target thread and needs to be executed.

[0124] It is understandable that, generally, the number of threads N is less than (or may be equal to) the total number M of streaming download tasks that need to be executed. For example, the number of threads N may be equal to 5, while the number M of streaming download tasks in the streaming download task set may be as high as 100. Therefore, each thread may be continuously assigned the streaming download tasks it needs to execute. A thread may process one streaming download task at a time. After a thread completes processing a streaming download task, it may be assigned the next streaming download task to be executed, and so on, until all streaming download tasks in the streaming download task set are completed.

[0125] Since the larger the data size of the compressed data block, the longer the download time will be, the method provided in the present application can allow each thread to give priority to processing the compressed data block with a larger data size, so that while some threads are downloading compressed data blocks with larger data sizes, other threads downloading compressed data blocks with smaller data sizes can process multiple compressed data blocks with smaller data sizes at the same time. This can reduce the overall time spent on downloading compressed data packets, thereby improving the overall efficiency of downloading compressed data packets and allowing each thread to process various streaming download tasks more evenly.

[0126] Step S103, calling N threads to execute their respective streaming download tasks, so as to download the compressed data blocks corresponding to each streaming download task from the data server, and perform real-time streaming decompression processing on the downloaded compressed data blocks; wherein, the target data is obtained after the M compressed data blocks are stream decompressed.

[0127] Optionally, the terminal device can call the above-mentioned N threads to execute their own streaming download tasks respectively, so as to download the compressed data blocks corresponding to each streaming download task from the data server through each thread respectively, and can perform real-time streaming decompression processing on the downloaded compressed data blocks, that is, the process of one thread executing a streaming download task can be the process of downloading and real-time decompressing the compressed data blocks corresponding to the streaming download task from the data server.

[0128] Here, the process of the target thread (any thread) executing the streaming download task is also used as an example for description. The streaming download task that the target thread needs to execute may include the target streaming download task, that is, the streaming download task assigned to the target thread may include the target streaming download task, and the target streaming download task may be any streaming download task assigned to the target thread for execution. Therefore, the process of the target thread executing the target streaming download task is used as an example for description.

[0129] The target streaming download task may encapsulate the offset information of the corresponding compressed data block in the compressed data packet. The offset information may be used to indicate the data position of the compressed data block corresponding to the target streaming download task in the compressed data packet. For example, the offset information may include (or be used to calculate) the data starting position and data ending position of the compressed data block corresponding to the target streaming download task in the compressed data packet. The compressed data block is the continuous compressed data segment between the data starting position and the data ending position in the compressed data packet.

[0130] Therefore, the terminal device can call the target thread to extract the offset information of the corresponding compressed data block in the compressed data packet from the target streaming download task, and can call the target thread to download the corresponding compressed data block from the compressed data packet at the data server through the extracted offset information, that is, download the compressed data at the position indicated by the offset information extracted from the compressed data packet (the compressed data between the above-mentioned data starting position and data ending position).

[0131] See Figure 3 , Figure 3 This is a schematic diagram of a scenario for executing a streaming download task set provided by an embodiment of the present application. Figure 3 As shown, Large can indicate that the data size of the corresponding compressed data block is large, Middle can indicate that the data size of the corresponding compressed data block is medium, and Small can indicate that the data size of the corresponding compressed data block is relatively small. Figure 3 The rectangular box used to represent the compressed data block can reflect the data size of the compressed data block. The longer the rectangular box is, the larger the data size of the compressed data block it represents.

[0132] Depend on Figure 3It can be seen that when the streaming download task set is completed depends largely on the execution time of the streaming download task corresponding to the compressed data block with the largest data size. The execution time of the streaming download task corresponding to the compressed data block with a larger data size will also be longer. By adopting the above-mentioned method provided by the present application, that is, each thread gives priority to executing the streaming download task corresponding to the compressed data block with a larger data size, each thread can process each streaming download task more evenly. While one thread is processing the streaming download task corresponding to the compressed data block with a large data size, other threads can also synchronously execute multiple streaming download tasks corresponding to the compressed data blocks with smaller data sizes, so that the time spent on processing the entire streaming download task set will be minimized and the network bandwidth can be utilized to the greatest extent.

[0133] Among them, when a thread executes a streaming download task, it can download the compressed data block corresponding to the streaming download task part of the data then part of the data, and each downloaded part of the data can be the local compressed data in the compressed data block, and the downloaded part of the data can be streamed decompressed in real time, and then the decompressed data corresponding to the part of the data can be written to the disk in real time (such as stored in the disk of the terminal device in real time). After the decompressed data corresponding to the part of the data is successfully written to the disk, the part of the data can be deleted, and the next part of the data can be downloaded for the compressed data block, and so on, until all the decompressed data corresponding to the compressed data block are written to the disk. This process is the process of streaming downloading and streaming decompression of compressed data blocks, that is, the execution of a streaming download task can include the execution of two operations: streaming downloading of the corresponding compressed data blocks and real-time streaming decompression of the streamed compressed data blocks. The specific process can also be referred to below Figure 9 Corresponding to the relevant description in the embodiment.

[0134] The above-mentioned target data can be obtained after all the M compressed data blocks are stream-decompressed. Specifically, after all the M compressed data blocks are stream-decompressed, the M decompressed data blocks corresponding to the M compressed data blocks are written to the disk. A decompressed data block corresponding to a compressed data block is the decompressed data that can be obtained after the compressed data block is decompressed. Therefore, the M decompressed data blocks written to the disk constitute the target data written to the disk. It can be seen that by adopting the streaming download and decompression method of the present application, the decompression processing of each compressed data block of the compressed data packet can be completed at the same time as the download of each compressed data block of the compressed data packet is completed, so as to obtain the decompressed data of the decompressed data packet (i.e., the target data). Therefore, by adopting the method provided by the present application, the time spent on decompressing each compressed data block of the downloaded compressed data packet can be reduced, thereby improving the efficiency of obtaining the target data.

[0135] See Figure 4 , Figure 4 This is a schematic diagram of a framework for downloading compressed data packets provided by an embodiment of the present application. Figure 4 As shown, in the initial basic pipeline, a large zip package can be produced (i.e., generated) first. The large zip package can be the initial compressed data packet obtained after compressing the target data. Then, the external resources of the initial compressed data packet can be trimmed, that is, irrelevant resources in the initial compressed data packet can be trimmed and removed to obtain a compressed data packet with resource trimming.

[0136] For example, in a game testing scenario, if only a certain part of the game's functions needs to be tested, then all resources in the initial compressed data packet except for the resources related to the part of the functions to be tested can be removed as external resources.

[0137] The compressed data packet for resource cropping can be compressed as a whole using the first compression method. In the present application, the compression format of some large files in the compressed data packet for resource cropping can be modified. For example, the compressed data belonging to the second file data in the compressed data packet can be expressed as the compression format corresponding to the second compression method. That is, the mixed compression method is used to compress the various file data in the target data, and the compressed data packet to be downloaded can be obtained. The compressed data packet can then be uploaded to Dolphin. Dolphin here can represent the data server. The compressed data packet is divided into the M compressed data blocks.

[0138] Furthermore, the terminal device can read the description information of each compressed data block from the data server, thereby generating a streaming download task corresponding to each compressed data block. The terminal device can also parse the information of each generated streaming download task (such as parsing the data size of the corresponding compressed data block), and can use the parsed results to sort the streaming download tasks in descending order of the data size of the corresponding compressed data block, and add the sorted streaming download tasks to the task list (which can be understood as adding them to the task pool).

[0139] The terminal device can also create N threads for executing streaming download tasks, and the N threads can be stored in a thread pool. The terminal device can assign streaming download tasks to the threads in the idle state among the N threads. Each thread can receive the streaming download task assigned to it and execute the assigned streaming download task. In the process of executing the streaming download task, each thread can realize the extraction (such as downloading) and streaming decompression of the corresponding compressed data blocks, and directly write the decompressed data to the disk in real time based on the baseline directory. After the execution of the above-mentioned M streaming download tasks is realized, the target data is written to the disk. The baseline directory can be obtained by parsing the description information of each compressed data block, and the baseline directory can contain the disk location of the decompressed data corresponding to each compressed data block.

[0140] Furthermore, the data server can be a server in a CDN network (content delivery network). When the terminal device of the present application sends a download request, the CDN network can redirect the download request to a server that is geographically closest to the terminal device. This server can then be used as the server for the terminal device to download the compressed data packet. In the subsequent download of the compressed data packet, data can be read and downloaded from the same server without switching to other servers in the CDN network for data reading and downloading, thereby ensuring the correct execution of the compressed data packet download service.

[0141] The relevant configuration parameters in this application (including the number of threads, the size and number of blocks of compressed file data that can be split into the second file data, etc.) can be adaptively adjusted according to the actual application scenario to achieve the optimal effect of data downloading and decompression.

[0142] The present application can generate a streaming download task set in response to a download request for a compressed data packet; the compressed data packet is stored in a data server, the compressed data packet is obtained after compressing the target data, the compressed data packet contains M compressed data blocks, the streaming download task set contains M streaming download tasks, and each compressed data block corresponds to a streaming download task; and N threads can be created for executing streaming download tasks, and based on the streaming download task set, each thread is assigned the streaming download task to be executed; N and M are both positive integers; thus, N threads can be called to execute their respective streaming download tasks, so as to download the compressed data blocks corresponding to each streaming download task from the data server respectively, and perform real-time streaming decompression on the downloaded compressed data blocks; wherein, the target data is obtained after the M compressed data blocks are stream-decompressed. It can be seen that the method proposed in this application can realize the rapid download of compressed data packets through multiple threads, and can realize real-time decompression processing of downloaded compressed data blocks through streaming download and decompression. Therefore, this application combines the multi-threading and process download and decompression methods to simultaneously improve the efficiency of compression and decompression processing of compressed data packets.

[0143] See Figure 5 , Figure 5 This is a flow chart of a method for generating a streaming download task set provided by an embodiment of the present application. Figure 5 As shown, the method may include:

[0144] Step S201: In response to a download request for a compressed data package, obtain a download address of the compressed data package.

[0145] Optionally, the terminal device can obtain the download address of the compressed data packet in response to a download request for the compressed data packet. Optionally, the download address may be pre-stored in the terminal device. In this case, the terminal device can directly extract the stored download address in response to the download request. Alternatively, the download address may not be pre-stored in the terminal device. In this case, the terminal device can obtain the download address of the compressed data packet from the data server after responding to the download request.

[0146] The download address may include the data address of the compressed data packet in the data server.

[0147] Step S202: Read the description information of each compressed data block from the data server based on the download address; each compressed data block has corresponding description information.

[0148] Optionally, the terminal device can read the description information of each compressed data block from the data server through the above download address. A compressed data block can have corresponding description information. The description information of a compressed data block can include compression-related information used to describe the compressed data block. The description information can be called ZipEntry.

[0149] Optionally, the description information of any compressed data block may include at least one of the following: the offset information of any compressed data block in the compressed data packet (the offset information is used to indicate the data position of any compressed data block in the compressed data packet); the data size of any compressed data block (i.e., the amount of data after decompression); the disk location (i.e., the location to be stored, which may be the location to be stored in the disk of the terminal device) of the data after any compressed data block is decompressed (i.e., the decompressed data corresponding to any compressed data block); the data identifier (such as the file name) of the data after any compressed data block is decompressed. Generally, the description information of any compressed data block may include all types of information here.

[0150] Optionally, the description information of each compressed data block in the compressed data packet may be present in the compressed data packet in the form of a linked list. The description information of each compressed data block in the form of a linked list may be located in front of the compressed data block to which it belongs (at the head position) in the compressed data packet. The description information of each compressed data block in the compressed data packet may also be located at the tail position of the compressed data packet. That is, the description information of each compressed data block may be present in two copies in the compressed data packet, one copy being present in the form of a linked list and the other copy being present at the tail position in the compressed data packet.

[0151] Therefore, the present application can read the description information of each compressed data block from the compressed data packet based on the linked list, or can also read the description information of each compressed data block from the tail position in the compressed data packet. The following describes these two methods in detail.

[0152] The process of reading the description information of each compressed data block based on the linked list may include: the terminal device can read the M description information of the above-mentioned M compressed data blocks in sequence from the linked list through the download address of the above-mentioned compressed data packet, and in the process of reading the description information, the reading of the compressed data block to which the read description information belongs can be skipped, thereby improving the efficiency of reading the description information of each compressed data block.

[0153] The specific process of reading the descriptive information of each compressed data block from the linked list may include: the terminal device may first read the first descriptive information in the linked list. After reading the first descriptive information, the information position of the first descriptive information in the compressed data packet may be obtained, and the linked list will return (this is the function of the linked list itself) an estimated position (i.e., a rough position) of the next descriptive information to be read. The terminal device may calculate the information position of the next descriptive information to be read (i.e., the second descriptive information) in the compressed data packet through the estimated position and the information position of the first descriptive information read, so that the terminal device may continue to read the second descriptive information based on the calculated position information, and so on, until all the descriptive information in the linked list is read.

[0154] When reading the description information of each compressed data block based on the linked list, the terminal device needs to establish an HTTP connection (which can be called a data connection) with the data server each time it reads a description information. Therefore, in the process of reading the description information of each compressed data block based on the linked list, the terminal device needs to establish multiple data connections with the data server.

[0155] See Figure 6 , Figure 6 This is a schematic diagram of a scenario for reading description information of compressed data blocks provided by an embodiment of the present application. Figure 6 As shown, a compressed data packet can contain multiple Data (such as Data1 to DataM, a total of M Data), and the tail position of the compressed data packet can have a CENTRAL DIRECTORY, which can contain the description information of each compressed data block. A Data can contain a compressed data block and the description information of the compressed data block.

[0156] Therefore, in this application, the description information of each compressed data block can be read sequentially based on the linked list, such as reading description information 1, description information 2, description information 3, ..., description information M in sequence, and in the process of reading the description information, the reading of the compressed data block immediately following the read description information can be skipped, thereby improving the reading efficiency of the M description information. If the following tail prediction method is used to read the description information of each compressed data block, the following first-size memory block can be used to read the compressed data in the CENTRAL DIRECTORY area of the compressed data packet. For details, please refer to the following description.

[0157] See also Figure 7 , Figure 7 This is a schematic diagram of a scenario for determining the information location of description information provided by an embodiment of the present application. Figure 7As shown, after the terminal device reads a description information based on the linked list, the linked list will return the estimated position of the next description information for reading the description information. The estimated position is the position before the next description information in the compressed data packet. Here, the estimated position can be the position of byte 4 in the compressed data packet. Therefore, the terminal device can start traversing the subsequent bytes from the byte at the estimated position (i.e., byte 4). Each time a byte is traversed, the terminal device can perform corresponding data verification (such as cyclic redundancy verification) on the byte traversed, combining the above estimated position and the position of the read description information. If the verification passes, it indicates that the position of the byte currently traversed is the starting position of the next description information to be read. The ending position of the next description information can be further calculated based on the starting position. The next description information can be read based on the starting position and the ending position. If the verification fails, it indicates that the position of the byte currently traversed is not the starting position of the next description information to be read. The terminal device can continue to traverse the next byte and perform corresponding data verification again. This process is repeated until the corresponding data verification of the byte traversed passes, and the position of the next description information to be read can be found. Here, the starting position of the next description information to be read may be at byte 7, and the next description information can be read from byte 7.

[0158] In addition, the present application innovatively proposes a method for reading data from the back to the front in a compressed data packet to read the description information of each compressed data block at the tail position in the compressed data packet. The process may include:

[0159] The terminal device can establish a data connection with a data server and, through the data connection, use a memory block of a first size to read compressed data in a compressed data packet from the data server one or more times from back to front (i.e., read the compressed data packet one or more times). The size of the compressed data obtained by any data read is the first size, i.e., compressed data of the first size can be obtained from the compressed data packet at one time. The first size can be any suitable data size set to meet actual needs.

[0160] Each time the terminal device uses the memory block of the first size to read the compressed data of the first size in the compressed data packet, it can determine whether all the compressed data read by the memory block before contains M description information. If not, the terminal device can continue to read the compressed data packet for the next time to read more compressed data from the tail position of the compressed data packet forward; if it does, it indicates that the description information of all the compressed data blocks has been read through the memory block, and the data reading of the remaining compressed data in the compressed data packet (i.e., the remaining unread compressed data) can be stopped, and the M description information in the read compressed data is the description information of the above-mentioned M compressed data blocks.

[0161] The above-mentioned method of reading description information can be called the tail prediction method. By adopting the tail prediction method, the data in the compressed data packet is read and judged multiple times based on a fixed-size memory block. In this way, the terminal device only needs to establish a data connection with the data server once to read the description information of all compressed data blocks, thereby reducing the overhead and number of times the terminal device establishes a data connection with the data server when reading the description information.

[0162] The specific method for reading the description information of each compressed data block can be set according to actual needs, and this application does not impose any restrictions on this.

[0163] See Figure 8 , Figure 8 This is a schematic diagram of a scenario for establishing a data connection provided by an embodiment of the present application. Because the original data connection used for reading data (such as the HTTP stream here) does not support addressable data downloading (such as downloading only part of the data in a compressed data packet through addressing, where the part of the data may be compressed data blocks), the present application can build an addressable intermediate layer (such as the Http Seek Able Stream here). Furthermore, the original data connection can be modified through the addressable intermediate layer to generate a new addressable data connection. This new addressable data connection can be used as the data connection for reading and downloading data in the present application.

[0164] like Figure 8 As shown, the original data connection used for reading the description information can be modified through the addressable intermediate layer in this application to obtain a recreated data connection. The recreated data connection can be an addressable data connection (such as the Zip Input Stream used for reading the description information here), and the description information in the compressed data packet can be read through the addressable data connection.

[0165] For example, for the original data connection used to download compressed data blocks, the present application can also adopt the addressable intermediate layer and use the offset information of the compressed data block to transform the original data connection to obtain a new addressable data connection. Through the new addressable data connection, the addressed download of the compressed data block from the data compression package can be realized.

[0166] Step S203: Based on the description information of each compressed data block, a streaming download task corresponding to each compressed data block is generated.

[0167] Optionally, the terminal device can generate streaming download tasks corresponding to each compressed data block based on the descriptive information of each compressed data block read above. One compressed data block can correspond to one streaming download task, and the streaming download task corresponding to one compressed data block can encapsulate the descriptive information of the compressed data block. Therefore, when each thread subsequently processes the assigned streaming download task, it can extract and use the information used to download the corresponding compressed data block (including the information in the descriptive information of the corresponding compressed data block, such as the offset information of the corresponding compressed data block in the compressed data packet, etc.) from the processed streaming download task. The data size of each compressed data block used in the above-mentioned sorting of the streaming download tasks corresponding to each compressed data block can also be obtained from the descriptive information of each compressed data block read above.

[0168] By adopting the above-mentioned method of the present application, that is, by reading the description information of each compressed data block, the corresponding generation of streaming download tasks corresponding to each compressed data block is realized. Subsequently, the above-mentioned N threads can be used to execute the generated streaming download tasks in parallel to realize multi-threaded download of compressed data packets, thereby ensuring efficient download of compressed data packets.

[0169] See Figure 9 , Figure 9 This is a flow chart of a data downloading and decompression method provided by an embodiment of the present application. Figure 9 As shown, the method may include:

[0170] Step S301, calling the target thread to execute the target streaming download task based on the cache block of the second size, so as to download the target compressed data block multiple times from the data server, and each data download is used to obtain a local compressed data of the target compressed data block, and the size of the local compressed data is less than or equal to the second size.

[0171] Optionally, any one of the above-mentioned N threads can be the target thread, and the streaming download task that the target thread needs to execute includes the target streaming download task. The target streaming download task can be any streaming download task assigned to the target thread for processing, and the compressed data block corresponding to the target streaming download task can be called the target compressed data block.

[0172] Since the principles of each thread executing its own streaming download task are the same, the embodiment of the present application will be specifically described by taking the process of the target thread executing the target streaming download task as an example.

[0173] The terminal device may have a cache block of a second size, which may be a cache block for downloading compressed data blocks. The second size may be determined according to the actual application scenario, such as the second size may be 4kb (kilobytes).

[0174] The present application can use the cache block of the second size to implement streaming download and streaming decompression of compressed data blocks, as described below.

[0175] The terminal device can invoke a target thread to execute a target streaming download task using the cache block of the second size, thereby downloading the target compressed data block from the data server multiple times. Each data download can be used to obtain a portion of compressed data of the target compressed data block, and the size of the portion of compressed data obtained by each data download is less than or equal to the second size. The multiple data downloads can achieve the download of all compressed data contained in the target compressed data block, that is, the multiple portion of compressed data obtained by the multiple data downloads can constitute the entire target compressed data block.

[0176] It is understood that, under normal circumstances, unless it is the last data download of the target compressed data block, the size of the downloaded partial compressed data can be equal to the second size. However, when the data size of the target compressed data block is not an integer multiple of the second size, the size of the partial compressed data obtained after the last data download of the target compressed data block is smaller than the second size. Therefore, it can be understood that when the data size of the target compressed data block is an integer multiple of the second size, the size of the partial compressed data obtained after the last data download of the target compressed data block can also be equal to the second size.

[0177] Each thread may have its own corresponding cache block of the second size, and each thread may implement parallel and streaming downloading of compressed data blocks corresponding to the executed streaming download task through its own corresponding cache block.

[0178] In step S302, the local compressed data obtained from each data download is subjected to real-time streaming decompression processing, and the local decompressed data obtained by decompressing the local compressed data is subjected to real-time disk writing processing; the multiple local decompressed data corresponding to the multiple local compressed data obtained from multiple data downloads constitute the decompressed data blocks corresponding to the target compressed data blocks.

[0179] Optionally, the terminal device may perform real-time streaming decompression processing on the local compressed data obtained from each data download, and may perform real-time disk processing on the local decompressed data obtained from the decompression of the local compressed data (e.g., storing the decompressed data to disk in real time). After a local compressed data obtained from a data download is decompressed, a corresponding local decompressed data may be obtained. The multiple local decompressed data corresponding to the multiple local compressed data obtained from the multiple data downloads of the target compressed data block may constitute the decompressed data block corresponding to the target compressed data block. The decompressed data block is the decompressed data obtained after the target compressed data block is decompressed.

[0180] Wherein, the above-mentioned multiple data downloads may include the i-th data download, i is a positive integer, and the i-th data download may be any data download except the last data download in the multiple data downloads. After the target compressed data block is downloaded for the i-th time through the cache block of the second size, the local compressed data obtained by the i-th data download is cached in the cache block. The terminal device may perform real-time streaming decompression processing on the local compressed data obtained by the i-th data download of the target compressed data block in the cache block to obtain local decompressed data corresponding to the local compressed data. The terminal device may write the obtained local decompressed data to the disk of the terminal device in real time. After the local decompressed data is successfully written to the disk, the terminal device may write the local decompressed data obtained by the i-th data download to the disk of the terminal device. The compressed data is deleted from the cache block, that is, the local compressed data cached in the cache block is cleared. Thus, the terminal device can use the cache block after data deletion (that is, the cache block after data clearing, that is, the cache block from which the local compressed data obtained by the i-th data download is deleted) to continue the i+1-th data download of the target compressed data block, that is, continue to download the next data of the target compressed data block, that is, continue to download the next local compressed data of the target compressed data block, and so on, until all the compressed data in the target compressed data block is downloaded and decompressed. It can be understood that after the cache block is full (for example, after the local compressed data of the second size is cached), the cached data can be immediately decompressed and written to the disk in real time.

[0181] Through the above principle, the terminal device can call N threads to implement streaming download and streaming decompression of M compressed data blocks, and finally achieve the storage of M decompressed data blocks corresponding to the M compressed data blocks. The M decompressed data blocks stored on the disk constitute the entire target data stored on the disk.

[0182] See Figure 10 , Figure 10 This is a schematic diagram of a data streaming download and decompression scenario provided by an embodiment of the present application. Figure 10 As shown, the target compressed data block may sequentially include partial compressed data 1, partial compressed data 2, partial compressed data 3, partial compressed data 4, and partial compressed data 5. The data sizes of partial compressed data 1 to 4 may all be the second size, and the data size of partial compressed data 5 may be smaller than the second size. It should be noted that the partial compressed data of the target compressed data block may not be obtained by segmentation (i.e., the target compressed data block is not segmented), but may be generated during the download process.

[0183] In the process of calling the target thread to download the target compressed data blocks using the above-mentioned second-size cache blocks, when the downloaded local compressed data 1 fills the second-size cache blocks, the local compressed data 1 in the cache blocks can be decompressed in real time to obtain the local decompressed data corresponding to the local compressed data 1, and the local decompressed data corresponding to the local compressed data 1 can be written to the disk in real time.

[0184] After the local decompressed data corresponding to the local compressed data 1 is successfully written to the disk, the local compressed data 1 cached in the cache block can be deleted (i.e., cleared), and the compressed data following the local compressed data 1 in the target compressed data block can be downloaded through the cache block from which the data was cleared. When the downloaded local compressed data 2 fills the cache block of the second size, the local compressed data 2 in the cache block can be decompressed in real time to obtain the local decompressed data corresponding to the local compressed data 2, and the local decompressed data corresponding to the local compressed data 2 can be written to the disk in real time.

[0185] Similarly, after the local decompressed data corresponding to the local compressed data 2 is successfully written to the disk, the local compressed data 2 cached in the cache block can be deleted (i.e., cleared), and the compressed data following the local compressed data 2 in the target compressed data block can be downloaded through the cache block from which the data was cleared. When the downloaded local compressed data 3 fills the cache block of the second size, the local compressed data 3 in the cache block can be decompressed in real time to obtain the local decompressed data corresponding to the local compressed data 3, and the local decompressed data corresponding to the local compressed data 3 can be written to the disk in real time.

[0186] After the local decompressed data corresponding to the local compressed data 3 is successfully written to the disk, the local compressed data 3 cached in the cache block can be deleted (i.e., cleared), and the compressed data following the local compressed data 3 in the target compressed data block can be downloaded through the cache block from which the data was cleared. When the downloaded local compressed data 4 fills the cache block of the second size, the local compressed data 4 in the cache block can be decompressed in real time to obtain the local decompressed data corresponding to the local compressed data 4, and the local decompressed data corresponding to the local compressed data 4 can be written to the disk in real time.

[0187] After the local decompressed data corresponding to the local compressed data 4 is successfully written to the disk, the local compressed data 4 cached in the cache block can be deleted (i.e., cleared), and the compressed data following the local compressed data 4 in the target compressed data block can be continued to be downloaded through the cache block where the data is cleared. When the downloaded local compressed data 5 fills the cache block of the second size, the local compressed data 5 in the cache block can be decompressed in real time to obtain the local decompressed data corresponding to the local compressed data 5, and the local decompressed data corresponding to the local compressed data 5 can be written to the disk in real time. At this point, the streaming download and streaming decompression of the entire target compressed data block are realized, and the local decompressed data corresponding to the local compressed data 1, the local decompressed data corresponding to the local compressed data 2, the local decompressed data corresponding to the local compressed data 3, the local decompressed data corresponding to the local compressed data 4, and the local decompressed data corresponding to the local compressed data 5 that are written to the disk constitute the target compressed data block written to the disk.

[0188] By adopting the above-mentioned streaming download and streaming decompression method of the present application, for the target thread, only a cache block of the second size is needed to realize the streaming download of the compressed data blocks corresponding to the target streaming download task to be executed (that is, local multiple consecutive downloads), and the downloaded local compressed data can be streamed and decompressed in real time. Therefore, for the terminal device, there is no need to free up a large amount of memory space to cache the integrated compressed data blocks that need to be downloaded, nor is there any need to free up a large amount of memory space to cache the decompressed data blocks after decompressing the entire compressed data blocks. Therefore, the memory space of the terminal device is greatly saved, and the downloading and decompression of the compressed data blocks are realized synchronously, so that when the download of each compressed data block is completed, the decompression of each downloaded compressed data block can also be realized synchronously, thereby reducing the time spent by the terminal device to decompress the downloaded compressed data packet.

[0189] The target data may be game installation data. After streaming decompression, the M compressed data blocks can be converted into M decompressed data blocks, each corresponding to a decompressed data block. These M decompressed data blocks constitute the installation data. Therefore, the terminal device can quickly install the corresponding game using the installation data and run the installed game.

[0190] During the development of terminal games, developers need to download and install the daily version of the compressed data package (i.e., the compressed data package of the game) to the terminal for experience every day. The compressed data package ranges from a few GB (gigabits) to tens of GB. Therefore, the method of the present application can solve the problems of slow download and decompression speed of the compressed data package and excessive memory space occupied by the terminal in this scenario, thereby improving the user experience of downloading and decompressing the compressed data package.

[0191] See Figure 11 , Figure 11 This is a schematic diagram of a multi-threaded download and decompression scenario provided by an embodiment of the present application. Figure 11 As shown, existing solutions, when using multi-threaded downloading of a large compressed package, need to download temporary files for each block of the compressed data package. After the download is complete, the temporary files for each block are merged to obtain the entire large compressed package. If the data size of the large compressed package is A, this process will require 2A of memory on the terminal device. Furthermore, the large compressed package needs to be decompressed to obtain the decompressed files of the large compressed package, which in turn requires more than 2A of memory on the terminal device. However, the multi-threaded plus streaming download and decompression solution of this application does not cause the terminal device to occupy memory during these two processes.

[0192] See also Figure 12 and Figure 13 , Figure 12 This is a comparative diagram of the time taken to download and decompress data provided by an embodiment of the present application. Figure 13: This is a comparative diagram of the space occupied during data download and decompression provided by an embodiment of the present application. The present application conducted a comparative experiment on the time consumption of multiple download and decompression methods, and the multiple download and decompression methods include a mixed multi-threaded streaming decompression method (that is, the first compression method and the second compression method are used in the present application to perform mixed compression on the compressed data packet, and a multi-threaded method is used for data streaming download and decompression), a multi-threaded streaming decompression method (that is, the present application does not use the first compression method and the second compression method to perform mixed compression on the compressed data packet, and a multi-threaded method is used for data streaming download and decompression), a single-threaded streaming decompression method (that is, a method of using one thread for data streaming download and decompression), a multi-threaded download and decompression method (that is, a method of using multiple threads for data download and decompression, which does not use streaming download and decompression) and a single-threaded download and decompression method (that is, a method of using one thread for data download and decompression, which does not use streaming download and decompression).

[0193] like Figure 12 As shown, the hybrid multi-threaded streaming decompression method provided by this application takes the least time to download and decompress the compressed data packet, taking only 2.9 minutes.

[0194] In addition, this application also conducted a comparative experiment on the space occupied by the various download and decompression methods. Figure 13 As shown, the target data can be the installation data of the APP (software), the APP size here can be the size of the memory space occupied after the APP is installed based on the installation data written to the disk, the decompressed Base size here can be the size of the memory space occupied by the target data obtained after streaming decompression, and the compressed package temporary space, that is, the space used to cache the compressed data in the compressed data packet during the download process.

[0195] The hybrid multi-threaded streaming decompression or multi-threaded streaming decompression provided by this application can achieve the minimum peak space occupancy of the terminal device, such as the minimum value can be 2.23GB plus 3.13GB here, a total of 5.36GB.

[0196] The above experimental results show that the method provided by this application can achieve the minimum time consumption for downloading and decompressing compressed data packets, and can achieve the minimum memory space occupied by the terminal device during the download and installation of compressed data packets.

[0197] See Figure 14 , Figure 14 This is a schematic diagram of the structure of a compressed data processing device provided in an embodiment of the present application. Figure 14 As shown, the compressed data processing device 140 may include: a generating module 1401 , an allocating module 1402 and a calling module 1403 .

[0198] Generating module 1401, configured to generate a streaming download task set in response to a download request for a compressed data packet; the compressed data packet is stored in a data server, is obtained by compressing target data, contains M compressed data blocks, and the streaming download task set contains M streaming download tasks, each compressed data block corresponds to one streaming download task, where M is a positive integer;

[0199] The allocation module 1402 is configured to create N threads for executing streaming download tasks and allocate the streaming download tasks to be executed to each thread based on the streaming download task set; N is a positive integer;

[0200] The calling module 1403 is used to call N threads to execute their respective streaming download tasks, so as to download the compressed data blocks corresponding to each streaming download task from the data server, and perform real-time streaming decompression processing on the downloaded compressed data blocks; wherein, the M compressed data blocks are subjected to streaming decompression processing to obtain the target data.

[0201] Optionally, the generation module 1401 generates a streaming download task set in response to a download request for a compressed data packet, including:

[0202] In response to a download request for a compressed data packet, obtaining a download address of the compressed data packet;

[0203] Based on the download address, the description information of each compressed data block is read from the data server; each compressed data block has corresponding description information;

[0204] Based on the description information of each compressed data block, a streaming download task corresponding to each compressed data block is generated.

[0205] Optionally, the description information of the M compressed data blocks is located at the tail position of the compressed data packet; the generation module 1401 reads the description information of each compressed data block from the data server based on the download address, including:

[0206] Establishing a data connection with a data server;

[0207] Based on the data connection, a memory block of the first size is used to read the compressed data in the compressed data packet from the data server one or more times in a back-to-front direction, and the size of the compressed data obtained by any data reading is the first size;

[0208] If the compressed data obtained by reading data using the memory block contains M description information, then the data reading of the remaining compressed data in the compressed data packet is stopped;

[0209] The description information of the M compressed data blocks includes M description information obtained by reading data.

[0210] Optionally, the description information of the M compressed data blocks exists in the compressed data packet in the form of a linked list;

[0211] The generation module 1401 reads the description information of each compressed data block from the data server based on the download address, including:

[0212] Based on the download address, M description information of the M compressed data blocks are read in sequence from the linked list, and in the process of reading the description information, reading of the compressed data block to which the read description information belongs is skipped.

[0213] Optionally, the description information of any compressed data block includes at least one of the following:

[0214] The offset information of any compressed data block in the compressed data packet; the data size of any compressed data block; the data size of any compressed data block after decompression; the disk location of the data after decompression of any compressed data block; the data identifier of the data after decompression of any compressed data block;

[0215] The offset information of any compressed data block in the compressed data packet is used to indicate the data position of any compressed data block in the compressed data packet.

[0216] Optionally, after generating the streaming download task set, the processing device 140 is further configured to:

[0217] Get the data size of each compressed data block in the compressed data packet;

[0218] Sorting the streaming download tasks corresponding to each compressed data block in descending order of the data size of each compressed data block to obtain sorted streaming download tasks;

[0219] The sorted streaming download tasks are added to a task pool; wherein the task pool is used to store unprocessed streaming download tasks.

[0220] Optionally, any one of the N threads is a target thread; the allocation module 1402 allocates the streaming download task to be executed to each thread based on the streaming download task set, including:

[0221] When the target thread is in an idle state, the streaming download task that is arranged at the front is taken out from the remaining streaming download tasks in the task pool;

[0222] Assign the extracted streaming download task to the target thread.

[0223] Optionally, any one of the N threads is a target thread, and the streaming download task to be executed by the target thread includes the target streaming download task;

[0224] The calling module 1403 calls N threads to respectively execute respective streaming download tasks, so as to download the compressed data blocks corresponding to the respective streaming download tasks from the data server, including:

[0225] The target thread is called to extract offset information of the corresponding compressed data block in the compressed data packet from the target streaming download task; the extracted offset information is used to indicate the data position of the compressed data block corresponding to the target streaming download task in the compressed data packet;

[0226] The target thread is called to download corresponding compressed data blocks from the compressed data packet at the data server based on the extracted offset information.

[0227] Optionally, the target data includes K file data, where K is a positive integer, and each file data in the K file data has its own file compression ratio;

[0228] The compressed data packet includes compressed file data obtained by compressing first file data using a first compression method, and compressed file data obtained by compressing second file data using a second compression method; the first file data is file data having a file compression ratio less than a preset ratio threshold, and the second file data is file data having a file compression ratio greater than or equal to the ratio threshold;

[0229] The compressed file data obtained by compressing the first file data using the first compression method does not support segmentation, and the compressed file data obtained by compressing the second file data using the second compression method supports segmentation.

[0230] Optionally, the M compressed data blocks include compressed file data of the first file data and a plurality of compressed file blocks obtained by segmenting the compressed file data of the second file data;

[0231] The compressed file data of a first file data is a compressed data block in the compressed data packet, and the compressed file block obtained by segmenting the compressed file data of a second file data is a compressed data block in the compressed data packet.

[0232] Optionally, any one of the K file data is target file data, and the target file data has an original file size before being compressed using the first compression method, and a compressed file size after being compressed using the first compression method;

[0233] The file compression ratio of the target file data is the ratio of the compressed file size to the original file size.

[0234] Optionally, any one of the N threads is a target thread, the streaming download task to be executed by the target thread includes a target streaming download task, and the compressed data block corresponding to the target streaming download task is a target compressed data block;

[0235] The calling module 1403 calls N threads to respectively execute respective streaming download tasks, so as to respectively download the compressed data blocks corresponding to the respective streaming download tasks from the data server, and performs real-time streaming decompression processing on the downloaded compressed data blocks, including:

[0236] Invoking a target thread to execute a target streaming download task based on a cache block of the second size, so as to download the target compressed data block multiple times from the data server, wherein each data download is used to obtain a local compressed data of the target compressed data block, and a size of the local compressed data is less than or equal to the second size; and

[0237] Perform real-time streaming decompression on the locally compressed data obtained from each data download, and write the locally decompressed data obtained from the decompression of the locally compressed data to disk in real time; the multiple locally decompressed data corresponding to the multiple locally compressed data obtained from multiple data downloads constitute the decompressed data blocks corresponding to the target compressed data blocks;

[0238] Among them, multiple data downloads include the i-th data download, i is a positive integer, and after the local decompressed data corresponding to the local compressed data obtained by the i-th data download is successfully written to the disk, the local compressed data obtained by the i-th data download is used to be deleted from the cache block, and the cache block after the data deletion is used to continue the i+1-th data download of the target compressed data block.

[0239] Optionally, the target data is game installation data. The M compressed data blocks are stream-decompressed to obtain M decompressed data blocks that are written to the disk. Each compressed data block corresponds to a decompressed data block. The M decompressed data blocks that are written to the disk constitute the installation data.

[0240] The processing device 140 is further configured to:

[0241] Install the game based on the installation data dropped to the disc and run the installed game.

[0242] According to one embodiment of the present application, Figure 2 The steps involved in the method for processing compressed data can be represented by Figure 14 The various modules in the compressed data processing device 140 shown are executed. For example, Figure 2The step S101 shown in FIG. Figure 14 The generation module 1401 in is executed, Figure 2 The step S102 shown in FIG. Figure 14 The allocation module 1402 is executed; Figure 2 The step S103 shown in FIG. Figure 14 The calling module 1403 in is executed.

[0243] The present application can generate a streaming download task set in response to a download request for a compressed data packet; the compressed data packet is stored in a data server, the compressed data packet is obtained after compressing the target data, the compressed data packet contains M compressed data blocks, the streaming download task set contains M streaming download tasks, and each compressed data block corresponds to a streaming download task; and N threads can be created for executing streaming download tasks, and based on the streaming download task set, each thread is assigned the streaming download task to be executed; N and M are both positive integers; thus, N threads can be called to execute their respective streaming download tasks, so as to download the compressed data blocks corresponding to each streaming download task from the data server respectively, and perform real-time streaming decompression on the downloaded compressed data blocks; wherein, the target data is obtained after the M compressed data blocks are stream-decompressed. It can be seen that the device proposed in this application can realize the rapid download of compressed data packets through multiple threads, and can realize real-time decompression processing of downloaded compressed data blocks through streaming download and decompression. Therefore, this application combines the multi-threading and process download and decompression methods to simultaneously improve the efficiency of compression and decompression processing of compressed data packets.

[0244] According to one embodiment of the present application, Figure 14 The various modules in the processing device 140 of compressed data shown can be individually or all combined into one or several units to constitute, or one (some) of the units can be further divided into multiple smaller sub-units in function, and the same operation can be achieved without affecting the realization of the technical effects of the embodiments of the present application. The above modules are divided based on logical functions. In actual applications, the functions of a module can also be implemented by multiple units, or the functions of multiple modules can be implemented by one unit. In other embodiments of the present application, the processing device 140 of compressed data can also include other units. In actual applications, these functions can also be implemented with the assistance of other units, and can be implemented by the collaboration of multiple units.

[0245] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program that has a predetermined function and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories) or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0246] According to one embodiment of the present application, a computer program capable of executing the steps involved in the corresponding methods shown in the various embodiments of the present application can be run on a general-purpose computer device (the computer device may include processing elements and storage elements such as a central processing unit (CPU), a random access memory medium (RAM), and a read-only memory medium (ROM)) to construct the following. Figure 14 The compressed data processing device 140 shown in FIG. The computer program can be recorded on a computer-readable recording medium, and can be loaded into the computer device through the computer-readable recording medium and executed therein.

[0247] See Figure 15 , Figure 15 This is a schematic diagram of the structure of a computer device provided in an embodiment of the present application. Figure 15 As shown, the computer device 1000 may include: a processor 1001, a network interface 1004 and a memory 1005. In addition, in some embodiments, the computer device 1000 may also include: a user interface 1003, and at least one communication bus 1002. The communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), a keyboard (Keyboard), and the user interface 1003 may optionally include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory, or a non-volatile memory (non-volatile memory), such as at least one disk memory. The memory 1005 may optionally be at least one storage device located away from the aforementioned processor 1001. As Figure 15 As shown, the memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module, and a device control application program.

[0248] exist Figure 15In the computer device 1000 shown, the network interface 1004 can provide network communication functions; the user interface 1003 is mainly used to provide an interface for user input; and the processor 1001 can be used to call the device control application stored in the memory 1005 to achieve:

[0249] In response to a download request for a compressed data packet, a streaming download task set is generated; the compressed data packet is stored in a data server, the compressed data packet is obtained by compressing the target data, the compressed data packet includes M compressed data blocks, and the streaming download task set includes M streaming download tasks, each compressed data block corresponds to a streaming download task, where N and M are both positive integers;

[0250] Create N threads for executing streaming download tasks, and assign each thread a streaming download task to execute based on the streaming download task set;

[0251] N threads are called to execute their respective streaming download tasks, so as to download the compressed data blocks corresponding to each streaming download task from the data server, and perform real-time streaming decompression processing on the downloaded compressed data blocks; wherein, the target data is obtained after the M compressed data blocks are stream-decompressed.

[0252] In a feasible implementation, the processor 1001 may also be configured to call a device control application stored in the memory 1005 to implement:

[0253] In response to a download request for a compressed data packet, obtaining a download address of the compressed data packet;

[0254] Based on the download address, the description information of each compressed data block is read from the data server; each compressed data block has corresponding description information;

[0255] Based on the description information of each compressed data block, a streaming download task corresponding to each compressed data block is generated.

[0256] In a feasible implementation, the description information of the M compressed data blocks is located at the end of the compressed data packet; the processor 1001 may also be configured to call a device control application stored in the memory 1005 to implement:

[0257] Establishing a data connection with a data server;

[0258] Based on the data connection, a memory block of the first size is used to read the compressed data in the compressed data packet from the data server one or more times in a back-to-front direction, and the size of the compressed data obtained by any data reading is the first size;

[0259] If the compressed data obtained by reading data using the memory block contains M description information, then the data reading of the remaining compressed data in the compressed data packet is stopped;

[0260] The description information of the M compressed data blocks includes M description information obtained by reading data.

[0261] In a feasible implementation, the description information of the M compressed data blocks exists in the compressed data packet in the form of a linked list;

[0262] The processor 1001 may also be configured to call a device control application stored in the memory 1005 to implement:

[0263] Based on the download address, M description information of the M compressed data blocks are read in sequence from the linked list, and in the process of reading the description information, reading of the compressed data block to which the read description information belongs is skipped.

[0264] In a feasible implementation manner, the description information of any compressed data block includes at least one of the following:

[0265] The offset information of any compressed data block in the compressed data packet; the data size of any compressed data block; the data size of any compressed data block after decompression; the disk location of the data after decompression of any compressed data block; the data identifier of the data after decompression of any compressed data block;

[0266] The offset information of any compressed data block in the compressed data packet is used to indicate the data position of any compressed data block in the compressed data packet.

[0267] In a feasible implementation, after generating the streaming download task set, the processor 1001 may also be configured to call the device control application stored in the memory 1005 to implement:

[0268] Get the data size of each compressed data block in the compressed data packet;

[0269] Sorting the streaming download tasks corresponding to each compressed data block in descending order of the data size of each compressed data block to obtain sorted streaming download tasks;

[0270] The sorted streaming download tasks are added to a task pool; wherein the task pool is used to store unprocessed streaming download tasks.

[0271] In a feasible implementation, any one of the N threads is a target thread; the processor 1001 may also be configured to call a device control application stored in the memory 1005 to implement:

[0272] When the target thread is in an idle state, the streaming download task that is arranged at the front is taken out from the remaining streaming download tasks in the task pool;

[0273] Assign the extracted streaming download task to the target thread.

[0274] In a feasible implementation, any one of the N threads is a target thread, and the streaming download task to be executed by the target thread includes a target streaming download task;

[0275] The processor 1001 may also be configured to call a device control application stored in the memory 1005 to implement:

[0276] The target thread is called to extract offset information of the corresponding compressed data block in the compressed data packet from the target streaming download task; the extracted offset information is used to indicate the data position of the compressed data block corresponding to the target streaming download task in the compressed data packet;

[0277] The target thread is called to download corresponding compressed data blocks from the compressed data packet at the data server based on the extracted offset information.

[0278] In a feasible implementation, the target data includes K file data, K is a positive integer, and each file data in the K file data has its own file compression ratio;

[0279] The compressed data packet includes compressed file data obtained by compressing first file data using a first compression method, and compressed file data obtained by compressing second file data using a second compression method; the first file data is file data having a file compression ratio less than a preset ratio threshold, and the second file data is file data having a file compression ratio greater than or equal to the ratio threshold;

[0280] The compressed file data obtained by compressing the first file data using the first compression method does not support segmentation, and the compressed file data obtained by compressing the second file data using the second compression method supports segmentation.

[0281] In a feasible implementation, the M compressed data blocks include compressed file data of the first file data and a plurality of compressed file blocks obtained by segmenting the compressed file data of the second file data;

[0282] The compressed file data of a first file data is a compressed data block in the compressed data packet, and the compressed file block obtained by segmenting the compressed file data of a second file data is a compressed data block in the compressed data packet.

[0283] In a feasible implementation, any one of the K file data is target file data, and the target file data has an original file size before being compressed using the first compression method, and a compressed file size after being compressed using the first compression method;

[0284] The file compression ratio of the target file data is the ratio of the compressed file size to the original file size.

[0285] In a feasible implementation, any one of the N threads is a target thread, the streaming download task to be executed by the target thread includes a target streaming download task, and the compressed data block corresponding to the target streaming download task is a target compressed data block;

[0286] The processor 1001 may also be configured to call a device control application stored in the memory 1005 to implement:

[0287] Invoking a target thread to execute a target streaming download task based on a cache block of the second size, so as to download the target compressed data block multiple times from the data server, wherein each data download is used to obtain a local compressed data of the target compressed data block, and a size of the local compressed data is less than or equal to the second size; and

[0288] Perform real-time streaming decompression on the locally compressed data obtained from each data download, and write the locally decompressed data obtained from the decompression of the locally compressed data to disk in real time; the multiple locally decompressed data corresponding to the multiple locally compressed data obtained from multiple data downloads constitute the decompressed data blocks corresponding to the target compressed data blocks;

[0289] Among them, multiple data downloads include the i-th data download, i is a positive integer, and after the local decompressed data corresponding to the local compressed data obtained by the i-th data download is successfully written to the disk, the local compressed data obtained by the i-th data download is used to be deleted from the cache block, and the cache block after the data deletion is used to continue the i+1-th data download of the target compressed data block.

[0290] In one feasible implementation, the target data is game installation data. M compressed data blocks are stream-decompressed to obtain M decompressed data blocks that are written to the disk. Each compressed data block corresponds to a decompressed data block. The M decompressed data blocks written to the disk constitute the installation data.

[0291] The processor 1001 may also be configured to call a device control application stored in the memory 1005 to implement:

[0292] Install the game based on the installation data dropped to the disc and run the installed game.

[0293] It should be understood that the computer device 1000 described in the embodiments of the present application can execute the description of the above-mentioned method for processing compressed data in each embodiment of the present application, and can also execute the above-mentioned method. Figure 14 The description of the compressed data processing device 140 in the corresponding embodiment will not be repeated here. In addition, the description of the beneficial effects of adopting the same method will not be repeated here either.

[0294] In addition, it should be noted that this application also provides a computer-readable storage medium, and the computer-readable storage medium stores a computer program. When a processor executes the computer program, it can perform the description of the method for processing compressed data in each embodiment of this application. Therefore, it will not be repeated here. In addition, the description of the beneficial effects of using the same method will not be repeated. For technical details not disclosed in the computer storage medium embodiment involved in this application, please refer to the description of the method embodiment of this application.

[0295] The computer-readable storage medium may be an internal storage unit of the computer device, such as a hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the computer device. Furthermore, the computer-readable storage medium may include both an internal storage unit of the computer device and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by the computer device. The computer-readable storage medium may also be used to temporarily store data that has been output or is to be output.

[0296] The present application provides a computer program product, which includes a computer program stored in a computer-readable storage medium. The processor of a computer device reads the computer program from the computer-readable storage medium, and the processor executes the computer program, causing the computer device to perform the description of the above-mentioned method for processing compressed data in each embodiment of the present application. Therefore, it will not be repeated here. In addition, the description of the beneficial effects of using the same method will not be repeated. For technical details not disclosed in the computer-readable storage medium embodiment involved in this application, please refer to the description of the method embodiment of this application.

[0297] The terms "first", "second", etc. in the description, claims, and drawings of the embodiments of the present application are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "comprising" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, apparatus, product, or device comprising a series of steps or units is not limited to the listed steps or modules, but may optionally include steps or modules not listed, or may optionally include other step units inherent to these processes, methods, apparatuses, products, or devices.

[0298] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0299] The above disclosure is only a preferred embodiment of the present application, and certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

Claims

1. A method for processing compressed data, characterized in that: The method comprises: In response to a download request for a compressed data packet, a streaming download task set is generated; the compressed data packet is stored in a data server, the compressed data packet is obtained by compressing target data, the compressed data packet includes M compressed data blocks, and the streaming download task set includes M streaming download tasks, each compressed data block corresponds to a streaming download task, where M is a positive integer; Create N threads for executing streaming download tasks, and assign a streaming download task to each thread based on the streaming download task set; N is a positive integer; The N threads are called to respectively execute their respective streaming download tasks, so as to respectively download the compressed data blocks corresponding to each streaming download task from the data server, and perform real-time streaming decompression processing on the downloaded compressed data blocks; wherein, the M compressed data blocks are subjected to streaming decompression processing to obtain the target data.

2. The method according to claim 1, wherein The step of generating a streaming download task set in response to a download request for a compressed data packet includes: In response to a download request for the compressed data packet, obtaining a download address of the compressed data packet; Based on the download address, reading the description information of each compressed data block from the data server; each compressed data block has corresponding description information; Based on the description information of each compressed data block, a streaming download task corresponding to each compressed data block is generated.

3. The method according to claim 2, wherein The description information of the M compressed data blocks is located at the tail position of the compressed data packet; and the reading of the description information of each compressed data block from the data server based on the download address includes: Establishing a data connection with the data server; Based on the data connection using a memory block of a first size, the compressed data in the compressed data packet is read from the data server one or more times in a back-to-front direction, and the size of the compressed data obtained by any data reading is the first size; If the compressed data obtained by reading data using the memory block contains M description information, then stopping the data reading of the remaining compressed data in the compressed data packet; The description information of the M compressed data blocks includes the M description information obtained by reading data.

4. The method according to claim 2, wherein The description information of the M compressed data blocks exists in the compressed data packet in the form of a linked list; The step of reading the description information of each compressed data block from the data server based on the download address includes: Based on the download address, the M description information of the M compressed data blocks are read in sequence from the linked list, and in the process of reading the description information, the reading of the compressed data block to which the read description information belongs is skipped.

5. The method according to claim 2, wherein The description information of any compressed data block includes at least one of the following: The offset information of any compressed data block in the compressed data packet; the data size of any compressed data block; the data size of any compressed data block after decompression; the disk location of the data of any compressed data block after decompression; a data identifier of the data after decompression processing of any compressed data block; The offset information of any compressed data block in the compressed data packet is used to indicate the data position of any compressed data block in the compressed data packet.

6. The method according to claim 1, wherein After generating the streaming download task set, the method further includes: Obtaining the data size of each compressed data block in the compressed data packet; sorting the streaming download tasks corresponding to each compressed data block in descending order of data size of each compressed data block to obtain sorted streaming download tasks; The sorted streaming download tasks are added to a task pool; wherein the task pool is used to store unprocessed streaming download tasks.

7. The method according to claim 6, wherein Any one of the N threads is a target thread; and allocating a streaming download task to be executed to each thread based on the streaming download task set includes: When the target thread is in an idle state, taking out the streaming download task that is arranged at the front from the remaining streaming download tasks in the task pool; Allocate the extracted streaming download task to the target thread.

8. The method according to claim 1, wherein Any one of the N threads is a target thread, and the streaming download task to be executed by the target thread includes a target streaming download task; The calling of the N threads to respectively execute respective streaming download tasks to respectively download the compressed data blocks corresponding to the respective streaming download tasks from the data server includes: Invoking the target thread to extract offset information of a corresponding compressed data block in the compressed data packet from the target streaming download task; the extracted offset information is used to indicate a data position of the compressed data block corresponding to the target streaming download task in the compressed data packet; The target thread is called to download corresponding compressed data blocks from the compressed data packet at the data server based on the extracted offset information.

9. The method according to claim 1, wherein The target data includes K file data, K is a positive integer, and each file data in the K file data has its own file compression ratio; The compressed data packet includes compressed file data obtained by compressing first file data using a first compression method, and compressed file data obtained by compressing second file data using a second compression method; the first file data is file data having a file compression ratio less than a preset ratio threshold, and the second file data is file data having a file compression ratio greater than or equal to the ratio threshold; The compressed file data obtained by compressing the first file data using the first compression method does not support segmentation, and the compressed file data obtained by compressing the second file data using the second compression method supports segmentation.

10. The method according to claim 9, wherein The M compressed data blocks include the compressed file data of the first file data and a plurality of compressed file blocks obtained by segmenting the compressed file data of the second file data; The compressed file data of a first file data is a compressed data block in the compressed data packet, and the compressed file block obtained by segmenting the compressed file data of a second file data is a compressed data block in the compressed data packet.

11. The method according to claim 9, wherein Any one of the K file data is target file data, and the target file data has an original file size before being compressed using the first compression method and a compressed file size after being compressed using the first compression method; The file compression ratio of the target file data is the ratio of the compressed file size to the original file size.

12. The method according to claim 1, wherein Any one of the N threads is a target thread, the streaming download task to be executed by the target thread includes a target streaming download task, and the compressed data block corresponding to the target streaming download task is a target compressed data block; The calling of the N threads to respectively execute respective streaming download tasks to respectively download the compressed data blocks corresponding to the respective streaming download tasks from the data server, and performing real-time streaming decompression processing on the downloaded compressed data blocks, includes: Invoking the target thread to execute the target streaming download task based on the cache block of the second size, so as to download the target compressed data block from the data server multiple times, wherein each data download is used to obtain a local compressed data of the target compressed data block, and the size of the local compressed data is less than or equal to the second size; and Performing real-time streaming decompression on the locally compressed data obtained from each data download, and writing the locally decompressed data obtained by decompressing the locally compressed data to a disk in real time; the multiple locally decompressed data corresponding to the multiple locally compressed data obtained from the multiple data downloads constitute the decompressed data blocks corresponding to the target compressed data blocks; Among them, the multiple data downloads include the i-th data download, i is a positive integer, and after the local decompressed data corresponding to the local compressed data obtained by the i-th data download is successfully written to the disk, the local compressed data obtained by the i-th data download is used to be deleted from the cache block, and the cache block after the data deletion is used to continue the i+1-th data download of the target compressed data block.

13. The method according to claim 1, wherein The target data is game installation data, the M compressed data blocks are stream-decompressed to obtain M decompressed data blocks that are written to the disk, each compressed data block corresponds to a decompressed data block, and the M decompressed data blocks that are written to the disk constitute the installation data that is written to the disk; The method further comprises: The game is installed based on the installation data downloaded to the disc, and the installed game is run.

14. A compressed data processing device, characterized in that: The device comprises: a generating module, configured to generate a streaming download task set in response to a download request for a compressed data packet; the compressed data packet is stored in a data server, the compressed data packet is obtained by compressing target data, the compressed data packet includes M compressed data blocks, the streaming download task set includes M streaming download tasks, each compressed data block corresponds to a streaming download task, and M is a positive integer; an allocation module, configured to create N threads for executing streaming download tasks, and allocate a streaming download task to be executed to each thread based on the streaming download task set; N is a positive integer; A calling module is used to call the N threads to execute their respective streaming download tasks, so as to download the compressed data blocks corresponding to each streaming download task from the data server, and perform real-time streaming decompression processing on the downloaded compressed data blocks; wherein, the M compressed data blocks are subjected to streaming decompression processing to obtain the target data.

15. A computer program product, characterized in that The invention comprises a computer program, which implements the steps of the method according to any one of claims 1 to 13 when executed by a processor.

16. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the method according to any one of claims 1 to 13.

17. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and the computer program is suitable for being loaded by a processor and executing the method according to any one of claims 1 to 13.

Images