Compressed data processing method, device, equipment, medium and program product

Abstract

The present invention relates to the technical field of data compression, and provides a method, device, equipment, medium and program product for processing compressed data, the method comprising: generating multiple header information reading commands and data reading commands corresponding to each header information according to a received decompression request; responding to a first header information reading command and reading the first header information, and responding to the corresponding first data reading command and reading the first compressed data after the first header information is read; responding to a second header information reading command and reading the second header information before the first header information is read or in the process of responding to the first data reading command, so that after the first compressed data is read, the second data reading command is continuously responded to and the second compressed data is read until all compressed data is read. By pre-fetching the header information, cross-parallel processing of different compressed data is realized, the decompression cycle is shortened, the decompression performance and bandwidth utilization are improved, and the waste of bandwidth resources is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of data compression, and in particular to a compressed data processing method, device, equipment, medium and program product. Background Art

[0002] At present, in the process of decompressing compressed data, the compressed code stream and the compressed header information of the compressed code stream are located in two different memory spaces, among which the compressed header information is mainly used to store the data size of each compressed code stream, and its data size is relatively small, and a read command for the compressed header information can return multiple consecutive compressed header information. During decompression, before reading the code stream data from the memory, it is necessary to read the compressed header information from the memory first, and after the compressed header information is returned, the data read command of the compressed code stream can be sent to read the compressed code stream data for decompression processing. For multiple different compressed header information, it is necessary to wait until the previous compressed header information and its corresponding compressed code stream data are read before continuing to send the header information read command to process the next compressed header information. The compressed code stream data is processed in a completely serial manner, which has a large delay, low decompression performance, and is prone to bandwidth waste. Summary of the invention

[0003] The present invention provides a compressed data processing method, device, equipment, medium and program product to solve the defects of the prior art that compressed code stream data is processed in a completely serial manner, the decompression performance is low, and bandwidth is easily wasted.

[0004] The present invention provides a compressed data processing method, comprising: receiving a decompression request, and generating a plurality of header information reading commands and data reading commands corresponding to each header information according to the decompression request; Responding to a first header information reading command and reading first header information; the first header information reading command is any one of the multiple header information reading commands; After the first header information is read, responding to the corresponding first data read command and reading the first compressed data; Before the first header information is read or in response to the first data read command, respond to the second header information read command and read the second header information, so that after the first compressed data is read, continuously respond to the second data read command corresponding to the second header information and read the second compressed data until the reading of the compressed data corresponding to each of the header information is completed; the second header information read command is any one of the multiple header information read commands except the first header information read command.

[0005] According to the compressed data processing method provided by the present invention, the step of responding to the first header information reading command and reading the first header information includes: Before receiving the target header information, sending the first header information reading command to a preset first command queue for caching; During the sending process of the first header information reading command, the first data reading command is sent to a preset second command queue for caching; Prioritizing responding to the header information reading command in the first command queue, responding to the first header information reading command, and reading the first header information; Wherein, the target header information is read in response to a target header information read command, and the target header information read command is a header information read command before the first header information command; The first header information is used as the target header information, and the step of sending the first header information read command to a preset first command queue for caching before receiving the target header information is returned and executed until the first header information read command is the last one of the multiple header information read commands.

[0006] According to the compressed data processing method provided by the present invention, before the first header information is read or in the process of responding to the first data read command, responding to the second header information read command and reading the second header information includes: Before the first header information is read out or in the process of responding to the first data read command, responding to a preset first command queue to respond to a second header information read command and read the second header information; The second header information reading command is sent to the first command queue for buffering before the first header information is received.

[0007] According to the compressed data processing method provided by the present invention, the step of responding to the first header information reading command and reading the first header information includes: Matching the header information address in the first header information read command with a plurality of preset header information cache columns; A first target cache column where the first header information corresponding to the first header information read command is located is determined according to the matching result, and the first header information is read from the first target cache column.

[0008] According to the compressed data processing method provided by the present invention, after the first header information is read, responding to the corresponding first data read command and reading the first compressed data includes: Responding to a first data read command corresponding to the first header information, matching a data address in the first data read command with a plurality of preset data cache columns; The second target cache column where the first compressed data corresponding to the first data read command is located is determined according to the matching result, and the first compressed data is read from the second target cache column.

[0009] According to the compressed data processing method provided by the present invention, the generating of multiple header information reading commands and data reading commands corresponding to each header information according to the decompression request includes: Parsing the header information address of the compressed data to be decompressed and the data address of the compressed data from the decompression request; Generate and cache multiple header information read commands according to the header information address, and generate and cache data read commands corresponding to each header information according to the data address; wherein any one of the multiple header information read commands corresponds to at least one header information, any one of the header information corresponds to at least one data read command, and any one of the data read commands corresponds to at least one compressed data to be decompressed.

[0010] The present invention also provides a compressed data processing device, comprising: A request receiving module, used for receiving a decompression request, and generating a plurality of header information reading commands and data reading commands corresponding to each header information according to the decompression request; A first command response module, configured to respond to a first header information reading command and read the first header information; the first header information reading command is any one of the multiple header information reading commands; A second command response module, configured to respond to the corresponding first data read command and read the first compressed data after the first header information is read; A third command response module is used to respond to a second header information read command and read the second header information before the first header information is read or in the process of responding to the first data read command, so as to continuously respond to the second data read command corresponding to the second header information and read the second compressed data after the first compressed data is read until the reading of the compressed data corresponding to each of the header information is completed; the second header information read command is any one of the multiple header information read commands except the first header information read command.

[0011] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the compressed data processing method described above is implemented.

[0012] The present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the compressed data processing method as described above is implemented.

[0013] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the compressed data processing method described above is implemented.

[0014] The compressed data processing method, device, equipment, medium and program product provided by the present invention generate multiple header information reading commands and data reading commands corresponding to each header information according to a decompression request, and respond to the first header information reading command to read the first header information, respond to the first data reading command after the first header information is read, read the first compressed data, and then respond to the second header information reading command and read the second header information before the first header information is read or in the process of responding to the first data reading command, so as to realize the pre-fetching of the second header information, so that after the first compressed data is read, it is possible to continuously respond to the second data reading command of the second header information and read the second compressed data until the reading of the compressed data corresponding to each header information is completed. Through the pre-fetching mechanism of the header information, the reading process of the header information and the compressed data is decoupled, and the cross-parallel processing of the compressed code stream data corresponding to multiple header information can be realized. Compared with the completely serial processing method, the decompression cycle is shortened, the decompression performance is improved, and the utilization rate of bandwidth resources is improved, and the waste of bandwidth resources is reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0016] Figure 1 It is a flow chart of the compressed data processing method provided by the present invention; Figure 2 It is a schematic diagram of the compressed data processing flow provided by the present invention; Figure 3 It is a structural schematic diagram of the compressed data processing device provided by the present invention; Figure 4 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0017] In the description of the embodiments of the present application, it should be noted that the terms "center", "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, which are only for the convenience of describing the embodiments of the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the embodiments of the present application. In addition, the terms "first", "second", and "third" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance.

[0018] In the description of the embodiments of the present application, it should be noted that, unless otherwise clearly specified and limited, the terms "connected" and "connection" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium. For ordinary technicians in this field, the specific meanings of the above terms in the embodiments of the present application can be understood according to specific circumstances.

[0019] In the embodiments of the present application, unless otherwise clearly specified and limited, a first feature being "above" or "below" a second feature may mean that the first and second features are in direct contact, or the first and second features are in indirect contact through an intermediate medium. Moreover, a first feature being "above", "above" or "above" a second feature may mean that the first feature is directly above or obliquely above the second feature, or simply means that the first feature is higher in level than the second feature. A first feature being "below", "below" or "below" a second feature may mean that the first feature is directly below or obliquely below the second feature, or simply means that the first feature is lower in level than the second feature.

[0020] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the embodiments of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0021] In order to solve the problems existing in the existing decompression processing of compressed data, an embodiment of the present invention provides a compressed data processing method, which adopts a cache (cache memory) caching mechanism to manage compressed header information, realize pre-fetching of compressed header information, improve decompression performance, and at the same time improve the utilization rate of bandwidth resources and reduce the waste of bandwidth resources.

[0022] Specifically, Figure 1 A schematic diagram of a compressed data processing method according to an embodiment of the present invention is provided. Figure 1 , the compressed data processing method provided by the embodiment of the present invention includes: Step 100, receiving a decompression request, and generating a plurality of header information reading commands and data reading commands corresponding to each header information according to the decompression request; Step 200, responding to a first header information reading command and reading first header information; the first header information reading command is any one of the multiple header information reading commands; Step 300, after the first header information is read, respond to the corresponding first data read command to read the first compressed data; Step 400, before the first header information is read or in the process of responding to the first data read command, respond to the second header information read command and read the second header information, so as to continuously respond to the second data read command corresponding to the second header information and read the second compressed data after the first compressed data is read until the reading of the compressed data corresponding to each of the header information is completed; the second header information read command is any one of the multiple header information read commands except the first header information read command.

[0023] A decompression request is received, the decompression request including the header information address and data address of the compressed data to be decompressed, and multiple header information reading commands of the compressed data to be decompressed and data reading commands corresponding to each header information are generated and cached according to the received decompression request. The header information reading command of the compressed data is used to read the compressed header information of the compressed data, and the data reading command is used to read part or all of the compressed code stream of the compressed data.

[0024] Optionally, the header information read command includes multiple, any one of which is generated based on the header information address of the compressed data, and the data read command includes multiple, any one of which is generated based on the data address of the compressed data. The multiple header information read commands include at least a first header information read command and a second header information read command, one header information read command corresponds to at least one header information, and one header information corresponds to at least one data read command. The first header information read command is used to read the first header information, and the second header information read command is used to read the second header information. Therefore, the multiple data read commands include at least a first data read command corresponding to the first header information, and a second data read command corresponding to the second header information.

[0025] Optionally, the header information read command has a first correspondence with the header information of the compressed data, the header information of the compressed data has a second correspondence with the data read command, and the data read command has a third correspondence with part of the compressed data. Optionally, the data read command has a third correspondence with the compressed code stream of the compressed data.

[0026] Optionally, the header information read command and the data read command may be generated simultaneously or sequentially in chronological order; multiple header information read commands may be generated simultaneously or sequentially in chronological order; multiple data read commands may be generated simultaneously or sequentially in chronological order, and there is no specific limitation on this. In one embodiment, a header information read command for compressed header information may be generated first according to the header information address of compressed data, and then a data read command corresponding to the header information may be generated.

[0027] The generated compressed data header information reading command and data reading command can be responded to in sequence to realize the reading and decompression processing of the compressed data. Specifically, the header information reading command can be responded to first, and the header information of the compressed data can be read from the memory according to the header information address in the header information reading command. Then, the compressed data reading command can be responded to according to the header information of the compressed data, so as to read the compressed data to be decompressed according to the data address in the data reading command, and the read compressed data can be decompressed, so as to respond to the received decompression request.

[0028] In this embodiment, a cache mechanism for header information read commands and data read commands is provided, and the generated header information read commands and data read commands can be cached. Optionally, for the same compressed part of the compressed data, the header information read command and the data read command can be cached in the same storage area or in different storage areas. Based on the cache mechanism for the header information read command and the data read command, the reading of the header information and the compressed data can be decoupled, and there is no need to generate and send the read command for the compressed data after the header information is returned. The header information read command and the data read command can be generated at the same time, and the decompression processing of the compressed data can be realized by giving priority to responding to the header information read command, thereby improving the decompression performance.

[0029] In the process of decompressing compressed data, there is a lack of command caching mechanism. Therefore, it is necessary to generate a header information read command and respond according to the decompression request, read the corresponding header information, generate a data read command and respond after the header information is returned, read the corresponding compressed data for decompression, and generate and respond to the next header information read command after the compressed data is returned. The generation and response of the header information read command and the generation and response of the data read command during the decompression process are completely serial processes with high coupling.

[0030] In the embodiment of the present invention, based on the command caching mechanism, the next header information read command can be responded to during the process of responding to the data read command, and the response to the header information read command no longer depends on the return of compressed data, so that the response to the header information read command and the response to the data read command are executed in parallel, reducing the coupling between the header information reading process and the compressed data reading process.

[0031] Optionally, if the header information read command and data read command of the same compressed data are cached in the same storage area, the header information read command is first generated and cached according to the header information address in the decompression request, and then the data read command is generated and cached. Then, based on the first-in-first-out principle, the header information read command cached in the storage area can be responded to first.

[0032] Optionally, if the header information read command and data read command of the same compressed data are cached in different storage areas respectively, for example, the header information read command is cached in a first storage area and the data read command is cached in a second storage area, then the header information read command cached in the first storage area is responded to first, and then the data read command cached in the second storage area is responded to according to the read header information, the compressed data is read and decompressed.

[0033] Optionally, based on the cache mechanism for header information read commands and data read commands, for different compressed data, when responding to the data read command of the previous compressed data, a header information read command of the subsequent compressed data can be generated and cached at the same time. Alternatively, when responding to the data read command of the previous compressed data, a header information read command of the subsequent compressed data can be responded to at the same time, that is, the previous compressed data and the header information of the subsequent compressed data are read at the same time, so as to realize parallel processing of different compressed data or different parts of compressed data, thereby improving decompression performance.

[0034] In some embodiments, for any first header information read command and second header information read command, the second header information read command is any one of the multiple header information read commands except the first header information read command, the first header information read command is the header information read command that is responded to first, and the second header information read command may be the header information read command that is responded to immediately after the first header information read command. Respond to the first header information read command and read the first header information, after the first header information is read, respond to the first data read command corresponding to the first header information, read the first compressed data for decompression processing. Before the first header information is read or in the process of responding to the first data read command, respond to the second header information read command and read the second header information, so that after the first compressed data is read, the second data read command corresponding to the second header information can be responded to uninterruptedly, and the second compressed data can be read for decompression processing. Wherein, the second header information read command is any one of the multiple header information read commands except the first header information read command.

[0035] That is, the second header information can be pre-fetched before the first header information is read or in the process of responding to the first data read command, and then after the first compressed data is read, the second data read command can be continuously responded to to read the second compressed data for decompression, thereby realizing cross-parallel processing of the first compressed data and the second compressed data. There is no need to wait for the first compressed data to be processed before responding to the second header information read command to process the second compressed data. Instead, before the first header information is read or in the process of reading the first compressed data, the second header information is read in response to the second header information read command, thereby shortening the overall decompression delay of the compressed data and improving the decompression performance.

[0036] Similarly, before the second header information is read or in response to the second data read command, a third header information read command is responded to among the plurality of header information read commands, the third header information is read, and a third data read command is responded to according to the third header information, and the third compressed data is read for decompression processing. The third header information read command is any one of the plurality of header information read commands except the first header information read command and the second header information read command.

[0037] In one embodiment, the process of reading the header information includes data reading and data return stages. Before the first header information is read, the second header information read command is responded to, which may be during the process of sending the read first header information. Before the first header information is read or during the process of responding to the first data read command, the second header information read command is responded to and the second header information is read, that is, starting from when the first header information is sent to before the first compressed data is read, at any time within this time period, the second header information read command can be responded to and the second header information can be read, thereby realizing the pre-fetching of the second header information. Therefore, after the first compressed data is read, the second data read command corresponding to the second header information can be continuously responded to, the second compressed data can be read and decompressed.

[0038] Optionally, the first header information read command and the second header information read command are adjacent and continuous, or the first header information read command and the second header information read command are adjacent and discontinuous, and this is also not specifically limited. In some embodiments, the header information read command and the data read command of the same compressed data are stored in the same storage area, and the header information read command and the data read command of different compressed data are also stored in the same storage area, then the header information read command and the data read command can be arranged in sequence according to the order of the first header information read command, the first data read command, the second header information read command and the second data read command, the header information read command and the data read command corresponding to the header information are adjacent and continuous, at this time, the first header information read command and the second header information read command are two adjacent and discontinuous header information read commands.

[0039] In this embodiment, according to the header information reading command generated by the decompression request and the data reading command corresponding to each header information, the first header information is read by responding to the first header information reading command, and after the first header information is read, the corresponding first data reading command is responded to to read the first compressed data; then before the first header information is read or in the process of responding to the first data reading command, the second header information reading command is responded to and the second header information is read to realize the pre-fetching of the second header information, so that after the first compressed data is read, the second data reading command corresponding to the second header information can be responded to uninterruptedly to read the second compressed data until the reading of the compressed data corresponding to each header information is completed. Through the pre-fetching mechanism of the header information, the reading process of the header information and the compressed data is decoupled, and the cross-parallel processing of the compressed code stream data corresponding to multiple header information can be realized. Compared with the completely serial processing method, the decompression cycle is shortened, the decompression performance is improved, and the utilization rate of bandwidth resources is improved, and the waste of bandwidth resources is reduced.

[0040] In one embodiment, the header information read command is generated according to the header information address of the compressed data, and the data read command is generated according to the data address of the compressed code stream of the compressed data based on the header information read command. Based on this, in step 100, multiple header information read commands and data read commands corresponding to each header information are generated according to the decompression request, specifically including: Step 101, parsing the header information address of the compressed data to be decompressed and the data address of the compressed data from the decompression request; Step 102, generate and cache multiple header information read commands according to the header information address, and generate and cache data read commands corresponding to each header information according to the data address; wherein any one of the multiple header information read commands corresponds to at least one header information, any one of the header information corresponds to at least one data read command, and any one of the data read commands corresponds to at least one compressed data to be decompressed.

[0041] The decompression request is parsed to obtain the header information address and data address of the compressed data to be decompressed, multiple header information read commands are generated and cached based on the header information address, and data read commands corresponding to each header information are generated and cached based on the data address of the compressed data, thereby obtaining multiple header information read commands corresponding to the decompression request and data read commands corresponding to each header information.

[0042] In one embodiment, the compressed data and the read command of its header information are stored in different storage areas, which are cache columns. In addition, the header information address corresponds to multiple header information, and one header information read command is used to read at least one header information. Therefore, the header information read command generated according to the header information address includes at least a first header information read command and a second header information read command. Similarly, the data address can correspond to multiple compressed data, or to multiple compressed parts of compressed data, and one header information corresponds to at least one data read command. The data read command generated according to the data address includes at least a first data read command and a second data read command. The decoupling of the reading process of the header information and the compressed data is realized based on the cache mechanism of the cache column for the header information read command and the data read command. The cache column includes a first command queue and a second command queue, wherein the first command queue is used to cache the header information read command, and the second command queue is used to cache the data read command. By giving priority to the first command queue, a priority response to the header information read command is realized. At the same time, based on the separation of the response to the header information read command and the data read command, cross-parallel execution of the processing process of different compressed data can be realized.

[0043] Specifically, in step 200, responding to the first header information reading command and reading the first header information may include: Step 201, before receiving the target header information, sending a first header information reading command to a preset first command queue for caching; Step 202, during the sending process of the first header information reading command, sending the first data reading command to a preset second command queue for caching; Step 203, preferentially responding to the header information reading command in the first command queue, responding to the first header information reading command, and reading the first header information; The target header information is read in response to a target header information read command, and the target header information read command is a header information read command sent to the first command queue for buffering before the first header information command; Step 204, taking the first header information as the target header information, returns and executes the step of sending the first header information read command to a preset first command queue for caching before receiving the target header information, until the first header information read command is the last one of the multiple header information read commands.

[0044] Before receiving the target header information, the first header information read command is sent to the preset first command queue for caching, and during the sending process of the first header information read command, the first data read command is sent to the preset second command queue for caching. The target header information is read in response to the target header information read command, and the target header information read command is a header information read command sent to the first command queue for caching before the first header information read command, and the target header information read command is also generated according to the header information address in the decompression request, and is sent to the first command queue for caching before the first header information read command.

[0045] For multiple header information, a first header information read command is first generated, and the first header information read command is sent to a preset first command queue for caching. The first header information read command is used to read the first header information among the multiple header information corresponding to the header information address. Then, the first header information is used as the target header information. Before the target header information is received, that is, before the first header information is returned, the second header information read command is sent to the first command queue for caching. At this time, the second header information read command is a new first header information read command. In this way, before the header information read by the previous header information read command is returned, the next header information read command is sent to the first command queue. Before the previous header information is read or in the data read command response process corresponding to the previous header information, the next header information read command cached in the first command queue is responded to, and the next header information is pre-fetched, so as to perform cross-parallel processing on the compressed code streams corresponding to different header information. Before the previous header information is returned, the next header information read command is sent in advance until the first header information read command sent is the last one among the multiple header information read commands corresponding to the header information address.

[0046] Optionally, before the first header information is returned, the second header information read command is sent to the first command queue for caching, specifically, after the first header information read command is sent to the first command queue for caching, the second header information read command is sent to the first command queue for caching, or, in the process of responding to the first header information read command, the second header information read command is sent to the first command queue for caching. The second header information read command is different from the first header information read command, and optionally, the first header information is the header information that is located before the second header information among the multiple header information.

[0047] Optionally, each of the multiple header information may correspond to one or more compressed data code streams. In step 400, before the first header information is read or in the process of responding to the first data read command, responding to the second header information read command to read the second header information may include: Step 401, before the first header information is read or in the process of responding to the first data read command, respond to a preset first command queue to respond to a second header information read command to read the second header information; The second header information reading command is sent to the first command queue for buffering before the first header information is received.

[0048] Before the first header information is read, or in the process of responding to the first data read command, respond to the first command queue to respond to the second header information read command in the first command queue, thereby reading the second header information. The second header information is sent to the first command queue for caching before the first header information is returned.

[0049] Optionally, after responding to the first header information read command and before the first header information is returned, the second header information read command is sent to a preset first command queue for caching. Then, after acquiring the first header information, the second command queue is responded to according to the first header information, thereby responding to the first data read command, and reading the first compressed data for decompression processing. Before the first header information is read or in the process of responding to the first data read command, the second header information read command is responded to, thereby pre-fetching the second header information before completing the processing of the first compressed data. Then, the second command queue is responded to according to the read second header information, thereby responding to the second data read command in the second command queue, and the second compressed data is read for decompression processing.

[0050] In some embodiments, the second command queue is different from the first command queue. In other embodiments, the second command queue and the first command queue are the same cache queue. When the second command queue and the first command queue are the same cache queue, the first header information read command and the first data read command are sent to the cache queue respectively, and then according to the principle of first-in-first-out, after responding to the first header information read command, the first data read command is responded to according to the read first header information. And after sending the first data read command to the cache queue, the second header information read command and the second data read command are sent to the cache queue in sequence, and before the first header information is read or in the process of responding to the first data read command, the second header information read command is responded to. In this way, the header information read command and the data read command of the same compressed part of the compressed data are sent to the cache queue as a command pair, and the reading process of the data read command of the previous command pair is executed in parallel with the header information read process of the next command pair, so as to realize the pre-fetching of the header information of the next compressed data to be decompressed, reduce the data processing delay, and improve the decompression performance.

[0051] Optionally, for multiple header information read commands and their corresponding data read commands, they may be first uniformly generated according to the header information address and the data address, and then sent to the command queues to wait for responses. For example, according to the header information address in the decompression request, multiple header information read commands are generated and cached, and according to the data address in the decompression request, data read commands corresponding to each header information read command are generated and cached. Then, according to the processing order of the compressed data, the first header information read command among the multiple header information read commands is sent to the first command queue for caching, and the first data read command corresponding to the first header information is sent to the second command queue for caching, and the first command queue is given priority to respond to the first header information read command to read the first header information. Before the first header information is returned, the second header information read command is sent to the first command queue for caching, and the second data read command corresponding to the second header information read command is sent to the second command queue for caching. Before the first header information is read, the second header information read command is sent to the first command queue for caching, and the second data read command corresponding to the second header information read command is sent to the second command queue for caching. Before the first header information is read, or in the process of responding to the first data read command according to the first header information and reading the first compressed data, the second header information read command is responded to and the second header information is pre-fetched. In this method, all header information read commands and data read commands are generated uniformly and then sent to the command queue in sequence for cross response.

[0052] Optionally, multiple header information read commands and their corresponding data read commands can also be generated according to the header information address and the data address, and immediately sent to the first command queue to wait for response after being generated. For example, according to the header information address in the decompression request, a first header information read command is generated and sent to the first command queue for caching, and a first data read command corresponding to the first header information is generated according to the data address in the decompression request, and sent to the second command queue for caching. Then, according to the processing order of the compressed data, the first command queue is responded to first to respond to the first header information read command and read the first header information. Before the first header information is returned, a second header information read command is generated according to the header information address, and the second header information read command is sent to the first command queue for caching, and a second data read command corresponding to the second header information is generated according to the data address, and the second data read command is sent to the second command queue for caching. Before the first header information is read, or in the process of responding to the first data read command according to the first header information and reading the first compressed data, the second header information read command is responded to and the second header information is pre-fetched. In this manner, when processing compressed data, a header information read command and a data read command for the compressed data are generated accordingly, and then sent to the command queue to cross-respond with the previous data read command.

[0053] In one embodiment, the response to the header information read command and the data read command is implemented by address matching. Specifically, in step 200, responding to the first header information read command and reading the first header information includes: Step 210, matching the header information address in the first header information read command with a plurality of preset header information cache columns; Step 220: Determine, according to the matching result, a first target cache column where the first header information corresponding to the first header information read command is located, and read the first header information from the first target cache column.

[0054] Prioritize responding to the first header information read command. When reading the corresponding first header information, specifically match the header information address in the first header information read command with multiple preset header information cache columns, determine the first target cache column where the first header information is located according to the matching result, and read the first header information from the first target cache column.

[0055] Similarly, in step 300, after the first header information is read, responding to the corresponding first data read command and reading the first compressed data includes: Step 310, matching a data address in a first data read command with a plurality of preset data cache columns according to a first data read command corresponding to the first header information response; Step 320 , determining the second target cache column where the first compressed data corresponding to the first data read command is located according to the matching result, and reading the first compressed data from the second target cache column.

[0056] After the first header information is read, in response to a first data read command corresponding to the first header information, the data address in the first data read command is matched with a plurality of preset data cache rows and columns, and the second target cache column where the first compressed data corresponding to the first data read command is located is determined according to the matching result, and the first compressed data is read from the second target cache column, and the compressed code stream of the read first compressed data is decompressed.

[0057] The response process to the first header information reading command and the first data reading command may refer to the description in the above embodiments, which will not be described again here.

[0058] Optionally, the response process to the second header information reading command is the same as the response process to the first header information reading command. Therefore, in step 400, responding to the second header information reading command to read the second header information of the compressed data may include: Step 410, in response to a second header information read command, matching the header information address in the second header information read command with the plurality of header information cache columns; Step 420: Determine, according to the matching result, the third target cache column where the second header information corresponding to the second header information read command is located, and read the second header information of the compressed data from the third target cache column.

[0059] Similarly, the response process of the second data read command may be the same as the response process of the first data read command. Therefore, in step 400, responding to the second data read command according to the second header information to read the second compressed data may include: Step 430, responding to a second data read command according to the second header information, matching a data address in the second data read command with the plurality of data cache columns; Step 440: Determine, according to the matching result, the fourth target cache column where the second compressed data corresponding to the second data read command is located, and read the second compressed data from the fourth target cache column.

[0060] Specifically, in response to the second header information read command, the header information address in the second header information read command is matched with a plurality of header information cache columns, a third target cache queue where the second header information is located is determined according to the matching result, and the second header information is read from the third cache queue. Then, in response to the second data read command according to the second header information, the data address in the second data read command is matched with a plurality of data cache columns, a fourth target cache queue where the second compressed data corresponding to the second data read command is located is determined according to the matching result, and the second compressed data is read from the fourth target cache column for decompression processing.

[0061] The response process to the second header information read command and the second data read command is the same as the response process to the first header information read command and the first data read command, and will not be repeated here.

[0062] In other embodiments, the header information read command and the data read command are cached in the same command queue, that is, the first command queue and the second command queue are the same cache queue. Optionally, the compressed data and its header information are cached in the same cache queue, that is, the header information cache queue and the compressed data cache queue are the same cache queue. That is, in the compressed data processing method provided in the embodiment of the present invention, two cache queues are provided, namely, a command cache queue and a data cache queue, wherein the command cache queue is used to cache the header information read command and the data read command, and the data cache queue is used to cache the compressed data and its header information. Based on this, for the compressed data to be decompressed in the decompression request, the header information read command is first generated according to the header information address in the decompression request and sent to the command cache queue for caching, and then the data read command corresponding to the header information is generated according to the data address in the decompression request, and sent to the command cache queue for caching, so as to ensure that for the same compressed data, the header information read command can be responded to first, and the header information can be pre-fetched. Through the command cache, cross-parallel processing of different compressed data can be achieved.

[0063] Specifically, refer to Figure 2 The timing flow of compressed data processing shown in the figure, after receiving the decompression request, according to the header information address and the compressed data address in the decompression request, before the first header information header data1 is returned, the second header information read command header cmd2 is sent in advance to the command queue for caching. The first header information header data1 is read from the header information cache column in response to the first header information read command header cmd1 that was generated and sent first. After header data1 is returned, the first compressed data is read in response to the first data read command according to header data1, and before the first header information is read or in the process of responding to the first data read command, the newly generated header information read command header cmd2 is responded to, wherein the first data read command includes Figure 2 data cmd11 and data cmd12 in the header information read command (i.e., the header information read command corresponds to multiple data read commands, such as Figure 2 In the header cmd1, the data read commands data cmd11 and data cmd12 are corresponding). The first compressed data includes Figure 2 data11 and data12 in it.

[0064] Furthermore, in the process of responding to the second data read command data cmd2, the second header information read command header cmd2 is responded to first, the second header information headerdata2 is read based on the second header information read command header cmd2, and the newly generated first header information read command header cmd3 is sent to the command cache queue in advance, and then before the second header information header data2 is read or in the process of responding to the second data read command, the newly generated first header information read command header cmd3 is responded to, thereby realizing the pre-fetching of the header information header data3. Furthermore, after reading the second header information header data2, the second data read command is responded to according to the second header information header data2, the corresponding second compressed data is read, and the read second compressed data is decompressed, thereby improving the data reading performance during the decompression process. Among them, the second data read command can be issued by Figure 2 The header information read command may be composed of multiple data read sub-commands such as data cmd21 and data cmd22, or the second data read command may include multiple data read sub-commands such as data cmd21 and data cmd22 (i.e., the header information read command corresponds to multiple data read commands, for example Figure 2 In the header cmd2, the data read commands correspond to data cmd21 and data cmd22).

[0065] In some embodiments, the response to the data read command includes a sending process of sending the data read command to the command queue and a data reading process of reading the compressed data. Figure 2As shown, before the second header information header data2 returns, the newly generated header information read command header cmd3 is sent to the command cache queue in advance, and in the process of responding to the data read command data cmd2 corresponding to headerdata2, the header information read command header cmd3 is responded to, and the corresponding header information header data3 is read. Alternatively, before the second header information header data2 returns, the newly generated header information read command header cmd3 is sent to the command cache queue in advance, and in the process of reading the corresponding compressed data according to the data read command data cmd2 corresponding to header data2, the header information read command header cmd3 is responded to, and the corresponding header information header data3 is read. Then, according to header data3, the data read command data cmd3 is responded to, and the corresponding compressed data such as data31 and data32 are read for decompression processing. Before header data3 returns, header cmd4 is sent to the command cache queue in advance, and so on, until the reading and decompression processing of the last compressed data is completed.

[0066] In other embodiments, the first command queue for caching header information read commands is different from the second command queue for caching data read commands, and the response to the header information read command and the response to the data read command are different threads. Specifically, the first thread is used to respond to the header information read command in the first command queue, and the second thread is used to respond to the data read command in the second command queue. Based on the caching mechanism for the header information read command and the data read command, the operation of the first thread and the second thread can be independent of each other, thereby realizing the decoupling of the header information and compressed data reading processes.

[0067] For any first header information read command and second header information read command, first send the first header information read command to the first command queue for caching, and send the first data read command corresponding to the first header information read command to the second command queue for caching; then give priority to responding to the first command queue, use the first thread to respond to the first header information read command, read the first header information, and before the first header information returns, send the second header information read command to the first command queue for caching, and send the second data read command corresponding to the second header information read command to the second command queue for caching. After the first header information returns, use the second thread to respond to the first data read command in the second command queue according to the first header information, and read the first compressed data for decompression processing. Before the first header information returns or in the process of responding to the first data read command, use the first thread to respond to the second header information read command in the first command queue to read the second header information. Any two header information read commands are processed in this way. Before the previous header information is read or in the process of responding to the data read command corresponding to the previous header information read command, the next header information read command is responded to to realize pre-fetching of the next header information, thereby performing cross-parallel decompression processing on different compressed data, shortening the decompression cycle, and improving the decompression efficiency. In addition, dual-thread parallel processing is adopted to respond to the header information read command and the data read command, thereby improving the utilization rate of bandwidth resources.

[0068] In this embodiment, the read command and compressed data are cached separately through the cache mechanism, and the command can be sent in advance before the data is returned to realize the pre-fetching of the header information, without waiting for the compressed data to be returned, and cross-parallel processing of different compressed data can be realized, reducing the data processing delay, thereby improving the data reading performance when decompressing the compressed data. Through the header information pre-fetching mechanism, cross-parallel processing of different compressed data is realized, which improves the overall performance of data decompression, and at the same time, improves the utilization rate of bandwidth resources and reduces the waste of bandwidth resources.

[0069] The compressed data processing device provided by the present invention is described below. The compressed data processing device described below and the compressed data processing method described above can be referenced to each other.

[0070] Reference Figure 3 , the compressed data processing device provided by the embodiment of the present invention includes: A request receiving module 10, configured to receive a decompression request and generate a plurality of header information reading commands and data reading commands corresponding to each header information according to the decompression request; A first command response module 20, configured to respond to a first header information reading command and read the first header information; the first header information reading command is any one of the multiple header information reading commands; A second command response module 30, configured to respond to the corresponding first data read command and read the first compressed data after the first header information is read; The third command response module 40 is used to respond to the second header information reading command and read the second header information before the first header information is read or in the process of responding to the first data reading command, so as to continuously respond to the second data reading command corresponding to the second header information and read the second compressed data after the first compressed data is read until the reading of the compressed data corresponding to each of the header information is completed; the second header information reading command is any one of the multiple header information reading commands except the first header information reading command.

[0071] In one embodiment, the first command response module 20 is further configured to: Before receiving the target header information, sending the first header information reading command to a preset first command queue for caching; During the sending process of the first header information reading command, the first data reading command is sent to a preset second command queue for caching; Prioritizing responding to the header information reading command in the first command queue, responding to the first header information reading command, and reading the first header information; The target header information is read in response to a target header information read command, and the target header information read command is a header information read command sent to the first command queue for buffering before the first header information command; The first header information is used as the target header information, and the step of sending the first header information read command to a preset first command queue for caching before receiving the target header information is returned and executed until the first header information read command is the last one of the multiple header information read commands.

[0072] In one embodiment, the second command response module 30 is further configured to: Before the first header information is read out or in the process of responding to the first data read command, responding to a preset first command queue to respond to a second header information read command and read the second header information; The second header information reading command is sent to the first command queue for buffering before the first header information is received.

[0073] In one embodiment, the first command response module 20 is further configured to: Matching the header information address in the first header information read command with a plurality of preset header information cache columns; A first target cache column where the first header information corresponding to the first header information read command is located is determined according to the matching result, and the first header information is read from the first target cache column.

[0074] In one embodiment, the second command response module 30 is further configured to: Responding to a first data read command corresponding to the first header information, matching a data address in the first data read command with a plurality of preset data cache columns; The second target cache column where the first compressed data corresponding to the first data read command is located is determined according to the matching result, and the first compressed data is read from the second target cache column.

[0075] In one embodiment, the request receiving module 10 is further used to: Parsing the header information address of the compressed data to be decompressed and the data address of the compressed data from the decompression request; Generate and cache multiple header information read commands according to the header information address, and generate and cache data read commands corresponding to each header information according to the data address; wherein any one of the multiple header information read commands corresponds to at least one header information, any one of the header information corresponds to at least one data read command, and any one of the data read commands corresponds to at least one compressed data to be decompressed.

[0076] Figure 4 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 4 As shown, the electronic device may include: a processor 410, a communication interface 420, a memory 430 and a communication bus 440, wherein the processor 410, the communication interface 420 and the memory 430 communicate with each other through the communication bus 440. The processor 410 may call the logic instructions in the memory 430 to execute the steps of the compressed data processing method, which includes: receiving a decompression request, and generating a plurality of header information reading commands and data reading commands corresponding to each header information according to the decompression request; Responding to a first header information reading command and reading first header information; the first header information reading command is any one of the multiple header information reading commands; After the first header information is read, responding to the corresponding first data read command and reading the first compressed data; Before the first header information is read or in response to the first data read command, respond to the second header information read command and read the second header information, so that after the first compressed data is read, continuously respond to the second data read command corresponding to the second header information and read the second compressed data until the reading of the compressed data corresponding to each of the header information is completed; the second header information read command is any one of the multiple header information read commands except the first header information read command.

[0077] In addition, the logic instructions in the above-mentioned memory 430 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0078] On the other hand, the present invention further provides a computer program product, the computer program product comprising a computer program, the computer program can be stored on a non-transitory computer-readable storage medium, when the computer program is executed by a processor, the computer can perform the steps of the compressed data processing method provided by the above methods, the method comprising: receiving a decompression request, and generating a plurality of header information reading commands and data reading commands corresponding to each header information according to the decompression request; Responding to a first header information reading command and reading first header information; the first header information reading command is any one of the multiple header information reading commands; After the first header information is read, responding to the corresponding first data read command and reading the first compressed data; Before the first header information is read or in response to the first data read command, respond to the second header information read command and read the second header information, so that after the first compressed data is read, continuously respond to the second data read command corresponding to the second header information and read the second compressed data until the reading of the compressed data corresponding to each of the header information is completed; the second header information read command is any one of the multiple header information read commands except the first header information read command.

[0079] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the compressed data processing method provided by the above methods are implemented, and the method includes: receiving a decompression request, and generating a plurality of header information reading commands and data reading commands corresponding to each header information according to the decompression request; Responding to a first header information reading command and reading first header information; the first header information reading command is any one of the multiple header information reading commands; After the first header information is read, responding to the corresponding first data read command and reading the first compressed data; Before the first header information is read or in response to the first data read command, respond to the second header information read command and read the second header information, so that after the first compressed data is read, continuously respond to the second data read command corresponding to the second header information and read the second compressed data until the reading of the compressed data corresponding to each of the header information is completed; the second header information read command is any one of the multiple header information read commands except the first header information read command.

[0080] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0081] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0082] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A compressed data processing method, characterized in that: include: receiving a decompression request, and generating a plurality of header information reading commands and data reading commands corresponding to each header information according to the decompression request; Responding to a first header information reading command and reading the first header information; The first header information reading command is any one of the multiple header information reading commands; After the first header information is read, responding to the corresponding first data read command and reading the first compressed data; Before the first header information is read or in response to the first data read command, respond to the second header information read command and read the second header information, so that after the first compressed data is read, continuously respond to the second data read command corresponding to the second header information and read the second compressed data until the reading of the compressed data corresponding to each of the header information is completed; the second header information read command is any one of the multiple header information read commands except the first header information read command.

2. The compressed data processing method according to claim 1, characterized in that: The responding to the first header information reading command and reading the first header information includes: Before receiving the target header information, sending the first header information reading command to a preset first command queue for caching; During the sending process of the first header information reading command, the first data reading command is sent to a preset second command queue for caching; Prioritizing responding to the header information reading command in the first command queue, responding to the first header information reading command, and reading the first header information; The target header information is read in response to a target header information read command, and the target header information read command is a header information read command sent to the first command queue for buffering before the first header information command; The first header information is used as the target header information, and the step of sending the first header information read command to a preset first command queue for caching before receiving the target header information is returned and executed until the first header information read command is the last one of the multiple header information read commands.

3. The compressed data processing method according to claim 1, characterized in that: The step of responding to a second header information reading command and reading the second header information before the first header information reading is completed or in the process of responding to the first data reading command comprises: Before the first header information is read out or in the process of responding to the first data read command, responding to a preset first command queue to respond to a second header information read command and read the second header information; The second header information reading command is sent to the first command queue for buffering before the first header information is received.

4. The compressed data processing method according to claim 1, characterized in that: The responding to the first header information reading command and reading the first header information includes: Matching the header information address in the first header information read command with a plurality of preset header information cache columns; A first target cache column where the first header information corresponding to the first header information read command is located is determined according to the matching result, and the first header information is read from the first target cache column.

5. The compressed data processing method according to claim 1, characterized in that: The step of responding to the corresponding first data read command and reading the first compressed data after the first header information is read includes: Responding to a first data read command corresponding to the first header information, matching a data address in the first data read command with a plurality of preset data cache columns; The second target cache column where the first compressed data corresponding to the first data read command is located is determined according to the matching result, and the first compressed data is read from the second target cache column.

6. The compressed data processing method according to claim 1, characterized in that: The step of generating a plurality of header information reading commands and data reading commands corresponding to each header information according to the decompression request includes: Parsing the header information address of the compressed data to be decompressed and the data address of the compressed data from the decompression request; Generate and cache multiple header information read commands according to the header information address, and generate and cache data read commands corresponding to each header information according to the data address; wherein any one of the multiple header information read commands corresponds to at least one header information, any one of the header information corresponds to at least one data read command, and any one of the data read commands corresponds to at least one compressed data to be decompressed.

7. A compressed data processing device, characterized in that: include: A request receiving module, used for receiving a decompression request, and generating a plurality of header information reading commands and data reading commands corresponding to each header information according to the decompression request; A first command response module, used to respond to a first header information reading command and read the first header information; The first header information reading command is any one of the multiple header information reading commands; A second command response module, used for responding to the corresponding first data read command and reading the first compressed data after the first header information is read; A third command response module is used to respond to a second header information read command and read the second header information before the first header information is read or in the process of responding to the first data read command, so as to continuously respond to the second data read command corresponding to the second header information and read the second compressed data after the first compressed data is read until the reading of the compressed data corresponding to each of the header information is completed; the second header information read command is any one of the multiple header information read commands except the first header information read command.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the compressed data processing method according to any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the compressed data processing method according to any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the compressed data processing method according to any one of claims 1 to 6 is implemented.

Images