Progressive image decoding method, apparatus, device and storage medium

Abstract

This application discloses a progressive image decoding method, apparatus, device, and storage medium, relating to the field of computer technology. The method includes: using a super-prior network to infer the edge information of the image to be decoded to generate an initial scale and initial mean; decoding based on a mask set of the image to be decoded; determining a new initial scale and initial mean based on the obtained cumulative tensor and edge information; determining the next mask corresponding to the initial mask as a new initial mask; jumping to the step of decoding the initial mask; repeating this process until all masks have been traversed; determining the latent variables corresponding to each decoding step based on the obtained cumulative tensors; constructing an entropy decoding sequence based on the latent variables and the complete mask graph; splitting the backbone network of the image decoder into a target number of sub-stages; determining the feature information of each sub-stage based on the entropy decoding sequence and the sub-stages; and upsampling and injecting residual information into the sub-stage feature information to obtain the target image. This improves the accuracy of progressive image decoding.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a progressive image decoding method, apparatus, device, and storage medium. Background Technology

[0002] Existing technologies are end-to-end learning image compression and progressive decoding techniques: using latent space tensor y and super-prior edge information z as the core data for compression transmission, and employing a geometric multi-rate pyramid approach to achieve progressive hierarchical decoding; the decoding network generally adopts a U-Net-like architecture, relying on skip connections of intermediate features in the encoder to construct a progressively complete decoding tower structure; and only after all latent variables have been fully decoded can a single forward inference of the network be performed to generate an image, without utilizing the step-by-step ordered characteristics of entropy decoding itself.

[0003] Existing end-to-end image compression and decoding methods only transmit quantized latent variables and prior edge information in the compressed bitstream, failing to carry intermediate features from the encoder. This prevents the decoder from using U-Net-style skip connections and from directly reproducing the multi-scale hierarchical decoding capability of the geometric multi-ratio pyramid. Furthermore, existing technologies do not utilize the natural structure of causal order and progressively denser cumulative tensors formed by masked stepwise entropy decoding at the same latent space resolution. The decoding network can only perform a single forward inference after all latent variables have been decoded, failing to align with the causal process of entropy decoding. It also cannot distinguish between placeholder zeros and real zero coefficients at undecoded positions, easily misleading the decoding network. Consequently, it is impossible to achieve parallel device pipelines for decoding and entropy calculation, mid-process image preview, or high-quality progressive decoding without encoder skip conditions.

[0004] As can be seen from the above, improving the accuracy of progressive image decoding is a problem that urgently needs to be solved. Summary of the Invention

[0005] In view of this, the purpose of this invention is to provide a progressive image decoding method, apparatus, device, and storage medium that can improve the accuracy of progressive image decoding. The specific solution is as follows: In a first aspect, this application provides a progressive image decoding method, including: The edge information corresponding to the image to be decoded is inferred using a super-prior network to generate an initial scale and an initial mean. An initial mask is determined based on the mask set corresponding to the image to be decoded, and the initial mask is decoded. The cumulative tensor is determined using the obtained decoding result and the initial scale and the initial mean. Based on the accumulated tensor and the edge information, a new initial scale and initial mean are determined. The next mask corresponding to the initial mask is determined as the new initial mask, and the process jumps to the step of decoding the initial mask until all masks are traversed to obtain each accumulated tensor. Based on each accumulated tensor, the latent variables corresponding to each decoding step are determined. A complete mask graph is constructed based on the mask corresponding to each decoding step. An entropy decoding sequence is constructed based on the latent variables and the complete mask graph. The backbone network of the image decoder is divided into sub-stages of the target number. The corresponding sub-stage feature information is determined based on the entropy decoding sequence and the sub-stages. The sub-stage feature information is upsampled and residual information is injected to obtain the target image. The target number is determined based on the total number of entropy decoding steps.

[0006] Optionally, the step of using a super-prior network to infer the edge information corresponding to the image to be decoded to generate an initial scale and an initial mean, determining an initial mask based on the mask set corresponding to the image to be decoded, decoding the initial mask, and determining the cumulative tensor using the obtained decoding result in combination with the initial scale and the initial mean includes: The compressed bitstream corresponding to the image to be decoded is extracted to obtain side information. The side information is then inferred using a super-prior network to generate the initial scale and initial mean in the entropy decoding process. The first mask in the mask set corresponding to the image to be decoded is determined as the initial mask, and the support set of the initial mask is entropy decoded to obtain the corresponding decoding result. Based on the decoding results and in combination with the initial scale and the initial mean, a sparse latent representation is determined, and the cumulative tensor is determined using the sparse latent representation.

[0007] Optionally, the step of determining a new initial scale and initial mean based on the accumulated tensor and the edge information, determining the next mask corresponding to the initial mask as the new initial mask, and jumping to the step of decoding the initial mask, until all masks are traversed to obtain each accumulated tensor, includes: The cumulative tensor and the edge information are used as prior conditions, and a new initial scale and initial mean are determined based on the prior conditions. Take the next mask corresponding to the initial mask from the mask set to obtain a new initial mask, and jump to the step of decoding the initial mask until all masks have been traversed to obtain each accumulated tensor.

[0008] Optionally, determining the latent variables corresponding to each decoding step based on each of the accumulated tensors includes: Determine whether a quantization operation is performed on the image to be decoded during image compression, and use the obtained determination result to determine the latent variables corresponding to each decoding step based on each of the cumulative tensors.

[0009] Optionally, the step of using the obtained judgment results and determining the latent variables corresponding to each decoding step based on each accumulated tensor includes: If the judgment result indicates that a quantization operation exists, then the cumulative tensors are dequantized using a preset dequantization coefficient to obtain the latent variables corresponding to each decoding step. If the judgment result indicates that there is no quantization operation, then each of the accumulated tensors is determined as the latent variable corresponding to each decoding step.

[0010] Optionally, the step of constructing a complete mask graph based on the mask corresponding to each decoding step, and constructing an entropy decoding sequence based on the latent variables and the complete mask graph, includes: The masks corresponding to each decoding step are summed element by element based on preset constraints to obtain a union mask; the preset constraints are conditions that constrain the range of values ​​of the mask. The union mask is compressed along the channel dimension to obtain a complete mask graph, and the latent variables and the complete mask graph are concatenated and combined to obtain an entropy decoding sequence.

[0011] Optionally, the step of splitting the backbone network of the image decoder into a target number of sub-stages, and determining the corresponding sub-stage feature information based on the entropy decoding sequence and the sub-stages, includes: The backbone network of the image decoder is split into a target number of sub-stages, and the entropy decoding sequence is used to perform feature mapping based on the target stem network to obtain the mapped sequence. The mapped sequence is then embedded sequentially into each of the sub-stages to obtain sub-stage feature information.

[0012] Secondly, this application provides a progressive image decoding apparatus, comprising: The cumulative tensor determination module is used to infer the edge information corresponding to the image to be decoded using a super-prior network to generate an initial scale and an initial mean, determine an initial mask based on the mask set corresponding to the image to be decoded, decode the initial mask, and determine the cumulative tensor using the obtained decoding result and the initial scale and the initial mean. The latent variable determination module is used to determine a new initial scale and initial mean based on the accumulated tensor and the edge information, determine the next mask corresponding to the initial mask as the new initial mask, and jump to the step of decoding the initial mask until all masks are traversed to obtain each accumulated tensor, and determine the latent variables corresponding to each decoding step based on each accumulated tensor. The image decoding determination module is used to construct a mask complete map based on the mask corresponding to each decoding step, construct an entropy decoding sequence based on the latent variables and the mask complete map, split the backbone network of the image decoder into a target number of sub-stages, determine the corresponding sub-stage feature information based on the entropy decoding sequence and the sub-stages, and upsample and inject residual information into the sub-stage feature information to obtain the target image; the target number is determined based on the total number of entropy decoding steps.

[0013] Thirdly, this application provides an electronic device, comprising: Memory, used to store computer programs; A processor is configured to execute the computer program to implement the aforementioned progressive image decoding method.

[0014] Fourthly, this application provides a computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the aforementioned progressive image decoding method.

[0015] This application utilizes a super-prior network to infer the edge information corresponding to the image to be decoded, thereby generating an initial scale and an initial mean. An initial mask is determined based on the mask set corresponding to the image to be decoded, and the initial mask is decoded. The obtained decoding result, combined with the initial scale and the initial mean, is used to determine a cumulative tensor. A new initial scale and initial mean are determined based on the cumulative tensor and the edge information. The next mask corresponding to the initial mask is determined as the new initial mask, and the process jumps to the step of decoding the initial mask until all masks are traversed to obtain each cumulative tensor. Latent variables corresponding to each decoding step are determined based on each cumulative tensor. A mask complete graph is constructed based on the masks corresponding to each decoding step. An entropy decoding sequence is constructed based on the latent variables and the mask complete graph. The backbone network of the image decoder is split into a target number of sub-stages. Corresponding sub-stage feature information is determined based on the entropy decoding sequence and the sub-stages. The sub-stage feature information is upsampled and residual information is injected to obtain the target image. The target number is determined based on the total number of entropy decoding steps.

[0016] As shown above, this application utilizes a super-prior network to infer edge information to generate the initial scale and initial mean. Using a mask as the spatial partitioning basis, entropy decoding is performed only on the area covered by the current mask, generating a sparse latent representation and determining the cumulative tensor. The initial scale and mean are updated with the current cumulative tensor and edge information, and the next mask is switched for iterative decoding. This process iterates through all masks to obtain multi-step cumulative tensors, and each cumulative tensor determines the latent variables corresponding to each decoding step. Each decoding step reuses previously decoded information, resulting in more accurate entropy modeling and lower compression decoding distortion. A complete mask graph is constructed from the multi-step masks, and an entropy decoding sequence is constructed by combining the latent variables and the complete graph. The decoder backbone network is divided into an equal number of sub-stages according to the total number of entropy decoding steps. The entropy decoding sequence is embedded in each sub-stage to obtain sub-stage features. In this way, the sub-stage features are upsampled and residual information is injected to output a high-quality decoded image, achieving high-quality progressive image decoding. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0018] Figure 1 This is a flowchart of a progressive image decoding method disclosed in this application; Figure 2 This is a schematic diagram of a progressive image decoding method disclosed in this application; Figure 3 This is a schematic diagram of a single upsampling operation disclosed in this application; Figure 4 This is a schematic diagram of a dual-branch target image disclosed in this application; Figure 5 This is a schematic diagram of the structure of a progressive image decoding device disclosed in this application; Figure 6 This is a structural diagram of an electronic device disclosed in this application. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] Currently, existing end-to-end image compression and decoding methods only transmit quantized latent variables and prior edge information, failing to carry intermediate encoder features. This prevents the decoder from using U-Net-style skip connections and directly reproducing the multi-scale hierarchical decoding capabilities of geometric multi-ratio pyramids. Furthermore, they do not utilize the natural structure of causal order and progressively denser cumulative tensors formed by masked step-by-step entropy decoding at the same latent space resolution. The decoding network can only perform a single forward inference after all latent variables have been decoded, failing to align with the causal process of entropy decoding and unable to distinguish between placeholder zeros and true zero coefficients at undecoded locations, easily misleading the decoding network. Therefore, this application provides a progressive image decoding method that upsamples sub-stage features and injects residual information to output high-quality decoded images, achieving high-quality progressive image decoding operations.

[0021] See Figure 1 As shown, this embodiment of the invention discloses a progressive image decoding method, including: Step S11: Use a super-prior network to infer the edge information corresponding to the image to be decoded to generate an initial scale and an initial mean. Determine an initial mask based on the mask set corresponding to the image to be decoded, and decode the initial mask. Use the obtained decoding result and the initial scale and the initial mean to determine the cumulative tensor.

[0022] In this embodiment, the side information carried in the compressed bitstream of the image to be decoded is input into the super-prior network for inference calculation to obtain the initial scale corresponding to the 0th step of entropy decoding. and initial mean From a fixed set of masks that match the latent spatial resolution of the image to be decoded, the first mask is selected as the initial mask. Integer or quantized indices are obtained by decoding on the support set, and then combined with the initial scale and the initial mean to synthesize a sparse latent representation. The sparse latent representation is used to perform incremental amplitude calculations, and the unmasked positions in the image to be decoded are set to 0 to obtain the cumulative tensor.

[0023] Specifically, the step of using a super-prior network to infer the edge information corresponding to the image to be decoded to generate an initial scale and an initial mean, determining an initial mask based on the mask set corresponding to the image to be decoded, decoding the initial mask, and determining the cumulative tensor using the obtained decoding result and the initial scale and the initial mean includes: extracting the compressed bitstream corresponding to the image to be decoded to obtain edge information; using a super-prior network to infer the edge information to generate an initial scale and an initial mean in the entropy decoding process; determining the first mask in the mask set corresponding to the image to be decoded as the initial mask; performing entropy decoding on the support set of the initial mask to obtain the corresponding decoding result; determining a sparse latent representation based on the decoding result and the initial scale and the initial mean; and determining the cumulative tensor using the sparse latent representation.

[0024] Step S12: Determine a new initial scale and initial mean based on the accumulated tensor and the edge information, determine the next mask corresponding to the initial mask as the new initial mask, and jump to the step of decoding the initial mask until all masks are traversed to obtain each accumulated tensor, and determine the latent variables corresponding to each decoding step based on each accumulated tensor.

[0025] In this embodiment, after obtaining the cumulative tensor, a new initial scale and initial mean are obtained iteratively based on the cumulative tensor and the edge information. The next mask corresponding to the initial mask is determined as the new initial mask. Then, the process jumps to the step of decoding the initial mask until all masks are traversed to obtain each cumulative tensor. In one specific implementation, the current scale of the current step is updated based on the cumulative tensor obtained in step k-1 and the prior information provided by the edge information. and current average and the mask corresponding to the current step. The decoding process yields the sparse latent representation for the current step. The sparse latent representation of the current step is summed with the cumulative tensor obtained in the previous step to obtain the cumulative tensor of the current step.

[0026] Specifically, the step of determining a new initial scale and initial mean based on the accumulated tensor and the edge information, determining the next mask corresponding to the initial mask as the new initial mask, and jumping to the step of decoding the initial mask, until all masks are traversed to obtain each accumulated tensor, includes: using the accumulated tensor and the edge information as prior conditions, and determining a new initial scale and initial mean based on the prior conditions; taking the next mask corresponding to the initial mask from the mask set to obtain a new initial mask, and jumping to the step of decoding the initial mask, until all masks are traversed to obtain each accumulated tensor. It can be understood that an inverse quantization coefficient s is introduced to perform inverse quantization on the accumulated tensor after each iteration of the aforementioned process. If there is an inverse quantization step at the decoding end, the formula corresponding to the latent variable is: ; in, These are the latent variables corresponding to each decoding step; This represents the sparse latent representation corresponding to each decoding step; This is the preset dequantization coefficient; if there is no dequantization step, then directly set... .

[0027] Specifically, determining the latent variables corresponding to each decoding step based on each of the accumulated tensors includes: determining whether a quantization operation exists during image compression of the image to be decoded, and using the obtained determination result to determine the latent variables corresponding to each decoding step based on each of the accumulated tensors. The step of using the obtained determination result to determine the latent variables corresponding to each decoding step based on each of the accumulated tensors includes: if the determination result indicates that a quantization operation exists, then performing dequantization processing on each of the accumulated tensors using a preset dequantization coefficient to obtain the latent variables corresponding to each decoding step; if the determination result indicates that no quantization operation exists, then determining each of the accumulated tensors as the latent variables corresponding to each decoding step.

[0028] Step S13: Construct a mask complete map based on the mask corresponding to each decoding step, construct an entropy decoding sequence based on the latent variables and the mask complete map, split the backbone network of the image decoder into sub-stages of the target number, determine the corresponding sub-stage feature information based on the entropy decoding sequence and the sub-stages, and upsample and inject residual information into the sub-stage feature information to obtain the target image; the target number is determined based on the total number of entropy decoding steps.

[0029] In this embodiment, all masks from step 0 to step k are summed element-wise, with the values ​​limited to 0 or 1, to generate a union mask. The corresponding formula is as follows: ; in, The union mask; Let be the mask for the i-th step, and let , For batches, For the number of channels, For height, The width is [value]. The union mask is then subjected to channel compression to reduce the multi-channel structure to a single channel, resulting in a complete mask image. The corresponding formula is as follows: ; in, For a complete mask diagram; The union mask; For batches, For the number of channels, For height, The width is [value]. The complete mask image is compared with the latent variable [value]. The sequences are concatenated to obtain the entropy decoding sequence. .

[0030] Specifically, the step of constructing a complete mask graph based on the masks corresponding to each decoding step, and constructing an entropy decoding sequence based on the latent variables and the complete mask graph, includes: summing the masks corresponding to each decoding step element-wise based on preset constraints to obtain a union mask; the preset constraints are conditions that constrain the range of values ​​of the mask; compressing the union mask in the channel dimension to obtain a complete mask graph, and concatenating and combining the latent variables and the complete mask graph to obtain an entropy decoding sequence.

[0031] Understandable Figure 2 This is a schematic diagram of a progressive image decoding method. If the total number of entropy decoding steps is K, then the backbone network of the decoder is divided into K sub-stages. Based on the entropy decoding sequence, the latent variables and mask complete map of each step are fed into the target STEM network for feature mapping. The corresponding formula is as follows: ; in, The mapped sequence; The target STEM network can be a STEM layer in a neural network; For splicing operations; Let be the latent variable at step k; This is the complete mask graph for the k-th step; For batches, For the number of channels, For height, The width is [value]. The mapped sequence is sequentially embedded into each of the sub-stages to obtain sub-stage feature information.

[0032] Specifically, the step of splitting the backbone network of the image decoder into a target number of sub-stages and determining the corresponding sub-stage feature information based on the entropy decoding sequence and the sub-stages includes: splitting the backbone network of the image decoder into a target number of sub-stages, performing feature mapping based on the entropy decoding sequence and using the target STEM network to obtain a mapped sequence; and embedding the mapped sequence into each of the sub-stages in sequence to obtain sub-stage feature information.

[0033] Furthermore, for the segment index of the sub-stage... , Given the total number of sub-stages, the feature information of each sub-stage is upsampled using the following formula: ; in, Features after upsampling; For upsampling operations, there can be various types of upsampling module layers; The sub-stage feature information; For batches, For the number of channels, For height, Width; Even multiples The value is greater than or equal to 2. This means that the feature information of the sub-stage is upsampled by a factor of 2 or greater. Based on the upsampled features, a transformation is performed using network layers to obtain the transformed features. The corresponding formula is as follows: ; in, The transformed features; For the ST segment network layer; The upsampled features are defined as follows. Residual information is injected into the transformed features only when st > 0 to obtain the injected features, as shown in the following formula: ; in, Features after injection; The transformed features; This is the residual fusion network for the current sub-stage; These are the features after changes from the previous stage. The network is then optimized based on the injected features in the final sub-stage to obtain the target decoded image.

[0034] In this embodiment, Figure 3 This is a schematic diagram of a single upsampling step. After obtaining the mapped sequence, fusion is performed based on the mapped sequence to obtain the fused features of all steps. The corresponding formula is as follows: ; in, The fusion feature; This is the sequence after mapping in step 0; For a learnable network; This is the sequence after the mapping at step k; The total number of steps is denoted as . The fused features are upsampled once and directly connected to the Stages network without inter-segment residual injection, and finally the target image is output.

[0035] Understandable Figure 4 This is a schematic diagram of a dual-branch approach for a target image, including a main branch and a preview branch. The main branch uses upsampling of at least 2 and injection of residual information to obtain the target image. The preview branch represents the latent variable. Only until At that time, all subsequent residual injection operations are discarded (set to 0) to obtain the preview image, without waiting for the entire decoding to be completed, and the intermediate preview image is output in real time.

[0036] As shown above, this application utilizes a super-prior network to infer edge information to generate the initial scale and initial mean. Using a mask as the spatial partitioning basis, entropy decoding is performed only on the area covered by the current mask, generating a sparse latent representation and determining the cumulative tensor. The initial scale and mean are updated with the current cumulative tensor and edge information, and the next mask is switched for iterative decoding. This process iterates through all masks to obtain multi-step cumulative tensors, and each cumulative tensor determines the latent variables corresponding to each decoding step. Each decoding step reuses previously decoded information, resulting in more accurate entropy modeling and lower compression decoding distortion. A complete mask graph is constructed from the multi-step masks, and an entropy decoding sequence is constructed by combining the latent variables and the complete graph. The decoder backbone network is divided into an equal number of sub-stages according to the total number of entropy decoding steps. The entropy decoding sequence is embedded in each sub-stage to obtain sub-stage features. In this way, the sub-stage features are upsampled and residual information is injected to output a high-quality decoded image, achieving high-quality progressive image decoding.

[0037] Accordingly, see Figure 5 As shown, this application also provides a progressive image decoding apparatus, comprising: The cumulative tensor determination module 11 is used to infer the edge information corresponding to the image to be decoded using a super-prior network to generate an initial scale and an initial mean, determine an initial mask based on the mask set corresponding to the image to be decoded, decode the initial mask, and determine the cumulative tensor using the obtained decoding result and the initial scale and the initial mean. The latent variable determination module 12 is used to determine a new initial scale and initial mean based on the accumulated tensor and the edge information, determine the next mask corresponding to the initial mask as the new initial mask, and jump to the step of decoding the initial mask until all masks are traversed to obtain each accumulated tensor, and determine the latent variables corresponding to each decoding step based on each accumulated tensor. The image decoding determination module 13 is used to construct a complete mask map based on the mask corresponding to each decoding step, construct an entropy decoding sequence based on the latent variables and the complete mask map, split the backbone network of the image decoder into a target number of sub-stages, determine the corresponding sub-stage feature information based on the entropy decoding sequence and the sub-stages, and upsample and inject residual information into the sub-stage feature information to obtain the target image; the target number is determined based on the total number of entropy decoding steps.

[0038] In some specific embodiments, the cumulative tensor determination module 11 may specifically include: An initial mean generation unit is used to extract the compressed bitstream corresponding to the image to be decoded to obtain side information, and to use a super prior network to infer the side information to generate the initial scale and initial mean in the entropy decoding process. The support set entropy decoding unit is used to determine the first mask in the mask set corresponding to the image to be decoded as the initial mask, and to perform entropy decoding on the support set of the initial mask to obtain the corresponding decoding result. The cumulative tensor determination unit is used to determine a sparse latent representation based on the decoding result and in combination with the initial scale and the initial mean, and to determine the cumulative tensor using the sparse latent representation.

[0039] In some specific embodiments, the latent variable determination module 12 may specifically include: The prior condition determination unit is used to take the cumulative tensor and the edge information as prior conditions, and determine a new initial scale and initial mean based on the prior conditions; The mask traversal unit is used to retrieve the next mask corresponding to the initial mask from the mask set to obtain a new initial mask, and then jump to the step of decoding the initial mask until all masks are traversed to obtain each accumulated tensor.

[0040] In some specific embodiments, the latent variable determination module 12 may specifically include: The latent variable determination submodule is used to determine whether there is a quantization operation when the image to be decoded is compressed, and to determine the latent variables corresponding to each decoding step based on the obtained judgment result and the cumulative tensor.

[0041] In some specific implementations, the latent variable determination submodule may specifically include: Tensor dequantization unit is used to dequantize each of the accumulated tensors using a preset dequantization coefficient if the judgment result indicates that a quantization operation exists, so as to obtain the latent variables corresponding to each decoding step. The latent variable determination unit is used to determine each of the accumulated tensors as the latent variables corresponding to each decoding step if the judgment result indicates that there is no quantization operation.

[0042] In some specific embodiments, the decoded image determination module 13 may specifically include: The mask summation unit is used to sum the masks corresponding to each decoding step element by element based on preset constraints to obtain a union mask; the preset constraints are conditions that constrain the range of values ​​of the mask. The complete graph combining unit is used to compress the union mask in the channel dimension to obtain a complete mask graph, and to concatenate and combine the latent variable and the complete mask graph to obtain an entropy decoding sequence.

[0043] In some specific embodiments, the decoded image determination module 13 may specifically include: The feature mapping unit is used to split the backbone network of the image decoder into a target number of sub-stages, perform feature mapping based on the entropy decoding sequence and using the target stem network to obtain the mapped sequence. A sequence embedding unit is used to embed the mapped sequence into each of the sub-stages in sequence to obtain sub-stage feature information.

[0044] Furthermore, embodiments of this application also disclose an electronic device, Figure 6 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the progressive image decoding method disclosed in any of the foregoing embodiments. Furthermore, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0045] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0046] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon can include operating system 221, computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0047] The operating system 221 is used to manage and control the various hardware devices on the electronic device 20 and the computer program 222, which may be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program capable of performing the progressive image decoding method executed by the electronic device 20 as disclosed in any of the foregoing embodiments, the computer program 222 may further include a computer program capable of performing other specific tasks.

[0048] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned progressive image decoding method. Specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.

[0049] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0050] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0051] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0052] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0053] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A progressive image decoding method, characterized in that, include: The edge information corresponding to the image to be decoded is inferred using a super-prior network to generate an initial scale and an initial mean. An initial mask is determined based on the mask set corresponding to the image to be decoded, and the initial mask is decoded. The cumulative tensor is determined using the obtained decoding result and the initial scale and the initial mean. Based on the accumulated tensor and the edge information, a new initial scale and initial mean are determined. The next mask corresponding to the initial mask is determined as the new initial mask, and the process jumps to the step of decoding the initial mask until all masks are traversed to obtain each accumulated tensor. Based on each accumulated tensor, the latent variables corresponding to each decoding step are determined. A complete mask graph is constructed based on the mask corresponding to each decoding step. An entropy decoding sequence is constructed based on the latent variables and the complete mask graph. The backbone network of the image decoder is divided into sub-stages of the target number. The corresponding sub-stage feature information is determined based on the entropy decoding sequence and the sub-stages. The sub-stage feature information is upsampled and residual information is injected to obtain the target image. The target number is determined based on the total number of entropy decoding steps.

2. The progressive image decoding method according to claim 1, characterized in that, The process involves using a super-prior network to infer the edge information corresponding to the image to be decoded, generating an initial scale and an initial mean, determining an initial mask based on the mask set corresponding to the image to be decoded, decoding the initial mask, and using the obtained decoding result in conjunction with the initial scale and the initial mean to determine the cumulative tensor, including: The compressed bitstream corresponding to the image to be decoded is extracted to obtain side information. The side information is then inferred using a super-prior network to generate the initial scale and initial mean in the entropy decoding process. The first mask in the mask set corresponding to the image to be decoded is determined as the initial mask, and the support set of the initial mask is entropy decoded to obtain the corresponding decoding result. Based on the decoding results and in combination with the initial scale and the initial mean, a sparse latent representation is determined, and the cumulative tensor is determined using the sparse latent representation.

3. The progressive image decoding method according to claim 1, characterized in that, The step of determining a new initial scale and initial mean based on the accumulated tensor and the edge information, determining the next mask corresponding to the initial mask as the new initial mask, and jumping to the step of decoding the initial mask, continues until all masks have been traversed to obtain each accumulated tensor, including: The cumulative tensor and the edge information are used as prior conditions, and a new initial scale and initial mean are determined based on the prior conditions. Take the next mask corresponding to the initial mask from the mask set to obtain a new initial mask, and jump to the step of decoding the initial mask until all masks have been traversed to obtain each accumulated tensor.

4. The progressive image decoding method according to claim 1, characterized in that, The determination of the latent variables corresponding to each decoding step based on each of the accumulated tensors includes: Determine whether a quantization operation is performed on the image to be decoded during image compression, and use the obtained determination result to determine the latent variables corresponding to each decoding step based on each of the cumulative tensors.

5. The progressive image decoding method according to claim 4, characterized in that, The step of using the obtained judgment results and determining the latent variables corresponding to each decoding step based on each accumulated tensor includes: If the judgment result indicates that a quantization operation exists, then the cumulative tensors are dequantized using a preset dequantization coefficient to obtain the latent variables corresponding to each decoding step. If the judgment result indicates that there is no quantization operation, then each of the accumulated tensors is determined as the latent variable corresponding to each decoding step.

6. The progressive image decoding method according to claim 1, characterized in that, The construction of a complete mask graph based on the masks corresponding to each decoding step, and the construction of an entropy decoding sequence based on the latent variables and the complete mask graph, include: The masks corresponding to each decoding step are summed element by element based on preset constraints to obtain a union mask; the preset constraints are conditions that constrain the range of values ​​of the mask. The union mask is compressed along the channel dimension to obtain a complete mask graph, and the latent variables and the complete mask graph are concatenated and combined to obtain an entropy decoding sequence.

7. The progressive image decoding method according to any one of claims 1 to 6, characterized in that, The step of splitting the backbone network of the image decoder into a target number of sub-stages, and determining the corresponding sub-stage feature information based on the entropy decoding sequence and the sub-stages, includes: The backbone network of the image decoder is split into a target number of sub-stages, and the entropy decoding sequence is used to perform feature mapping based on the target stem network to obtain the mapped sequence. The mapped sequence is then embedded sequentially into each of the sub-stages to obtain sub-stage feature information.

8. A progressive image decoding apparatus, characterized in that, include: The cumulative tensor determination module is used to infer the edge information corresponding to the image to be decoded using a super-prior network to generate an initial scale and an initial mean, determine an initial mask based on the mask set corresponding to the image to be decoded, decode the initial mask, and determine the cumulative tensor using the obtained decoding result and the initial scale and the initial mean. The latent variable determination module is used to determine a new initial scale and initial mean based on the accumulated tensor and the edge information, determine the next mask corresponding to the initial mask as the new initial mask, and jump to the step of decoding the initial mask until all masks are traversed to obtain each accumulated tensor, and determine the latent variables corresponding to each decoding step based on each accumulated tensor. The image decoding determination module is used to construct a mask complete map based on the mask corresponding to each decoding step, construct an entropy decoding sequence based on the latent variables and the mask complete map, split the backbone network of the image decoder into a target number of sub-stages, determine the corresponding sub-stage feature information based on the entropy decoding sequence and the sub-stages, and upsample and inject residual information into the sub-stage feature information to obtain the target image; the target number is determined based on the total number of entropy decoding steps.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the progressive image decoding method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, Used to store a computer program, wherein the computer program, when executed by a processor, implements the progressive image decoding method as described in any one of claims 1 to 7.

Images