Image encoding, decoding, model training and bit stream storage and transmission method

Abstract

The embodiment of the invention provides an image coding method, an image decoding method, an image model training method, an image bit stream storage method and an image bit stream transmission method. The scheme comprises: obtaining a potential representation of a to-be-coded image; quantizing the potential representation to obtain a quantized potential representation; dividing the quantized potential representation into a plurality of slices in a channel dimension; constructing a plurality of groups based on the plurality of slices; wherein each group comprises elements from all the slices, and the elements belonging to the same group are distributed in each slice according to a predefined cross-slice structured spatial index mode; performing entropy coding on the plurality of groups to obtain a bit stream of the to-be-coded image; wherein when entropy coding is carried out on the target group, probability distribution parameters of elements in the target group are estimated based on the coded group.

Description

Technical Field

[0001] This specification relates to the field of image compression technology, and particularly to image encoding methods, image decoding methods, and training methods for image encoding / decoding models. This specification also relates to image encoding apparatus, image decoding apparatus, training apparatus for image encoding / decoding models, a computing device, a computer-readable storage medium, a method for storing bitstreams, a method for transmitting bitstreams, and a computer-readable storage medium. Background Technology

[0002] With the rapid development of the Internet and multimedia technologies, the demand for image data storage and transmission is increasing daily. Image compression technology, as a key technology for reducing storage space and transmission bandwidth, has always been a research hotspot in the fields of computer vision and signal processing. Although traditional image compression standards such as JPEG and JPEG2000 have been widely used, there is still room for improvement in compression efficiency and reconstruction quality.

[0003] In recent years, deep learning-based image compression methods have made significant progress. These methods typically employ an end-to-end neural network architecture, including modules for analytical transforms, quantization, and entropy coding, enabling the learning of more efficient image representations. Among these, the entropy model is a crucial component determining compression performance, responsible for estimating the probability distribution of the underlying representation to achieve more efficient arithmetic coding. Summary of the Invention

[0004] In view of the above, one or more embodiments of this specification provide an image encoding method, an image decoding method, a training method for an image encoding / decoding model, an apparatus, a device, and a computer-readable medium, as well as a method for storing a bit stream, a method for transmitting a bit stream, and a computer-readable storage medium for storing a bit stream, to more accurately estimate the probability distribution of the latent representation and improve the rate-distortion performance of the image encoding / decoding system.

[0005] According to a first aspect of one or more embodiments of this specification, an image encoding method is provided, comprising: Obtain the latent representation of the image to be encoded; The latent representation is quantized to obtain a quantized latent representation; The quantized latent representation is divided into multiple slices along the channel dimension; Based on the multiple slices, multiple groups are constructed; wherein each group contains elements from all the slices, and elements belonging to the same group are distributed within each slice according to a predefined cross-slice structured spatial index pattern; Entropy coding is performed on the multiple groups to obtain the bitstream of the image to be encoded; wherein, when entropy coding is performed on the target group, the probability distribution parameters of the elements in the target group are estimated based on the encoded groups.

[0006] According to a second aspect of one or more embodiments of this specification, an image decoding method is provided, comprising: Receive the bitstream of the image to be decoded; Entropy decoding is performed on multiple sets of data in the bitstream to obtain a grouped feature tensor; wherein, when entropy decoding is performed on the target group, the probability distribution parameters of the elements in the target group are estimated based on the decoded groups; The group feature tensors of all decoded groups are merged into a quantized latent representation; The quantized latent representation is then subjected to a synthetic transformation to obtain the reconstructed image.

[0007] According to a third aspect of one or more embodiments of this specification, a method for training an image encoding / decoding model is provided, the image encoding / decoding model including an analysis transform network, a context entropy model, and a synthesis transform network; the method includes: Acquire training images; The training images are input into the analysis and transformation network to obtain the latent representation; The latent representation is quantized to obtain a quantized latent representation; Probabilistic modeling is performed using the context entropy model to estimate the probability distribution parameters of the quantized latent representation and calculate the bitrate loss; wherein, the probabilistic modeling includes: dividing the quantized latent representation into multiple slices along the channel dimension; constructing multiple groups based on the multiple slices, each group containing elements from all the slices, and elements belonging to the same group being distributed within each slice according to a predefined cross-slice structured spatial indexing pattern; when performing probabilistic modeling on a target group, estimating the probability distribution parameters of elements in the target group based on information from the processed groups; The quantized latent representation is input into the synthetic transform network to obtain a reconstructed image, and the distortion loss between the reconstructed image and the training image is calculated. The parameters of the image encoding / decoding model are updated based on the bit rate loss and the distortion loss.

[0008] According to a fourth aspect of one or more embodiments of this specification, an image encoding apparatus is provided, comprising: The analysis and transformation module is used to obtain the latent representation of the image to be encoded; The quantization module is used to quantize the latent representation to obtain a quantized latent representation; A grouping module is used to divide the quantized latent representation into multiple slices along the channel dimension; and to construct multiple groups based on the multiple slices; wherein each group contains elements from all the slices, and elements belonging to the same group are distributed within each slice according to a predefined cross-slice structured spatial indexing pattern; The entropy coding module performs entropy coding on the multiple groups to obtain the bitstream of the image to be encoded; wherein, when entropy coding is performed on the target group, the probability distribution parameters of the elements in the target group are estimated based on the already encoded groups.

[0009] According to a fifth aspect of one or more embodiments of this specification, an image decoding apparatus is provided, comprising: The receiving module is used to receive the bit stream of the image to be decoded; The entropy decoding module is used to entropy decode multiple sets of data in the bitstream to obtain grouped feature tensors; wherein, when entropy decoding a target group, the probability distribution parameters of the elements in the target group are estimated based on the decoded groups; and the grouped feature tensors of all decoded groups are merged into a quantized latent representation. The synthesis transformation module is used to perform a synthesis transformation on the quantized latent representation to obtain a reconstructed image.

[0010] According to a sixth aspect of one or more embodiments of this specification, a training apparatus for an image encoding / decoding model is provided, comprising: The training sample acquisition module is used to acquire training images; An analytical transformation network is used to input the training image into the analytical transformation network to obtain a latent representation; The quantization module is used to quantize the latent representation to obtain a quantized latent representation; A context entropy model is used for probabilistic modeling to estimate the probability distribution parameters of the quantized latent representation and calculate the bitrate loss; wherein the probabilistic modeling includes: dividing the quantized latent representation into multiple slices along the channel dimension; constructing multiple groups based on the multiple slices, each group containing elements from all the slices, and elements belonging to the same group are distributed within each slice according to a predefined cross-slice structured spatial indexing pattern; when performing probabilistic modeling on a target group, estimating the probability distribution parameters of the elements in the target group based on information from the processed groups; A synthetic transform network is used to input the quantized latent representation into the synthetic transform network to obtain a reconstructed image, and to calculate the distortion loss between the reconstructed image and the training image; The model update module is used to update the parameters of the image encoding / decoding model based on the bitrate loss and the distortion loss.

[0011] According to a seventh aspect of one or more embodiments of this specification, a computing device is provided, including a memory, a processor, and computer instructions stored in the memory and executable on the processor, wherein the processor executes the computer instructions to implement the steps of the methods provided in the embodiments of this specification.

[0012] According to an eighth aspect of one or more embodiments of this specification, a computer-readable storage medium is provided that stores computer instructions which, when executed by a processor, implement the steps of the methods provided in the embodiments of this specification.

[0013] According to a ninth aspect of one or more embodiments of this specification, a method for storing a bitstream is provided, comprising storing the bitstream in a storage medium, the bitstream being generated by an image encoding method provided in embodiments of this specification.

[0014] According to a tenth aspect of one or more embodiments of this specification, a method for transmitting a bit stream is provided, comprising transmitting the bit stream, the bit stream being generated by an image encoding method provided in embodiments of this specification.

[0015] According to an eleventh aspect of one or more embodiments of this specification, a computer-readable storage medium is provided that stores a bit stream thereon, the bit stream being generated by an image encoding method provided in the embodiments of this specification.

[0016] One embodiment of this specification achieves at least the following beneficial effects: By dividing the quantized latent representation into multiple slices and constructing groups according to a predefined cross-slice structured spatial indexing pattern, the elements in each group are sampled in a distributed and balanced manner in both spatial and channel dimensions. This ensures that when encoding any target group, the encoded context information (from other groups) it can refer to contains diverse features from different regions and feature channels of the image, greatly enriching the diversity and comprehensiveness of the context information. Consequently, when using this context to estimate probability distribution parameters, a more accurate conditional probability model can be obtained. Therefore, the entire encoding method can generate a shorter data bitstream when outputting reconstructed images of the same visual quality, thus achieving better rate-distortion performance. At the same time, the structured grouping method also facilitates parallel computation optimization during the encoding and decoding process, further improving encoding and decoding efficiency. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments or prior art of this specification, the drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A schematic diagram of a network architecture for an image encoding / decoding model provided in an embodiment of this specification; Figure 2 A flowchart illustrating a training method for an image encoding / decoding model provided in an embodiment of this specification; Figure 3 A flowchart illustrating an image encoding method provided in an embodiment of this specification; Figure 4 This is a schematic diagram illustrating the selection of elements from slices corresponding to a window in a grouping manner, as provided in an embodiment of this specification. Figure 5 This is a schematic diagram of the structure of a local context module provided in an embodiment of this specification; Figure 6 This is a schematic diagram of the structure of a global context module provided in an embodiment of this specification; Figure 7 This is a schematic diagram of the probabilistic modeling process of a context entropy model provided in the embodiments of this specification; Figure 8 A flowchart illustrating an image decoding method provided in an embodiment of this specification; Figure 9 The embodiments provided in this specification correspond to Figure 2 A schematic diagram of the structure of a training device for an image encoding and decoding model; Figure 10 The embodiments provided in this specification correspond to Figure 3 A schematic diagram of the structure of an image encoding device; Figure 11 The embodiments provided in this specification correspond to Figure 8 A schematic diagram of the structure of an image decoding device; Figure 12 This is a structural block diagram of a computing device provided as an embodiment of this specification. Detailed Implementation

[0019] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this specification.

[0020] This specification uses specific terms to describe embodiments thereof. Terms such as "an embodiment," "one embodiment," and / or "some embodiments" refer to a particular feature, structure, or characteristic associated with at least one embodiment of this specification. Therefore, it should be emphasized and noted that references to "an embodiment," "one embodiment," or "an alternative embodiment" in different locations throughout this specification do not necessarily refer to the same embodiment. Furthermore, those skilled in the art can combine and integrate the different embodiments or examples described herein, as well as the features of those different embodiments or examples, without contradiction.

[0021] The terminology used in one or more embodiments of this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of this specification. The singular forms “a,” “an,” “an,” “the,” and “the” as used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this specification includes any or all possible combinations of one or more associated listed items.

[0022] The terms “comprising,” “including,” or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, product, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, product, or apparatus. Without further limitation, the presence of additional identical or equivalent elements in the process, method, product, or apparatus that includes said elements is not excluded.

[0023] Although the terms "first," "second," etc., may be used to describe various information in one or more embodiments of this specification, this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, "first" may also be referred to as "second," and similarly, "second" may also be referred to as "first," without departing from the scope of one or more embodiments of this specification. Ordinal numbers such as "first," "second," etc., do not necessarily indicate order; often they are used to facilitate the distinction of objects. For example, "first server" and "second server" usually refer to two servers. To distinguish these two servers, they are described as "first server" and "second server." Of course, sometimes these two servers may be the same server.

[0024] Depending on the context, the word "if" as used here can be interpreted as "when," "when," or "in response to determination."

[0025] In this specification, unless explicitly stated otherwise, "receiving and sending data" does not necessarily mean direct receiving and sending; it can also mean indirect receiving and sending. For example, A receiving data sent by B can be understood as A directly receiving the data sent by B, or it can be understood as A indirectly receiving the data sent by B through other entities such as C. Similarly, B sending data to A can be understood as B sending the data directly to A, or it can be understood as B indirectly sending the data to A through other entities such as C. Here, C can be one entity, or it can be two or more entities.

[0026] In this specification, unless explicitly stated otherwise, the relationships between structures can be direct or indirect. For example, when describing "A is connected to B," unless it is explicitly stated that A and B are directly connected, it should be understood that A can be directly connected to B or indirectly connected to B. Similarly, when describing "A is on top of B," unless it is explicitly stated that A is directly above B (AB is adjacent and A is above B), it should be understood that A can be directly above B or indirectly above B (AB is separated by other elements, and A is above B). And so on.

[0027] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this specification are all information and data authorized by the user or fully authorized by all parties. The collection, use and processing of related data shall comply with the relevant laws, regulations and standards of the relevant regions, and corresponding operation entry points shall be provided for users to choose to authorize or refuse.

[0028] With the rapid development of the internet and multimedia technologies, the storage and transmission of image data face increasingly severe challenges. Image compression, as a key technology, needs to reduce the bit rate as much as possible while ensuring visual quality. Traditional coding standards (such as JPEG and JPEG2000) and recent deep learning-based end-to-end compression methods all rely on entropy models to probabilistically estimate the compressed representation (latent representation) to achieve efficient arithmetic coding. However, current deep learning-based entropy models still have significant limitations in their context modeling methods. For example, there are problems such as insufficient context utilization and limited coding parallelism.

[0029] Therefore, how to improve an image compression method that can fully and efficiently integrate multi-dimensional contextual information while maintaining excellent rate-distortion performance has become an urgent technical problem to be solved.

[0030] To address the shortcomings of related technologies, the embodiments of this specification divide the quantized latent representation into multiple slices along the channel dimension and construct multiple groups based on these slices. Each group contains elements from all slices, and the elements are distributed according to a predefined cross-slice structured spatial indexing pattern. This breaks away from the traditional single partitioning method of spatial or channel dimensions, achieving an interwoven and dispersed context sampling structure in both channels and space. This allows each group to obtain rich and balanced reference information from different regions and feature channels of the image during encoding, significantly improving the sufficiency of context modeling. Furthermore, when entropy encoding multiple groups, the probability distribution parameters of the target group are estimated based on the encoded groups. This allows the aforementioned constructed context sampling structure to be effectively utilized in sequential encoding. By fusing more comprehensive contextual cues, a more accurate probability estimation of the target symbol is achieved, thereby reducing the bitrate of entropy encoding. Thus, a higher compression ratio (lower bitrate) can be achieved under the same image distortion conditions, significantly reducing image storage and transmission overhead. Simultaneously, thanks to the structured grouping design, the parallelism of the encoding and decoding process can be improved, enhancing encoding and decoding efficiency.

[0031] The technical solutions provided in the various embodiments of this specification are described in detail below with reference to the accompanying drawings.

[0032] Figure 1 This is a schematic diagram of a network architecture for an image encoding / decoding model provided in an embodiment of this specification.

[0033] like Figure 1 As shown, the following operations can be performed at the encoding end: First, encoder 104 can analyze the transformation. Map the input image x to its latent representation y. Analyze the transformation. A deep convolutional neural network is used to extract high-level semantic features of images. Then, the latent representation y can be quantized by 10⁶ using the quantization module to obtain the quantized latent representation. .

[0034] Furthermore, the quantized latent representation can be determined using the context entropy model 118. The probability distribution parameter of the elements is 120. In the field of image / video compression, the entropy model is a module used to estimate the probability distribution of a signal for efficient entropy coding. The context entropy model is used to represent the entropy model's conditional probability estimation based on contextual information (i.e., the encoded / decoded portions).

[0035] To improve the efficiency of entropy coding, it is necessary to accurately estimate the quantization latent representation. The probability distribution of elements in the model. In embodiments of this specification, the context entropy model 118 is determined by quantizing the latent representation. Cross-slice structured grouping is performed, and the probability distribution parameters of each group are estimated sequentially using information from the processed groups, thereby achieving more efficient conditional entropy coding. This enables the multiple groups to achieve uniform three-dimensional coverage, facilitating the capture of redundant information across channels and spaces, thus improving the accuracy of probability distribution prediction. Furthermore, the context entropy model 118 uses an estimated Gaussian distribution. To approximate the latent representation The true distribution Among them, the mean (μ) and variance (μ) ) is the probability distribution parameter 120 estimated by the context entropy model 118.

[0036] Optionally, such as Figure 1 As shown, an auxiliary information encoder can be used. 122. Extract auxiliary information z from the latent representation y. Then, quantization 124 can be performed on the auxiliary information z to transform it into quantized auxiliary information. This can be achieved through the total decomposition entropy model (…). Figure 1 (Not shown) Determine quantification auxiliary information The probability distribution data of the elements in the middle.

[0037] Entropy encoders (such as arithmetic encoders (AE)) 126 quantize auxiliary information based on the probability distribution data. The encoding is an auxiliary bitstream 128. An entropy decoder (such as an arithmetic decoder (AD)) 130 recovers the quantization auxiliary information from the auxiliary bitstream 128 using a total decomposition entropy model (not shown). Auxiliary information decoder 132 pairs of quantification auxiliary information Processing (with auxiliary information encoder) (corresponding to the processing of 122), outputting auxiliary prior features. Auxiliary prior features The introduction of quantization latent representation The probability modeling provides additional prior information to more accurately estimate the probability distribution parameters120, thereby further improving the efficiency of entropy coding.

[0038] Furthermore, an entropy encoder (such as an arithmetic encoder (AE))108 can be used, based on probability distribution parameters120, to quantize the latent representation. Entropy encoding is performed to generate bitstream 110.

[0039] like Figure 1 As shown, the following operations can be performed on the decoding end: First, entropy decoding can be performed on the received bitstream. When using auxiliary information, a total decomposition entropy model (not shown) is used as the codebook, and its parameters are available at the decoding end. An entropy decoder (such as an arithmetic decoder (AD)) 130 recovers the quantization auxiliary information from the auxiliary bitstream 128 using the total decomposition entropy model (not shown). It is then fed into the auxiliary information decoder. 132 to obtain additional auxiliary prior features Then, auxiliary prior features were used. For reference, the quantized latent representation can be estimated based on the context entropy model 118. The probability distribution parameters are 120 (mean μ and variance). Furthermore, an entropy decoder (such as an arithmetic decoder (AD)) 112 can be used to entropy decode the bitstream 110 based on the estimated probability distribution parameters 120, thereby recovering the quantized latent representation from the bitstream 110. .

[0040] Furthermore, decoder 114 can be synthesized through transformation. Quantify the latent representation Transform into reconstructed image Synthesis and transformation at the decoding end Analysis and transformation at the encoding end The structure is symmetrical.

[0041] The embodiments of this specification provide image encoding methods, image decoding methods, and training methods for image encoding / decoding models. These embodiments also relate to image encoding devices, image decoding devices, training devices for image encoding / decoding models, a computing device, a computer-readable storage medium, a method for storing bit streams, a method for transmitting bit streams, and a computer-readable storage medium storing bit streams. Each of these will be described in detail in the following embodiments.

[0042] Figure 2 This is a flowchart illustrating a training method for an image encoding / decoding model provided in an embodiment of this specification.

[0043] From a programming perspective, the entity executing the process can be a program hosted on a server. It can be understood that this method can be executed by any device, equipment, platform, or cluster of devices with computing and processing capabilities.

[0044] In the embodiments of this specification, the image encoding / decoding model is an end-to-end image compression model based on deep learning, which at least includes an analytical transform network (such as...) for feature extraction. Figure 1 The encoder 104), and the context entropy model used to estimate the probability distribution of the latent representation (such as...) Figure 1 118), and synthetic transformation networks for image reconstruction (such as Figure 1 (decoder 114).

[0045] like Figure 2 As shown, the process may include the following steps: Step 202: Obtain training images.

[0046] Specifically, a training image dataset can be obtained. For each image in the dataset, the following can be performed: Figure 2 The method shown obtains the loss of each training image in the dataset, calculates the comprehensive loss, and then updates the parameters in the image encoding / decoding model based on the comprehensive loss.

[0047] In practical applications, the training images can come from independent images or from image frames in GIFs or videos.

[0048] Step 204: Input the training image into the analysis transformation network to obtain the latent representation.

[0049] In an optional embodiment, the analysis transformation network may be a deep convolutional neural network used to extract high-level semantic features of the image. Specifically, it may include a rearrangement unit, a feature extraction unit, and a downsampling unit.

[0050] Furthermore, the feature extraction unit may include multiple stacked residual blocks. Each residual block may contain two convolutional layers and an activation function, and employ a residual connection structure. Further, an activation function may be set between the two convolutional layers in the residual block. For example, the activation function may be ReLU or GELU.

[0051] In practical applications, for a given input image x, the transformation can be analyzed. Transform it into a latent representation .

[0052] Specifically, the analysis transformation typically includes the following steps: Pixel rearrangement: First, through a reversible spatial-channel dimensional recombination operation, the spatial resolution and number of channels of the feature map are significantly changed without loss of information. For example, in one specific embodiment, the width and height of the input image x can be reduced to 1 / 8 of their original values, while the number of channels is increased to 64 times; for a 3-channel RGB image, the number of channels will increase from 3 to 192. Through pixel rearrangement, the spatial information of the image can be effectively transformed and allocated to the channel dimension, constructing a higher-dimensional and more compact intermediate representation for subsequent deep feature extraction.

[0053] Feature Extraction: Next, the reconstructed feature map is fed into a deep network consisting of multiple stacked basic network modules for deep feature extraction. For example, these basic modules can be residual blocks, each containing two convolutional layers and an activation function, utilizing residual connections to alleviate gradient problems during deep network training. Through this feature extraction step, leveraging the powerful expressive capabilities of deep neural networks, high-level semantic and structural features of the image are further learned and extracted from the pixel-rearranged features.

[0054] Downsampling: Finally, the features are downsampled spatially through convolutional operations to further compress the feature map size, resulting in the latent representation y. For example, a convolutional layer with a stride of 2 can be used to reduce the width and height of the feature map to half its original size before input to that layer. This downsampling step significantly reduces the spatial resolution of the feature map while preserving key information, resulting in a highly compressed latent representation suitable for subsequent quantization and entropy coding.

[0055] Step 206: Quantize the latent representation to obtain a quantized latent representation.

[0056] Step 206 is used to convert the continuous-valued latent representation y into a discrete quantized latent representation. This facilitates subsequent entropy encoding and bitstream generation. Specifically, this step uses a uniform scalar quantizer to independently quantize each element in the latent representation y, with the quantization step size pre-set or trained according to rate-distortion performance requirements.

[0057] During the training phase, to ensure the quantization operation is differentiable and thus support end-to-end gradient backpropagation, this embodiment uses additive uniform noise to approximate the quantization process. That is, during training, noise in the interval [...] is added to the latent representation y. 0.5,0.5][ Noise is uniformly distributed within the range of 0.5 to simulate the error introduced by quantization, without performing a rounding operation. This method allows the model to learn the impact of quantization on reconstruction quality and bitrate while maintaining gradient flow.

[0058] Step 206: Quantization of the latent representation obtained after quantization This will be used as input for subsequent entropy encoding.

[0059] Step 208: Perform probabilistic modeling using the context entropy model to estimate the probability distribution parameters of the quantized latent representation and calculate the bit rate loss; wherein, the probabilistic modeling includes: dividing the quantized latent representation into multiple slices along the channel dimension; constructing multiple groups based on the multiple slices, each group containing elements from all the slices, and elements belonging to the same group being distributed within each slice according to a predefined cross-slice structured spatial indexing pattern; when performing probabilistic modeling on the target group, estimating the probability distribution parameters of the elements in the target group based on the information of the processed groups.

[0060] To improve the efficiency of entropy coding, it is necessary to accurately estimate the quantization latent representation. The probability distribution of each element in the representation. The entropy encoding in step 208 is used to provide an accurate, context-based conditional probability distribution for subsequent arithmetic encoding, thereby minimizing the representation quantization latent representation. The required theoretical bit rate (i.e., entropy).

[0061] In the embodiments described herein, a context modeling method based on cross-slice structured grouping is employed to dynamically estimate the quantized latent representation using the contextual information of the encoded groups. The distribution parameters of the current element (such as the mean (μ) and variance (μ) of the Gaussian distribution) Thus, the quantization of potential representations can be achieved. Highly efficient compression encoding.

[0062] Furthermore, when estimating the probability distribution parameters of elements in the target group, the probability distribution parameters of each element in the same group can be calculated in parallel.

[0063] In one or more embodiments of this specification, the context entropy model includes a local context module, a global context module, and an entropy estimation module; the local context module is used to extract local context features from the information of the processed groups; the global context module is used to extract global context features from the information of the processed groups; and the entropy estimation module is used to estimate the probability distribution parameters based on the local context features and the global context features.

[0064] Furthermore, the process of probabilistic modeling through the context entropy model to estimate the probability distribution parameters of the quantized latent representation can be systematically decomposed into the following sub-steps.

[0065] First, structured data restructuring and grouping are performed. The context entropy model quantizes the latent representation of the input. The model is divided into multiple slices along the channel dimension (e.g., uniformly). Based on these slices, multiple groupings are further constructed. Each group contains elements from all slices, and the spatial locations of elements belonging to the same group within each slice follow a predefined cross-slice structured indexing pattern. This cross-slice structured distribution pattern ensures that elements within each group are broadly sampled from different channels and dispersed spatial regions of the latent representation. Therefore, when encoding any group, its contextual information incorporates diverse features, providing a rich and balanced reference base for probabilistic modeling. Grouping is processed sequentially; later-processed groups can rely on information from earlier-processed groups.

[0066] Secondly, multi-level contextual features are extracted. When probabilistically modeling the target group, information from all processed (i.e., encoded) groups can be utilized. Specifically, this can be achieved by constructing an encoded contextual feature tensor: elements from the processed groups are filled into a zero-based tensor according to their original positions, forming a partially known context map. This context map is then input into two sub-modules of the context entropy model—the local context module and the global context module. The local context module, built on a convolutional neural network (such as stacked residual blocks), is used to extract detailed dependencies and spatial local patterns in the neighborhood of the target elements from the encoded contextual feature tensor. The global context module, built on an attention mechanism, captures long-distance semantic dependencies and global structural associations between encoded elements across the entire feature map. The local and global context modules can work in parallel, parsing contextual information at both the micro and macro levels, respectively, and outputting complementary local and global contextual features.

[0067] Finally, the features are fused and the probability distribution parameters are estimated. The extracted local and global context features, along with any possible auxiliary prior features (such as features decoded from auxiliary information), are fed into the entropy estimation module. The entropy estimation module can be a small neural network (such as a feedforward network or a shallow convolutional network) used to map the fused high-dimensional context feature vector to an estimate of the specific probability distribution parameters for each element in the target group. In one embodiment of this specification, this distribution is set to a Gaussian distribution (therefore, the context entropy model can also be called a Gaussian distribution entropy model), so the entropy estimation module can use an estimated Gaussian distribution. To approximate The true distribution To perform arithmetic encoding, specifically, the entropy estimation module can output the estimated mean (μ) and variance (μ) for each element. )parameter.

[0068] Based on these estimated distribution parameters N(μ,σ2) that are highly adapted to the current specific context, the theoretical number of bits required to encode each element in the target group can be accurately calculated. The sum of the expected values ​​of these bits can constitute the bit rate loss estimate of the current group (and the entire image), which will be used to optimize the model parameters during model training.

[0069] Furthermore, the estimated Gaussian distribution is obtained. Then, quantify the latent representation The bitrate (bitrate loss) can be estimated using the following formula: .

[0070] In practical entropy coding, an arithmetic encoder can be used based on the estimated probability distribution. right Encode the data to achieve a compression ratio close to the theoretical entropy.

[0071] Step 210: Input the quantized latent representation into the synthetic transform network to obtain the reconstructed image, and calculate the distortion loss between the reconstructed image and the training image.

[0072] Step 210 can be derived from the compressed quantized latent representation. Reconstruct a visually similar image to the original training image x. And calculate the metric to reconstruct the image. Information loss (i.e. distortion) between the original training image x and the original training image x.

[0073] In the embodiments of this specification, the synthesis transform network (decoder) is functionally symmetrical to the analysis transform network (encoder) in step 202. The synthesis transform network receives the quantized latent representation. As input, it performs a series of inverse spatial and semantic transformations, gradually mapping it back to the original image pixel space.

[0074] Optionally, the synthetic transformation may sequentially include an upsampling operation, a feature recovery operation, and a pixel rearrangement operation.

[0075] Upsampling is used to initially improve spatial resolution. Specifically, it involves quantizing the latent representation of the input. An upsampling operation is performed in the spatial dimension to reverse the downsampling effect in the analysis transformation. This upsampling operation can be implemented using learnable upsampling methods such as subpixel convolution or deconvolution. For example, in one specific implementation, the latent representation is quantized through subpixel convolution. The spatial resolution (width and height) is doubled, and corresponding adjustments are made in the channel dimension to obtain a higher-resolution intermediate feature map that carries more spatial details. This upsampling operation can restore the basic spatial structure lost due to compression.

[0076] Feature restoration reconstructs semantics and details using deep networks. Specifically, the upsampled feature map is input into a deep network consisting of multiple stacked residual blocks for further feature restoration and enhancement. Each residual block can contain convolutional layers, normalization layers, and activation functions, and cross-layer connections alleviate the gradient vanishing problem, effectively promoting the learning and fusion of deep features. These residual blocks process features layer by layer, gradually restoring and enriching the texture, edge, and semantic information of the image. In the last residual block, the deep network can perform a channel dimension mapping, for example, adjusting the number of channels in the feature map to 192 to meet the input channel requirements of subsequent pixel rearrangement operations.

[0077] Pixel rearrangement completes spatial-channel reconstruction to output the final image. Specifically, pixel rearrangement is performed on the feature map output by the feature recovery module. This pixel rearrangement is the reverse process of pixel rearrangement at the encoding end. It significantly improves resolution and compresses the number of channels by rearranging the distribution of feature map pixels in the spatial and channel dimensions. Specifically, in one embodiment, pixel rearrangement expands the width and height of the input feature map by 8 times, while reducing the number of channels from 192 to 3 (corresponding to RGB three channels). Through this pixel rearrangement operation, the feature map can be converted into a reconstructed image with standard color channels and the same size as the original input image. This completes the end-to-end mapping from compressed latent representation to pixel-level reconstructed image.

[0078] In the embodiments of this specification, during model training, the quantized latent representation is... The encoding and decoding process may include: at the encoding end, using an arithmetic encoder based on a probability distribution. Grouping Each element in the group is encoded to generate a bitstream; since elements in the same group have the same context information, their probability distributions can be calculated in parallel, but the encoding operations are performed serially; multiple groups (e.g., 8 groups) are entropy encoded sequentially. 0→ 1→ 2→ ... → 7. At the decoding end, entropy decoding is performed in the same order to recover the quantization value of each group, while updating the context information for use by subsequent groups.

[0079] In the embodiments of this specification, distortion loss is used to quantify the loss of visual information caused by the compression process, and distortion loss is a metric guiding model optimization. Furthermore, the embodiments of this specification may employ one or more complementary distortion metrics. The distortion loss may be at least one of mean squared error loss, MS-SSIM loss, or LPIPS loss.

[0080] Optionally, the mean squared error (MSE) can be calculated. Specifically, the mean squared difference between corresponding pixel values ​​in the two images is calculated, i.e. The optimization objective of mean squared error is directly aimed at pixel-level accuracy and can be used as a basic distortion metric during training.

[0081] Optionally, a multi-scale structural similarity index (MS-SSIM) loss can be calculated. Specifically, the MS-SSIM loss, based on the characteristics of the human visual system, evaluates the perceptual similarity of images at multiple scales, including brightness, contrast, and structure. Optimizing the MS-SSIM loss helps improve the naturalness and structural fidelity of the reconstructed image in subjective vision.

[0082] Optionally, a Learning-Aware Image Patch Similarity (LPIPS) loss can be computed. Specifically, features extracted by a pre-trained deep neural network are used to calculate the distance between two images in a deep feature space. The LPIPS loss better captures semantic-level differences, promoting consistency between the reconstructed image and the original image in terms of texture detail and high-level semantics.

[0083] During training, one or more selected distortion losses can be used together to calculate the overall distortion loss.

[0084] Step 212: Update the parameters of the image encoding / decoding model based on the bitrate loss and the distortion loss.

[0085] Specifically, the distortion loss calculated in step 210 and the bitrate loss calculated in step 208 can jointly constitute the rate-distortion optimization objective. Step 212 may specifically include: calculating the total loss based on the bitrate loss and the distortion loss; the total loss is the weighted sum of the bitrate loss and the distortion loss; and updating the parameters of the image encoding / decoding model based on the total loss.

[0086] Furthermore, the rate loss and the distortion loss can be weighted using Lagrange multipliers λ. The training objective is to minimize the rate-distortion loss: ,in, Let R represent the distortion loss and R represent the bitrate loss. When calculating the distortion loss using mean squared error, the loss function is: .

[0087] Furthermore, by minimizing the rate-distortion loss through backpropagation, the model can automatically learn to achieve a balance between bit rate and distortion, ultimately obtaining a practical image encoding and decoding model that combines high compression efficiency and high reconstruction quality.

[0088] In practical applications, during training, all learnable parameters in the image encoding / decoding model can be updated using backpropagation and an optimizer (such as Adam). Specifically, learnable parameters can include all learnable parameters of the analytical transform network, the contextual entropy model (including local context modules, global context modules, and entropy estimation modules), and the synthetic transform network. More specifically, learnable parameters include, but are not limited to: the convolutional weights and biases in the analytical transform network, the parameters in the contextual entropy model (such as the convolutional layers of the local context module, the attention projection layers of the global context module, and the network weights of the entropy estimation module), and the upsampling layer and convolutional layer parameters in the synthetic transform network.

[0089] During model training, quantization operations are non-differentiable. Therefore, the embodiments in this specification employ the following strategy to ensure effective gradient propagation, thereby updating the learnable parameters of the analytical transform network and the synthetic transform network: During forward propagation, uniform scalar quantization is performed on the latent representation y, for example, by obtaining the quantized latent representation through a rounding function. =round(y); During backpropagation, a straight-through estimator or additive uniform noise approximation can be used to replace the gradient of the quantization operation, allowing the gradient to bypass this non-differentiable stage and continue propagating backward. The quantization operation itself does not contain learnable parameters; the above approximation methods are only used in the training phase to ensure that the entire encoding / decoding model can be jointly optimized in an end-to-end manner.

[0090] In one or more embodiments of this specification, before calculating the bit rate loss, the method further includes: extracting auxiliary information from the latent representation; and quantizing the auxiliary information to obtain quantized auxiliary information.

[0091] Furthermore, the training method for the image encoding / decoding model further includes: decoding the quantization auxiliary information to obtain auxiliary prior features; the probability modeling is also based on the auxiliary prior features. Specifically, the quantization auxiliary information is processed by an auxiliary information decoder to obtain auxiliary prior features.

[0092] Specifically, the auxiliary information is extracted from the latent representation by an auxiliary information encoder, which includes at least one downsampling operation. The bitrate of each group is calculated based on the probability distribution parameters of that group. The total decomposition entropy model can directly output the probability of the quantized auxiliary information without relying on any prior information.

[0093] Furthermore, the training method of the image encoding / decoding model, specifically the calculation of bitrate loss, includes: calculating a first bitrate of the quantized latent representation based on the probability distribution parameters; the first bitrate is the sum of the bitrates of all groups; estimating the auxiliary probability distribution of the quantized auxiliary information using a total decomposition entropy model, and calculating a second bitrate of the quantized auxiliary information based on the auxiliary probability distribution; and determining the bitrate loss based on the first bitrate and the second bitrate.

[0094] Specifically, the total bitrate during training is the sum of the bitrates of all groups plus the bitrate of the auxiliary information:

[0095] in, This represents the bitrate of the i-th packet. The bitrate representing auxiliary information.

[0096] In the embodiments of this specification, when training the model, a suitable distortion function is selected in conjunction with the bit rate loss to construct a loss function according to the task requirements, thereby improving the rate-distortion performance of the image compression model under the distortion index.

[0097] In each training batch: forward propagation computes the reconstructed image. And estimate bitrate loss Then, we assume that mean squared error is used as the distortion function to measure the distortion loss between each pixel of the reconstructed image and the input image. ;after, To control the trade-off between bitrate loss and distortion loss, the total loss is calculated. More specifically, Then, the network parameters are updated through backpropagation based on the total loss. The model training is complete and ready for deployment once the training termination condition is met (e.g., the total loss is less than a preset loss threshold or a preset iteration threshold is reached).

[0098] While one or more embodiments of this specification provide method steps as described in the embodiments or flowcharts, it is understood that the order of steps listed in the embodiments or flowcharts is merely one possible execution order among many steps and does not represent the only possible execution order. The order of some steps may be adjusted according to actual needs, or some steps may be omitted. When the claims involve method steps, changes in the order of such steps, or parallel execution between steps, are also within the scope of protection of the claims.

[0099] Figure 2 The method described herein integrates a cross-slice structured grouping probabilistic modeling mechanism into an end-to-end rate-distortion optimization framework. During training, the contextual entropy model learns to utilize a contextual reference pattern that is interwoven and dispersed across channel and spatial dimensions. In this way, the model can fuse more comprehensive and diverse contextual information during probability estimation, thereby modeling the statistical characteristics of the latent representation more accurately. This accurate modeling can reduce the theoretical lower bound of the bitrate loss. At the same time, the distortion loss term ensures reconstruction fidelity. Finally, through joint optimization, the model parameters are guided to a better balance point, enabling the trained model to more intelligently balance bitrate and distortion in practical compression applications, achieving higher compression efficiency. In other words, the image encoding and decoding model trained based on the embodiments of this specification can achieve better rate-distortion performance, that is, under the same bitrate conditions, it can obtain reconstructed images with higher visual quality, or generate smaller compressed bitstreams while maintaining the same reconstruction quality, significantly improving model performance.

[0100] The various technical features in the above embodiments can be combined arbitrarily, as long as there is no conflict or contradiction between the combinations of features. However, due to space limitations, they have not been described one by one. Therefore, the arbitrary combination of various technical features in the above embodiments is also within the scope of this specification.

[0101] Figure 3 This is a flowchart illustrating an image encoding method provided in an embodiment of this specification.

[0102] From a programming perspective, the entity executing the process can be a program hosted on a server. It can be understood that this method can be executed by any device, equipment, platform, or cluster of devices with computing and processing capabilities.

[0103] like Figure 3 As shown, the process may include the following steps: Step 302: Obtain the latent representation of the image to be encoded.

[0104] Furthermore, the acquisition of the latent representation of the image to be encoded includes: transforming the image to be encoded into a latent representation through analysis transformation; the analysis transformation sequentially includes pixel rearrangement operation, feature extraction operation, and downsampling operation.

[0105] Furthermore, the feature extraction operation is implemented through multiple stacked residual blocks, each residual block containing two convolutional layers and an activation function, and employing a residual connection structure. Further, an activation function, either ReLU or GELU, is placed between the two convolutional layers in each residual block.

[0106] The specific implementation of the analytical transform in the image coding method can be consistent with the specific implementation of the analytical transform in the training method of the image encoding and decoding model. For details, please refer to the description above, which will not be repeated here.

[0107] For example, the analysis transformation can perform the following steps in sequence: 1) reduce the spatial resolution of the image to be encoded to 1 / 8 of the original and increase the number of channels to 64 times the original by pixel rearrangement operation; 2) input the rearranged feature map into the stacked residual block for feature extraction; 3) downsample the extracted features by a convolutional layer with a stride of 2 to obtain the latent representation.

[0108] Step 304: Quantize the latent representation to obtain a quantized latent representation.

[0109] Furthermore, the quantization operation can employ a uniform scalar quantizer. The specific implementation of quantization in the image encoding method can be consistent with the specific implementation of quantization in the training method of the image encoding / decoding model; details can be found in the description above and will not be repeated here.

[0110] Step 306: Divide the quantized latent representation into multiple slices along the channel dimension.

[0111] Furthermore, the multiple slices can be evenly divided along the channel dimension.

[0112] For example, the latent representation can be Divide the channel dimension into 8 uniform slices, {S0, S1, S2, S3, S4, S5, S6, S7}, assuming... If the number of channels is C, then the number of channels in each slice is C / 8.

[0113] Step 308: Based on the multiple slices, construct multiple groups; wherein each group contains elements from all the slices, and elements belonging to the same group are distributed within each slice according to a predefined cross-slice structured spatial index pattern.

[0114] The cross-slice structured spatial indexing pattern can be a pre-designed sampling rule. This sampling rule reflects structured, non-random cross-dimensional sampling to ensure a dispersed distribution of elements. The cross-slice structured spatial indexing pattern can be used to determine the position of elements selected in each window of each slice for each group. Based on this sampling rule, for a given group, the spatial position of elements collected from each slice is predetermined; and this series of spatial positions exhibits a periodic and predictable transformation relationship between different slices. This relationship ensures that elements in the same group can widely come from different channels and different spatial regions of the latent representation, thereby achieving a full mixing and balanced utilization of contextual information in both channel and spatial dimensions.

[0115] Based on the embodiments described in this specification, by employing a cross-slice structured indexing pattern, the data locality of potential representations in both channel and spatial dimensions can be broken, providing richer and more evenly distributed encoded references with contextual information for sequential entropy coding. Unlike the spatial limitations of traditional row-by-row or region-by-region coding, predefined rules ensure that each group can sample elements from all channels (slices) and various regions (windows) in the space. This allows each group to reference encoded contextual information as richly and diversely as possible in both channel and spatial dimensions during encoding. This helps the entropy model more accurately estimate the probability distribution of the current symbol, thereby improving compression efficiency.

[0116] In one or more embodiments of this specification, the step of constructing multiple groups based on the multiple slices specifically includes: dividing each slice into multiple windows according to a predefined window size in the spatial dimension; for each window, assigning a set of sampling positions to each group according to the cross-slice structured spatial indexing pattern, wherein each set of sampling positions contains at least one position in each slice; and selecting elements from the corresponding windows of the corresponding slices according to the sampling positions to form each group.

[0117] Furthermore, the multiple windows can be non-overlapping. For the same group, the selected element positions are usually different within the same window of different slices; for different groups, the selected element positions are usually different within the same window of the same slice. Based on this, it can be ensured that the sampling positions are evenly distributed in both channel and spatial dimensions.

[0118] In one or more embodiments of this specification, the cross-slice structured spatial indexing pattern can satisfy the following condition: for any given group, the sampling position of a window in the k-th slice is associated with the sampling position of the corresponding window in the (k+1)-th slice of the group, based on a predefined cross-slice displacement mapping rule.

[0119] Furthermore, the cross-slice structured spatial indexing mode is implemented as a cross-slice cyclic shift mode; the set of index positions selected by all groups within any window covers all index positions within the arbitrary window.

[0120] As an optional implementation, the cross-slice structured spatial indexing pattern can specifically be a cross-slice cyclic shift pattern. Assume the number of slices is N (e.g., 8), and each window contains M positions (e.g., 4×4=16). Define the set of basic sampling positions {P0, P1, …, P_{N-1}} for a certain window of group 0 (G0) in the N slices, where P_k represents the sampling position of group G0 in slice S_k. Then, the sampling position of the corresponding window of group G_i in slice S_k can be either the basic sampling position P_{(ki) mod N} of group G0 in slice S_{(ki) mod N}, or the basic sampling position P_{(k+i) mod N} of group G0 in slice S_{(k+i) mod N}. It can be seen that for each group, the sampling positions of its elements can have cyclic dependency characteristics in both the slice index "k" and the group index "i". Based on this cross-slice cyclic shift pattern, the periodicity of the sampling positions on the slice index can be ensured.

[0121] To make it easier to understand, an example is given below.

[0122] Figure 4 This is a schematic diagram illustrating a grouping method for selecting elements from slices corresponding to a window, as provided in an embodiment of this specification.

[0123] For example, a latent representation of size H×W×C is uniformly divided into 8 slices (S0 to S7) along the channel dimension. For each slice, it is divided into multiple non-overlapping windows of size 4×4 along the spatial dimension, each window containing 16 elements with position indices from (0,0) to (3,3). Divided into 8 groups, denoted as { 0, 1, 2, 3, 4, 5, 6, 7}.

[0124] Define group G0 ( 0) The set of sampling locations from each slice is P = {p0, p1, ..., p7}, where p_k represents the grouping. 0 in slice S kThe sampling location in the group. 1 in slice S k The sampling location in the data can be grouped. 0 in slice S k+i The sampling position is p(k+i). Taking i=1, k=2 as an example, the groups are... 1. The sampling position in slice S2 can be grouped. 0 is sampled at position p3 in slice S3.

[0125] More specifically, for each 4×4 window, 8 sets of sampling positions are defined, each set corresponding to a group; each set of sampling positions contains 8 position pairs, and each position pair corresponds to a slice.

[0126] Specifically, group G0 selects positions (0,0) and (2,2) within the window of slice S0, and positions (1,1) and (3,3) within the window of slice S1, and so on. Thus, the 0th group can be obtained. 0 contains the following elements: elements taken from slice S0 at positions {(0,0), (2,2)}; elements taken from slice S1 at positions {(1,1), (3,3)}; elements taken from slice S2 at positions {(0,2), (2,0)}; elements taken from slice S3 at positions {(1,3), (3,1)}; elements taken from slice S4 at positions {(0,1), (1,0)}; elements taken from slice S5 at positions {(2,3), (3,2)}; elements taken from slice S6 at positions {(0,3), (1,2)}; and elements taken from slice S7 at positions {(2,1), (3,0)}. For example... Figure 4 As shown in (a), group G0 ( 0) Sampling positions from each slice within a window.

[0127] The sampling positions of group G1 are the result of cyclically shifting the sampling positions of group G0 along the slice dimension. That is, group G1 selects positions (1,1) and (3,3) within the window of slice S0, and positions (0,2) and (2,0) within the window of slice S1, and so on. Thus, the first group can be obtained. 1. Contains the following elements: elements at positions {(1,1), (3,3)} from slice S0; elements at positions {(0,2), (2,0)} from slice S1; elements at positions {(1,3), (3,1)} from slice S2; elements at positions {(0,1), (1,0)} from slice S3; elements at positions {(2,3), (3,2)} from slice S4; elements at positions {(0,3), (1,2)} from slice S5; elements at positions {(2,1), (3,0)} from slice S6; and elements at positions {(0,0), (2,2)} from slice S7. For example... Figure 4 As shown in (b), group G1 ( 1) Sampling positions from each slice within the same window.

[0128] Similarly, other groups will not be discussed further.

[0129] In this example, the M positions of each window are evenly distributed among N groups (e.g., 16 positions, 8 groups, 2 positions per group). This is an efficient and non-redundant partitioning method that ensures no positional information is wasted and all data participates in building the context.

[0130] In this example, the cross-slice cyclic shifting pattern ensures that: the 16 positions within each window are evenly distributed among 8 groups; each group has representative elements in all windows across all slices; and the sampling positions of the same group in different slices follow a predictable transformation relationship.

[0131] Therefore, the cross-slice structured spatial indexing mode (cross-slice cyclic shift mode) based on the embodiments of this specification can achieve the effect of spatial dispersion of elements within a group. Specifically, for any group, all sampling points in the entire image space will not be concentrated in a certain local area, thus achieving spatial dispersion within the group; moreover, this dispersion is the result of cross-channel (slice) collaboration. The context information (from other encoded groups) of an element in a group at a certain spatial position comes from different spatial positions in other slices. This cross-dimensional dispersion cannot be achieved by traditional spatial checkerboard partitioning.

[0132] Those skilled in the art will understand that cyclic shifting is a preferred, but not the only, method for achieving structured and spatially dispersed distribution across slices. The spatial indexing pattern for cross-slice cyclic shifting illustrated above should not be construed as the sole limitation on the embodiments of this specification. Any predefined indexing pattern capable of achieving structured dispersion of elements within the same group across channel and spatial dimensions falls within the scope of protection conceived in the embodiments of this specification. For example, the transformation relationship of sampling positions can be generated based on mathematical functions (such as modular arithmetic), permutation matrices, or other deterministic algorithms. For example, the transformation relationship of sampling positions between different slices can also follow a pseudo-random sequence (which can be predefined and fixed), a Latin square matrix, or other mathematical transformations, as long as it guarantees dispersion and regularity.

[0133] Based on the embodiments of this specification, by employing a cross-slice structured spatial indexing pattern to construct groups for inter-group sequential entropy coding, at least the following technical effects are achieved: 1) Enhanced context richness: When encoding a group, the encoded context information comes from different channels and spatial locations, providing more comprehensive reference information; 2) Enhanced parallelism: Elements in the same group have similar encoding contexts, and their probability distribution parameters can be calculated in parallel; 3) Guaranteed balance: Avoids excessive concentration of information in certain regions or channels, ensuring a balance in encoding efficiency.

[0134] Step 310: Perform entropy coding on the multiple groups to obtain the bitstream of the image to be encoded; wherein, when performing entropy coding on the target group, the probability distribution parameters of the elements in the target group are estimated based on the encoded groups. The bitstream can be transmitted to a receiving end (such as a decoding end).

[0135] In the embodiments of this specification, when performing entropy encoding, the probability distribution parameters of each element in the same group can be calculated in parallel. Furthermore, each element within a group can be entropy encoded sequentially.

[0136] In the embodiments of this specification, sequential entropy encoding can be performed, adjusting the encoding probability distribution of the target group based on the already encoded groups. Specifically, for each group, contextual features are extracted through a context modeling module based on the already encoded groups, and the probability distribution of the target group is estimated using the contextual features. When encoding the target group, the probability distribution parameters of the elements in the target group are estimated based on the already encoded groups. When encoding the i-th group, its probability distribution parameters depend on the previous i-1 encoded groups.

[0137] In one or more embodiments of this specification, the entropy encoding of the plurality of groups specifically includes: estimating the probability distribution parameters of elements in the target group based on the encoded groups; and performing entropy encoding on the target group based on the probability distribution parameters.

[0138] Here, "based on encoded groups" can refer to "based on quantized latent representations". The values ​​of the elements that have already been encoded and written into the bitstream. For example, when encoding the i-th block, the information of the encoded block can refer to... 0, 1, ..., i-1 The elements in.

[0139] Furthermore, the probability distribution parameters may include the mean (μ) and variance (μ) of a Gaussian distribution. ).

[0140] In one or more embodiments of this specification, estimating the probability distribution parameters of elements in the target group based on the encoded group specifically includes: constructing an encoded context feature tensor based on the encoded group; and estimating the probability distribution parameters of elements in the target group based on the encoded context feature tensor.

[0141] Furthermore, the step of constructing an encoded context feature tensor based on the encoded groups specifically includes: initializing a feature tensor with the same size as the latent representation; filling the feature tensor with the elements of the encoded groups according to their positions in the latent representation, and setting the unencoded positions to predetermined values ​​to obtain the encoded context feature tensor.

[0142] Furthermore, the predetermined value can be 0.

[0143] Furthermore, for the first group (e.g., group 0), since there are no encoded groups, all positions in the encoded context feature tensor are predetermined values.

[0144] Following the example above, for the i-th group i When encoding This requires constructing an encoded context feature tensor. First, initialize a tensor with the quantized latent representation. Feature tensors of the same size are initialized with all elements to 0. Then, the previously encoded groups are... The elements are filled into the feature tensor according to their positions in the original latent representation, with unencoded positions remaining at 0. Finally, the filled feature tensor is fed into the context module (see the detailed description below, including the local context module and the global context module). For group 0, since there is no group reference, both the local and global contexts are set to 0.

[0145] In one or more embodiments of this specification, the context entropy model includes a context module (containing a local context module and a global context module) and an entropy estimation module. The local context module is used to extract local context features from the processed grouped information (encoded context feature tensors); the global context module is used to extract global context features from the processed grouped information (encoded context feature tensors); the entropy estimation module is used to estimate the probability distribution parameters based on the local context features and the global context features. Further, the entropy estimation module is a parameter prediction network. The entropy estimation module includes residual blocks.

[0146] Furthermore, estimating the probability distribution parameters of elements in the target group based on the encoded group specifically includes: extracting local context features from the encoded group using a local context module; extracting global context features from the encoded group using a global context module; and estimating the probability distribution parameters of elements in the target group based on the local context features and the global context features using an entropy estimation module.

[0147] Continuing with the example above, on the one hand, the encoded context feature tensor is input into the local context module, and the output of the local context module can be denoted as the local context feature. This includes local spatial neighborhood information based on the encoded grouped elements. On the other hand, the encoded context feature tensor is input to the global context module, and the output of the global context module can be denoted as the global context feature. It contains global dependency information between encoded grouped elements.

[0148] Furthermore, after obtaining the local context features of the i-th group... and global context features With optional auxiliary prior features, the Gaussian distribution can be estimated using the entropy estimation module. To approximate the true distribution, where and These are the estimated mean and variance, respectively.

[0149] In one or more embodiments of this specification, the local context module includes stacked residual blocks; the global context module is constructed based on an attention mechanism.

[0150] In one or more embodiments of this specification, specific implementations of the local context module are further provided.

[0151] Furthermore, the local context module includes stacked residual blocks; the residual blocks include a first residual unit and a second residual unit connected in series; wherein: the first residual unit includes a first linear layer, a depthwise separable convolutional layer and a second linear layer arranged in sequence, and a first residual connection is provided between the input and output of the first residual unit; the second residual unit includes a third linear layer and a fourth linear layer arranged in sequence, and a second residual connection is provided between the input and output of the second residual unit.

[0152] A residual block is a basic network unit with a residual connection structure. Residual connections represent the operation in a neural network where the input of a layer or a segment of a module bypasses intermediate computational layers and is added (or concatenated) to the outputs of those layers, thus addressing gradient vanishing and network degradation problems. The core formula for residual connections is input = F(input) + input, where F represents a series of intermediate transformations. In the embodiments of this specification, the at least one residual block is a stacked structure, meaning these residual blocks are sequentially connected in the depth direction, with the output of one residual block serving as the input of the next.

[0153] Depthwise separable convolutional layers are a lightweight convolutional operation that includes two stages: depthwise convolution and pointwise convolution. Depthwise convolution processes each input channel independently, while pointwise convolution is responsible for channel fusion, significantly reducing computational complexity and the number of parameters.

[0154] The second residual unit is a feedforward neural network, a network structure composed of multiple fully connected layers, used for nonlinear transformation and feature recombination in a fixed-dimensional space.

[0155] In practical applications, a first activation function can be set between the first linear layer and the depthwise separable convolutional layer. A second activation function can also be set between the third linear layer and the fourth linear layer.

[0156] Figure 5 This is a schematic diagram of the structure of a local context module provided in an embodiment of this specification.

[0157] like Figure 5 As shown, the input is the encoded context feature tensor X.

[0158] In the first residual unit: First, the input tensor X passes through the first linear layer, projecting it onto an intermediate dimension suitable for convolution processing; then, it undergoes a nonlinear transformation introduced by the first activation function (such as ReLU, GELU, etc.); after that, it is processed through a depthwise separable convolutional layer, where the depthwise convolution part processes each feature channel independently along the sequence / spatial dimension to capture local dependencies; the pointwise convolution part performs linear combination along the channel dimension to fuse information from different channels; then, the features are mapped back to the original dimension through the second linear layer to obtain the local feature enhancement representation F(X); simultaneously, the original input X is added element-wise to F(X) through the first residual connection to form the output of the first residual unit: X' = X + F(X).

[0159] In the second residual unit: First, the output X' of the first residual unit is input into the third linear layer, which typically expands the feature dimension (e.g., to 2-4 times); then, it undergoes non-linear activation through the second activation function; after that, the expanded dimension is compressed back to the original dimension through the fourth linear layer to obtain Y; at the same time, the output X' of the first residual unit is added element-wise to Y through the second residual connection to obtain the final output: X'' = X' + Y.

[0160] In one or more embodiments of this specification, a specific implementation of the global context module is further provided.

[0161] Furthermore, the global context module is built based on an attention mechanism and is responsible for capturing long-distance dependencies between encoded elements.

[0162] Attention mechanisms refer to a class of techniques where a model dynamically assigns weights to different positions and focuses on key information when processing an input sequence. Specifically, attention weights are generated by calculating the similarity between the query vector Q and the key vector K, and then the value vector V is weighted and summed, enabling the model to dynamically focus on different parts of the input.

[0163] Furthermore, the attention mechanism can be a self-attention mechanism. A self-attention mechanism is a specific type of attention mechanism where the query vector Q, key vector K, and value vector V all come from the same input sequence. Self-attention mechanisms excel at capturing long-range dependencies between elements within a sequence without relying on external information. In the embodiments of this specification, in the global context module constructed based on the self-attention mechanism, the query vector Q, key vector K, and value vector V all come from the same encoded context feature tensor. A standard implementation of the self-attention mechanism can be as follows: First, the encoded context feature tensor is linearly projected to obtain the query vector Q, key vector K, and value vector V; then, the attention weights are calculated. , where d is the dimension of the key vector.

[0164] Optionally, the attention mechanism can be a multi-head attention mechanism. A multi-head attention mechanism divides the input features into multiple attention heads, each of which independently computes its self-attention. The outputs of each attention head are then concatenated and linearly transformed to obtain the final result. Multi-head attention allows the model to capture different types of dependencies in parallel across different subspaces. In practical applications, when the Q, K, and V values ​​of all heads come from the same input sequence (such as the core layer of a Transformer encoder), it is a multi-head self-attention mechanism; when Q comes from one input (such as the current output of the decoder), and K and V come from another input (such as the output of the encoder), it is a multi-head cross attention mechanism.

[0165] Optionally, the attention mechanism can be a linear attention mechanism. Specifically, the linear attention mechanism performs a non-linear mapping between the query vector and the key vector through a kernel function, and uses the associative law of matrix multiplication to change the computational order, reducing the computational complexity from O(n²) of standard attention to O(n), where n is the length of the input sequence. The calculation of attention before changing the computational order is as follows: After changing the calculation order, it becomes By changing the order, the complexity can be reduced from Reduce to ,in , High as a characteristic, The width of the feature is [value]; the rest of the structure is the same as standard attention. Furthermore, the kernel function in a linear attention mechanism can use the ReLU activation function.

[0166] In practical applications, the computation process of the linear attention mechanism includes: projecting the encoded context feature tensors to obtain the query vector Q, key vector K, and value vector V; and using a kernel function. Mapping Q and K, we get (Q) and (K); The attention output is calculated as follows: The change in the order of operations linearizes the computational complexity.

[0167] Furthermore, the global context module is constructed by stacking multiple layers of attention modules to perform deeper dependency modeling. Here, "multi-layered" can refer to concatenating multiple attention modules (such as standard attention modules, multi-head attention modules, or linear attention modules). Specifically, the output of one module serves as the input of the next, thereby constructing a deeper network for more complex and abstract dependency modeling.

[0168] Figure 6 This is a schematic diagram of the structure of a global context module provided in an embodiment of this specification.

[0169] like Figure 6 As shown, the global context module employs a linear attention mechanism.

[0170] like Figure 6 As shown, the input vector These will be projected into three different vector spaces, resulting in a query vector Q, a key vector K, and a value vector V. (Top branch: Input) After dimension mapping via a linear layer, and then ReLU activation, the output is... Middle branch: Input After linear layer mapping, ReLU activation is performed, followed by transpose operation, and the output is... Below branch: Input It passes through only one linear layer and directly outputs the value vector V. Then, it first calculates... Multiplication with V, then Multiply the result of the previous step by the result to obtain the output of the global context module.

[0171] In one or more embodiments of this specification, auxiliary information (auxiliary prior features) may also be used in the process of estimating probability distribution parameters through probability modeling.

[0172] Furthermore, before estimating the probability distribution parameters of elements in the target group based on the encoded group, the method further includes: using an auxiliary information encoder to extract auxiliary information from the latent representation; quantizing the auxiliary information to obtain quantized auxiliary information; and using an auxiliary information decoder to decode the quantized auxiliary information to obtain auxiliary prior features. Specifically, estimating the probability distribution parameters of elements in the target group based on the encoded group includes: estimating the probability distribution parameters of elements in the target group based on the encoded group and the auxiliary prior features.

[0173] Furthermore, the auxiliary information encoder may include stacked residual blocks and residual downsampling layers. For example, the auxiliary information encoder may downsample the input twice, resulting in a smaller size of the quantized auxiliary information to achieve lower bit rate overhead.

[0174] Furthermore, the auxiliary information decoder may include stacked residual blocks and residual upsampling layers. For example, the auxiliary information decoder may process the quantized auxiliary information obtained from entropy decoding. Perform two upsampling operations to adjust its spatial resolution and quantization latent representation. Consistent.

[0175] Furthermore, the image encoding method further includes: entropy encoding the auxiliary information to obtain auxiliary encoding information; and sending the auxiliary encoding information to the decoding end in the bit stream so that the decoding end can assist in estimating probability distribution parameters when performing probability modeling during entropy decoding of the bit stream.

[0176] In practical applications, the following steps can be performed at the encoding end (sending end).

[0177] Extraction and Quantization: The encoding end also includes an auxiliary information encoder h_a, used to extract auxiliary information z from the latent representation y. The auxiliary information z is then quantized to obtain quantized auxiliary information. .

[0178] Independent entropy coding: quantizing auxiliary information An auxiliary bitstream is independently entropically encoded using a total decomposition entropy model (a priori-free model; an independent, context-independent entropy model). This auxiliary bitstream is used in conjunction with the quantized latent representation. The bitstream obtained from entropy encoding is transmitted to the decoding end (receiving end). The total entropy decomposition model provides quantization auxiliary information. It provides its own probability distribution so that it can be independently and efficiently entropy encoded.

[0179] Generate prior conditions: At the encoding end, quantize auxiliary information. This can be fed into the auxiliary information decoder h_s. The auxiliary information decoder h_s processes the quantized auxiliary information. Upsampling and other transformations are performed to output auxiliary prior features ψ. The auxiliary information decoder h_s quantizes the auxiliary information. Decoding and semantic enhancement are performed to transform it into a latent representation with quantization. Spatially aligned (consistent spatial resolution) feature maps rich in global semantic information. The auxiliary prior feature ψ output by the auxiliary information decoder h_s is a feature tensor used as conditional input to help estimate the quantized latent representation. The probability distribution parameters (mean μ and variance) ).

[0180] Probabilistic modeling: for the i-th group i Determine the probability distribution parameters (mean μ and variance). During estimation, the auxiliary prior feature ψ can be used as an additional conditional input, along with the local context features L extracted from the encoded groups. i Global context features G i They are input together into the entropy parameter estimation network.

[0181] Figure 7 This is a schematic diagram of the probabilistic modeling process of a context entropy model provided in the embodiments of this specification.

[0182] like Figure 7 As shown, on the one hand, global context features can be extracted from the encoded groups using the global context module, and local context features can be extracted from the encoded groups using the local context module. On the other hand, auxiliary prior features can be obtained. Thus, the global context features, local context features, and auxiliary prior features can be fused and input into the parameter prediction network (entropy estimation module). This network predicts the final probability distribution parameters (μ and δ²) at each position in the latent representation, thereby obtaining a Gaussian distribution and achieving probabilistic modeling.

[0183] While one or more embodiments of this specification provide method steps as described in the embodiments or flowcharts, it is understood that the order of steps listed in the embodiments or flowcharts is merely one possible execution order among many steps and does not represent the only possible execution order. The order of some steps may be adjusted according to actual needs, or some steps may be omitted. When the claims involve method steps, changes in the order of such steps, or parallel execution between steps, are also within the scope of protection of the claims.

[0184] Figure 3 The proposed method divides the quantized latent representation into multiple slices and constructs groups based on a predefined cross-slice structured spatial indexing pattern. This ensures that elements in each group are sampled evenly and distributed across both spatial and channel dimensions. Consequently, when encoding any target group, the encoded context information (from other groups) it can reference includes diverse features from different regions and feature channels of the image, greatly enriching the diversity and comprehensiveness of the context information. This allows for a more accurate conditional probability model when estimating probability distribution parameters using this context. As a result, the entire encoding method can generate a shorter data bitstream when outputting reconstructed images of the same visual quality, thus achieving better rate-distortion performance. At the same time, the structured grouping method also facilitates parallel computation optimization during the encoding and decoding process, further improving encoding and decoding efficiency.

[0185] The various technical features in the above embodiments can be combined arbitrarily, as long as there is no conflict or contradiction between the combinations of features. However, due to space limitations, they have not been described one by one. Therefore, the arbitrary combination of various technical features in the above embodiments is also within the scope of this specification.

[0186] Figure 8 This is a flowchart illustrating an image decoding method provided in an embodiment of this specification.

[0187] From a programming perspective, the entity executing the process can be a program hosted on a server. It can be understood that this method can be executed by any device, equipment, platform, or cluster of devices with computing and processing capabilities.

[0188] like Figure 8 As shown, the process may include the following steps: Step 802: Receive the bit stream of the image to be decoded.

[0189] Step 804: Perform entropy decoding on multiple sets of data in the bitstream to obtain grouped feature tensors; wherein, when performing entropy decoding on the target group, estimate the probability distribution parameters of the elements in the target group based on the decoded groups.

[0190] Step 806: Merge the group feature tensors of all decoded groups into a quantized latent representation.

[0191] Step 808: Perform a synthetic transformation on the quantized latent representation to obtain the reconstructed image.

[0192] The specific implementation of the image decoding method can be found in the corresponding image encoding method, and can refer to the relevant descriptions in the training method of the image encoding and decoding model.

[0193] In one or more embodiments of this specification, the entropy decoding of the bitstream specifically includes: estimating the probability distribution parameters of elements in the target group based on the decoded groups; and performing entropy decoding on the target group based on the probability distribution parameters.

[0194] In one or more embodiments of this specification, estimating the probability distribution parameters of elements in a target group based on the decoded group specifically includes: extracting local context features from the decoded group using a local context module; extracting global context features from the decoded group using a global context module; and estimating the probability distribution parameters of elements in the target group based on the local context features and the global context features using an entropy estimation module.

[0195] Furthermore, the local context module includes stacked residual blocks; each residual block includes a first residual unit and a second residual unit connected in series; wherein: the first residual unit includes a first linear layer, a depthwise separable convolutional layer, and a second linear layer arranged sequentially, and a first residual connection is provided between the input and output of the first residual unit; the second residual unit includes a third linear layer and a fourth linear layer arranged sequentially, and a second residual connection is provided between the input and output of the second residual unit. The global context module can be constructed based on an attention mechanism.

[0196] In one or more embodiments of this specification, before estimating the probability distribution parameters of elements in the target group based on the decoded group, the method further includes: entropy decoding quantization auxiliary information from the bitstream; and using an auxiliary information decoder to decode the quantization auxiliary information to obtain auxiliary prior features. Specifically, estimating the probability distribution parameters of elements in the target group based on the decoded group includes: estimating the probability distribution parameters of elements in the target group based on the decoded group and the auxiliary prior features.

[0197] In practical applications, the following steps can be performed at the decoding end (receiving end): First, entropy decoding is performed on the received bitstream, and the quantization auxiliary information is decoded from the bitstream according to the probability output by the total decomposition entropy model. Then, the quantification auxiliary information can be used. The auxiliary information decoder h_s is input to obtain the auxiliary prior features ψ; then, based on the additional conditional information provided by the auxiliary prior features ψ, the quantized latent representation can be estimated through the context entropy model. The probability distribution parameters (mean μ and variance) ), recovering the quantized latent representation from the bitstream Based on this embodiment, the auxiliary prior feature ψ can more accurately reconstruct the quantized latent representation at the decoding end. It provides important global or structured clues, thereby effectively improving the quantization latent representation without significantly increasing the bitrate overhead. Compression efficiency.

[0198] In one or more embodiments of this specification, during entropy decoding, the probability distribution parameters of each element in the same group are calculated in parallel. The decoding of each element in the same group can be performed sequentially.

[0199] In one or more embodiments of this specification, the synthetic transformation of the quantized latent representation specifically includes: transforming the quantized latent representation into a reconstructed image through a synthetic transformation; the synthetic transformation sequentially includes an upsampling operation, a feature recovery operation, and a pixel rearrangement operation.

[0200] Furthermore, the feature recovery operation is implemented through multiple stacked residual blocks, each containing two convolutional layers and an activation function, and employing a residual connection structure.

[0201] The specific implementation of the analytical transform in the image coding method can be consistent with the specific implementation of the analytical transform in the training method of the image encoding and decoding model. For details, please refer to the description above, which will not be repeated here.

[0202] For example, the synthetic transformation sequentially performs the following steps: 1) increasing the spatial resolution of the quantized latent representation to twice its original value through an upsampling operation; 2) inputting the upsampled feature map into a stacked residual block for feature recovery; 3) increasing the spatial resolution of the recovered feature map to eight times its original value and reducing the number of channels to 1 / 64 of its original value through a pixel rearrangement operation to obtain the reconstructed image.

[0203] While one or more embodiments of this specification provide method steps as described in the embodiments or flowcharts, it is understood that the order of steps listed in the embodiments or flowcharts is merely one possible execution order among many steps and does not represent the only possible execution order. The order of some steps may be adjusted according to actual needs, or some steps may be omitted. When the claims involve method steps, changes in the order of such steps, or parallel execution between steps, are also within the scope of protection of the claims.

[0204] Figure 8 The proposed method achieves efficient and accurate data recovery and image reconstruction at the decoding end by employing a cross-slice structured grouping context modeling mechanism symmetrical to the encoding end. Specifically, during entropy decoding, this method utilizes information from already decoded groups, collaboratively extracting multi-level contextual features through local context modules (such as stacked residual blocks) and global context modules (such as attention mechanisms). Combined with auxiliary prior features decoded from the bitstream, it dynamically and accurately estimates the probability distribution parameters of the target group, thereby ensuring lossless or high-fidelity recovery of the quantized latent representation from the compressed bitstream. Furthermore, the probability distribution parameters of elements within the same group can be calculated in parallel, improving overall decoding efficiency. This method fully leverages the structured information in the compressed bitstream at the decoding end, achieving fast decoding while maintaining high reconstruction quality. This, in conjunction with the encoding method, achieves superior overall rate-distortion performance and faster processing speed.

[0205] The various technical features in the above embodiments can be combined arbitrarily, as long as there is no conflict or contradiction between the combinations of features. However, due to space limitations, they have not been described one by one. Therefore, the arbitrary combination of various technical features in the above embodiments is also within the scope of this specification.

[0206] Based on the same idea, embodiments of this specification also provide apparatus corresponding to the above methods.

[0207] Figure 9 The embodiments provided in this specification correspond to Figure 2 A schematic diagram of the structure of a training device for an image encoding and decoding model.

[0208] like Figure 9 As shown, the device may include: Training sample acquisition module 902 is used to acquire training images; An analysis transformation network 904 is used to input the training image into the analysis transformation network to obtain a latent representation; Quantization module 906 is used to quantize the latent representation to obtain a quantized latent representation; Context entropy model 908 is used for probabilistic modeling to estimate the probability distribution parameters of the quantized latent representation and calculate the bitrate loss; wherein, the probabilistic modeling includes: dividing the quantized latent representation into multiple slices in the channel dimension; constructing multiple groups based on the multiple slices, each group containing elements from all the slices, and elements belonging to the same group are distributed within each slice according to a predefined cross-slice structured spatial indexing pattern; when performing probabilistic modeling on a target group, estimating the probability distribution parameters of the elements in the target group based on information from the processed groups; A synthetic transformation network 910 is used to input the quantized latent representation into the synthetic transformation network to obtain a reconstructed image, and to calculate the distortion loss between the reconstructed image and the training image; The parameter update module 912 is used to update the parameters of the image encoding / decoding model based on the bit rate loss and the distortion loss.

[0209] based on Figure 9 The embodiments of this specification also provide some specific implementation schemes of the method, which are described below.

[0210] Optionally, the context entropy model includes a local context module, a global context module, and an entropy estimation module; the local context module is used to extract local context features from the information of the processed groups; the global context module is used to extract global context features from the information of the processed groups; and the entropy estimation module is used to estimate the probability distribution parameters based on the local context features and the global context features.

[0211] Optionally, before calculating the bitrate loss, the method further includes: extracting auxiliary information from the latent representation; quantizing the auxiliary information to obtain quantized auxiliary information; the calculation of bitrate loss specifically includes: calculating a first bitrate of the quantized latent representation based on the probability distribution parameters; the first bitrate being the sum of the bitrates of all packets; estimating the auxiliary probability distribution of the quantized auxiliary information using a total decomposition entropy model, and calculating a second bitrate of the quantized auxiliary information based on the auxiliary probability distribution; and determining the bitrate loss based on the first bitrate and the second bitrate.

[0212] Optionally, the training device for the image encoding / decoding model is further configured to: decode the quantization auxiliary information to obtain auxiliary prior features; and the probability modeling is also based on the auxiliary prior features.

[0213] It is understood that the modules mentioned above refer to computer programs or program segments used to perform one or more specific functions. Furthermore, the distinction between these modules does not imply that the actual program code must also be separate.

[0214] The above is a schematic scheme of an image codec model training device according to this embodiment. It should be noted that the technical solution of this image codec model training device and the technical solution of the image codec model training method described above belong to the same concept. For details not described in detail in the technical solution of the image codec model training device, please refer to the description of the technical solution of the image codec model training method described above.

[0215] Figure 10 The embodiments provided in this specification correspond to Figure 3 A schematic diagram of the structure of an image encoding device.

[0216] like Figure 10 As shown, the device may include: The analysis and transformation module 1002 is used to obtain the latent representation of the image to be encoded; Quantization module 1004 is used to quantize the latent representation to obtain a quantized latent representation; Grouping module 1006 is used to divide the quantized latent representation into multiple slices in the channel dimension; and to construct multiple groups based on the multiple slices; wherein each group contains elements from all the slices, and elements belonging to the same group are distributed in each slice according to a predefined cross-slice structured spatial index pattern. The entropy coding module 1008 performs entropy coding on the multiple groups to obtain the bit stream of the image to be encoded; wherein, when entropy coding is performed on the target group, the probability distribution parameters of the elements in the target group are estimated based on the encoded groups.

[0217] based on Figure 10 The embodiments of this specification also provide some specific implementation schemes of the method, which are described below.

[0218] Optionally, constructing multiple groups based on the multiple slices specifically includes: dividing each slice into multiple windows according to a predefined window size in the spatial dimension; for each window, assigning a set of sampling positions to each group according to the cross-slice structured spatial indexing pattern; each set of sampling positions contains at least one position in each slice; and selecting elements from the corresponding window of the corresponding slice according to the sampling positions to form each group.

[0219] Optionally, the cross-slice structured spatial indexing mode is implemented as a cross-slice cyclic shift mode; the set of index positions selected by all groups within any window covers all index positions within the arbitrary window.

[0220] Optionally, the entropy coding of the plurality of groups specifically includes: estimating the probability distribution parameters of elements in the target group based on the encoded groups; and performing entropy coding on the target group based on the probability distribution parameters.

[0221] Optionally, estimating the probability distribution parameters of elements in the target group based on the encoded group specifically includes: extracting local context features from the encoded group using a local context module; extracting global context features from the encoded group using a global context module; and estimating the probability distribution parameters of elements in the target group based on the local context features and the global context features using an entropy estimation module.

[0222] Optionally, the local context module includes stacked residual blocks; the global context module is constructed based on an attention mechanism.

[0223] Optionally, before estimating the probability distribution parameters of elements in the target group based on the encoded group, the method further includes: An auxiliary information encoder is used to extract auxiliary information from the latent representation; The auxiliary information is quantized to obtain quantized auxiliary information; An auxiliary information decoder is used to decode the quantized auxiliary information to obtain auxiliary prior features; The step of estimating the probability distribution parameters of elements in the target group based on the encoded group specifically includes: Based on the encoded grouping and the auxiliary prior features, the probability distribution parameters of the elements in the target grouping are estimated.

[0224] Optionally, when performing entropy encoding, the probability distribution parameters of each element in the same group are calculated in parallel.

[0225] It is understood that the modules mentioned above refer to computer programs or program segments used to perform one or more specific functions. Furthermore, the distinction between these modules does not imply that the actual program code must also be separate.

[0226] The above is an illustrative scheme of an image encoding device according to this embodiment. It should be noted that the technical solution of this image encoding device and the technical solution of the image encoding method described above belong to the same concept. For details not described in detail in the technical solution of the image encoding device, please refer to the description of the technical solution of the image encoding method described above.

[0227] Figure 11 The embodiments provided in this specification correspond to Figure 8 A schematic diagram of the structure of an image decoding device.

[0228] like Figure 11 As shown, the device may include: The receiving module 1102 is used to receive the bit stream of the image to be decoded; The entropy decoding module 1104 is used to perform entropy decoding on multiple sets of data in the bitstream to obtain grouped feature tensors; wherein, when performing entropy decoding on a target group, the probability distribution parameters of the elements in the target group are estimated based on the decoded groups; and the grouped feature tensors of all decoded groups are merged into a quantized latent representation. The synthesis transformation module 1106 is used to perform a synthesis transformation on the quantized latent representation to obtain a reconstructed image.

[0229] based on Figure 11 The embodiments of this specification also provide some specific implementation schemes of the method, which are described below.

[0230] Optionally, the entropy decoding of the bitstream specifically includes: Based on the decoded group, estimate the probability distribution parameters of the elements in the target group; Entropy decoding is performed on the target group based on the probability distribution parameters.

[0231] Optionally, estimating the probability distribution parameters of elements in the target group based on the decoded group specifically includes: Local context features are extracted from the decoded packets using the local context module; Global context features are extracted from the decoded packets using the global context module; An entropy estimation module is used to estimate the probability distribution parameters of elements in the target group based on the local context features and the global context features.

[0232] Optionally, before estimating the probability distribution parameters of elements in the target group based on the decoded group, the method further includes: Entropy decoding is performed on the bitstream to extract quantization auxiliary information; An auxiliary information decoder is used to decode the quantized auxiliary information to obtain auxiliary prior features; The estimation of probability distribution parameters of elements in the target group based on the decoded group specifically includes: Based on the decoded group and the auxiliary prior features, the probability distribution parameters of the elements in the target group are estimated.

[0233] Optionally, during entropy decoding, the probability distribution parameters of each element in the same group are calculated in parallel.

[0234] It is understood that the modules mentioned above refer to computer programs or program segments used to perform one or more specific functions. Furthermore, the distinction between these modules does not imply that the actual program code must also be separate.

[0235] The above is an illustrative scheme of an image encoding device according to this embodiment. It should be noted that the technical solution of this image encoding device and the technical solution of the image encoding method described above belong to the same concept. For details not described in detail in the technical solution of the image encoding device, please refer to the description of the technical solution of the image encoding method described above.

[0236] For ease of description, the above devices are described by dividing them into various modules or units based on their functions. Of course, when implementing one or more of these specifications, the functions of each module or unit can be implemented in the same or different software and / or hardware, or a module that performs the same function can be implemented by a combination of multiple sub-modules or sub-units, etc. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed.

[0237] Based on the same idea, this specification also provides devices corresponding to the above methods in its embodiments.

[0238] Figure 12 This is a structural block diagram of a computing device provided as an embodiment of this specification.

[0239] The computing device 1200 includes: Memory 1210 and processor 1220; The memory 1210 is used to store computer programs / instructions, and the processor 1220 is used to execute the computer programs / instructions. When the computer programs / instructions are executed by the processor 1220, they implement the steps of the image encoding method, the image decoding method, or the training method of the image encoding / decoding model.

[0240] Specifically, the components of the computing device 1200 include, but are not limited to, a memory 1210 and a processor 1220. The processor 1220 is connected to the memory 1210 via a bus 1230, and the database 1250 is used to store data.

[0241] The computing device 1200 also includes an access device 1240, which enables the computing device 1200 to communicate via one or more networks 1260. Examples of these networks include Public Switched Telephone Network (PSTN), Local Area Network (LAN), Wide Area Network (WAN), Personal Area Network (PAN), or combinations of communication networks such as the Internet. The access device 1240 may include one or more of any type of wired or wireless network interface (e.g., a network interface card (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) wireless interface, a Wi-MAX (Worldwide Interoperability for Microwave Access) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, a Near Field Communication (NFC) interface, and so on.

[0242] In one embodiment of this specification, the above-described components of the computing device 1200 and Figure 12 Other components, not shown, can also be connected to each other, for example, via a bus. It should be understood that... Figure 12 The block diagram of the computing device shown is for illustrative purposes only and is not intended to limit the scope of this application. Those skilled in the art can add or replace other components as needed.

[0243] The computing device 1200 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or personal computers (PCs). The computing device 1200 can also be a mobile or stationary server.

[0244] The processor 1220 executes the computer instructions to implement the steps of the image encoding method, the image decoding method, or the training method of the image encoding / decoding model.

[0245] The above is an illustrative scheme of a computing device according to this embodiment. It should be noted that the technical solution of this computing device belongs to the same concept as the technical solution of the image encoding method, the image decoding method, or the training method of the image encoding / decoding model. For details not described in detail in the technical solution of the computing device, please refer to the description of the technical solution of the image encoding method, the image decoding method, or the training method of the image encoding / decoding model.

[0246] An embodiment of this specification also provides a computer-readable storage medium storing computer instructions that, when executed by a processor, implement the steps of the image encoding method, the image decoding method, or the training method of the image encoding / decoding model as described above.

[0247] The above is an illustrative scheme of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of this storage medium belongs to the same concept as the technical solutions of the image encoding method, the image decoding method, or the training method of the image encoding / decoding model described above. Details not described in detail in the technical solution of the storage medium can be found in the descriptions of the technical solutions of the image encoding method, the image decoding method, or the training method of the image encoding / decoding model described above.

[0248] An embodiment of this specification also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the image encoding method, the image decoding method, or the training method of the image encoding / decoding model as described above.

[0249] The above is an illustrative scheme of a computer program product according to this embodiment. It should be noted that the technical solution of this computer program product belongs to the same concept as the technical solution of the image encoding method, the image decoding method, or the training method of the image encoding / decoding model described above. For details not described in detail in the technical solution of the computer program product, please refer to the description of the technical solution of the image encoding method, the image decoding method, or the training method of the image encoding / decoding model described above.

[0250] An embodiment of this specification also provides a method for storing a bitstream, comprising storing the bitstream in a storage medium, the bitstream being generated by the image encoding method described above.

[0251] The above is an illustrative scheme of a method for storing a bitstream according to this embodiment. It should be noted that the technical solution of this method belongs to the same concept as the technical solution of the image encoding method described above. For details not described in detail in the technical solution of the bitstream storage method, please refer to the description of the technical solution of the image encoding method described above.

[0252] An embodiment of this specification also provides a method for transmitting a bit stream, including transmitting a bit stream generated by the image encoding method described above.

[0253] The above is an illustrative scheme of a method for transmitting a bit stream according to this embodiment. It should be noted that the technical solution of this method belongs to the same concept as the technical solution of the image encoding method described above. For details not described in detail in the technical solution of the bit stream transmission method, please refer to the description of the technical solution of the image encoding method described above.

[0254] An embodiment of this specification also provides a computer-readable storage medium storing a bitstream generated by the image encoding method described above.

[0255] The above is an illustrative scheme of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of this computer-readable storage medium and the technical solution of the image encoding method described above belong to the same concept. For details not described in detail in the technical solution of the computer-readable storage medium, please refer to the description of the technical solution of the image encoding method described above.

[0256] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for the apparatus and device embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiments. The apparatus, device and method provided in the embodiments of this specification are corresponding to each other, and therefore the apparatus and device also have similar beneficial technical effects as the corresponding methods. Since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the corresponding apparatus and device will not be repeated here.

[0257] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0258] In the 1990s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to methodology). However, with technological advancements, many methodological improvements today can be considered direct improvements to hardware circuit structures. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that a methodological improvement cannot be implemented using hardware physical modules. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program a digital system themselves to "integrate" it onto a PLD, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must also be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also understand that by simply performing some logic programming on the method flow using one of these hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0259] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0260] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0261] For ease of description, the above devices are described separately by function as various units. Of course, in implementing this application, the functions of each unit can be implemented in one or more software and / or hardware.

[0262] Those skilled in the art will understand that one or more embodiments of this specification can be provided as a method, system, or computer program product. Therefore, the invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0263] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0264] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0265] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0266] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0267] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0268] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital character versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0269] This application can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0270] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. An image encoding method, comprising: Obtain the latent representation of the image to be encoded; The latent representation is quantized to obtain a quantized latent representation; The quantized latent representation is divided into multiple slices along the channel dimension; Based on the multiple slices, multiple groups are constructed; wherein each group contains elements from all the slices, and elements belonging to the same group are distributed within each slice according to a predefined cross-slice structured spatial index pattern; Entropy coding is performed on the multiple groups to obtain the bitstream of the image to be encoded; wherein, when entropy coding is performed on the target group, the probability distribution parameters of the elements in the target group are estimated based on the encoded groups.

2. The method as described in claim 1, wherein constructing multiple groups based on the multiple slices specifically includes: In the spatial dimension, each slice is divided into multiple windows according to a predefined window size; For each window, a set of sampling positions is assigned to each group according to the cross-slice structured spatial indexing pattern; Each set of sampling locations includes at least one location in each slice; Elements are selected from the corresponding windows of the corresponding slices according to the sampling positions to form various groups.

3. The method as described in claim 2, wherein the cross-slice structured spatial indexing mode is implemented as a cross-slice cyclic shift mode; the set of index positions selected by all groups within any window covers all index positions within the arbitrary window.

4. The method as described in claim 1, wherein the entropy coding of the plurality of groups specifically includes: Based on the encoded groups, estimate the probability distribution parameters of the elements in the target group; Entropy encoding is performed on the target group based on the probability distribution parameters.

5. The method of claim 4, wherein estimating the probability distribution parameters of elements in the target group based on the encoded group specifically includes: Local context features are extracted from the encoded groups using the local context module; Global context features are extracted from the encoded groups using the global context module; An entropy estimation module is used to estimate the probability distribution parameters of elements in the target group based on the local context features and the global context features.

6. The method of claim 5, wherein the local context module comprises stacked residual blocks; and the global context module is constructed based on an attention mechanism.

7. The method of claim 4, further comprising, before estimating the probability distribution parameters of elements in the target group based on the encoded group: An auxiliary information encoder is used to extract auxiliary information from the latent representation; The auxiliary information is quantized to obtain quantized auxiliary information; An auxiliary information decoder is used to decode the quantized auxiliary information to obtain auxiliary prior features; The step of estimating the probability distribution parameters of elements in the target group based on the encoded group specifically includes: Based on the encoded grouping and the auxiliary prior features, the probability distribution parameters of the elements in the target grouping are estimated.

8. In the method of any one of claims 1 to 7, when performing entropy encoding, the probability distribution parameters of each element in the same group are calculated in parallel.

9. An image decoding method, comprising: Receive the bitstream of the image to be decoded; Entropy decoding is performed on multiple sets of data in the bitstream to obtain a grouped feature tensor; wherein, when entropy decoding is performed on the target group, the probability distribution parameters of the elements in the target group are estimated based on the decoded groups; The group feature tensors of all decoded groups are merged into a quantized latent representation; The quantized latent representation is then subjected to a synthetic transformation to obtain the reconstructed image.

10. The method of claim 9, wherein the entropy decoding of the bitstream specifically includes: Based on the decoded group, estimate the probability distribution parameters of the elements in the target group; Entropy decoding is performed on the target group based on the probability distribution parameters.

11. The method of claim 10, wherein estimating the probability distribution parameters of elements in the target group based on the decoded group specifically includes: Local context features are extracted from the decoded packets using the local context module; Global context features are extracted from the decoded packets using the global context module; An entropy estimation module is used to estimate the probability distribution parameters of elements in the target group based on the local context features and the global context features.

12. The method of claim 10, further comprising, before estimating the probability distribution parameters of elements in the target group based on the decoded group: Entropy decoding is performed on the bitstream to extract quantization auxiliary information; An auxiliary information decoder is used to decode the quantized auxiliary information to obtain auxiliary prior features; The estimation of probability distribution parameters of elements in the target group based on the decoded group specifically includes: Based on the decoded group and the auxiliary prior features, the probability distribution parameters of the elements in the target group are estimated.

13. In the method of any one of claims 9 to 12, when performing entropy decoding, the probability distribution parameters of each element in the same group are calculated in parallel.

14. A training method for an image encoding / decoding model, the image encoding / decoding model comprising an analysis transform network, a context entropy model, and a synthesis transform network; the method comprising: Acquire training images; The training images are input into the analysis and transformation network to obtain the latent representation; The latent representation is quantized to obtain a quantized latent representation; Probabilistic modeling is performed using the context entropy model to estimate the probability distribution parameters of the quantized latent representation and calculate the bitrate loss; wherein, the probabilistic modeling includes: dividing the quantized latent representation into multiple slices along the channel dimension; constructing multiple groups based on the multiple slices, each group containing elements from all the slices, and elements belonging to the same group being distributed within each slice according to a predefined cross-slice structured spatial indexing pattern; when performing probabilistic modeling on a target group, estimating the probability distribution parameters of elements in the target group based on information from the processed groups; The quantized latent representation is input into the synthetic transform network to obtain a reconstructed image, and the distortion loss between the reconstructed image and the training image is calculated. The parameters of the image encoding / decoding model are updated based on the bit rate loss and the distortion loss.

15. The method of claim 14, wherein the context entropy model comprises a local context module, a global context module, and an entropy estimation module; the local context module is used to extract local context features from the information of the processed groups; the global context module is used to extract global context features from the information of the processed groups; and the entropy estimation module is used to estimate the probability distribution parameters based on the local context features and the global context features.

16. The method of claim 14, further comprising, before calculating the bit rate loss: Extract auxiliary information from the latent representation; The auxiliary information is quantized to obtain quantized auxiliary information; The calculation of bit rate loss specifically includes: Based on the probability distribution parameters, a first code rate of the quantized latent representation is calculated; the first code rate is the sum of the code rates of all packets. The auxiliary probability distribution of the quantization auxiliary information is estimated using a total decomposition entropy model, and the second code rate of the quantization auxiliary information is calculated based on the auxiliary probability distribution. The bitrate loss is determined based on the first bitrate and the second bitrate.

17. The method of claim 16, further comprising: The quantization auxiliary information is decoded to obtain auxiliary prior features; The probabilistic modeling is also based on the auxiliary prior features.

18. An image encoding apparatus, comprising: The analysis and transformation module is used to obtain the latent representation of the image to be encoded; The quantization module is used to quantize the latent representation to obtain a quantized latent representation; A grouping module is used to divide the quantized latent representation into multiple slices along the channel dimension; Based on the multiple slices, multiple groups are constructed; wherein each group contains elements from all the slices, and elements belonging to the same group are distributed within each slice according to a predefined cross-slice structured spatial index pattern; The entropy coding module performs entropy coding on the multiple groups to obtain the bitstream of the image to be encoded; wherein, when entropy coding is performed on the target group, the probability distribution parameters of the elements in the target group are estimated based on the already encoded groups.

19. An image decoding apparatus, comprising: The receiving module is used to receive the bit stream of the image to be decoded; The entropy decoding module is used to entropy decode multiple sets of data in the bitstream to obtain grouped feature tensors; wherein, when entropy decoding a target group, the probability distribution parameters of the elements in the target group are estimated based on the decoded groups; and the grouped feature tensors of all decoded groups are merged into a quantized latent representation. The synthesis transformation module is used to perform a synthesis transformation on the quantized latent representation to obtain a reconstructed image.

20. A training device for an image encoding / decoding model, comprising: The training sample acquisition module is used to acquire training images; An analytical transformation network is used to input the training image into the analytical transformation network to obtain a latent representation; The quantization module is used to quantize the latent representation to obtain a quantized latent representation; A context entropy model is used for probabilistic modeling to estimate the probability distribution parameters of the quantized latent representation and calculate the bitrate loss; wherein the probabilistic modeling includes: dividing the quantized latent representation into multiple slices along the channel dimension; constructing multiple groups based on the multiple slices, each group containing elements from all the slices, and elements belonging to the same group are distributed within each slice according to a predefined cross-slice structured spatial indexing pattern; when performing probabilistic modeling on a target group, estimating the probability distribution parameters of the elements in the target group based on information from the processed groups; A synthetic transform network is used to input the quantized latent representation into the synthetic transform network to obtain a reconstructed image, and to calculate the distortion loss between the reconstructed image and the training image; The model update module is used to update the parameters of the image encoding / decoding model based on the bitrate loss and the distortion loss.

21. A computing device, comprising: Memory and processor; The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions, which, when executed by the processor, implement the steps of the method according to any one of claims 1 to 17.

22. A computer-readable storage medium storing computer instructions that, when executed by a processor, implement the steps of the method according to any one of claims 1 to 17.

23. A method for storing a bit stream, comprising storing the bit stream in a storage medium, the bit stream being generated by the method of any one of claims 1 to 8.

24. A method for transmitting a bit stream, comprising transmitting the bit stream, the bit stream being generated by the method of any one of claims 1 to 8.

25. A computer-readable storage medium having a bit stream stored thereon, the bit stream being generated by the method of any one of claims 1 to 8.

Images