Data encoding and decoding methods and related devices

Abstract

The present application provides a data encoding and decoding method and related devices, and relates to the field of data processing. In the encoding method, side information feature extraction is performed on a first feature map of current data, and then quantization processing is performed to obtain a first quantized feature map. Entropy encoding is performed based on the first quantized feature map to obtain a first bitstream of current data. A scaling coefficient is obtained based on the first quantized feature map, and scaling processing is performed on a second feature map based on the scaling coefficient, and then quantization processing is performed to obtain a second quantized feature map. Scaling processing is performed on a first probability distribution parameter based on the scaling coefficient to obtain a second probability distribution parameter, and then entropy encoding is performed on the second quantized feature map based on the second probability distribution parameter to obtain a second bitstream of current data. The second feature map and the first probability distribution parameter are scaled by using the same scaling coefficient so that the degree of coincidence between the second probability distribution parameter and the second quantized feature map is higher, thereby improving the encoding accuracy of the second quantized feature map.

Description

Technical Field

[0001] This application claims priority to Chinese Patent Application No. 202210801030.7, titled "Data Encoding and Decoding Methods and Related Devices", filed with the China National Intellectual Property Administration on July 8, 2022, the entire disclosure of which is incorporated herein by reference.

[0002] Embodiments of this application relate to the field of data processing, and in particular, to data encoding and decoding methods and related devices.

Background Art

[0003] The purpose of data compression technology is to reduce redundant information in data so that data can be stored and transmitted in a more efficient format. In other words, data compression is a lossy or lossless representation of the original data in fewer bits. Data can be compressed because there is redundancy in the data. The purpose of data compression is to reduce the number of bits required to represent data by eliminating data redundancy.

[0004] How to improve the compression performance of data compression algorithms is a hot topic being studied by those skilled in the art.

Summary of the Invention

[0005] This application provides data encoding and decoding methods and related devices to improve data compression performance.

[0006] According to a first aspect, a data encoding method is provided, which is executed by a data encoding device. The method includes the following steps.

[0007] Side information feature extraction is performed on the first feature map of the current data to obtain a side information feature map. Next, quantization is performed on the side information feature map to obtain a first quantized feature map. Entropy encoding is performed on the first quantized feature map to obtain the first bitstream of the current data. Scaling is performed on the second feature map based on the scaling coefficient to obtain a scaled feature map. Quantization is performed on the scaled feature map to obtain a second quantized feature map. Scaling coefficients are obtained based on the first quantized feature map. Scaling is performed on the first probability distribution parameter based on the scaling coefficient to obtain the second probability distribution parameter. Entropy encoding is performed on the second quantized feature map based on the second probability distribution parameter to obtain a second bitstream of the current data.

[0008] The data includes at least one of the following: image data, video data, motion vector data of video data, audio data, point cloud data, or text data. In this embodiment, the first feature map is a feature map obtained by performing feature extraction on the complete current data, and the second feature map is a feature map obtained based on the current data.

[0009] Side Information means that existing Information Y is used to assist in the encoding of Information X so that the encoding length of Information X can be shortened. In other words, redundancy in Information X is reduced. Information Y is Side Information. In this embodiment of the present application, Side Information is information extracted from the first feature map and used to assist in the encoding and decoding of the first feature map. Furthermore, the bitstream is the bitstream generated after the encoding process.

[0010] In this solution, entropy encoding is performed based on a first quantized feature map to obtain a first bitstream of the current data. The first bitstream is the bitstream obtained after entropy encoding has been performed on the first quantized feature map. Furthermore, scaling coefficients may be obtained through estimation based on the first quantized feature map. In this way, a scaling process may be performed on a second feature map based on the scaling coefficients to obtain a scaled feature map. Next, a quantization process is performed on the scaled feature map to obtain a second quantized feature map. A scaling process is performed on the first probability distribution parameter based on the scaling coefficients to obtain a second probability distribution parameter. Finally, entropy encoding is performed on the second quantized feature map based on the second probability distribution parameter to obtain a second bitstream of the current data. The second bitstream is the bitstream obtained after entropy encoding has been performed on the second quantized feature map. The first bitstream and the second bitstream are used together as the total bitstream of the current data. Data encoding methods in the prior art only include the step of performing a scaling process on a feature map. In this solution, the second feature map and the first probability distribution parameter are scaled by using the same scaling coefficient so that the degree of agreement between the second probability distribution parameter and the second quantized feature map is higher, thereby improving the encoding accuracy of the second quantized feature map, i.e., improving data compression performance.

[0011] In some possible embodiments of the first aspect, the second feature map is a residual feature map obtained based on the current data, the third feature map being obtained by performing feature extraction on the current data.

[0012] In this embodiment, the third feature map is a feature map obtained by performing feature extraction on the complete current data, and the third feature map may be different from or the same as the first feature map. The residual of the third feature map, i.e., the residual feature map, may be obtained based on the third feature map and the first probability distribution parameter, and the residual feature map is used as the second feature map of the current data.

[0013] In this solution, during encoding, a scaling process is performed on the residual feature map of the current data to obtain a scaled feature map, and then a quantization process is performed on the scaled feature map so that the quantization loss of the scaled feature map is reduced. That is, the information loss of the second bitstream is reduced. This helps to improve the data quality of the reconstructed data obtained through decoding based on the second bitstream. Furthermore, compared to the encoding network structure in the prior art, this embodiment of the present application uses an encoding network structure that includes scaling of the residual feature map so that the network parameters of the entire encoding network, including the network for generating the first bitstream and the network for generating the second bitstream, can be optimized after the training of the entire encoding network is completed. Therefore, by using the encoding network structure in this embodiment of the present application, the amount of data in the total bitstream of the current data can be reduced and the encoding efficiency can be improved. In other words, the scaling of the residual feature map and the scaling of the first probability distribution parameter are combined, thereby further improving data compression performance.

[0014] In some possible embodiments of the first aspect, the second feature map is a feature map obtained by performing feature extraction on the current data. The first feature map is the same as or different from the second feature map.

[0015] In some possible embodiments of the first aspect, the first probability distribution parameters include the mean and / or variance.

[0016] In some possible embodiments of the first aspect, the first probability distribution parameter is obtained based on a first quantization feature map.

[0017] In some possible embodiments of the first aspect, the first probability distribution parameter is a pre-defined probability distribution parameter.

[0018] In some possible embodiments of the first aspect, the data encoding method further comprises the step of transmitting a first bitstream and a second bitstream.

[0019] After obtaining a first bitstream and a second bitstream of the current data using the data encoding method in this solution, the first bitstream and the second bitstream may be transmitted to another device according to requirements, thereby allowing the other device to process the first bitstream and the second bitstream.

[0020] In some possible embodiments of the first aspect, the first bitstream and the second bitstream are stored in the form of a bitstream file.

[0021] According to a second aspect, the present invention further provides a data encoding method which is performed by a data encoding device. The method comprises the following steps:

[0022] Side information feature extraction is performed on the first feature map of the current data to obtain a side information feature map; quantization is performed on the side information feature map to obtain a first quantized feature map. Entropy encoding is performed on the first quantized feature map to obtain the first bitstream of the current data. Scaling is performed on the residual feature map based on the scaling coefficient to obtain a scaled feature map, and quantization is performed on the scaled feature map to obtain a second quantized feature map. A residual feature map is obtained based on the second feature map of the current data and the first probability distribution parameters, and scaling coefficients are obtained based on the first quantized feature map. Entropy encoding is performed on the second quantized feature map based on the first probability distribution parameters to obtain a second bitstream of the current data.

[0023] In this solution, during encoding, a scaling process is performed on the residual feature map of the current data to obtain a scaled feature map, and then a quantization process is performed on the scaled feature map so that the quantization loss of the scaled feature map is reduced. That is, the information loss of the second bitstream is reduced. This helps to improve the data quality of the reconstructed data obtained through decoding based on the second bitstream. Furthermore, compared to the encoding network structure in the prior art, this embodiment of the present application uses an encoding network structure that includes scaling on the residual feature map so that the network parameters of the entire encoding network, including the network for generating the first bitstream and the network for generating the second bitstream, can be optimized after the entire encoding network has been trained. Therefore, by using the encoding network structure in this embodiment of the present application, the amount of data in the total bitstream of the current data can be reduced and the encoding efficiency can be improved. In general, the encoding method in this embodiment can further improve data compression performance.

[0024] In some possible embodiments of the second aspect, the first feature map is the same as or different from the second feature map.

[0025] In some possible embodiments of the second aspect, the data includes at least one of the following: image data, video data, motion vector data of video data, audio data, point cloud data, or text data.

[0026] In some possible embodiments of the second aspect, the first probability distribution parameters include the mean and / or variance.

[0027] In some possible embodiments of the second aspect, the first probability distribution parameter is obtained based on the first quantization feature map.

[0028] In some possible embodiments of the second aspect, the first probability distribution parameter is a pre-set probability distribution parameter.

[0029] According to a third aspect, the present application further provides a data decoding method, which is executed by a data decoding device. The method includes the following steps.

[0030] Perform entropy decoding based on the first bit stream of the current data to obtain a third feature map. Obtain a scaling coefficient based on the third feature map. Perform scaling processing on the first probability distribution parameter based on the scaling coefficient to obtain a second probability distribution parameter. Perform entropy decoding on the second bit stream of the current data based on the second probability distribution parameter to obtain a fourth feature map. Perform scaling processing on the fourth feature map based on the scaling coefficient to obtain a fifth feature map. Obtain the reconstructed data of the current data based on the fifth feature map.

[0031] If scaling is performed on the second feature map and the first probability distribution parameter during data encoding, the same scaling coefficients are used to process the first probability distribution parameter and the fourth feature map during data decoding to ensure the decoding accuracy of the fourth feature map. Furthermore, descaling may be performed on the fourth feature map based on the scaling coefficients to obtain a fifth feature map, and reconstructed data may be obtained based on the fifth feature map, thereby improving the accuracy and quality of the reconstructed data. In other words, by combining scaling on the first probability distribution parameter and descaling on the fourth feature map, the accuracy and quality of data decoding can be improved.

[0032] In some possible embodiments of the third aspect, the fourth feature map is a residual feature map, the fifth feature map is a scaled residual feature map, and the reconstructed data of the current data is obtained based on the fifth feature map: Adding the first probability distribution parameter to the fifth feature map to obtain the sixth feature map; and obtaining reconstructed data of the current data based on the sixth feature map. Includes.

[0033] When scaling and quantization operations are performed on the residual feature map during data encoding, the data volume of the first and second bitstreams in the data decoding method of this solution is smaller than that of the prior art. Therefore, the corresponding decoding processing load is smaller, and this solution can effectively improve decoding efficiency. Furthermore, the information loss in the second bitstream is smaller, thereby resulting in higher data quality for the reconstructed data obtained using this solution.

[0034] In some possible embodiments of the third aspect, the first probability distribution parameters include the mean and / or variance.

[0035] In some possible embodiments of the third aspect, the data includes at least one of the following: image data, video data, motion vector data of video data, audio data, point cloud data, or text data.

[0036] In some possible embodiments of the third aspect, the data decoding method further comprises the steps of receiving a first bitstream and a second bitstream of the current data.

[0037] In some possible embodiments of the third aspect, the data decoding method further comprises the step of obtaining a first probability distribution parameter based on a third feature map.

[0038] In some possible embodiments of the third aspect, the first probability distribution parameter is a pre-set probability distribution parameter.

[0039] According to a fourth aspect, the present invention further provides a data decoding method, which is performed by a data decoding device. The method comprises the following steps:

[0040] Entropy decoding is performed on the first bitstream of the current data to obtain a third feature map. Scaling coefficients are obtained based on the third feature map. Entropy decoding is performed on the second bitstream of the current data based on the first probability distribution parameters to obtain a fourth feature map. Scaling is performed on the fourth feature map based on the scaling coefficients to obtain a fifth feature map. A sixth feature map is obtained based on the first probability distribution parameters and the fifth feature map. Reconstructed data of the current data is obtained based on the sixth feature map.

[0041] When scaling and quantization operations are performed on the residual feature map during data encoding, the data volume of the first and second bitstreams in the data decoding method of this solution is smaller than that of the prior art. Therefore, the corresponding decoding processing load is smaller, and this solution can effectively improve decoding efficiency. Furthermore, the information loss in the second bitstream is smaller, thereby resulting in higher data quality for the reconstructed data obtained using this solution.

[0042] In some possible embodiments of the fourth aspect, the first feature map is the same as or different from the second feature map.

[0043] In some possible embodiments of the fourth aspect, the data includes at least one of the following: image data, video data, motion vector data of video data, audio data, point cloud data, or text data.

[0044] In some possible embodiments of the fourth aspect, the first probability distribution parameters include the mean and / or variance.

[0045] In some possible embodiments of the fourth aspect, the data decoding method further comprises the step of obtaining a first probability distribution parameter based on a third feature map.

[0046] In some possible embodiments of the fourth aspect, the first probability distribution parameter is a pre-set probability distribution parameter.

[0047] According to a fifth aspect, the present invention further provides a data encoder comprising a processing circuit configured to perform a data encoding method according to one of the embodiments of the first or second aspect.

[0048] According to a sixth aspect, the present invention further provides a computer-readable storage medium, the storage medium storing a bitstream, the bitstream being generated according to a data encoding method according to one of the embodiments of the first or second aspect.

[0049] According to a seventh aspect, the present invention further provides a data decoder comprising a processing circuit configured to perform a data decoding method according to any one of the embodiments of the third or fourth aspect.

[0050] According to the eighth aspect, the present invention further provides a computer program product comprising program code. When the program code is executed on a computer or processor, the computer program product is configured to perform a method according to any one of the embodiments of the first, second, third, or fourth aspect.

[0051] According to the ninth aspect, the present invention relates to a data encoder: One or more processors; and A computer-readable storage medium coupled to one or more processors, wherein the computer-readable storage medium stores a program, and when the program is executed by one or more processors, the data encoder is able to perform a data encoding method according to either one of the embodiments of the first or second embodiment. We further provide a data encoder equipped with the following features.

[0052] According to the tenth aspect, the present invention relates to a data decoder, which: One or more processors; and A computer-readable storage medium coupled to one or more processors, wherein the computer-readable storage medium stores a program, and when the program is executed by one or more processors, a data decoder is able to perform a data decoding method according to any one of the embodiments of the third or fourth aspect. Further providing is a data decoder equipped with the following features.

[0053] According to an eleventh aspect, the present invention further provides a computer-readable storage medium comprising program code. When the program code is executed by a computer device, the computer-readable storage medium is configured to perform a method according to any one of the embodiments of the first, second, third, or fourth aspect.

[0054] According to a twelfth aspect, the present invention further provides a computer-readable storage medium for storing a bitstream containing program code. Once the program code is executed by one or more processors, a decoder can perform a data decoding method according to any one of the embodiments of the third or fourth aspect.

[0055] According to a thirteenth aspect, the present invention further provides a chip comprising a processor and a data interface, the processor reading instructions stored in memory through the data interface and performing a method according to any one of the embodiments of the first, second, third, or fourth aspects.

[0056] Optionally, in one implementation, the chip may further provide memory, which stores instructions, and the processor is configured to execute the instructions stored in memory. When an instruction is executed, the processor is configured to perform a method according to any one of the embodiments of the first, second, third, or fourth embodiment. [Brief explanation of the drawing]

[0057] The following describes the attached drawings used in the embodiments of this application.

[0058] [Figure 1A] This is a diagram showing the architecture of a data coding system according to one embodiment of the present invention.

[0059] [Figure 1B] This is a block diagram of an example of a data coding system for implementing one embodiment of the present invention.

[0060] [Figure 1C] This is a block diagram of another example of a data coding system for implementing one embodiment of the present invention.

[0061] [Figure 2] This is a block diagram of an example of a data coding device for implementing one embodiment of the present invention.

[0062] [Figure 3] This is a block diagram of an example of a data coding device for implementing one embodiment of the present invention.

[0063] [Figure 4A] This is a diagram showing the structure of a data encoder according to one embodiment of the present invention.

[0064] [Figure 4B] This is a diagram showing the structure of a data decoder according to one embodiment of the present invention.

[0065] [Figure 4C] This is a diagram showing the structure of another data encoder according to one embodiment of the present invention.

[0066] [Figure 4D] This is a diagram showing the structure of another data encoder according to one embodiment of the present invention.

[0067] [Figure 4E] This is a diagram showing the structure of another data decoder according to one embodiment of the present invention.

[0068] [Figure 4F] This is a diagram showing the structure of another data encoder according to one embodiment of the present invention.

[0069] [Figure 4G] This is a diagram showing the structure of another data encoder according to one embodiment of the present invention.

[0070] [Figure 4H] This is a diagram showing the structure of another data decoder according to one embodiment of the present invention.

[0071] [Figure 4I] This is a diagram showing the structure of another data encoder according to one embodiment of the present invention.

[0072] [Figure 5A] This is a diagram showing the structure of an encoding network according to one embodiment of the present invention.

[0073] [Figure 5B] This is a diagram showing the structure of another encoding network according to one embodiment of the present invention.

[0074] [Figure 5C] This is a diagram showing the structure of a super pre-encoding network according to one embodiment of the present invention.

[0075] [Figure 5D] This is a diagram showing the structure of an ultra-pre-decoding network according to one embodiment of the present invention.

[0076] [Figure 5E] This is a diagram showing the structure of a decoding network according to one embodiment of the present invention.

[0077] [Figure 5F] This is a diagram showing the structure of a nonlinear unit according to one embodiment of the present invention.

[0078] [Figure 6A] This is a diagram showing the structure of another data coding system according to one embodiment of the present invention.

[0079] [Figure 6B]This is a diagram showing the structure of another data coding system according to one embodiment of the present invention.

[0080] [Figure 7] This is a schematic flowchart of a data encoding method according to one embodiment of the present invention.

[0081] [Figure 8] This is a schematic flowchart of another data encoding method according to one embodiment of the present invention.

[0082] [Figure 9] This is a schematic flowchart of a data decoding method according to one embodiment of the present invention.

[0083] [Figure 10] This is a schematic flowchart of another data decoding method according to one embodiment of the present invention. [Modes for carrying out the invention]

[0084] The technical solution of this application will be described below with reference to the attached drawings.

[0085] The embodiments of this application relate to applications. Therefore, for ease of understanding, the relevant concepts, such as the relevant terms used in the embodiments of this application, will be explained below.

[0086] In the embodiments of this application, expressions such as “example” or “for example” indicate that an example, illustration, or explanation is being given. None of the embodiments or design schemes described as “example” or “for example” in this application are described as being more preferable or having more advantages than other embodiments or design schemes. More precisely, the use of expressions such as “example” or “for example” is intended to present relative concepts in a particular manner.

[0087] In embodiments of this application, “at least one” means one or more, and “multiple” means two or more. “At least one of the following items (elements)” or similar expressions refer to any combination of these items, including a single item (element) or any combination of multiple items (elements). For example, at least one of a, b, or c may refer to: a, b, c, (a and b), (a and c), (b and c), or (a, b, and c), where a, b, and c may be singular or plural. The term “and / or” indicates a relational relationship between related objects, indicating that three relationships may exist. For example, A and / or B may refer to the following three cases: namely, only A exists, both A and B exist, and only B exists, where A and B may be singular or plural. The letter “ / ” generally indicates an “or” relationship between related objects. The sequence numbers of the steps in the embodiments of this application (e.g., step S1 and step S21) are used solely to distinguish between different steps and do not limit the order in which the steps are executed.

[0088] Furthermore, unless otherwise specified, ordinal numbers such as "first" and "second" in the embodiments of this application are used to distinguish between multiple objects and are not intended to limit the order, chronological order, priority, or importance of the multiple objects. For example, the designations "first device" and "second device" are for ease of explanation only and do not indicate any difference between the first device and the second device in terms of structure or importance. In some embodiments, the first device and the second device may be the same device.

[0089] Depending on the context ,below The term "when" as used in the embodiment may be interpreted as meaning "in the case of," "after," "according to the determination," or "according to the detection." .belowThe descriptions are merely optional embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc., made within the concepts and principles of the present application shall be within the scope of protection of the present application.

[0090] For ease of understanding, terms and concepts related to the embodiments of this application will be explained first.

[0091] (1) Quantization

[0092] Quantization is used to convert a continuous signal into a discrete signal. In the compression process, quantization means converting continuous features into discrete features. In entropy encoding, the probability values ​​of a probability distribution are typically converted from continuous to discrete values.

[0093] (2) Entropy encoding

[0094] Entropy encoding is an encoding process that follows the principle of entropy, ensuring that no information is lost. Information entropy is the average amount of information (a measure of uncertainty) in the source. Common entropy encodings include Shannon coding, Huffman coding, run-length coding, LZW encoding, and arithmetic coding. The LZW encoding algorithm, also known as the string table compression algorithm, performs compression by establishing a string table and representing long strings using short codes.

[0095] (3) Neural Networks

[0096] A neural network may comprise neurons. A neuron may be an operational unit that takes xs and a 1 intercept as inputs. The output of an operational unit may be as follows:

number

[0097] Here, s = 1, 2, ..., or n, where n is a natural number greater than 1, Ws is the weight of xs, b is the neuron's bias, and f is the neuron's activation function, which is used to introduce a nonlinear feature into the neural network and convert the input signal within the neuron into an output signal. The output signal of the activation function may be used as the input to the next convolutional layer. The activation function may be a sigmoid function. A neural network is a network formed by linking multiple single neurons together. That is, the output of one neuron may be the input of another neuron. The input of each neuron may be connected to the local receptive field of the previous layer to extract features of the local receptive field. The local receptive field may be a region containing multiple neurons.

[0098] (4) Deep neural networks

[0099] A deep neural network (DNN), also known as a multilayer neural network, can be understood as a neural network with multiple hidden layers. A DNN is divided based on the location of different layers, so that the neural network within the DNN can be classified into three types: input layers, hidden layers, and output layers. Generally, the first layer is the input layer, the last layer is the output layer, and the intermediate layers are the hidden layers. The layers are sufficiently connected to each other. Specifically, any neuron in the i-th layer is always connected to any neuron in the (i+1)-th layer.

[0100] While DNNs may appear complex, the function of each layer is not. Simply put, a DNN is a linear relational representation as follows:

number

number

number

number

number

number

number

number

number

[0101] In conclusion, the coefficient from the k-th neuron in the (L-1)th layer to the j-th neuron in the L-th layer is:

number

[0102] Note that the input layer has no parameter W. In deep neural networks, more hidden layers increase the network's ability to describe complex real-world cases. Theoretically, models with more parameters have higher complexity and greater "ability." This indicates that the model can complete more complex learning tasks. Training a deep neural network is the process of learning the weight matrix, and the ultimate goal of training is to obtain the weight matrix of all layers of the trained deep neural network (the weight matrix formed by the vectors W in many layers).

[0103] (5) Convolutional Neural Networks

[0104] A Convolutional Neural Network (CNN) is a deep neural network having a convolutional structure. A Convolutional Neural Network comprises a feature extraction unit having convolutional layers and subsampling layers, the feature extraction unit may be considered a filter. A convolutional layer is a neuron layer within a Convolutional Neural Network that performs a convolutional operation on an input signal. In the convolutional layer of a Convolutional Neural Network, a single neuron may be connected to only a few neighboring layer neurons. A single convolutional layer typically has multiple feature planes, each feature plane may contain several neurons in a rectangular configuration. Neurons in the same feature plane share weights. In this specification, shared weights are the convolutional kernel. Weight sharing may be understood as the image information extraction method being position-independent. The convolutional kernel may be initialized in the form of a random-size matrix. In the process of training a Convolutional Neural Network, the convolutional kernel may acquire appropriate weights through learning. Furthermore, a direct benefit of weight sharing is that it reduces the number of connections between layers in a convolutional neural network, thereby lowering the risk of overfitting.

[0105] (6) Loss function

[0106] In the process of training a deep neural network, the output of the deep neural network is expected to be as close as possible to the predicted value that is actually expected. Therefore, the network's current predicted value can be compared to the expected target value, and then the weight vectors of each layer of the neural network are updated based on the difference between the predicted value and the target value (of course, there is usually an initialization process before the first update, specifically, the parameters of all layers of the deep neural network are pre-configured). For example, if the network's predicted value is large, the weight vectors are adjusted to decrease the predicted value, and this adjustment is carried out continuously until the deep neural network can predict the expected target value or a value very close to the expected target value. Therefore, it is necessary to pre-define "a method for obtaining the difference between the predicted value and the target value through comparison." This is the loss function or objective function. The loss function and objective function are important equations used to measure the difference between the predicted value and the target value. The loss function is used as an example. A larger output value (Loss) of the loss function indicates a larger difference. Therefore, training a deep neural network is a process of minimizing loss as much as possible.

[0107] (7) Backpropagation algorithm

[0108] During the training process, the neural network may correct the values ​​of its initial neural network model parameters by using a backpropagation (BP) algorithm so that the reconstruction error loss of the neural network model becomes smaller and smaller. Specifically, the input signal is passed forward until the error loss is generated in the output, and the parameters of the initial neural network model are updated through backpropagation of information about the error loss, thereby converging the error loss. The backpropagation algorithm is a backpropagation motion centered on the error loss, intended to obtain the parameters, such as the weight matrix, of the optimal neural network model.

[0109] In the prior art, there is a need to improve the compression performance of data compression algorithms. For example, data occupies a large number of bits after compression. In view of this, the embodiments of the present invention provide a data encoding and decoding method that effectively improves data compression performance.

[0110] For example, the data in the data encoding method and / or data decoding method in embodiments of the present application includes at least one of the following: image data, video data, motion vector (MV) data of video data, audio data, point cloud data, or text data. Image data may be one image or at least two images. Video data is a sequence of images and is essentially formed by a group of consecutive images. For example, with respect to image data or video data, the data encoding method and / or data decoding method in embodiments of the present application may be applied to at least one image in the image data or video data to perform encoding and decoding on at least one image. As another example, with respect to encoding and decoding video data, the data encoding method and data decoding method in embodiments of the present application may be used to process frames of image A to obtain a reconstructed frame corresponding to image A. Then, the reconstructed frame is used to predict the next frame of image A to obtain a predicted image of the next frame of the image. Then, the difference between the next frame of the image and the predicted image is compressed. The reconstruction result obtained during decoding is the sum of the predicted image and reconstruction residual for the next frame of the image. With respect to video data, the pixel data for each video frame is typically encoded as blocks of pixels (also referred to herein as “pixel blocks,” “encoding units,” and “macroblocks”). Motion vectors are used to describe the offset vector of the position of a macroblock in the video frame relative to the position of a macroblock in the reference frame. Motion vector data is at least one motion vector obtained based on the video data.

[0111] Point cloud data is a set of vectors in a three-dimensional coordinate system. Scan data is recorded in the form of points, each point having a three-dimensional coordinate, and some points may have color information or reflectance intensity information. In addition to geometric location, some point cloud data also have color information. Color information is typically obtained by acquiring a color image using a camera, and then the color information of the pixels at the corresponding location is assigned to the corresponding point in the point cloud. Intensity information is the echo intensity collected by a receiving device for a laser scanner. Intensity information is related to the surface material, roughness, the angle of incidence of the target, the radiant energy of the device, and the laser wavelength. Furthermore, text data is data containing text, and the data format of text data may be a TXT file, PDF file, Word file, Excel file, etc. Furthermore, application scenarios of the data encoding method and / or data decoding method in embodiments of this application include cloud storage services, cloud monitoring, live streaming, etc.

[0112] Figure 1A is a diagram of the architecture of a data coding system according to one embodiment of the present invention. For example, the data coding system in Figure 1A comprises a data encoder and a data decoder. The data encoder has an encoding unit, an entropy encoding unit, and a storage unit. The data decoder has a decoding unit, an entropy decoding unit, and a load unit.

[0113] The encoding unit is configured to convert the data to be processed into feature data with lower redundancy and to obtain the probability distribution corresponding to the feature data. The data to be processed includes at least one of the following: image data, video data, motion vector data of video data, audio data, point cloud data, or text data.

[0114] The entropy encoding unit is configured to perform lossless encoding on feature data based on probabilities corresponding to the feature data, thereby further reducing the amount of data transmitted during the compression process.

[0115] The storage unit is configured to save the data files generated by the entropy encoding unit to the corresponding storage location.

[0116] The load unit is configured to load data files from their corresponding storage locations and input the data files into the entropy decoding unit.

[0117] The entropy decoding unit is configured to perform entropy decoding on a data file and retrieve the processed data.

[0118] The decoding unit is configured to perform an inverse transformation on the processed data output by the entropy decoding unit and parse the processed data into reconstructed data.

[0119] For example, after a data acquisition device collects data to be processed, it performs a compression process on the data. The specific process is as follows: The encoding unit processes the data to be processed to obtain the features to be encoded and their corresponding probability distributions, inputs the features to be encoded and their probability distributions to the entropy encoding unit for processing to obtain a bitstream file, and the storage unit saves the bitstream file. When the bitstream file is decompressed, the specific process is as follows: The load unit loads the bitstream file from storage and inputs the bitstream file to the entropy decoding unit, and the entropy decoding unit and the decoding unit cooperate to obtain reconstructed data corresponding to the bitstream file. Furthermore, the reconstructed data may be output, for example, for display purposes.

[0120] In the following embodiments of the coding system 10, the encoder 20 and decoder 30 are described with reference to Figures 1B and 1C.

[0121] Figure 1B is a block diagram of an example of a coding system 10 that may use the technology of the present invention, for example, a data coding system 10 (or simply referred to as coding system 10). The data encoder 20 (or simply referred to as encoder 20) and data decoder 30 (or simply referred to as decoder 30) in coding system 10 represent devices, etc., that may be configured to perform the technology based on the various examples described herein.

[0122] As shown in Figure 1B, the coding system 10 includes a source device 12. The source device 12 is configured to provide encoded data 21, such as an encoded image, encoded video, or encoded audio, to a destination device 14 that is configured to decode the encoded data 21.

[0123] The source device 12 has an encoder 20 and, in addition, may optionally have a data source 16, a preprocessor (or preprocessing unit) 18, and a communication interface (or communication unit) 22.

[0124] The data source 16 may include, or may not include, any type of data acquisition device and / or any type of data generation device configured to acquire data. In this embodiment, the data includes at least one of the following: image data, video data, motion vector data of video data, audio data, point cloud data, or text data.

[0125] For example, the data is image data. In this case, the data source 16 may include, or may not include, any type of image capture device configured to capture real-world images, and / or any type of image generation device, such as a computer graphics processing unit configured to generate computer-animated images, or any type of device configured to acquire and / or provide real-world images, computer-generated images (e.g., screen content or virtual reality (VR) images), and / or any combination thereof (e.g., augmented reality (AR) images). The data source may be any type of memory or storage that stores any of the aforementioned images.

[0126] For example, the data is video data. In this case, the data source 16 may include, or may not include, any type of video recording device and / or any type of video generating device configured to capture real-world images and generate video, such as a computer graphics processor configured to generate computer animation or any type of device configured to acquire and / or provide real-world video or computer-generated video (e.g., video acquired through screen recording). The data source may be any type of memory or storage that stores any of the aforementioned videos.

[0127] For example, the data is audio data. In this case, the data source 16 may include, or may not include, any type of audio capture device configured to capture real-world sounds and generate audio, and / or any type of audio generation device, such as an audio processor configured to generate virtual audio (e.g., virtual human voice) or any type of device configured to acquire and / or provide real-world audio or computer-generated audio (e.g., audio acquired through screen recording). The data source may be any type of memory or storage for storing any of the aforementioned audio.

[0128] For example, the data is point cloud data. In this case, the data source 16 may include or may include any type of device configured to acquire point cloud data, such as a three-dimensional laser scanner or an imaging scanner.

[0129] To distinguish the processing performed by the preprocessor (or preprocessing unit) 18, the data 17 may also be referred to as the original data 17.

[0130] The preprocessor 18 is configured to receive (original) data 17, preprocess the data 17, and obtain preprocessed data 19. For example, the data is image data. In this case, the preprocessing performed by the preprocessor 18 may include cropping, color format conversion (e.g., conversion from RGB to YCbCr), color correction, or noise reduction. It can be understood that the preprocessing unit 18 may be an optional component.

[0131] The encoder 20 is configured to receive the preprocessed data 19 and provide the encoded data 21 (further explanation is given below based on Figures 4A, 4C, 4D, 4F, 4G, 4I, etc.).

[0132] The communication interface 22 of the source device 12 may be configured to receive encoded data 21 and transmit the encoded data 21 (or any further processed version thereof) to another device such as the destination device 14 or any other device via the communication channel 13 for storage or direct reconstruction.

[0133] The destination device 14 includes a decoder 30 and, in addition, may optionally include a communication interface (or communication unit) 28, a post-processor (or post-processing unit) 32, and a display device 34.

[0134] The communication interface 28 of the destination device 14 is configured to receive encoded data 21 (or any further processed version thereof) directly from the source device 12 or from any other source device such as a storage device, for example, a storage device for encoded data, and to provide the encoded data 21 to the decoder 30.

[0135] Communication interfaces 22 and 28 may be configured to transmit or receive encoded data 21 through a direct communication link between the source device 12 and the destination device 14, for example, through a direct wired or wireless connection, or through any type of network, for example, a wired or wireless network or any combination thereof, or any type of private and public network or any combination thereof.

[0136] For example, the communication interface 22 may be configured to encapsulate the encoded data 21 in a preferred format such as a packet, and / or process the encoded data 21 through any type of transmission encoding or processing for transmission over a communication link or communication network.

[0137] Communication interface 28 corresponds to communication interface 22 and may be configured, for example, to receive transmitted data, process the transmitted data through any type of corresponding transmission decoding or processing and / or deencapsulation, and obtain encoded data 21.

[0138] Communication interfaces 22 and 28 may both be configured as unidirectional or bidirectional communication interfaces, as indicated by the arrows pointing from source device 12 to destination device 14, corresponding to communication channel 13 in Figure 1B, and may be configured to send and receive messages, establish connections, and confirm and exchange any other information such as encoded data.

[0139] The decoder 30 is configured to receive encoded data 21 and provide decoded data (or reconstructed data) 31 (further explanation is given below based on Figures 4B, 4E, 4H, etc.).

[0140] The post-processor 32 is configured to post-process the decoded data 31 to obtain post-processed data 33. For example, the data is image data, and the post-processing performed by the post-processing unit 32 may include any other processing, such as color format conversion (e.g., conversion from YCbCr to RGB), color correction, cropping, resampling, or generation of post-processed data 33 for display by a display device 34 or the like.

[0141] The display device 34 is configured to receive post-processed data 33 for displaying the data to a user, viewer, etc. The display device 34 may include or be any type of display configured to represent the reconstructed data, such as an integrated or external display screen or display. For example, the display screen may include a liquid crystal display (LCD), an organic light-emitting diode (OLED) display, a plasma display, a projector, a micro-LED display, a liquid crystal on silicon (LCoS), a digital light processor (DLP), or any other type of display screen.

[0142] The coding system 10 further comprises a training engine 25. The training engine 25 is configured to train the neural network in the encoder 20 or decoder 30 so that the encoder 20 can obtain encoded data 21 when data 17 or preprocessed data 19 is input, or the decoder 30 can obtain decoded data 31 when encoded data 21 is input. Optionally, the input data further includes superprior information.

[0143] Training data may be stored in a database (not shown in Figure 1B), and the training engine 25 acquires a neural network through training based on the training data. The neural network is the neural network in the encoder 20 or decoder 30. Note that the source of the training data is not limited to the embodiments of this application. For example, the training data may be acquired from the cloud or another location to perform neural network training.

[0144] The neural network acquired by the training engine 25 through training may be applied to the coding system 10 or the coding system 40, for example, to the source device 12 (e.g., encoder 20) or destination device 14 (e.g., decoder 30) shown in Figure 1B. For example, the training engine 25 may acquire the neural network through training on the cloud, and then the coding system 10 may download the neural network from the cloud and use it.

[0145] Figure 1B shows that the source device 12 and the destination device 14 are separate devices, but the device embodiment may comprise both the source device 12 and the destination device 14, or may comprise the functions of both the source device 12 and the destination device 14, that is, it may comprise both the source device 12 or its corresponding function and the destination device 14 or its corresponding function. In such embodiments, the source device 12 or its corresponding function and the destination device 14 or its corresponding function may be implemented by using the same hardware and / or software, or by using separate hardware and / or software, or any combination thereof.

[0146] Those skilled in the art will see from the above description that the presence and (precise) division of different units or functions of the source device 12 and / or destination device 14 shown in Figure 1B may vary depending on the actual device and application.

[0147] The encoder 20 or the decoder 30 or both may be implemented using the processing circuitry shown in Figure 1C, for example, one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, hardware, dedicated video coding processors, or any combination thereof. The encoder 20 may be implemented using the processing circuitry 43 to embody various modules described with reference to the encoder 20 in Figure 1C and / or any other encoder systems or subsystems described herein. The decoder 30 may be implemented using the processing circuitry 43 to embody various modules described with reference to the decoder 30 in Figure 1C and / or any other decoder systems or subsystems described herein. The processing circuitry 43 may be configured to perform various operations described below. As shown in Figure 3, when some techniques are implemented in software, the device may store the software instructions in a suitable non-temporary computer-readable storage medium and execute the hardware instructions by using one or more processors to implement the techniques of this application. Either the encoder 20 or the decoder 30 may be integrated into a single device as part of a codec (CODEC), as shown in Figure 1C.

[0148] The source device 12 and destination device 14 may include any type of handheld or stationary device, such as a notebook or laptop computer, mobile phone, smartphone, tablet or tablet computer, camera, desktop computer, server, set-top box, television, display device, digital media player, video gaming console, video streaming device (such as a content service server or content distribution server), broadcast receiver device, broadcast transmitter device, etc., and may or may not use any type of operating system. In some cases, the source device 12 and destination device 14 may include components for wireless communication. Thus, the source device 12 and destination device 14 may be wireless communication devices.

[0149] In some cases, the coding system 10 shown in Figure 1B is merely an example, and the technology provided herein is applicable to coding configurations, which do not necessarily involve any data communication between the encoding device and the decoding device. In another example, data is retrieved from local storage and transmitted over a network, for example. The encoding device may encode the data and store it in storage, and / or the decoding device may retrieve the data from storage and decode the data. In some examples, encoding and decoding do not communicate with each other but are performed simply by devices that encode data to storage and / or retrieve data from storage and decode the data.

[0150] Figure 1C is an illustrative diagram of an example coding system 40 comprising an encoder 20 and / or decoder 30 according to an exemplary embodiment. The coding system 40 may include the encoder 20 and decoder 30 (and / or an encoder / decoder implemented using a processing circuit 43), an antenna 42, one or more memories 44, and / or a display device 45. For example, when the data is image data, the coding system may further include an imaging device 41.

[0151] As shown in Figure 1C, the imaging device 41, antenna 42, processing circuit 43, encoder 20, decoder 30, memory 44, and / or display device 45 can communicate with each other. In different examples, the coding system 40 may consist only of the encoder 20 or only of the decoder 30. Of course, the coding system 40 is not limited to the configuration shown in Figure 1C and may consist of more or fewer components than those shown in Figure 1C.

[0152] In some examples, the antenna 42 may be configured to transmit or receive an encoded bitstream of data. Furthermore, in some examples, the display device 45 may be configured to present the reconstructed data. The processing circuit 43 may include application-specific integrated circuit (ASIC) logic, a graphics processing unit, a general-purpose processor, etc. Furthermore, the memory 44 may be any type of memory, such as volatile memory (e.g., static random access memory (SRAM) or dynamic random access memory (DRAM)), non-volatile memory (e.g., flash memory), etc. In an unspecified example, the memory 44 may be implemented by a cache memory. In another example, the processing circuit 43 may include memory (e.g., a cache) for implementing an image buffer.

[0153] In some examples, the coding system 40 may further include a decoder 30 coupled to an antenna 42 and configured to decode the encoded bitstream. A display device 45 is configured to present the reconstructed data.

[0154] In this embodiment of the present application, with reference to the example described with reference to the encoder 20, it should be understood that the decoder 30 may be configured to perform the reverse process.

[0155] Figure 2 shows a data coding device 200 according to one embodiment of the present application. The data coding device 200 is suitable for carrying out the disclosed embodiments described herein. In one embodiment, the data coding device 200 may be a decoder, for example, the decoder 30 in Figure 1C, or an encoder, for example, the encoder 20 in Figure 1C.

[0156] The data coding device 200 includes: an inlet port 210 (or input port 210) and a receiver unit (Rx) 220 configured to receive data; a processor, logic unit, or central processing unit (CPU) 230 configured to process the data, for example, the processor 230 in this specification may be a neural network processor 230; a transmitter unit (Tx) 240 and an exit port 250 (or output port 250) configured to transmit data; and a memory 260 configured to store the data. For example, the data coding device 200 may further include optical-to-electrical (OE) components and electrical-to-optical (EO) components coupled to the inlet port 210, receiver unit 220, transmitter unit 240, and exit port 250 for the entry or exit of optical or electrical signals.

[0157] The processor 230 is implemented using hardware and software. The processor 230 may be implemented as one or more processor chips, cores (e.g., a multi-core processor), FPGAs, ASICs, and DSPs. The processor 230 communicates with the input port 210, the receiver unit 220, the transmitter unit 240, the output port 250, and the memory 260. The processor 230 includes a coding module 270 (e.g., a coding module 270 based on a neural network NN). The coding module 270 implements the embodiments disclosed above. For example, the coding module 270 performs, processes, prepares, or provides various coding operations. Thus, the coding module 270 brings substantial improvement to the functionality of the data coding device 200 and affects the switching of the data coding device 200 between different states. Alternatively, the coding module 270 is implemented using instructions stored in the memory 260 and executed by the processor 230.

[0158] Memory 260 includes one or more magnetic disks, tape drives, and solid-state drives, and may be used as an overflow data storage device, and is configured to store such programs when they are selected for execution, as well as instructions and data read during program execution. Memory 260 may be volatile and / or non-volatile, and may be read-only memory (ROM), random access memory (RAM), ternary content-addressable memory (TCAM), and / or static random access memory (SRAM).

[0159] Figure 3 is a simplified block diagram of a data coding device 300 according to an exemplary embodiment. The device 300 may be used as either or both of the source device 12 and the destination device 14 in Figure 1B.

[0160] The processor 302 in the device 300 may be a central processing unit. Alternatively, the processor 302 may be any other type of device or multiple devices capable of manipulating or processing existing or future-developed information. The disclosed implementation can be carried out using a single processor, for example, the processor 302 shown in Figure 3, but higher speed and efficiency can be achieved by using more than one processor.

[0161] In one implementation, the memory 304 in the device 300 may be a read-only memory (ROM) device or a random access memory (RAM) device. Any other suitable type of storage device may be used as memory 304. Memory 304 may include code and data 306 accessed by the processor 302 through the bus 312. Memory 304 may further include an operating system 308 and an application 310. Application 310 includes at least one program that enables the processor 302 to perform the methods described herein. For example, application 310 may include applications 1 through N, further including a data coding application that performs the methods described herein, namely, data encoding and / or data decoding.

[0162] The device 300 may further include one or more output devices, for example, a display 314. In one example, the display 314 may be a touch-sensitive display, where the display is combined with a touch-sensitive element that can be configured to sense touch input. The display 314 may be coupled to the processor 302 via the bus 312.

[0163] Although bus 312 in device 300 is shown as a single bus in this specification, bus 312 may include multiple buses. Furthermore, secondary storage may be directly coupled to another component of device 300, or it may be accessed over a network, and may include a single integrated unit such as a storage card or multiple units such as multiple storage cards. Thus, device 300 may have a wide variety of configurations.

[0164] Figure 4A is a diagram showing the structure of a data encoder according to one embodiment of the present invention. In the example shown in Figure 4A, the data encoder 20 comprises an input terminal (or input interface) 401, an encoding network (Encoder) 402, a hyperencoder network (HyperEncoder) 403, a quantization unit 404, a quantization unit 405, an entropy encoding unit 406, an entropy encoding unit 407, an entropy estimation unit (Entropy) 408, a hyperentropy estimation unit (HyperEntropy) 409, and a hyperdecoder network (HyperDecoder) 410. The quantization units 404 and 405 may be the same quantization unit or two independent quantization units. Similarly, the entropy encoding units 406 and 407 may be the same entropy encoding unit or two independent entropy encoding units. The entropy estimation unit 408 is also called the entropy parameter model unit, and the ultra-prior entropy estimation unit 409 is an entropy parameter model unit that uses a pre-set distribution.

[0165] The data to be encoded may be received through the data encoder 20, input terminal 401, etc. The data to be encoded includes at least one of the following: image data, video data, motion vector data of video data, audio data, point cloud data, or text data.

[0166] The encoding network 402 is configured to extract features from the data to be encoded and obtain a first feature map y1 and a second feature map y2. The first feature map y1 is different from the second feature map y2. The method for obtaining the first feature map y1 and the second feature map y2 is not particularly limited.

[0167] For example, the data is image data. For example, compared to the original image, the first and second feature maps output by the encoding network 402 may have changed sizes and have redundant information removed, thereby making entropy encoding easier to perform.

[0168] The ultra-pre-encoding network 403 is configured to extract more concise information from the first feature map y1 to obtain a side information feature map z. For example, the size of the side information feature map z is smaller than that of the first feature map y1.

[0169] The quantization unit 404 performs quantization on the side information feature map z to obtain integer feature data, i.e., the first quantized feature map.

number

[0170] The ultra-pre-decoding network 410 uses the first quantized feature map

number

[0171] Furthermore, for example, the ultra-pre-decoding network 410 uses the first quantized feature map

number

[0172] The residual of the second feature map y2, i.e., the residual feature map C, may be obtained based on the second feature map and the first probability distribution parameter X. Specifically, the residual feature map C may be obtained by subtracting X from y2. For example, the residual feature map C is obtained by subtracting the mean μ from the second feature map y2.

[0173] A scaling process is performed on the residual feature map C based on the scaling coefficient △ to obtain the scaled residual feature map C. Specifically, the scaled residual feature map C may be obtained by dividing the residual feature map C by the scaling coefficient △.

[0174] The quantization unit 405 is configured to perform a quantization process on the scaled residual feature map C to obtain integer feature data, i.e., a second quantized feature map.

[0175] The ultra-prior entropy estimation unit 409 generates a first quantization feature map based on a pre-set distribution.

number

[0176] The entropy encoding unit 406 uses the first quantization feature map estimated by the ultra-prior entropy estimation unit 409.

number

number

[0177] The entropy estimation unit 408 is configured to obtain the probability distribution of a second quantized feature map based on a first probability distribution parameter X.

[0178] The entropy encoding unit 407 is configured to perform entropy encoding on a second quantization feature map based on probabilities estimated by the entropy estimation unit 408 to obtain a second bitstream B. The first bitstream A and the second bitstream B are used as a total bitstream of the data to be encoded. The data encoder may output the total bitstream of the data through an output terminal (or output interface) (not shown in Figure 4A).

[0179] Figure 4B is a diagram of the structure of a data decoder according to one embodiment of the present invention. The data decoder shown in Figure 4B is configured to decode the total bitstream of data acquired through processing by the data encoder shown in Figure 4A. The data decoder 30 comprises an entropy estimation unit 408, an entropy decoding unit 414, a super-pre-entropy estimation unit 409, an entropy decoding unit 413, a super-pre-decoding network 410, a decoding network 412, and an output terminal (or output interface) 411.

[0180] The data decoder 30 may obtain the total bitstream to be decoded, i.e., the first bitstream A and the second bitstream B, through the input terminal (or input interface) (not shown in Figure 4B).

[0181] The ultra-prior entropy estimation unit 409 generates a first quantization feature map based on a pre-set distribution.

number

[0182] The entropy decoding unit 413 is configured to perform entropy decoding on the first bitstream A based on probabilities estimated by the ultra-prior entropy estimation unit 409 to obtain a third feature map. The entropy decoding unit 413 performs entropy decoding by using a distribution that matches the distribution used by the entropy encoding unit 406.

[0183] The ultra-pre-decoding network 410 is configured to generate a scaling coefficient Δ and a first probability distribution parameter X based on a third feature map. The first probability distribution parameter X represents the probability distribution of the fourth feature map, where X includes, but is not limited to, a mean μ and / or variance σ. The first probability distribution parameter X is a matrix with the same scale as the fourth feature map. Each element in the matrix represents a probability distribution parameter of one of the elements in the fourth feature map. Each probability distribution parameter of an element includes, but is not limited to, a mean and / or variance. That is, one fourth feature map corresponds to one mean matrix and / or one variance matrix.

[0184] Furthermore, for example, the ultra-pre-decoding network 410 may be configured solely to generate a scaling coefficient △ based on a third feature map, and the first probability distribution parameter X is a pre-set probability distribution parameter depending on the actual situation. This is not particularly limited.

[0185] The entropy estimation unit 408 is configured to obtain the probability distribution of the fourth feature map based on the first probability distribution parameter X.

[0186] The entropy decoding unit 414 is configured to perform entropy decoding on the second bitstream B based on the probabilities estimated by the entropy estimation unit 408 to obtain a fourth feature map.

[0187] The fifth feature map may be obtained based on the scaling factor △ and the fourth feature map. That is, the fifth feature map is , the fourth special This can be obtained by multiplying the feature map by a scaling factor △. In this case, the relevant description in Figure 5B may be referenced. In this case, the scaling process is a scaling-down process.

[0188] The sixth feature map may be obtained based on the fifth feature map and the first probability distribution parameter X. That is, the sixth feature map may be obtained by adding the first probability distribution parameter X to the fifth feature map. For example, corresponding to Figure 4A, in this case, the sixth feature map may be obtained by adding the mean μ to the fifth feature map.

[0189] The decoding network 412 is configured to reverse-map the sixth feature map to reconstruct the data.

[0190] The output terminal 411 is configured to output reconstructed data.

[0191] Figure 4C is a diagram of the structure of another data encoder according to one embodiment of the present invention. The data encoder shown in Figure 4C has the same configuration as the data encoder shown in Figure 4A. The difference is that the encoding network 402 is configured to perform feature extraction on the data to be processed to obtain a feature map y (i.e., in this case, the first feature map y1 is the same as the second feature map y2). In this case, the data decoder configured to decode the total bitstream of data obtained through the processing in Figure 4C has the same structure as that in Figure 4B.

[0192] Figure 4D is a diagram of the structure of another data encoder according to one embodiment of the present invention. The difference between the data encoder shown in Figure 4D and the data encoder shown in Figure 4A is that in Figure 4D, there are scaling and quantization operations on the residual feature map and scaling operations on the probability distribution parameter. Specifically, after the ultra-pre-decoding network 410 obtains the first probability distribution parameter X and the scaling coefficient △, it performs a scaling operation on the first probability distribution parameter X based on the scaling coefficient. That is, it divides the first probability distribution parameter X by the scaling coefficient △ to obtain the second probability distribution parameter (X / △). The entropy encoding unit 407 is configured to cooperate with the entropy estimation unit 408 to perform entropy encoding on the second quantization feature map based on the second probability distribution parameter (X / △) to obtain the second bitstream B.

[0193] Figure 4E is a diagram of the structure of another data decoder according to one embodiment of the present invention. The data decoder shown in Figure 4E is configured to decode the total bitstream of data acquired through processing by the data encoder shown in Figure 4D. The data decoder shown in Figure 4E has the same configuration as the data decoder shown in Figure 4B. The difference is that after the ultra-pre-decoding network 410 acquires the first probability distribution parameter X and the scaling coefficient △, it performs a scaling operation on the first probability distribution parameter X based on the scaling coefficient. That is, the first probability distribution parameter X is divided by the scaling coefficient △ to obtain the second probability distribution parameter (X / △). The entropy decoding unit 414 is configured to cooperate with the entropy estimation unit 408 to perform entropy decoding on the second bitstream B based on the second probability distribution parameter (X / △) to obtain a fourth feature map.

[0194] Figure 4F is a diagram of the structure of another data encoder according to one embodiment of the present invention. The difference between the data encoder shown in Figure 4F and the data encoder shown in Figure 4D is that in this case, the first feature map y1 is the same as the second feature map y2. That is, the encoding network 402 performs feature extraction on the data to be processed to obtain the feature map y. In this case, the data decoder configured to decode the total bitstream of data obtained through the processing in Figure 4F has the same structure as that in Figure 4E.

[0195] Figure 4G is a diagram of the structure of another data encoder according to one embodiment of the present invention. The difference from the data encoder shown in Figure 4D is that the data encoder shown in Figure 4G does not generate a residual feature map and does not perform scaling operations on the residual feature map. In Figure 4G, after the encoding network 402 obtains a second feature map y2 of the data to be processed, it performs a scaling operation on the second feature map y2 based on the scaling coefficient △. That is, it divides the second feature map y2 by the scaling coefficient △ to obtain a scaled feature map. The quantization unit 405 then performs a quantization operation on the scaled feature map to obtain a second quantized feature map. Figure 4H is a diagram of the structure of another data decoder according to one embodiment of the present invention. The data decoder shown in Figure 4H is configured to decode the total bitstream of data acquired through processing by the data encoder shown in Figure 4G. The data decoder 30 comprises an entropy estimation unit 408, an entropy decoding unit 414, a super-pre-entropy estimation unit 409, an entropy decoding unit 413, a super-pre-decoding network 410, a decoding network 412, and an output terminal (or output interface) 411.

[0196] The data decoder 30 may obtain the total bitstream to be decoded, i.e., the first bitstream A and the second bitstream B, through the input terminal (or input interface) (not shown in Figure 4H).

[0197] The entropy decoding unit 413 is configured to work in cooperation with the ultra-prior entropy estimation unit 409 to perform entropy decoding on the first bitstream A to obtain a third feature map.

[0198] The ultra-pre-decoding network 410 is configured to generate a scaling coefficient Δ and a first probability distribution parameter X based on a third feature map. The first probability distribution parameter X represents the probability distribution of the fourth feature map, where X includes, but is not limited to, a mean μ and / or variance σ.

[0199] Furthermore, for example, the ultra-pre-decoding network 410 may be configured solely to generate a scaling coefficient △ based on a third feature map, and the first probability distribution parameter X is a pre-set probability distribution parameter depending on the actual situation. This is not particularly limited.

[0200] A scaling process is performed on the first probability distribution parameter X based on the scaling coefficient △. That is, the first probability distribution parameter X is divided by the scaling coefficient △ to obtain the second probability distribution parameter (X / △).

[0201] The entropy decoding unit 414 is configured to work in cooperation with the entropy estimation unit 408 to perform entropy decoding on the second bitstream B based on the second probability distribution parameter (X / △) to obtain a fourth feature map.

[0202] The fifth feature map may be obtained based on the scaling factor △ and the fourth feature map, that is, by multiplying the fourth feature map by the scaling factor △.

[0203] The decoding network 412 is configured to reverse-map the fifth feature map to reconstruct the data.

[0204] The output terminal 411 is configured to output reconstructed data.

[0205] Figure 4I is a diagram of the structure of another data encoder according to one embodiment of the present invention. The data encoder shown in Figure 4I is similar to the one shown in Figure 4G. The difference in Figure 4I is that the first feature map y1 is the same as the second feature map y2. That is, the encoding network 402 performs feature extraction on the data to be processed to obtain the feature map y. The data decoder, configured to decode the total bitstream of data obtained through processing by the data encoder shown in Figure 4I, has the same structure as that in Figure 4H. Further details will not be described again.

[0206] It should be noted herein that at least one of the above-mentioned encoding network, super-prior encoding network, quantization unit, entropy encoding unit, entropy decoding unit, super-prior entropy estimation unit, entropy estimation unit, super-prior decoding network, and decoding network may be implemented using a neural network, such as a convolutional neural network.

[0207] For example, the specific structure of the encoding network 402 in the data encoder shown in Figure 4C, Figure 4F, or Figure 4I may refer to the structure shown in Figure 5A. The encoding network 402 has a first convolutional (Conv) layer, a first nonlinear unit (ResAU) layer, a second convolutional layer, a second nonlinear unit (ResAU) layer, a third convolutional layer, a third nonlinear unit (ResAU) layer, and a fourth convolutional layer. For example, the specific parameters of the first, second, third, and / or fourth convolutional layers are 192 × 5 × 5 / 2, where the number of channels is 192, the size of the convolutional kernel is 5 × 5, and the stride is 2. The feature map y of the data to be processed may be obtained by using the encoding network shown in Figure 5A.

[0208] For example, the specific structure of the encoding network 402 in the data encoder shown in Figure 4A, Figure 4D, or Figure 4G can be referenced from the structure shown in Figure 5B. The encoding network 402 has a first encoder, a second encoder, and a third encoder. The first encoder is configured to perform feature extraction on the data to be processed first. The second and third encoders are configured to perform feature extraction again on the feature maps obtained through the extraction by the first encoder to obtain the first feature map y1 and the second feature map y2, respectively.

[0209] As another example, for the specific structure of the encoding network 402 in the data encoder shown in Figure 4A, Figure 4D, or Figure 4G, feature extraction may be performed using the encoding network shown in Figure 5A to obtain a feature map y, and then channel separation may be performed on the feature map y to obtain a first feature map y1 and a second feature map y2. The specific method of channel separation is not limited. For example, the feature map y obtained in Figure 5A has 384 channels, and the channel a (where a is less than 384) of the feature map y may be used as the first feature map y1, and the remaining (384-a) channels of the feature map y may be used as the second feature map y2. For example, the first 192 channels of the feature map y may be used as the first feature map y1, and the last 192 channels of the feature map y may be used as the second feature map y2.

[0210] For example, for the specific structure of the ultra-pre-encoding network in the data encoder shown in Figures 4A, 4C, 4D, 4F, 4G, or 4I, the structure shown in Figure 5C may be referenced. The ultra-pre-encoding network includes a first leaky ReLU layer, a first convolutional layer, a second leaky ReLU layer, a second convolutional layer, a third leaky ReLU layer, and a third convolutional layer. For example, the specific parameters of the first convolutional layer are 192 × 3 × 3, where the number of channels is 192 and the size of the convolutional kernel is 3 × 3. The specific parameters of the second and / or third convolutional layer are 192 × 5 × 5 / 2, where the number of channels is 192, the size of the convolutional kernel is 5 × 5, and the stride is 2.

[0211] For example, for the specific structure of the ultra-pre-decoding network in any one of Figures 4A to 4I, the structure shown in Figure 5D may be referenced. The ultra-pre-decoding network includes a first convolutional layer, a first leaky ReLU layer, a second convolutional layer, a second leaky ReLU layer, and a third convolutional layer. For example, the specific parameters of the first convolutional layer are 384 × 3 × 3, where the number of channels is 384 and the size of the convolutional kernel is 3 × 3. The specific parameters of the second convolutional layer are 288 × 5 × 5 / 2↑, where the number of channels is 288, the size of the convolutional kernel is 5 × 5, and the stride is 2. The specific parameters of the third convolutional layer are 192 × 5 × 5 / 2↑, where the number of channels is 192, the size of the convolutional kernel is 5 × 5, and the stride is 2.

[0212] For example, the specific structure of the decoding network in the data decoder shown in Figure 4B, Figure 4E, or Figure 4H may refer to the structure shown in Figure 5E. The decoding network includes a first convolutional layer, a first ResAU layer, a second convolutional layer, a second ResAU layer, a third convolutional layer, a third ResAU layer, and a fourth convolutional layer. For example, the specific parameters of the first, second, and / or third convolutional layers are 192 × 5 × 5 / 2↑, where the number of channels is 192, the size of the convolutional kernel is 5 × 5, and the stride is 2. The specific parameters of the fourth convolutional layer are 384 × 5 × 5 / 2↑, where the number of channels is 384, the size of the convolutional kernel is 5 × 5, and the stride is 2.

[0213] For example, Figure 5F may be referenced for the specific structure of the ResAU layer in the structures shown in Figure 5A and / or Figure 5E. The ResAU layer includes a leaky ReLU layer, a convolutional layer, and a Tanh layer.

[0214] In the following, as an example, we will use the data coding system shown in Figures 4A and 4B (hereinafter referred to as the first data coding system) and compare it with the data coding system shown in Figure 6A (hereinafter referred to as the second data coding system) for illustrative purposes. The first data coding system has scaling operations on residual feature maps, while the second data coding system has scaling operations on feature maps.

[0215] For example, the data is image data. The Kodak test set is used as the data to be processed. The test set contains 24 PNG images with a resolution of 768×512 or 512×768. Each test image is processed using the first and second data coding systems separately to obtain BPP (Bits per pixel), Peak Signal-to-Noise Ratio (PSNR), and BD rate (Bjoentegaard-Delta bit Rate). See Table 1 for details. The BPP and PSNR in Table 1 are the average values ​​for the 24 images. BPP represents the average number of bits used by a pixel, and a smaller BPP indicates a smaller compression bitrate. The Peak Signal-to-Noise Ratio is an objective criterion for evaluating image quality, and a higher Peak Signal-to-Noise Ratio indicates better image quality. The BD rate shows the bitrate reduction (compression rate increase) in the two comparison methods under the same image quality.

[0216] Table 1 shows that the results of the first data coding system are better than those of the second data coding system. The BD rate of the first data coding system is better, and the performance is improved by 3.87%. It can be seen that scaling and quantization of residual feature maps can improve compression performance. Table 1: Performance comparison of the first and second data coding systems [Table 1]

[0217] In the following, as an example, we will use the data coding system shown in Figure 6B (hereinafter referred to as the third data coding system) and compare it with the second data coding system shown in Figure 6A for illustrative purposes. The second data coding system involves scaling operations on feature maps, while the third data coding system involves scaling operations on both feature maps and probability distribution parameters. For example, in the third data coding system, both the mean μ and the variance σ are scaled.

[0218] For example, the data is image data. The Kodak test set is used as the data to be processed. The test set contains 24 PNG images with a resolution of 768×512 or 512×768. Each test image is processed using a second data coding system and a third data coding system separately to obtain BPP, PSNR, and BD rate. See Table 2 for details. Similarly, the BPP and PSNR in Table 2 are the average values ​​of the 24 images.

[0219] Table 2 shows that the results of the third data coding system are better than those of the second data coding system. The BD rate of the third data coding system is better, and the performance is improved by 2.71%. It can be seen that scaling with respect to the probability distribution parameter can improve compression performance. Table 2: Performance comparison of the second and third data coding systems [Table 2]

[0220] In the following, for illustrative purposes, the first data coding system will be compared with the data coding system (hereinafter referred to as the fourth data coding system), which includes Figures 4D and 4E. The first data coding system involves scaling operations on residual feature maps, while the fourth data coding system involves scaling operations on both residual feature maps and probability distribution parameters. For example, in the fourth data coding system, the variance σ is scaled.

[0221] For example, the data is image data. The Kodak test set is used as the data to be processed. The test set contains 24 PNG images with a resolution of 768×512 or 512×768. Each test image is processed using the first and fourth data coding systems separately to obtain BPP, PSNR, and BD rates. See Table 3 for details. Similarly, the BPP and PSNR in Table 3 are the average values ​​of the 24 images.

[0222] Table 3 shows that the results of the fourth data coding system are better than those of the first data coding system. The BD rate of the fourth data coding system is better, and the performance is improved by 0.65%. It can be seen that scaling with respect to the probability distribution parameter can improve compression performance. Table 3: Performance comparison of the first and fourth data coding systems [Table 3]

[0223] Figure 7 is a schematic flowchart of a data encoding method according to one embodiment of the present invention. The data encoding method 700 is performed by the data encoder 20. The method shown in Figure 7 is described as a series of steps or operations. It should be understood that the steps or operations of the method may be performed in various orders and / or simultaneously, and are not limited to the execution order shown in Figure 7.

[0224] As shown in Figure 7, the data encoding method 700 comprises the following steps.

[0225] 701: Perform side information feature extraction on the first feature map of the current data to obtain a side information feature map.

[0226] The data includes at least one of the following: image data, video data, motion vector data of video data, audio data, point cloud data, or text data. Side information means that existing information Y is used to assist in the encoding of information X so that the encoding length of information X can be shortened. In other words, redundancy in information X is reduced. Information Y is side information. In this embodiment of the present application, side information is information extracted from a first feature map and used to assist in the encoding and decoding of the first feature map.

[0227] 702: Perform quantization on the side information feature map to obtain the first quantized feature map.

[0228] 703: Perform entropy encoding on the first quantized feature map to obtain the first bitstream of the current data.

[0229] The bitstream is the bitstream generated after the encoding process. In this case, the first bitstream is the bitstream obtained after entropy encoding has been performed on the first quantized feature map.

[0230] 704: A scaling process is performed on the residual feature map based on the scaling coefficient to obtain a scaled feature map, and a quantization process is performed on the scaled feature map to obtain a second quantized feature map. A residual feature map is obtained based on the second feature map and the first probability distribution parameters of the current data, and the scaling coefficient is obtained based on the first quantized feature map. In this case, the scaling process is a scale-down process.

[0231] In this embodiment, the first feature map and the second feature map are different feature maps obtained by performing feature extraction on the complete current data.

[0232] 705: Perform entropy encoding on the second quantized feature map based on the first probability distribution parameter to obtain a second bitstream of the current data.

[0233] The first probability distribution parameter is a matrix with the same scale as the second quantization feature map. Each element in the matrix represents a probability distribution parameter of one of the elements in the second quantization feature map. Each probability distribution parameter of an element includes, but is not limited to, a mean and / or variance. That is, one second quantization feature map corresponds to one mean matrix and / or one variance matrix. The second bitstream is the bitstream obtained after entropy encoding has been performed on the second quantization feature map.

[0234] In this solution, during encoding, a scaling process is performed on the residual feature map of the current data to obtain a scaled feature map, and then a quantization process is performed on the scaled feature map so that the quantization loss of the scaled feature map is reduced. That is, the information loss of the second bitstream is reduced. This helps to improve the data quality of the reconstructed data obtained through decoding based on the second bitstream. Furthermore, compared to the encoding network structure in the prior art, this embodiment of the present application uses an encoding network structure that includes scaling on the residual feature map so that the network parameters of the entire encoding network, including the network for generating the first bitstream and the network for generating the second bitstream, can be optimized after the entire encoding network has been trained. Therefore, by using the encoding network structure in this embodiment of the present application, the amount of data in the total bitstream of the current data can be reduced and the encoding efficiency can be improved. In general, the encoding method in this embodiment can further improve data compression performance.

[0235] For example, the first quantized feature map may be input to a super-pre-decoding network, and the first probability distribution parameter and scaling coefficient may be obtained through prediction. The first probability distribution parameter represents the probability distribution of the second quantized feature map.

[0236] As another example, the first quantized feature map may be input to a super-pre-decoding network, and the scaling coefficients may be obtained through prediction. The first probability distribution parameter is a pre-configured probability distribution parameter.

[0237] In some possible embodiments, the first probability distribution parameters include the mean and / or variance.

[0238] In some possible embodiments, the first feature map is the same as the second feature map.

[0239] FIG. 8 is a schematic flowchart of another data encoding method according to an embodiment of the present application. The data encoding method 800 is executed by the data encoder 20. The method shown in FIG. 8 is described as a series of steps or operations. It should be understood that the steps or operations of the method may be executed in various orders and / or simultaneously, and are not limited to the execution order shown in FIG. 8.

[0240] As shown in FIG. 8, the data encoding method 800 includes the following steps.

[0241] 801: Execute side information feature extraction on the first feature map of the current data to obtain a side information feature map.

[0242] The data includes at least one of the following: image data, video data, motion vector data of video data, audio data, point cloud data, or text data. Side information means that existing information Y is used to assist in encoding information X so that the encoding length of information X can be made shorter. In other words, the redundancy in information X is reduced. Information Y is side information. In this embodiment of the present application, the side information is information extracted from the first feature map and used to assist in encoding and decoding the first feature map.

[0243] 802: Execute quantization processing on the side information feature map to obtain a first quantized feature map.

[0244] 803: Execute entropy encoding on the first quantized feature map to obtain a first bitstream of the current data.

[0245] The bitstream is the bitstream generated after the encoding process. The first bitstream is the bitstream obtained after entropy encoding has been performed on the first quantized feature map.

[0246] 804: Perform a scaling operation on the second feature map based on the scaling factor to obtain the scaled feature map.

[0247] The scaling factor is obtained based on the first quantization feature map. In this embodiment, the first and second feature maps are different feature maps obtained by performing feature extraction on the complete current data. For example, when the first feature map is different from the second feature map, the relevant description in Figure 5B can be referenced for a specific method of obtaining the first feature map (y1 in Figure 5B) and the second feature map (y2 in Figure 5B). In this case, the scaling process is a scale-down process.

[0248] 805: Perform quantization on the scaled feature map to obtain a second quantized feature map.

[0249] 806: Perform a scaling operation on the first probability distribution parameter based on the scaling coefficient to obtain the second probability distribution parameter.

[0250] The first probability distribution parameter is obtained based on the first quantization feature map. Alternatively, the first probability distribution parameter is a pre-defined probability distribution parameter. The first probability distribution parameter is a matrix with the same scale as the second quantization feature map. Each element in the matrix represents one of the probability distribution parameters in the second quantization feature map. Each probability distribution parameter of an element includes, but is not limited to, a mean and / or variance. That is, one second quantization feature map corresponds to one mean matrix and / or one variance matrix.

[0251] 807: Perform entropy encoding on the second quantized feature map based on the second probability distribution parameter to obtain a second bitstream of the current data.

[0252] The second bitstream is the bitstream obtained after entropy encoding has been performed on the second quantized feature map.

[0253] In this solution, entropy encoding is performed based on a first quantized feature map to obtain a first bitstream of the current data. Furthermore, scaling coefficients and a first probability distribution parameter may be obtained through estimation based on the first quantized feature map. In this way, a scaling process may be performed on a second feature map based on the scaling coefficient to obtain a scaled feature map. Next, a quantization process is performed on the scaled feature map to obtain a second quantized feature map. A scaling process is performed on the first probability distribution parameter based on the scaling coefficient to obtain a second probability distribution parameter. Finally, entropy encoding is performed on the second quantized feature map based on the second probability distribution parameter to obtain a second bitstream of the current data. The first bitstream and the second bitstream are used together as a total bitstream of the current data. Data encoding methods in the prior art only include the step of performing a scaling process on a feature map. In this solution, the second feature map and the first probability distribution parameter are scaled by using the same scaling coefficient so that there is a higher degree of agreement between the second probability distribution parameter and the second quantized feature map, thereby improving the encoding accuracy of the second quantized feature map, i.e., improving data compression performance. In some possible embodiments, the second feature map is a residual feature map of the current data obtained based on the third feature map and the first probability distribution parameter of the current data, and the third feature map is obtained by performing feature extraction on the current data.

[0254] In this embodiment, the second feature map is a feature map obtained based on the current data, i.e., a residual feature map. In this embodiment, the third feature map is a different feature map from the first feature map, obtained by performing feature extraction on the complete current data. The residual of the third feature map may be obtained based on the third feature map and the first probability distribution parameter. That is, the residual feature map may be obtained by subtracting the first probability distribution parameter from the third feature map, and the residual feature map is used as the second feature map of the current data. For example, when the first feature map is different from the third feature map, specific methods for obtaining the first feature map (y1 in Figure 5B) and the third feature map (y2 in Figure 5B) can be found in the relevant description in Figure 5B.

[0255] In this solution, during encoding, a scaling process is performed on the residual feature map of the current data to obtain a scaled feature map, and then a quantization process is performed on the scaled feature map so that the quantization loss of the scaled feature map is reduced. That is, the information loss of the second bitstream is reduced. This helps to improve the data quality of the reconstructed data obtained through decoding based on the second bitstream. Furthermore, compared to the encoding network structure in the prior art, this embodiment of the present application uses an encoding network structure that includes scaling of the residual feature map so that the network parameters of the entire encoding network, including the network for generating the first bitstream and the network for generating the second bitstream, can be optimized after the training of the entire encoding network is completed. Therefore, by using the encoding network structure in this embodiment of the present application, the amount of data in the total bitstream of the current data can be reduced and the encoding efficiency can be improved. In other words, the scaling of the residual feature map and the scaling of the first probability distribution parameter are combined, thereby further improving data compression performance.

[0256] Furthermore, in some possible embodiments, the first feature map is identical to the third feature map. For example, specific methods for obtaining the first and third feature maps can be found in the relevant description in Figure 5A.

[0257] In some possible embodiments, the second feature map is a feature map obtained by performing feature extraction on the current data. The first feature map is the same as or different from the second feature map. For example, specific methods for obtaining the first and second feature maps can be seen in the relevant description in Figure 5A.

[0258] In some possible embodiments, the first probability distribution parameter includes an average and / or a variance.

[0259] In some possible embodiments, the data encoding method: further comprises the step of transmitting a first bit stream and a second bit stream. is further provided.

[0260] After the first bit stream and the second bit stream of the current data are obtained by using the data encoding method in this solution means, the first bit stream and the second bit stream may be transmitted to another device according to requirements, whereby the other device can process the first bit stream and the second bit stream.

[0261] In some possible embodiments, the first bit stream and the second bit stream are stored in the form of a bit stream file.

[0262] FIG. 9 is a schematic flowchart of a data decoding method according to an embodiment of the present application. The data decoding method 900 is executed by a data decoder 30. The method shown in FIG. 9 is described as a series of steps or operations. It should be understood that the steps or operations of the method may be executed in various orders and / or simultaneously, and are not limited to the execution order shown in FIG. 9.

[0263] As shown in FIG. 9, the data decoding method 900 comprises the following steps.

[0264] 901: Perform entropy decoding based on the first bit stream of the current data to obtain a third feature map.

[0265] 902: Obtain a scaling coefficient based on the third feature map.

[0266] 903: Entropy decoding is performed on the second bitstream of the current data based on the first probability distribution parameter to obtain a fourth feature map.

[0267] The first probability distribution parameter is a matrix with the same scale as the fourth feature map. Each element in the matrix represents a probability distribution parameter of one of the elements in the fourth feature map. Each probability distribution parameter of an element includes, but is not limited to, a mean and / or variance. That is, one fourth feature map corresponds to one mean matrix and / or one variance matrix.

[0268] 904: A scaling process is performed on the fourth feature map based on the scaling factor to obtain the fifth feature map. In this case, the scaling process is a scale-up process.

[0269] 905: Obtain a sixth feature map based on the first probability distribution parameter and the fifth feature map.

[0270] 906: Obtain reconstructed data of the current data based on the sixth feature map.

[0271] When scaling and quantization operations are performed on the residual feature map during data encoding, the data volume of the first and second bitstreams in the data decoding method of this solution is smaller than that of the prior art. Therefore, the corresponding decoding processing load is smaller, and this solution can effectively improve decoding efficiency. Furthermore, the information loss in the second bitstream is smaller, thereby resulting in higher data quality for the reconstructed data obtained using this solution.

[0272] In some possible embodiments, the first feature map is the same as or different from the second feature map.

[0273] In some possible embodiments, the data includes at least one of the following: image data, video data, motion vector data of video data, audio data, point cloud data, or text data.

[0274] In some possible embodiments, the first probability distribution parameters include the mean and / or variance.

[0275] In some possible embodiments, the data decoding method is: Step 1: Obtaining the first probability distribution parameter based on the third feature map. To further prepare.

[0276] In some possible embodiments, the first probability distribution parameter is a pre-defined probability distribution parameter.

[0277] Figure 10 is a schematic flowchart of another data decoding method according to one embodiment of the present invention. The data decoding method 1000 is performed by the data decoder 30. The method shown in Figure 10 is described as a series of steps or operations. It should be understood that the steps or operations of the method may be performed in various orders and / or simultaneously, and are not limited to the execution order shown in Figure 10.

[0278] As shown in Figure 10, the data decoding method 1000 comprises the following steps.

[0279] 1001: Perform entropy decoding based on the first bitstream of the current data to obtain a third feature map.

[0280] 1002: Obtain the scaling factor based on the third feature map.

[0281] 1003: Perform a scaling operation on the first probability distribution parameter based on the scaling coefficient to obtain the second probability distribution parameter.

[0282] The first probability distribution parameter is a matrix with the same scale as the fourth feature map. Each element in the matrix represents one of the probability distribution parameters of an element in the fourth feature map. Each probability distribution parameter of an element includes, but is not limited to, a mean and / or variance. That is, one fourth feature map corresponds to one mean matrix and / or one variance matrix.

[0283] 1004: Entropy decoding is performed on the second bitstream of the current data based on the second probability distribution parameter to obtain a fourth feature map.

[0284] 1005: A scaling process is performed on the fourth feature map based on the scaling factor to obtain the fifth feature map. In this case, the scaling process is a scale-up process.

[0285] 1006: Obtain reconstructed data of the current data based on the fifth feature map.

[0286] If scaling is performed on the second feature map and the first probability distribution parameter during data encoding, the same scaling coefficients are used to process the first probability distribution parameter and the fourth feature map during data decoding to ensure the decoding accuracy of the fourth feature map. Furthermore, descaling may be performed on the fourth feature map based on the scaling coefficients to obtain a fifth feature map, and reconstructed data may be obtained based on the fifth feature map, thereby improving the accuracy and quality of the reconstructed data. In other words, by combining scaling on the first probability distribution parameter and descaling on the fourth feature map, the accuracy and quality of data decoding can be improved.

[0287] In some possible embodiments, the fourth feature map is a residual feature map, and the step of obtaining reconstructed data of the current data based on the fifth feature map is: The step of obtaining a sixth feature map by adding the first probability distribution parameter to the fifth feature map; and The sixth step is to obtain reconstructed data of the current data based on the feature map. It has.

[0288] When scaling and quantization operations are performed on the residual feature map during data encoding, the data volume of the first and second bitstreams in the data decoding method of this solution is smaller than that of the prior art. Therefore, the corresponding decoding processing load is smaller, and this solution can effectively improve decoding efficiency. Furthermore, the information loss in the second bitstream is smaller, thereby resulting in higher data quality for the reconstructed data obtained using this solution.

[0289] In some possible embodiments, the first probability distribution parameters include the mean and / or variance.

[0290] In some possible embodiments, the data includes at least one of the following: image data, video data, motion vector data of video data, audio data, point cloud data, or text data.

[0291] In some possible embodiments, the data decoding method is: The stage of receiving the first and second bitstreams of the current data. To further prepare.

[0292] In some possible embodiments, the data decoding method further comprises the step of obtaining a first probability distribution parameter based on a third feature map.

[0293] In some possible embodiments, the first probability distribution parameter is a pre-defined probability distribution parameter.

[0294] When the functions of a method according to any one embodiment of the present application are implemented in the form of a software function unit and sold or used as a standalone product, the functions may be stored on a computer-readable storage medium. Based on such understanding, some of the data encoding method and / or data decoding method in the present application, or some of the technical solutions that essentially or contribute to the prior art, may be implemented in the form of a computer program product. The computer program product includes a number of instructions stored on a storage medium for instructing an electronic device to perform all or some of the steps of the method described in the embodiments of the present application. The aforementioned storage medium includes any medium capable of storing program code, such as a USB flash drive, removable hard disk, read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0295] The present invention further provides a computer-readable storage medium, the storage medium storing a bitstream, the bitstream being generated according to a data encoding method according to any one of the embodiments described above.

[0296] The present invention further provides a computer-readable storage medium for storing a bitstream containing program code. Once the program code is executed by one or more processors, a decoder can perform a data decoding method according to any one of the embodiments described above.

[0297] One embodiment of the present invention further provides a chip used in an electronic device, the chip comprising one or more processors, the processors configured to call computer instructions to enable the electronic device to perform a data encoding method and / or data decoding method according to any one of the embodiments described above.

[0298] One embodiment of the present invention further provides a computer program product comprising instructions. When the computer program product is executed on an electronic device, the electronic device becomes capable of performing a data encoding method and / or a data decoding method according to any one of the embodiments described above.

[0299] It can be understood that all computer storage media, chips, and computer program products provided above are configured to perform a data encoding method and / or data decoding method according to any one of the embodiments described above. Therefore, for the beneficial effects that can be achieved, please refer to the beneficial effects of the data encoding method and / or data decoding method according to any one of the embodiments described above. Further details will not be described again here.

[0300] Those skilled in the art will recognize that the units and algorithmic stages in the examples described with reference to the embodiments disclosed herein can be implemented by electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed by hardware or software will depend on the specific application and the design constraints of the technical solution. Those skilled in the art may use different methods to implement the functions described for each specific application, but such implementations should not be considered to exceed the scope of this application.

[0301] In some embodiments provided herein, it should be understood that the disclosed apparatus and methods may be implemented in other ways. For example, the described embodiments of the apparatus are merely examples. For example, the division into multiple units is merely a logical functional division, and other divisions may occur during actual implementation. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not performed. In addition, the indicated or described interconnections, direct connections, or communication connections may be implemented through some interfaces. Indirect connections or communication connections between apparatus or units may be implemented electronically, mechanically, or in other forms.

[0302] Each unit described as a separate part may or may not be physically separate, and the part shown as a unit may or may not be a physical unit; in other words, it may be located in one place or distributed across multiple network units. Some or all of the units may be selected according to the actual requirements for achieving the objectives of the solution of the embodiment.

[0303] Furthermore, the functional units in the embodiments of this patent application may be integrated into a single processing unit, or each of these units may exist physically independently, or two or more units may be integrated into a single unit.

[0304] The foregoing description merely represents a specific implementation of the present application and is not intended to limit the scope of protection. Any modifications or substitutions that are readily conceivable by a person skilled in the art within the scope of the technical scope disclosed herein shall be included in the scope of protection. Therefore, the scope of protection of this application shall be subject to the scope of protection of the claims. 。 [Item 1] A data encoding method, the method comprising the following steps: This stage involves performing side information feature extraction on the current data's first feature map to obtain a side information feature map; A step of performing a quantization process on the aforementioned side information feature map to obtain a first quantized feature map; A step of performing entropy encoding on the first quantized feature map to obtain a first bitstream of the current data; The step involves performing a scaling process on a second feature map based on a scaling factor to obtain a scaled feature map, where the scaling factor is obtained based on the first quantized feature map; A step of performing a quantization process on the scaled feature map to obtain the second quantized feature map; A step of performing a scaling process on a first probability distribution parameter based on the scaling coefficient to obtain a second probability distribution parameter; and The step of performing entropy encoding on the second quantization feature map based on the second probability distribution parameter to obtain a second bitstream of the current data. A method that includes [a certain feature]. [Item 2] The method according to item 1, wherein the second feature map is a residual feature map obtained based on the third feature map of the current data and the first probability distribution parameters, and the third feature map is obtained by performing feature extraction on the current data. [Item 3] The method according to item 1, wherein the second feature map is a feature map obtained by performing feature extraction on the current data. [Item 4] The method described in item 2, wherein the first feature map is the same as the third feature map. [Item 5] The method described in item 3, wherein the first feature map is the same as the second feature map. [Item 6] The first probability distribution parameter is the method described in any one of items 1 to 5, including the mean and / or variance. [Item 7] The aforementioned method is: The step of transmitting the first bitstream and the second bitstream. The method according to any one of items 1 to 6, further comprising: [Item 8] The data described above includes at least one of the following: image data, video data, motion vector data of the video data, audio data, point cloud data, or text data, according to the method described in any one of items 1 to 7. [Item 9] A data decoding method, the method comprising the following steps: The next step is to perform entropy decoding based on the first bitstream of the current data to obtain a third feature map; A step to obtain scaling factors based on the third feature map; A step of obtaining a second probability distribution parameter by performing a scaling process on the first probability distribution parameter based on the scaling coefficient; A step of obtaining a fourth feature map by performing entropy decoding on a second bitstream of the current data based on the second probability distribution parameter; A step of performing a scaling process on the fourth feature map based on the scaling coefficient to obtain a fifth feature map; and Step 1: Obtaining reconstructed data of the current data based on the fifth feature map. A method for providing this. [Item 10] The fourth feature map is a residual feature map, the fifth feature map is a scaled residual feature map, and the step of obtaining the reconstructed data of the current data based on the fifth feature map is: A step of obtaining a sixth feature map by adding the first probability distribution parameter to the fifth feature map; and Step 6: Obtaining the reconstructed data of the current data based on the feature map of the sixth feature. The method described in item 9, having the characteristics of item 9. [Item 11] The first probability distribution parameter is the method described in item 9 or 10, including the mean and / or variance. [Item 12] The data described above includes at least one of the following: image data, video data, motion vector data of the video data, audio data, point cloud data, or text data, according to the method described in any one of items 9 to 11. [Item 13] The aforementioned method is: The step of receiving the first bitstream and the second bitstream of the current data. The method according to any one of items 9 to 12, further comprising: [Item 14] A data encoder comprising a processing circuit configured to perform a data encoding method as described in any one of items 1 through 8. [Item 15] A computer-readable storage medium, wherein the storage medium stores a bitstream, and the bitstream is generated according to a data encoding method described in any one of items 1 to 8. [Item 16] A data decoder comprising a processing circuit configured to perform the data decoding method described in any one of items 9 to 13. [Item 17] A computer program product comprising program code, wherein when the program code is executed on a computer or processor, the computer program product is configured to perform the method described in any one of items 1 to 13. [Item 18] A data encoder, One or more processors; and A computer-readable storage medium coupled to one or more processors, wherein the computer-readable storage medium stores a program, and when the program is executed by one or more processors, the data encoder is able to perform the data encoding method described in any one of items 1 to 8. A data encoder equipped with the following features. [Item 19] A data decoder, One or more processors; and A computer-readable storage medium coupled to one or more processors, wherein the computer-readable storage medium stores a program, and when the program is executed by the one or more processors, the data decoder is able to perform the data decoding method described in any one of items 9 to 13. A data decoder equipped with the following features. [Item 20] A computer-readable storage medium comprising program code, wherein the program code is configured to, when executed by a computer device, perform the method described in any one of items 1 to 13. [Item 21] A computer-readable storage medium for storing a bitstream containing program code, wherein the program code, when executed by one or more processors, enables a decoder to perform the data decoding method described in any one of items 9 to 13.

Claims

1. A data encoding method, the method comprising the following steps: This step involves performing side information feature extraction on the current data's first feature map to obtain a side information feature map; A step of performing a quantization process on the aforementioned side information feature map to obtain a first quantized feature map; The step of performing entropy encoding on the first quantized feature map to obtain a first bitstream of the current data; The step involves performing a scaling process on a second feature map based on a scaling factor to obtain a scaled feature map, where the scaling factor is obtained based on the first quantized feature map; A step of performing a quantization process on the scaled feature map to obtain a second quantized feature map; A step of performing a scaling process on a first probability distribution parameter based on the scaling coefficient to obtain a second probability distribution parameter; and The step of performing entropy encoding on the second quantization feature map based on the second probability distribution parameter to obtain a second bitstream of the current data. A method that includes [a certain feature].

2. The method according to claim 1, wherein the second feature map is a residual feature map of the current data obtained based on the third feature map of the current data and the first probability distribution parameter, and the third feature map is obtained by performing feature extraction on the current data.

3. The method according to claim 1, wherein the second feature map is a feature map obtained by performing feature extraction on the current data.

4. The method according to claim 2, wherein the first feature map is the same as the third feature map.

5. The method according to claim 3, wherein the first feature map is the same as the second feature map.

6. The method according to any one of claims 1 to 5, wherein the first probability distribution parameter includes a mean and / or variance.

7. The aforementioned method is: The first bitstream and the second bitstream TRANSATING stage The method according to any one of claims 1 to 5, further comprising:

8. The method according to any one of claims 1 to 5, wherein the current data includes at least one of the following: image data, video data, motion vector data of the video data, audio data, point cloud data, or text data.

9. A data decoding method, wherein the method comprises the following steps: The next step is to perform entropy decoding based on the first bitstream of the current data to obtain a third feature map; Steps to obtain scaling factors based on the third feature map described above; A step of obtaining a second probability distribution parameter by performing a scaling process on the first probability distribution parameter based on the scaling coefficient; A step of obtaining a fourth feature map by performing entropy decoding on the second bitstream of the current data based on the second probability distribution parameter; A step of performing a scaling process on the fourth feature map based on the scaling coefficient to obtain a fifth feature map; and Step 1: Obtain reconstruction data of the current data based on the fifth feature map. A method for providing this.

10. The fourth feature map is a residual feature map, the fifth feature map is a scaled residual feature map, and the step of obtaining the reconstructed data of the current data based on the fifth feature map is: A step of obtaining a sixth feature map by adding the first probability distribution parameter to the fifth feature map; and Steps to obtain the reconstructed data of the current data based on the sixth feature map. The method according to claim 9, having the following characteristics.

11. The method according to claim 9, wherein the first probability distribution parameter includes the mean and / or variance.

12. The method according to any one of claims 9 to 11, wherein the current data includes at least one of the following: image data, video data, motion vector data of the video data, audio data, point cloud data, or text data.

13. The aforementioned method is: The step of receiving the first bitstream and the second bitstream of the current data. The method according to any one of claims 9 to 11, further comprising:

14. A data encoder comprising a processing circuit configured to perform the data encoding method according to any one of claims 1 to 5.

15. A data decoder comprising a processing circuit configured to perform the data decoding method according to any one of claims 9 to 11.

16. A computer program for causing a processor to perform the method according to any one of claims 1 to 5, or any one of claims 9 to 11.

17. A data encoder, One or more processors; and A computer-readable storage medium coupled to one or more processors, wherein the computer-readable storage medium stores a program, and when the program is executed by one or more processors, the data encoder is able to perform the data encoding method according to any one of claims 1 to 5. A data encoder equipped with the following features.

18. A data decoder, One or more processors; and A computer-readable storage medium coupled to one or more processors, wherein the computer-readable storage medium stores a program, and when the program is executed by the one or more processors, the data decoder is able to perform the data decoding method according to any one of claims 9 to 11. A data decoder equipped with the following features.

19. A computer-readable storage medium comprising program code, wherein, when the program code is executed by a computer device, the computer-readable storage medium is configured to perform the method according to any one of claims 1 to 5 or any one of claims 9 to 11.

Images