Image encoding method, image decoding method, device and computer storage medium

Abstract

The present application discloses an image encoding method, an image decoding method, an apparatus, and a computer storage medium. The image encoding method includes: obtaining an image to be encoded and extracting original feature data of the image to be encoded; performing mapping processing on the original feature data to obtain mapped feature data, wherein the data volume of the mapped feature data is less than the data volume of the original feature data; performing feature dimension transformation on the mapped feature data of the current feature dimension to obtain encoded feature data of the feature dimension supported by a preset encoding tool; and encoding the encoded feature data using the preset encoding tool to obtain a feature code stream of the image to be encoded. The image encoding method of the present application provides a complete image encoding process that can encode various types of image feature data, thereby improving the generalization of image encoding.

Description

Technical Field

[0001] The present application relates to the field of feature compression technology, and in particular to an image encoding method, an image decoding method, an apparatus, and a computer storage medium. Background Art

[0002] Traditional image compression technology is designed for human vision. However, with the superior performance of deep neural networks in various machine vision tasks, such as image classification, object detection, and semantic segmentation, a large number of machine vision-based artificial intelligence applications have emerged. To ensure that the performance of machine vision tasks is not compromised by the image compression process, an analysis-first-then-compression model is adopted to meet machine vision requirements. Specifically, at the image acquisition end, a lossless image is directly passed through a neural network for feature extraction. The extracted features are then encoded and transmitted. At the decoding end, the decoded features are directly input into the subsequent network structure to complete various machine vision tasks. Therefore, to conserve transmission bandwidth resources, research on image encoding methods specifically for machine vision is needed.

[0003] Currently, feature compression algorithms fall into two main categories: traditional compression methods and learning-based schemes. The main traditional compression methods include the following: First, by replacing high-precision data types with low-precision data types, the space occupied by the original feature data is reduced. However, this is not true feature data compression, but rather achieved from the perspective of computer storage. Second, through dimensionality reduction methods such as PCA (Principal Component Analysis), the main data component information of the original feature data is extracted, allowing the low-dimensional data to roughly represent the original data information, which is a lossy compression method. Third, the hybrid encoder scheme first quantizes the deep features and then lossily encodes the quantized features using encoders such as High Efficiency Video Coding (HEVC) and H.266 / VVC. The disadvantage of this scheme is that the lossy compression degradation of the hybrid encoder is inconsistent with the degradation of the feature performance when performing visual analysis tasks, resulting in the features failing to provide important information required for visual analysis. Summary of the Invention

[0004] The present application provides an image encoding method, an image decoding method, an apparatus, and a computer storage medium.

[0005] A technical solution adopted by the present application is to provide an image coding method, the image coding method comprising:

[0006] Acquire an image to be encoded, and extract original feature data of the image to be encoded;

[0007] Performing mapping processing on the original feature data to obtain mapped feature data, wherein the data volume of the mapped feature data is smaller than the data volume of the original feature data;

[0008] Performing feature dimension transformation on the mapped feature data of the current feature dimension to obtain encoded feature data of the feature dimension supported by a preset encoding tool;

[0009] The encoding feature data is encoded using the preset encoding tool to obtain a feature code stream of the image to be encoded.

[0010] Wherein, after extracting the original feature data of the image to be encoded, and before performing mapping processing on the original feature data and obtaining the mapped feature data, the image encoding method further includes at least one of the following processing:

[0011] Reordering the original feature data according to a preset data rule;

[0012] Performing dimensionality reduction processing on the original feature data using a preset principal component analysis matrix;

[0013] Downsampling is performed on the original feature data.

[0014] The mapping process of the original feature data to obtain mapped feature data includes:

[0015] Get the maximum value of all original feature data and the floating-point data of each original feature data;

[0016] According to a preset mapping function, the floating-point data of each original feature data is mapped using the maximum value to obtain integer data of each original feature data as mapped feature data.

[0017] The method of mapping the floating-point data of each original feature data using the maximum value according to a preset mapping function to obtain the integer data of each original feature data as the mapped feature data includes:

[0018] According to the preset mapping function, obtaining a mapping range;

[0019] Determining initial mapping feature data for each original feature data using the mapping range and the maximum value;

[0020] The floating-point data of each initial mapping feature data is processed by the rounding function in the preset mapping function, the first integer part value of the floating-point data is retained, the decimal part value of the floating-point data is rounded off to obtain the second integer part value, the first integer part value and the second integer part value are added together to obtain the integer data of each original feature data as the mapping feature data.

[0021] The mapping process of the original feature data to obtain mapped feature data includes:

[0022] Multiplying the original feature data by a preset multiple to determine the initial mapping feature data;

[0023] The floating-point data of the initial mapping feature data is rounded off, the third integer part value of the floating-point data is retained, the decimal part value of the floating-point data is rounded off to obtain the fourth integer part value, the third integer part value and the fourth integer part value are added together to obtain the integer data of each original feature data as the mapping feature data.

[0024] The mapping process of the original feature data to obtain mapped feature data includes:

[0025] Cluster all original feature data to obtain the similarity between any two original feature data;

[0026] Mapping a plurality of original feature data with similarity greater than or equal to a preset similarity threshold to the same original feature data;

[0027] The result after mapping all original feature data is used as the mapped feature data.

[0028] The step of performing feature dimension transformation on the mapped feature data of the current feature dimension to obtain coded feature data of the feature dimension supported by a preset coding tool includes:

[0029] Converting the mapped feature data of the current feature dimension into spatial domain data or temporal domain data of the feature dimension supported by the preset encoding tool;

[0030] The preset encoding tool is associated with the data format of the original feature data.

[0031] The step of encoding the encoding feature data using the preset encoding tool to obtain the feature code stream of the image to be encoded includes:

[0032] Encoding the encoding feature data using the preset encoding tool to obtain an initial feature code stream of the image to be encoded;

[0033] Adding feature encoding process information to the initial feature code stream to form a feature code stream of the image to be encoded;

[0034] The feature encoding process information includes at least one or more of a mapping mode index, a mapping parameter, a dimension conversion mode index, a dimension conversion parameter, an encoding tool index, and an encoding tool parameter.

[0035] Another technical solution adopted by the present application is to provide an image decoding method, the image encoding method comprising:

[0036] Get the feature code stream;

[0037] Decoding the feature code stream to obtain decoded feature data;

[0038] Performing an inverse feature dimension transformation on the decoded feature data according to the feature dimensions supported by the preset encoding tool to obtain mapped feature data;

[0039] Performing inverse mapping processing on the mapped feature data to obtain original feature data of the feature code stream, wherein the data volume of the original feature data is greater than the data volume of the mapped feature data;

[0040] The original feature data is reconstructed to obtain a reconstructed image.

[0041] Another technical solution adopted by the present application is to provide an image coding device, the image coding device including a memory and a processor coupled to the memory;

[0042] The memory is used to store program data, and the processor is used to execute the program data to implement the above-mentioned image encoding method.

[0043] Another technical solution adopted by the present application is to provide an image decoding device, the image decoding device comprising a memory and a processor coupled to the memory;

[0044] The memory is used to store program data, and the processor is used to execute the program data to implement the above-mentioned image decoding method.

[0045] Another technical solution adopted by the present application is to provide a computer storage medium, which is used to store program data. When the program data is executed by a computer, it is used to implement the above-mentioned image encoding method and / or image decoding method.

[0046] The beneficial effects of the present application are as follows: an image encoding device obtains an image to be encoded and extracts original feature data of the image to be encoded; maps the original feature data to obtain mapped feature data, wherein the data volume of the mapped feature data is less than the data volume of the original feature data; performs feature dimension transformation on the mapped feature data of the current feature dimension to obtain encoded feature data of the feature dimension supported by a preset encoding tool; and uses the preset encoding tool to encode the encoded feature data to obtain a feature code stream of the original feature data. The image encoding method of the present application provides a complete image encoding process that can encode various types of image feature data, thereby improving the generalization of image encoding. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0048] Figure 1 This is a flow chart of an embodiment of an image encoding method provided by the present application;

[0049] Figure 2 This is a schematic diagram of the overall process of the image encoding method provided by this application;

[0050] Figure 3 This is a schematic diagram of the encoding code stream structure provided by this application;

[0051] Figure 4 This is a flowchart of an embodiment of an image decoding method provided by the present application;

[0052] Figure 5 This is a schematic diagram of the overall process of the image decoding method provided by this application;

[0053] Figure 6 is a structural diagram of an embodiment of an image encoding device provided by the present application;

[0054] Figure 7 is a structural diagram of an embodiment of an image decoding device provided by the present application;

[0055] Figure 8 It is a structural diagram of an embodiment of a computer storage medium provided by this application. DETAILED DESCRIPTION

[0056] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0057] Please refer to the following for details: Figure 1 and Figure 2 , Figure 1 This is a flow chart of an embodiment of the image encoding method provided by this application. Figure 2 It is a schematic diagram of the overall process of the image encoding method provided by this application.

[0058] like Figure 1 As shown, the image encoding method of the embodiment of the present application includes the following steps:

[0059] Step S11: Acquire the image to be encoded and extract original feature data of the image to be encoded.

[0060] In the embodiment of the present application, the image encoding device obtains an input image to be encoded, and then performs feature extraction on the image to obtain original feature data of the image. The feature extraction method can be implemented using a common feature extraction network, etc., which will not be described in detail here.

[0061] In the embodiment of the present application, the features of the feature data include but are not limited to: 1. The feature dimension is 1*N, and N is a positive integer; 2. All feature data are floating point numbers.

[0062] After the original feature data is input, the image encoding device may use a preprocessing unit to preprocess the original feature data so that the preprocessed feature data can be more easily compressed.

[0063] Specifically, the role of the preprocessing unit is to organize the original feature data so that the organized original feature data can be more conducive to the feature mapping, feature dimension conversion, and feature encoding processes in subsequent steps. Among them, the preprocessing unit of the embodiment of the present application preprocesses the original feature data in a manner including but not limited to dimensionality reduction, redundancy removal, energy concentration, and other processing methods. The following are three feasible preprocessing methods:

[0064] 1. Reordering: Sort the original feature data according to a certain rule, which can be from large to small, small to large, staggered, small in the middle and large on both sides, etc. This application does not limit the specific sorting method of reordering.

[0065] 2. Dimensionality reduction: reducing the number of original feature data. It can be multiplying the feature data by a PCA (Principal Component Analysis) matrix to obtain a simplified 1*M feature data (M < N), or downsampling the original feature data, or other methods that can reduce the number of feature data. In the embodiments of the present application, PCA is adopted to linearly transform the original data into a set of linearly independent representations in each dimension, which can be used to extract the main feature components of the data and is often used for dimensionality reduction of high-dimensional data.

[0066] 3. DCT (Discrete Cosine Transform) / DST (Discrete Sine Transform) / FFT (Fast Fourier Transform) transform: The frequency-domain transform can be performed on the original feature data to concentrate the data energy in the low-frequency region.

[0067] Step S12: Perform a mapping process on the original feature data to obtain mapped feature data, where the amount of data of the mapped feature data is less than that of the original feature data.

[0068] In the embodiments of the present application, in order to further reduce the amount of data of the original feature data, the image coding device can also use a mapping unit to perform spatial mapping or precision mapping on the preprocessed original feature data, or use both spatial mapping and precision mapping simultaneously to further reduce the amount of data.

[0069] Specifically, the mapping unit performs processing such as spatial mapping or precision mapping on the preprocessed original feature data, including but not limited to: mapping floating-point numbers to integers, performing many-to-one mapping on feature data, etc. The following lists three feasible mapping methods:

[0070] 1. Mapping method one: First, obtain the maximum value in the original feature data, Then perform mapping according to the following formula:

[0071] F′ = Round(F * 2 t-1 / V max ) + 2 t-1

[0072] where F′ is the mapped integer data, F is the floating-point data before mapping, and t is the mapping range. Finally, the mapping unit can also truncate the mapped integer data within the numerical range, that is, limit the numerical range of the mapped integer data through the following formula:

[0073] F′ = clip(F′, 0, 2 t-1 ).

[0074] 2. Mapping Method 2: Magnify the floating-point data by A times and then round it to integer data. Specifically, the mapping unit can magnify the floating-point data of each original feature data by a preset multiple; then, retain the first integer portion of each amplified floating-point data value, round the decimal portion of the floating-point data value to obtain a second integer portion value, and add the first integer portion value and the second integer portion value to obtain the integer data of each original feature data as the mapped feature data.

[0075] 3. Mapping Method Three: Cluster the preprocessed feature data and perform mapping based on similarity. Specifically, the mapping unit can cluster all raw feature data to determine the similarity between any two raw feature data points. It then maps multiple raw feature data points with similarities greater than or equal to a preset similarity threshold to the same raw feature data point. Finally, the mapping result of all raw feature data points is used as the mapped feature data. By clustering and performing a many-to-one mapping of feature data, the mapping unit can effectively reduce the amount of raw feature data points, thereby reducing the total volume of raw feature data.

[0076] Among them, the above-mentioned mapping method 1 and mapping method 2 belong to precision mapping processing, and mapping method 3 belongs to spatial mapping processing. In one embodiment, the image encoding device can use only the precision mapping processing method, only the spatial mapping processing method, or simultaneously use the precision mapping processing method and the spatial mapping processing method.

[0077] Step S13: performing feature dimension transformation on the mapped feature data of the current feature dimension to obtain encoded feature data of the feature dimension supported by the preset encoding tool.

[0078] In an embodiment of the present application, in order to adapt to the original feature data of different feature dimensions and various encoding tools of different feature dimensions, the image encoding device can also use a dimension conversion unit to convert the feature data mapped in step S12 into a dimension, so that the feature data after dimension conversion can be feature encoded using a preset encoding tool.

[0079] Specifically, the dimension conversion method of the dimension conversion unit includes but is not limited to the following methods:

[0080] 1. Convert to spatial domain data: transform the mapped data from 1*N to W*H spatial domain data, 1*N=W*H. The spatial domain data can be encoded using an image codec.

[0081] 2. Convert to time domain data: transform the mapped data from m*N to n*W*H time domain data, where m*N=n*W*H. The time domain data can be encoded using a video codec.

[0082] It should be noted that the preset encoding tool in the embodiment of the present application can be specified by the staff based on the original feature data, or can be determined by the image encoding device by identifying the feature type and feature properties of the original feature data.

[0083] Step S14: Encode the encoding feature data using a preset encoding tool to obtain a feature code stream of the image to be encoded.

[0084] In the embodiment of the present application, the image encoding device may use a coding unit to encode the dimensionally converted feature data using a preset coding tool to obtain an initial feature code stream.

[0085] Specifically, the encoding unit compresses the dimensionally converted data into a code stream, which can be compressed using existing encoding tools. The encoding tools used in this application include but are not limited to:

[0086] 1.H.26x series encoders.

[0087] 2. Image encoders, including JPEG, BMP, TIFF, PNG and other encoders.

[0088] 3. AVS, SVAC, VP8, VP9, ​​AV1 and other encoders.

[0089] 4. Various self-developed codecs.

[0090] Furthermore, the image encoding device can also use the code stream organization unit to add feature encoding process information such as preprocessing method index, preprocessing parameters, mapping method index, mapping parameters, dimensionality conversion method index, dimensionality conversion parameters, encoder index and encoder parameters to the initial feature code stream to form the final feature code stream of the original feature data.

[0091] The image encoding device writes any one or more of the above-mentioned feature encoding process information into the feature code stream, so that the image encoding device can reconstruct the feature code stream according to the feature encoding process information in subsequent feature decoding to obtain the original feature data.

[0092] Specifically, the code stream organization unit may add key information in the pre-processing unit, the mapping unit, the dimension conversion unit, and the encoding unit to the initial code stream to obtain the final code stream.

[0093] like Figure 3As shown, the information added by the codestream organization unit includes, but is not limited to, header information, preprocessing information, mapping information, dimensionality conversion information, encoding information, and initial codestream information. This information can be added to the codestream using various methods, including, but not limited to, fixed-length coding, zero-order unsigned exponential Golomb coding, zero-order signed exponential Golomb coding, mapped exponential Golomb coding, and truncated (truncated) exponential Golomb coding.

[0094] The following describes each of the key information:

[0095] 1. Header Information: The header information contains the toolset flag information used in this encoding process, which is used to indicate which toolset is used in this encoding process, including pre_flag, map_flag, dim_flag, and enc_flag. If the value of each flag is 0, it means that the toolset is not enabled, otherwise it means that the toolset is enabled.

[0096] 2. Preprocessing information: If pre_flag = 1, it means that the preprocessing function is turned on. The preprocessing information will store preprocessing related information, including pre_index (preprocessing method index value), parameter values ​​​​required for preprocessing (pca_matrix_index (PCA matrix index value), sort_index (sort index value), matrix_index (transformation matrix index value), etc.).

[0097] 3. Mapping information: If map_flag = 1, it means that the mapping function is turned on. The mapping information will store mapping related information, including map_index (mapping method index value), parameter values ​​​​required for mapping (max_value (maximum value), amp_coefficient (amplification coefficient), standard_index (classification standard index value), etc.).

[0098] 4. Dimension conversion information: If dim_flag = 1, it means that the dimension conversion function is turned on. The dimension conversion information will store dimension conversion related information, including the final log2_width, log2_height (logarithm of width and height to 2), num_frame (final number of data frames), etc.

[0099] 5. Coding information: If enc_flag = 1, it means that the encoding function is turned on. The encoding information will store encoder-related information, including enc_index (codec index value), codec configuration parameter values ​​(bitrate (bit rate), delta_qp (quantization coefficient), etc.

[0100] In an embodiment of the present application, an image encoding device obtains an image to be encoded and extracts original feature data of the image to be encoded; performs mapping processing on the original feature data to obtain mapped feature data, wherein the data volume of the mapped feature data is less than the data volume of the original feature data; performs feature dimension transformation on the mapped feature data of the current feature dimension to obtain encoded feature data of the feature dimension supported by a preset encoding tool; and uses the preset encoding tool to encode the encoded feature data to obtain a feature code stream of the image to be encoded. The image encoding method of the present application provides a complete feature encoding process that can encode and decode various types of image feature data, thereby improving the generalization of image encoding.

[0101] The following continues to introduce the process of decoding the feature code stream by the image decoding device. It should be noted that the encoding process of the image encoding device and the decoding process of the image decoding device are essentially inverse processes of each other. That is, the encoding technology used in the encoding process introduced in the above embodiment can be deduced from the decoding technology used in the decoding process through reverse engineering, and they are not listed one by one here.

[0102] Please continue reading Figure 4 and Figure 5 , Figure 4 This is a flow chart of an embodiment of the image decoding method provided by this application. Figure 5 This is a schematic diagram of the overall process of the image decoding method provided by this application.

[0103] like Figure 4 As shown, the image decoding method of the embodiment of the present application includes the following steps:

[0104] Step S21: Acquire a feature code stream.

[0105] In an embodiment of the present application, the image decoding device uses a parsing unit to extract header information from the complete feature code stream, determine which toolsets are enabled, then parse the corresponding information from the enabled functions, and finally extract the initial code stream.

[0106] Step S22: Decode the feature code stream to obtain decoded feature data.

[0107] In the embodiment of the present application, when the encoding function is turned on, the image encoding device uses a decoding unit to decode the initial code stream based on the encoding information and reconstruct the feature data after dimension conversion.

[0108] Step S23: performing a feature dimension inverse transformation on the decoded feature data according to the feature dimension supported by the preset encoding tool to obtain mapped feature data.

[0109] In an embodiment of the present application, when the dimensionality conversion function is enabled, the image encoding apparatus utilizes the dimensionality inverse conversion unit to perform dimensionality inverse conversion on the dimensionality converted feature data based on the dimensionality conversion information, and reconstructs the mapped feature data.

[0110] Step S24: performing inverse mapping processing on the mapped feature data to obtain original feature data of the feature code stream, wherein the data volume of the original feature data is greater than the data volume of the mapped feature data.

[0111] Step S25: reconstruct the original feature data to obtain a reconstructed image.

[0112] When the mapping function is enabled, the image coding apparatus utilizes the inverse mapping unit to inversely map the mapped feature data based on the mapping information, and reconstructs the preprocessed feature data.

[0113] Furthermore, when the preprocessing function is enabled, the image encoding apparatus utilizes the inverse preprocessing unit to inversely map the preprocessed feature data based on the preprocessing information to reconstruct the original feature data.

[0114] The following further analyzes the encoding process of the image encoding device and the decoding process of the image decoding device through three embodiments:

[0115] Example 1: Based on vtm13.0, codec without pre-processing.

[0116] The feature data is 1*2048 floating point data.

[0117] 1. Coding:

[0118] Step 1: First get the maximum value, Then map according to the formula, F′=Round(F*2 t-1 / V max )+2 t-1 , where F′ is the integer data after mapping, F is the floating point data before mapping, and t is the mapping range; finally, truncation is performed, F′=clip(F′,0,2 t-1 ), record the mapping information.

[0119] Step 2: Convert the integer data dimension into spatial domain data (64*32) and record the dimension conversion information.

[0120] Step 3: Call the vtm13.0 encoder to encode the spatial domain data, obtain the original code stream, and record the encoding information.

[0121] Step 4: The header information, mapping information, dimension conversion information, encoding information and the original bitstream are spliced ​​into a complete bitstream, as follows:

[0122] Header information: Determine the values ​​of four flags: pre_flag = 0, map_flag = 1, dim_flag = 1, and enc_flag = 1. The four flags are encoded using 1-bit fixed-length coding.

[0123] Mapping information: Use 2-bit fixed-length code to encode map_index. If the value is 0, use 32 bits (float type) to encode max_value. If the value is 1, use 4-bit fixed-length code to encode amp_coefficient. If the value is 2, it means that non-uniform quantization is used for mapping. If the value is 3, it is a reserved value to be determined.

[0124] Dimension conversion information: Use 3-bit fixed-length encoding to encode log2_width and log2_height, and then use zero-order signed exponential Golomb code to encode num_frame.

[0125] Coding information: Use a 4-bit fixed-length code to record the enc index, which indicates which encoder was used for encoding. Use a zero-order signed exponential Golomb code to record the encoder delta_qp and bitrate.

[0126] 2. Decoding:

[0127] Step 1: Parse the bitstream to obtain header information, mapping information, dimension conversion information, encoding information, and the original bitstream, as follows:

[0128] (1) Parse header information: parse pre_flag, map_flag, dim_flag and enc_flag in turn to confirm which toolsets are needed for decoding.

[0129] (2) Parsing mapping information: If map_flag = 1, parse the value of map_index and determine which mapping method to use. If map_index is 0, read and parse the value of max_value; if map_index is 1, parse amp_coefficient, and the mapping method is amplification followed by rounding; if map_index is 2, it indicates that non-uniform quantization is used for mapping; if the value is 3, it is a reserved value and is to be determined.

[0130] (3) Parsing dimension conversion information: If dim_flag = 1, parse the values ​​of log2_width, log2_height and num_frame.

[0131] (4) Parsing coding information: If enc_flag=1, parse the value of enc_index and the encoder parameter values ​​delta_qp and bitrate.

[0132] Step 2: Call vtm13.0 to decode the original code stream to obtain the reconstructed spatial domain data.

[0133] Step 3: Perform inverse dimensionality transformation on the spatial domain data to obtain 1*2048 integer data.

[0134] Step 4: Integer data is mapped into floating-point data, which is the reconstructed feature data.

[0135] Example 2: A codec based on JPEG using a PCA matrix for preprocessing.

[0136] The feature data is 1*2048 floating point data.

[0137] 1. Coding:

[0138] Step 1: Use the PCA matrix to preprocess the feature data, obtain the feature data after dimensionality reduction, and record the preprocessing information.

[0139] Step 2: Get the maximum value of feature data after dimensionality reduction. Then map according to the formula, F′=Round(F*2 t-1 / V max )+2 t-1 , where F' is the integer data after mapping, F is the floating point data before mapping, and t is the mapping range; finally, truncation is performed, F'=clip(F',0,2 t-1 ), record the mapping information.

[0140] Step 3: Convert the integer data dimension into spatial domain data (32*32) and record the dimension conversion information.

[0141] Step 4: Call the jpeg encoder to encode the spatial domain data, obtain the original code stream, and record the encoding information.

[0142] Step 5: The header information, preprocessing information, mapping information, dimension conversion information, encoding information and the original bitstream are spliced ​​into a complete bitstream, as follows:

[0143] Header information: Determine the values ​​of four flags: pre_flag = 0, map_flag = 1, dim_flag = 1, and enc_flag = 1. The four flags are encoded using 1-bit fixed-length coding.

[0144] Preprocessing information: Use zero-order unsigned exponential Golomb code to encode pre_index. If the value is 0, it means that a single PCA matrix is ​​used for dimensionality reduction for preprocessing; if the value is 1, it means that multiple PCA matrices are used for dimensionality reduction for preprocessing, and pca_matrix_index needs to be encoded using zero-order unsigned exponential Golomb code; if the value is 2, it means that reordering is used for preprocessing, and sort_index needs to be encoded; if the value is 3, it means that FFT / DCT / DST is used for transformation, and matrix_index needs to be encoded.

[0145] Mapping information: Use 2-bit fixed-length code to encode map_index. If the value is 0, use 32 bits (float type) to encode max_value. If the value is 1, use 4-bit fixed-length code to encode amp_coefficient. If the value is 2, it means that non-uniform quantization is used for mapping. If the value is 3, it is a reserved value to be determined.

[0146] Dimension conversion information: Use 3-bit fixed-length encoding to encode log2_width and log2_height, and then use zero-order signed exponential Golomb code to encode num_frame.

[0147] Encoding information: Use a 4-bit fixed-length code to record enc_index, which indicates which encoder was used for encoding. Use a zero-order signed exponential Golomb code to record the encoder delta_qp and bitrate.

[0148] 2. Decoding:

[0149] Step 1: Parse the bitstream to obtain header information, preprocessing information, mapping information, dimension conversion information, encoding information, and the original bitstream, as follows:

[0150] Parse header information: parse pre_flag, map_flag, dim_flag, and enc_flag in turn to determine which toolsets are needed for decoding.

[0151] Parse preprocessing information: If pre_flag = 1, parse pre_index, which is the index value of the method used. If the value is 0, it means that a single PCA matrix is ​​used for dimensionality reduction for preprocessing; if the value is 1, it means that multiple PCA matrices are used for dimensionality reduction for preprocessing, and it is necessary to continue to parse pca_matrix_index to determine the PCA matrix; if the value is 2, it means that the reordering method is used for preprocessing, and it is necessary to continue to parse sort_index to determine the final sorting; if the value is 3, it means that the matrix transformation method is used for preprocessing, and it is necessary to parse matrix_index to determine the transformation matrix.

[0152] Parse mapping information: If map_flag = 1, parse the value of map_index and determine which mapping method to use. If map_index is 0, read and parse the value of max_value; if map_index is 1, parse amp_coefficient, and the mapping method is amplification followed by rounding; if map_index is 2, it indicates that non-uniform quantization is used for mapping; if the value is 3, it is a reserved value and is to be determined.

[0153] Parse dimension conversion information: If dim_flag = 1, parse the values ​​of log2_width, log2_height and num_frame.

[0154] Parse the encoding information: If enc_flag = 1, parse the value of enc_index and the encoder parameter values ​​delta_qp and bitrate.

[0155] Step 2: Call jpeg to decode the original code stream to obtain the reconstructed spatial domain data.

[0156] Step 3: Perform inverse dimensionality transformation on the spatial domain data to obtain 1*1024 integer data.

[0157] Step 4: Integer data is mapped into floating-point data, which is the preprocessed feature data.

[0158] Step 5: Use the PCA matrix to perform inverse preprocessing on the reconstructed feature data after dimensionality reduction to obtain the reconstructed feature data.

[0159] Example 3: A codec based on vtm13.0 that uses multiple pca matrices for preprocessing.

[0160] The feature data is 1*2048 floating point data.

[0161] 1. Coding:

[0162] Step 1: Use different PCA matrices to preprocess the feature data in sequence to obtain the feature data after dimensionality reduction. Use steps 2, 3, 4, and 5 in sequence for each feature data after dimensionality reduction to obtain the final bitstream. From these final bitstreams, select the bitstream with the highest quality of reconstructed feature data and save it to the preprocessing information.

[0163] Step 2: Get the maximum value of feature data after dimensionality reduction. Then map according to the formula, F′=Round(F*2 t-1 / V max )+2 t-1 , where F' is the integer data after mapping, F is the floating point data before mapping, and t is the mapping range; finally, truncation is performed, F'=clip(F', 0, 2 t-1 ), record the mapping information.

[0164] Step 3: Convert the integer data dimension into spatial domain data (32*32) and record the dimension conversion information.

[0165] Step 4: Call the vtm13.0 encoder to encode the spatial domain data, obtain the original code stream, and record the encoding information.

[0166] Step 5: The header information, preprocessing information, mapping information, dimension conversion information, encoding information and the original bitstream are spliced ​​into a complete bitstream, as follows:

[0167] Header information: Determine the values ​​of four flags: pre_flag = 0, map_flag = 1, dim_flag = 1, and enc_flag = 1. The four flags are encoded using 1-bit fixed-length coding.

[0168] Preprocessing information: Use zero-order unsigned exponential Golomb code to encode pre_index. If the value is 0, it means that a single PCA matrix is ​​used for dimensionality reduction for preprocessing; if the value is 1, it means that multiple PCA matrices are used for dimensionality reduction for preprocessing, and pca_matrix_index needs to be encoded using zero-order unsigned exponential Golomb code; if the value is 2, it means that reordering is used for preprocessing, and sort_index needs to be encoded; if the value is 3, it means that FFT / DCT / DST is used for transformation, and matrix_index needs to be encoded.

[0169] Mapping information: Use 2-bit fixed-length code to encode map_index. If the value is 0, use 32 bits (foat type) to encode max_value. If the value is 1, use 4-bit fixed-length code to encode amp_coefficient. If the value is 2, it means that non-uniform quantization is used for mapping. If the value is 3, it is a reserved value to be determined.

[0170] Dimension conversion information: Use 3-bit fixed-length encoding to encode log2_width and log2_height, and then use zero-order signed exponential Golomb code to encode num_frame.

[0171] Encoding information: Use a 4-bit fixed-length code to record enc_index, which indicates which encoder was used for encoding. Use a zero-order signed exponential Golomb code to record the encoder delta_qp and bitrate.

[0172] 2. Decoding:

[0173] Step 1: Parse the bitstream to obtain preprocessing information, mapping information, dimension conversion information, encoding information, and the original bitstream, as follows:

[0174] Parse header information: parse pre_flag, map_flag, dim_flag, and enc_flag in turn to determine which toolsets are needed for decoding.

[0175] Parse preprocessing information: If pre_flag = 1, parse pre_index, which is the index value of the method used. If the value is 0, it means that a single PCA matrix is ​​used for dimensionality reduction for preprocessing; if the value is 1, it means that multiple PCA matrices are used for dimensionality reduction for preprocessing, and it is necessary to continue to parse pca_matrix_index to determine the PCA matrix; if the value is 2, it means that the reordering method is used for preprocessing, and it is necessary to continue to parse sort_index to determine the final sorting; if the value is 3, it means that the matrix transformation method is used for preprocessing, and it is necessary to parse matrix_index to determine the transformation matrix.

[0176] Parse mapping information: If map_flag = 1, parse the value of map_index and determine which mapping method to use. If map_index is 0, read and parse the value of max_value; if map_index is 1, parse amp_coefficient, and the mapping method is amplification followed by rounding; if map_index is 2, it indicates that non-uniform quantization is used for mapping; if the value is 3, it is a reserved value and is to be determined.

[0177] Parse dimension conversion information: If dim_flag = 1, parse the values ​​of log2_width, log2_height and num_frame.

[0178] Parse the encoding information: If enc_flag = 1, parse the value of enc_index and the encoder parameter values ​​delta_qp and bitrate.

[0179] Step 2: Call vtm13.0 to decode the original code stream to obtain the reconstructed spatial domain data.

[0180] Step 3: Perform inverse dimensionality transformation on the spatial domain data to obtain 1*1024 integer data.

[0181] Step 4: Integer data is mapped into floating-point data, which is the preprocessed feature data.

[0182] Step 5: Use the PCA matrix to perform inverse preprocessing on the reconstructed feature data after dimensionality reduction to obtain the reconstructed feature data.

[0183] This application proposes an image encoding method and an image decoding method, which use a variety of preprocessing methods to flexibly deal with various types of feature data; they can also perform dimensionality conversion on the feature data to make it compatible with existing codecs and keep pace with the times; they can also use a variety of mapping methods to compress the feature data to different degrees, and control the accuracy of the final encoding.

[0184] The above embodiment is only one common case of the present application and does not limit the technical scope of the present application. Therefore, any minor modifications, equivalent changes or modifications made to the above content based on the essence of the present application solution are still within the scope of the technical solution of the present application.

[0185] Please continue to see Figure 6 , Figure 6 FIG. 5 is a schematic diagram of the structure of an embodiment of an image coding apparatus provided by the present application. The image coding apparatus 500 of the embodiment of the present application includes a processor 51 , a memory 52 , an input / output device 53 , and a bus 54 .

[0186] The processor 51 , the memory 52 , and the input / output device 53 are respectively connected to a bus 54 . The memory 52 stores program data, and the processor 51 is used to execute the program data to implement the image encoding method described in the above embodiment.

[0187] In the embodiment of the present application, the processor 51 may also be referred to as a CPU (Central Processing Unit). The processor 51 may be an integrated circuit chip having signal processing capabilities. The processor 51 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor may be a microprocessor, or the processor 51 may be any conventional processor.

[0188] Please continue to see Figure 7 , Figure 7 FIG. 6 is a schematic diagram of the structure of an image decoding apparatus according to an embodiment of the present application. The image decoding apparatus 600 according to the embodiment of the present application includes a processor 61 , a memory 62 , an input / output device 63 , and a bus 64 .

[0189] The processor 61 , the memory 62 , and the input / output device 63 are respectively connected to the bus 64 . The memory 62 stores program data, and the processor 61 is used to execute the program data to implement the image decoding method described in the above embodiment.

[0190] This application also provides a computer storage medium, please continue to refer to Figure 8 , Figure 8 It is a structural diagram of an embodiment of a computer storage medium provided in the present application. The computer storage medium 700 stores program data 71. When the program data 71 is executed by the processor, it is used to implement the image encoding method and / or image decoding method of the above embodiment.

[0191] When the embodiments of the present application are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0192] The above description is only an implementation method of the present application and does not limit the patent scope of the present application. Equivalent structures or equivalent process changes made by utilizing the contents of the description and drawings of this application, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. An image coding method, characterized in that: The image encoding method comprises: Acquire an image to be encoded, and extract original feature data of the image to be encoded; Performing mapping processing on the original feature data to obtain mapped feature data, wherein the data volume of the mapped feature data is smaller than the data volume of the original feature data; Performing feature dimension transformation on the mapped feature data of the current feature dimension to obtain encoded feature data of the feature dimension supported by a preset encoding tool, wherein the encoded feature data is spatial domain data or temporal domain data; The encoding feature data is encoded using the preset encoding tool to obtain a feature code stream of the image to be encoded, wherein the preset encoding tool is an image codec or a video codec responsive to spatial domain data, or the preset encoding tool is a video codec responsive to temporal domain data.

2. The image encoding method according to claim 1, wherein After extracting the original feature data of the image to be encoded, and before performing mapping processing on the original feature data and obtaining the mapped feature data, the image encoding method further includes at least one of the following processing: Reordering the original feature data according to a preset data rule; Performing dimensionality reduction processing on the original feature data using a preset principal component analysis matrix; Downsampling is performed on the original feature data.

3. The image encoding method according to claim 1, wherein The mapping process is performed on the original feature data to obtain mapped feature data, including: Get the maximum value of all original feature data and the floating-point data of each original feature data; According to a preset mapping function, the floating-point data of each original feature data is mapped using the maximum value to obtain integer data of each original feature data as mapped feature data.

4. The image encoding method according to claim 3, wherein The method of mapping the floating-point data of each original feature data using the maximum value according to a preset mapping function to obtain integer data of each original feature data as mapped feature data includes: According to the preset mapping function, obtaining a mapping range; Determining initial mapping feature data for each original feature data using the mapping range and the maximum value; The floating-point data of each initial mapping feature data is processed by the rounding function in the preset mapping function, the first integer part value of the floating-point data is retained, the decimal part value of the floating-point data is rounded off to obtain the second integer part value, the first integer part value and the second integer part value are added together to obtain the integer data of each original feature data as the mapping feature data.

5. The image encoding method according to claim 1, wherein The mapping process is performed on the original feature data to obtain mapped feature data, including: Multiplying the original feature data by a preset multiple to determine the initial mapping feature data; The floating-point data of the initial mapping feature data is rounded off, the third integer part value of the floating-point data is retained, the decimal part value of the floating-point data is rounded off to obtain the fourth integer part value, the third integer part value and the fourth integer part value are added together to obtain the integer data of the original feature data as the mapping feature data.

6. The image encoding method according to claim 1, wherein The mapping process is performed on the original feature data to obtain mapped feature data, including: Cluster all original feature data to obtain the similarity between any two original feature data; Mapping a plurality of original feature data with similarity greater than or equal to a preset similarity threshold to the same original feature data; The result after mapping all original feature data is used as the mapped feature data.

7. The image encoding method according to claim 1, wherein The performing feature dimension transformation on the mapped feature data of the current feature dimension to obtain coded feature data of the feature dimension supported by a preset coding tool includes: Converting the mapped feature data of the current feature dimension into spatial domain data or temporal domain data of the feature dimension supported by the preset encoding tool; The preset encoding tool is associated with the data format of the original feature data.

8. The image encoding method according to claim 1, wherein The step of encoding the encoding feature data using the preset encoding tool to obtain a feature code stream of the image to be encoded includes: Encoding the encoding feature data using the preset encoding tool to obtain an initial feature code stream of the image to be encoded; Adding feature encoding process information to the initial feature code stream to form a feature code stream of the image to be encoded; The feature encoding process information includes at least one or more of a mapping mode index, a mapping parameter, a dimension conversion mode index, a dimension conversion parameter, an encoding tool index, and an encoding tool parameter.

9. An image decoding method, characterized in that: The image decoding method comprises: Get the feature code stream; Decoding the feature code stream to obtain decoded feature data; Performing an inverse feature dimension transformation on the decoded feature data according to a feature dimension supported by a preset encoding tool to obtain mapped feature data, wherein the decoded feature data is spatial domain data or temporal domain data, wherein the preset encoding tool is an image codec or a video codec responsive to spatial domain data, or the preset encoding tool is a video codec responsive to temporal domain data; Performing inverse mapping processing on the mapped feature data to obtain original feature data of the feature code stream, wherein the data volume of the original feature data is greater than the data volume of the mapped feature data; The original feature data is reconstructed to obtain a reconstructed image.

10. An image encoding device, characterized in that The image encoding device includes a memory and a processor coupled to the memory; The memory is used to store program data, and the processor is used to execute the program data to implement the image encoding method according to any one of claims 1 to 8.

11. An image decoding device, characterized in that: The image decoding device includes a memory and a processor coupled to the memory; The memory is used to store program data, and the processor is used to execute the program data to implement the image decoding method according to claim 9.

12. A computer storage medium, characterized in that The computer storage medium is used to store program data, and when the program data is executed by a computer, it is used to implement the image encoding method according to any one of claims 1 to 8 and / or the image decoding method according to claim 9.

Images