Image processing method and device, electronic equipment and storage medium

Abstract

The present disclosure relates to an image processing method and device, electronic equipment and storage medium, the method comprising: obtaining first integer feature information and first coding side information of a first image to be encoded; obtaining an entropy coding parameter according to the first integer feature information and the first coding side information; and performing coding processing according to the entropy coding parameter, the first coding side information and the first integer feature information to obtain a first coding result of the first image. According to the image processing method of the embodiment of the present disclosure, the floating point number and other parameters output by the neural network are determined through integer fixed-point processing, and the obtained parameters can be kept consistent when the neural network is used across platforms for encoding, thereby obtaining cross-platform coding capability and reducing the possibility of decoding image distortion when applied across platforms.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of computer, and particularly relates to an image processing method and device, electronic equipment and storage medium. BACKGROUND

[0002] The image coding based on deep learning is limited by the calculation non-determinacy of IEEE754 floating-point number system. On different software and hardware platforms, the results of floating-point number operation cannot remain consistent for the same numerical input. The above-mentioned situation may cause the decoded image to be distorted, which is not conducive to the implementation of cross-platform deep visual coding and transmission. SUMMARY

[0003] The present disclosure provides an image processing method and device, electronic equipment and storage medium.

[0004] According to an aspect of the present disclosure, an image processing method is provided, comprising: obtaining first integer feature information and first coding side information of a first image to be coded; obtaining entropy coding parameters according to the first integer feature information and the first coding side information, the entropy coding parameters being coding information obtained by a neural network after integer fixed-point processing, the integer fixed-point processing being used to make the output information of the neural network be integer information; and performing coding processing according to the entropy coding parameters, the first coding side information and the first integer feature information to obtain a first coding result of the first image.

[0005] The image processing method according to the embodiment of the present disclosure, through the integer fixed-point processing, makes the floating-point number and other parameters output by the neural network be determined values, and when the neural network is used for coding across platforms, the obtained parameters can remain consistent, so that the cross-platform coding capability is obtained, and the possibility of the decoded image being distorted when applied across platforms is reduced.

[0006] In a possible implementation manner, the obtaining of the first integer feature information and the first coding side information of the first image to be coded comprises: inputting the first image into a first coding network to obtain first feature information; inputting the first feature information into a second coding network to obtain third feature information; performing integer processing on the third feature information to obtain the first coding side information; and performing integer processing on the first feature information to obtain the first integer feature information.

[0007] In this way, the first coding side information and the first integer feature information each of which is an integer can be obtained, so as to reduce coding redundancy and reduce the uncertainty of floating-point number operation.

[0008] In a possible implementation, the obtaining the entropy coding parameter according to the first integer characteristic information and the first coding side information comprises: inputting the first coding side information into the first decoding network after integer quantization processing to obtain second characteristic information; and obtaining the entropy coding parameter according to the second characteristic information and the first integer characteristic information.

[0009] In this way, the second characteristic information can be obtained through the first decoding network after integer quantization processing, so as to reduce the uncertainty of the calculation result and improve the accuracy of the entropy coding parameter.

[0010] In a possible implementation, the obtaining the entropy coding parameter according to the second characteristic information and the first integer characteristic information comprises: obtaining first transcendental characteristic information and second transcendental characteristic information according to the second characteristic information and the first integer characteristic information, wherein the first transcendental characteristic information and the second transcendental characteristic information correspond to part of the first integer characteristic information respectively; obtaining first entropy coding estimation parameter and second entropy coding estimation parameter according to the first transcendental characteristic information, the second transcendental characteristic information and the parameter prediction network after integer quantization processing; and obtaining the entropy coding parameter according to the first entropy coding estimation parameter and the second entropy coding estimation parameter.

[0011] In this way, the parameter prediction processing can be performed on the second characteristic information based on the first integer characteristic information, so as to improve the accuracy of the obtained entropy coding parameter.

[0012] In a possible implementation, the obtaining the first transcendental characteristic information and the second transcendental characteristic information according to the second characteristic information and the first integer characteristic information comprises: decomposing the first integer characteristic information to obtain second integer characteristic information and third integer characteristic information; obtaining the first transcendental characteristic information according to the second integer characteristic information and the second characteristic information; and obtaining the second transcendental characteristic information according to the third integer characteristic information and the second characteristic information.

[0013] In this way, the two integer characteristic information (the second integer characteristic information and the third integer characteristic information) obtained respectively have adjacent elements to the other characteristic information, so as to facilitate obtaining the relationship between the elements in the two integer characteristic information and obtaining the semantic information.

[0014] In a possible implementation, the decomposing the first integer characteristic information to obtain the second integer characteristic information and the third integer characteristic information comprises: performing chessboard decomposition on the first integer characteristic information to obtain the second integer characteristic information and the third integer characteristic information.

[0015] In this way, the first transcendental feature information and the second transcendental feature information can be obtained through the corresponding relationship of the elements in the first integer feature information after chessboard decomposition, the relationship between the elements of the first transcendental feature information and the second transcendental feature information is facilitated to be obtained, and the semantic information is facilitated to be obtained.

[0016] In a possible implementation, according to the first transcendental feature information, the second transcendental feature information, and the parameter prediction network after the integer quantization processing, the first entropy coding estimation parameter and the second entropy coding estimation parameter are obtained, including: inputting the first transcendental feature information into the parameter prediction network after the integer quantization processing to obtain the first entropy coding estimation parameter; inputting the second integer feature information into the semantic prediction network after the integer quantization processing to obtain the first semantic feature information; and inputting the first semantic feature information and the second transcendental feature information into the parameter prediction network after the integer quantization processing to obtain the second entropy coding estimation parameter.

[0017] In this way, the first semantic feature information can be obtained by considering the information of the relationship between adjacent elements when determining the entropy coding parameter, and the accuracy of the entropy coding parameter can be improved.

[0018] In a possible implementation, the encoding processing according to the entropy coding parameter, the first encoding side information, and the first integer feature information to obtain the first encoding result of the first image includes: inputting the entropy coding parameter into a probability estimation network after probability quantization processing to obtain a first probability cumulative parameter, where the probability quantization processing makes the first probability cumulative parameter a determined parameter; performing entropy coding processing according to the first probability cumulative parameter and the first integer feature information to obtain an encoded feature; and obtaining the first encoding result according to the encoded feature and the first encoding side information.

[0019] In this way, the first probability cumulative parameter output by the probability estimation network after probability quantization processing is determined, and the encoding is performed based on the first probability cumulative function, so that the encoding result can have determinacy, higher compression rate, and the ability to perform cross-platform coding and decoding, and the probability of image distortion during decoding is reduced.

[0020] In a possible implementation, the method further includes: obtaining second encoding side information from the second encoding result to be decoded; performing decoding processing according to the second encoding side information to obtain fourth integer feature information; and performing decoding processing on the fourth integer feature information to obtain a second image.

[0021] In this way, the fourth feature information can be obtained based on the first decoding network after the integer quantization processing, so that the fourth feature information is determined information, and the probability of decoding distortion is reduced.

[0022] In a possible implementation, the decoding according to the second encoding side information to obtain fourth integer feature information comprises: inputting the second encoding side information into the first decoding network after integer fixed-point processing to obtain fourth feature information; obtaining fifth integer feature information according to the fourth feature information, where the fifth integer feature information corresponds to part of the fourth feature information; and obtaining the fourth integer feature information according to the fifth integer feature information and the fourth feature information.

[0023] In a possible implementation, the obtaining of the fifth integer feature information according to the fourth feature information comprises: decomposing the fourth feature information to obtain third transcendental feature information, where the third transcendental feature information is part of the fourth feature information; inputting the third transcendental feature information into the parameter prediction network after integer fixed-point processing to obtain third entropy encoding estimation parameters; inputting the third entropy encoding estimation parameters into the probability estimation network after probability fixed-point processing to obtain second probability cumulative parameters; and obtaining the fifth integer feature information by decoding the second encoding result according to the second probability cumulative parameters.

[0024] In this way, the fifth integer feature information corresponding to part of the integer feature information can be obtained by the parameter prediction network after integer fixed-point processing and the probability estimation network after probability fixed-point processing, and the certainty of the fifth integer feature information is improved.

[0025] In a possible implementation, the obtaining of the fourth integer feature information according to the fifth integer feature information and the fourth feature information comprises: decomposing the fourth feature information to obtain fourth transcendental feature information, where the fourth transcendental feature information is part of the fourth feature information; obtaining fourth entropy encoding estimation parameters according to the fifth integer feature information and the fourth transcendental feature information; obtaining sixth integer feature information according to the fourth entropy encoding estimation parameters; and obtaining the fourth integer feature information according to the fifth integer feature information and the sixth integer feature information.

[0026] In this way, the sixth integer feature information can be obtained based on the information about the relationship between adjacent elements, and then the fourth integer feature information is obtained, so that the accuracy of the fourth integer feature information is improved.

[0027] In a possible implementation, the obtaining the fourth entropy coding estimation parameter according to the fifth integer characteristic information and the fourth transcendental characteristic information comprises: inputting the fifth integer characteristic information into the semantic prediction network after integer quantization processing to obtain second semantic characteristic information; and inputting the second semantic characteristic information and the fourth transcendental characteristic information into the parameter prediction network after integer quantization processing to obtain the fourth entropy coding estimation parameter.

[0028] In this way, the second semantic characteristic information, that is, information representing the relationship between adjacent elements, can be obtained, so that the accuracy of the fourth entropy coding estimation parameter is improved.

[0029] In a possible implementation, the method further comprises: performing rounding processing on the convolution kernel parameters of the plurality of network levels of the first decoding network according to the maximum value and the minimum value of the convolution kernel parameters to obtain the convolution kernel parameters of the plurality of network levels after rounding processing; and determining the maximum value and the minimum value in the operation result of the preset calibration data for the plurality of network levels of the first decoding network respectively, wherein the inputting the first encoded side information into the first decoding network after integer quantization processing to obtain the second characteristic information comprises: obtaining input information of the plurality of network levels according to output information of the plurality of network levels of the first decoding network after integer quantization processing and the maximum value and the minimum value in the operation result, the output information of the plurality of network levels comprising processing results of the plurality of network levels of the first decoding network after integer quantization processing on the first encoded side information; and obtaining the second characteristic information according to the input information of the network level and the convolution kernel parameters of the plurality of network levels after rounding processing.

[0030] In this way, the uncertainty of floating-point operation can be reduced through integer quantization processing, an application basis is provided for cross-platform applications, the structure of the neural network is not changed, the performance of the neural network is ensured, the training difficulty and the design difficulty are not increased, and the compression rate of the encoding is not reduced.

[0031] In a possible implementation, the method further comprises: performing quadratic fitting on the normal distribution function to obtain a fitting coefficient; and obtaining the probability estimation network after probability quantization processing according to the fitting coefficient.

[0032] In this way, the first probability cumulative parameter output by the probability estimation network can be determined through probability quantization processing, so that the ability of cross-platform use is obtained, the possibility of image distortion after decoding is reduced, the structure of the neural network is not changed, the training difficulty and the design difficulty are not increased, the probability estimation network is not deleted, and the compression rate of the encoding is not reduced.

[0033] According to an aspect of the present disclosure, an image processing apparatus is provided, comprising: a first information acquisition module configured to acquire first integer feature information and first encoding side information of a first image to be encoded; an entropy encoding parameter acquisition module configured to acquire an entropy encoding parameter according to the first integer feature information and the first encoding side information, the entropy encoding parameter being encoding information obtained by a neural network after integer fixed-point processing, the integer fixed-point processing being configured to make the output information of the neural network be integer information; and an encoding module configured to perform encoding processing according to the entropy encoding parameter, the first encoding side information and the first integer feature information, and obtain a first encoding result of the first image.

[0034] In a possible implementation, the first information acquisition module is further configured to: input the first image into a first encoding network to acquire the first feature information; input the first feature information into a second encoding network to obtain third feature information; perform rounding processing on the third feature information to obtain the first encoding side information; and perform rounding processing on the first feature information to obtain the first integer feature information.

[0035] In a possible implementation, the entropy encoding parameter acquisition module is further configured to: input the first encoding side information into a first decoding network after integer fixed-point processing to obtain second feature information; and acquire the entropy encoding parameter according to the second feature information and the first integer feature information.

[0036] In a possible implementation, the entropy encoding parameter acquisition module is further configured to: the acquiring the entropy encoding parameter according to the second feature information and the first integer feature information comprises: acquiring first transcendental feature information and second transcendental feature information according to the second feature information and the first integer feature information, the first transcendental feature information and the second transcendental feature information respectively corresponding to part of the first integer feature information; acquiring first entropy encoding estimation parameter and second entropy encoding estimation parameter according to the first transcendental feature information, the second transcendental feature information and a parameter prediction network after the integer fixed-point processing; and acquiring the entropy encoding parameter according to the first entropy encoding estimation parameter and the second entropy encoding estimation parameter.

[0037] In a possible implementation, the entropy encoding parameter acquisition module is further configured to: decompose the first integer feature information to obtain second integer feature information and third integer feature information; acquire the first transcendental feature information according to the second integer feature information and the second feature information; and acquire the second transcendental feature information according to the third integer feature information and the second feature information.

[0038] In a possible implementation, the entropy coding parameter obtaining module is further configured to: perform chessboard decomposition on the first integer feature information to obtain the second integer feature information and the third integer feature information.

[0039] In a possible implementation, the entropy coding parameter obtaining module is further configured to: input the first transcendental feature information into the parameter prediction network after integer quantization processing to obtain first entropy coding estimation parameters; input the second integer feature information into the semantic prediction network after integer quantization processing to obtain first semantic feature information; and input the first semantic feature information and the second transcendental feature information into the parameter prediction network after integer quantization processing to obtain second entropy coding estimation parameters.

[0040] In a possible implementation, the encoding module is further configured to: input the entropy coding parameters into the probability estimation network after probability quantization processing to obtain first probability accumulation parameters, where the probability quantization processing makes the first probability accumulation parameters deterministic parameters; perform entropy coding processing according to the first probability accumulation parameters and the first integer feature information to obtain encoded features; and obtain the first encoding result according to the encoded features and the first encoding side information.

[0041] In a possible implementation, the apparatus further includes a decoding module configured to: obtain second encoding side information from a second encoding result to be decoded; perform decoding processing according to the second encoding side information to obtain fourth integer feature information; and perform decoding processing on the fourth integer feature information to obtain a second image.

[0042] In a possible implementation, the decoding module is further configured to: input the second encoding side information into the first decoding network after integer quantization processing to obtain fourth feature information; obtain fifth integer feature information according to the fourth feature information, where the fifth integer feature information corresponds to part of the fourth feature information; and obtain the fourth integer feature information according to the fifth integer feature information and the fourth feature information.

[0043] In a possible implementation, the decoding module is further configured to: perform decomposition on the fourth feature information to obtain third transcendental feature information, where the third transcendental feature information is part of the fourth feature information; input the third transcendental feature information into the parameter prediction network after integer quantization processing to obtain third entropy coding estimation parameters; input the third entropy coding estimation parameters into the probability estimation network after probability quantization processing to obtain second probability accumulation parameters; and perform decoding processing on the second encoding result according to the second probability accumulation parameters to obtain fifth integer feature information.

[0044] In a possible implementation, the decoding module is further configured to: decompose the fourth feature information to obtain fourth transcendental feature information, where the fourth transcendental feature information is part of the fourth feature information; obtain fourth entropy coding estimation parameters according to the fifth integer feature information and the fourth transcendental feature information; obtain sixth integer feature information according to the fourth entropy coding estimation parameters; and obtain the fourth integer feature information according to the fifth integer feature information and the sixth integer feature information.

[0045] In a possible implementation, the decoding module is further configured to: input the fifth integer feature information into an integer quantized semantic prediction network to obtain second semantic feature information; and input the second semantic feature information and the fourth transcendental feature information into an integer quantized parameter prediction network to obtain the fourth entropy coding estimation parameters.

[0046] In a possible implementation, the apparatus further includes an integer quantization module configured to: perform integer processing on convolution kernel parameters of multiple network levels of the first decoding network according to maximum values and minimum values of the convolution kernel parameters, to obtain multiple network level convolution kernel parameters after the integer processing; and determine multiple network levels of the first decoding network respectively, and the entropy coding parameter obtaining module is further configured to: obtain input information of the multiple network levels according to output information of the multiple network levels of the first decoding network after the integer quantization processing and the maximum values and the minimum values in the operation result, where the output information of the multiple network levels includes processing results of the multiple network levels of the first decoding network after the integer quantization processing on the first encoding side information; and obtain the second feature information according to the input information of the network levels and the multiple network level convolution kernel parameters after the integer processing.

[0047] In a possible implementation, the apparatus further includes a probability quantization module configured to: perform quadratic fitting on a normal distribution function to obtain fitting coefficients; and obtain the probability quantization processed probability estimation network according to the fitting coefficients.

[0048] According to an aspect of the present disclosure, an electronic device is provided, including: a processor; a memory for storing processor-executable instructions; and wherein the processor is configured to invoke the instructions stored in the memory to perform the above method.

[0049] According to an aspect of the present disclosure, a computer readable storage medium is provided, which stores computer program instructions, and the computer program instructions are executed by a processor to implement the above method.

[0050] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. Other features and aspects of the present disclosure will become apparent from the following detailed description, taken in conjunction with the accompanying drawings, which illustrate, by way of example, the principles of the disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0051] The accompanying drawings, which are incorporated herein and constitute part of the specification, illustrate embodiments consistent with the present disclosure and, together with the description, further serve to explain the principles of the present disclosure.

[0052] Figure 1 A flow chart of an image processing method according to an embodiment of the present disclosure is shown;

[0053] Figure 2 A schematic diagram of chessboard decomposition according to an embodiment of the present disclosure is shown;

[0054] Figure 3A And Figure 3B An application schematic diagram of an image processing method according to an embodiment of the present disclosure is shown;

[0055] Figure 4 A block diagram of an image processing apparatus according to an embodiment of the present disclosure is shown;

[0056] Figure 5 A block diagram of an electronic device according to an embodiment of the present disclosure is shown;

[0057] Figure 6 A block diagram of an electronic device according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0058] Various exemplary embodiments, features and aspects of the present disclosure will be explained in detail below with reference to the accompanying drawings. Like reference numerals in the drawings denote like elements or components. Although various aspects of the embodiments are illustrated in the drawings, the drawings are not necessarily drawn to scale unless specifically indicated.

[0059] The word "exemplary" is used herein to mean "serving as an example, instance, or illustration." Any implementation described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other implementations.

[0060] The term "and / or", used herein only to describe association relationship of associated objects, means that there can be three relationships, for example, A and / or B, which can represent the existence of A alone, the existence of A and B at the same time, and the existence of B alone. In addition, the term "at least one" herein means any one of the plurality or any combination of at least two of the plurality, for example, including at least one of A, B and C, which can mean including any one or more elements selected from the set consisting of A, B and C.

[0061] In addition, for a better illustration of the present disclosure, numerous specific details are given in the following detailed description. Those skilled in the art will understand that the present disclosure can be practiced without certain specific details. In some instances, well-known methods, means, elements and circuits have not been described in detail in order to emphasize the principles of the present disclosure.

[0062] In the process of image coding by deep learning neural network, due to the uncertainty of floating point number and probability calculation result, the application of the image coding method is limited, that is, in the process of cross-platform use, the decoded image may be distorted due to the deviation of the calculation result. In the related art, changing the deep learning neural network to a full integer operation network may limit the function of the neural network, and the design and training are more difficult, and deleting the probability estimation network may cause the coding compression to decrease. In view of the problem, the present disclosure provides an image processing method as follows.

[0063] Figure 1 A flowchart of an image processing method according to an embodiment of the present disclosure is shown as follows. Figure 1 As shown in the figure, the method comprises:

[0064] In step S11, first integer feature information and first coding edge information of a first image to be coded are obtained.

[0065] In step S12, entropy coding parameters are obtained according to the first integer feature information and the first coding edge information, the entropy coding parameters being coding information obtained by an integer fixed-point processing neural network, the integer fixed-point processing being used to make the output information of the neural network be integer information.

[0066] In step S13, coding processing is performed according to the entropy coding parameters, the first coding edge information and the first integer feature information, and a first coding result of the first image is obtained.

[0067] According to the image processing method of the embodiment of the present disclosure, the floating point number and other parameters output by the neural network are made to be determined values through integer fixed-point processing, and when the neural network is used for coding across platforms, the obtained parameters can be kept consistent, thereby obtaining cross-platform coding capability and reducing the possibility of image distortion when decoding across platforms.

[0068] In a possible implementation, the image processing method can be executed by a terminal device or other processing device. The terminal device can be a user equipment (UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, or the like. The other processing device can be a server or a cloud server, or the like. In some possible implementations, the method can be implemented by a processor invoking computer readable instructions stored in a memory.

[0069] In a possible implementation, to address the above problems, the present disclosure can be processed by integer quantization processing, so that the parameters and / or results obtained in the neural network operation process are determined and are not affected by the uncertainty of floating-point numbers. The determined results can enable the coding and decoding method based on the deep learning neural network to be applied across platforms, and reduce the inconsistency of the coding and decoding results and the consequences of image distortion. Further, the integer quantization processing does not change the structure of the neural network, does not reduce the performance of the neural network, and does not increase the complexity of the neural network.

[0070] In a possible implementation, in the encoding process, taking a first image to be encoded as an example, the neural network processed by the integer quantization processing is used for encoding, which can ensure the determinacy of the encoding result without reducing the encoding performance. The first image can be any image, for example, an image obtained by shooting, a computer-generated image, or any video frame in a video. The present disclosure does not limit the type of the first image.

[0071] In a possible implementation, in step S11, the first integer feature information and the first encoding side information of the first image to be encoded can be obtained. The encoding side information is used to reduce information redundancy in the encoding process, and the first encoding side information is the encoding side information in the encoding process of the first image. The first integer feature information is feature information with an integer parameter value, which is obtained by performing feature extraction on the first image.

[0072] In a possible implementation, step S11 can include: inputting the first image into a first encoding network to obtain first feature information; inputting the first feature information into a second encoding network to obtain third feature information; performing rounding processing on the third feature information to obtain the first encoding side information; and performing rounding processing on the first feature information to obtain the first integer feature information.

[0073] In a possible implementation, the first image is input into a first encoding network, which can perform encoding processing such as downsampling and convolution on the first image to reduce the resolution of the first image, but can improve the dimension, receptive field, and other parameters of the first image. The obtained first feature information is multi-dimensional feature information (that is, a tensor), and each dimension of the feature information can be represented as a feature map or a feature matrix, that is, the first feature information can include a plurality of feature maps or feature matrices. The numerical values of the elements in the first feature information can be floating-point numbers, for example, the numerical values of the elements in the feature matrix can be floating-point numbers. The first feature information can also be other forms of information, such as a vector, and the type of the first feature information is not limited in the present disclosure.

[0074] The first feature information is converted into first integer feature information, that is, the floating-point numbers of the numerical values of the elements in the first feature information are converted into integers, to reduce the uncertainty of floating-point number operations in further operations.

[0075] In a possible implementation, the first feature information can be further processed to obtain first encoding side information used in encoding and capable of reducing encoding redundancy. The first feature information can be input into a second encoding network to obtain third feature information. The second encoding network can be a deep learning neural network, which can perform further encoding processing such as downsampling and convolution on the first feature information to obtain the third feature information. The numerical values of the elements in the third feature information can be floating-point numbers.

[0076] In a possible implementation, to reduce the uncertainty of floating-point number operations, the third feature information can be rounded, that is, the numerical values of the elements of the third feature information are rounded to obtain first encoding side information with integer numerical values of the elements. In an example, the rounding processing can include down rounding, up rounding, approximate rounding, and the like, and can also be performed through the following formula (1):

[0077]

[0078] wherein W is the numerical value (floating-point number) of any element in the third feature information, maxW is the maximum value of the elements in the third feature information, and minW is the minimum value of the elements in the third feature information. represents rounding operations such as down rounding, up rounding, approximate rounding, and the like, and q is the integer value of W rounded through the formula (1).

[0079] In a possible implementation, the first feature information is processed by rounding similarly to the rounding of the third feature information. For example, the values of the elements of the first feature information are rounded down, rounded up, rounded approximately, rounded by formula (1), and the like. The present disclosure does not limit the specific rounding method. After rounding, the first integer feature information is obtained, in which the values of the elements are all integers.

[0080] In this way, the first encoding side information and the first integer feature information, in which the values of the elements are all integers, are obtained, so as to reduce the coding redundancy and reduce the uncertainty of floating-point operation.

[0081] In a possible implementation, after the first encoding side information and the first integer feature information, in which the values of the elements are all integers, are obtained in step S12, the entropy coding parameter used for coding can be obtained based on the two kinds of information. The coding can include entropy coding, which is a coding process according to the entropy principle without losing any information. The entropy coding parameter is information used in the entropy coding process of the first image.

[0082] In a possible implementation, step S12 can include: inputting the first encoding side information into the first decoding network after integer fixed-point processing to obtain second feature information; and obtaining the entropy coding parameter based on the second feature information and the first integer feature information.

[0083] In this way, the second feature information is obtained through the first decoding network after integer fixed-point processing, so as to reduce the uncertainty of the calculation result and improve the accuracy of the entropy coding parameter.

[0084] In a possible implementation, the first encoding side information can be processed by the first decoding network after integer fixed-point processing to obtain the second feature information. As described above, the integer fixed-point processing can reduce the uncertainty of the calculation result of the floating-point number, so that the output result of the neural network is a determined parameter, that is, there is a determined corresponding relationship between the input information and the output information of the neural network, and the output information is not determined according to probability and the like, so that the output information has a certain uncertainty, or the output information has a certain uncertainty due to the value and precision of the floating-point number. In addition, the first encoding side information in which the values of the elements are all integers is processed, so as to further reduce the uncertainty of the output information.

[0085] In a possible implementation, the integer quantization processing can be performed on parameters (for example, convolution kernel parameters) of the neural network, for example, rounding the parameters of the neural network according to a certain rule, so as to reduce the uncertainty of the output result caused by the value and accuracy of the floating point number in the operation process of the neural network. Further, the input information of each network level of the neural network can also be rounded according to a certain rule, so as to further reduce the uncertainty of the output result.

[0086] In a possible implementation, the method further includes: rounding the convolution kernel parameters according to the maximum and minimum values of the convolution kernel parameters of the plurality of network levels of the first decoding network, to obtain the convolution kernel parameters of the plurality of network levels after rounding; and determining the maximum and minimum values in the operation result of the preset calibration data for each network level of the first decoding network.

[0087] In a possible implementation, the first decoding network can be a deep learning neural network, for example, a convolutional neural network, and the first decoding network can include a plurality of convolution layers, and the structure of the first decoding network is not limited in the present disclosure. As described above, the network parameters (for example, convolution kernel parameters) of the first decoding network can be rounded according to a certain rule, for example, rounding up, rounding down, approximate rounding, etc., or rounding according to the above formula (1).

[0088] In an example, the convolution kernel parameters can be in the form of a matrix, for example, a 2x2 matrix, a 3x3 matrix, etc. The values of the elements in the matrix can be rounded, for example, according to the formula (1). For the convolution kernel parameters of a certain convolution layer, the maximum and minimum values can be determined, and the values of the rounded elements can be solved according to the formula (1), to obtain the convolution kernel parameters of the convolution layer after rounding. Further, the convolution kernel parameters of each network level after rounding can be obtained in this way.

[0089] Further, the quantization result of each rounded convolution kernel parameter can also be obtained The quantization result can represent the change range of the rounded value, as shown in the following formula (2):

[0090]

[0091] wherein q' is the rounded convolution kernel parameter, maxW' is the maximum value of the convolution kernel parameter before rounding, and minW' is the minimum value of the convolution kernel parameter before rounding.

[0092] In a possible implementation, the input information of each network level can also be rounded according to a certain rule to further reduce the uncertainty of the output information and prevent data overflow in the operation process. In an example, the data length of the rounded convolution kernel parameter is 8 bits, and the input information of each convolution layer after rounding can be rounded so that the input information is an 8-bit integer in the value range of [0, 255], so that no data overflow occurs during operation.

[0093] In a possible implementation, the parameters used when rounding the input information of each convolution layer after rounding can be determined according to the operation results of each convolution layer when the preset calibration data x is processed by the neural network that does not perform the above rounding. In an example, the parameters used when rounding are the maximum value max x and the minimum value min x in each element of the operation result (for example, the processing result is in the form of a tensor) of each network level (for example, a convolution layer) when the preset calibration data is processed by the neural network that does not perform rounding. The calibration data can be randomly generated image data or preset image data that satisfies a certain rule, and the present disclosure does not limit the calibration data.

[0094] In a possible implementation, further, the first encoding information can be input to the first decoding network after integer fixed-point processing, and the input information of each convolution layer after rounding of the first decoding network can be rounded by using the above parameters (the maximum value max x and the minimum value min x), and the input information after rounding can be processed by the convolution layer after rounding. The first decoding network after integer fixed-point processing can finally output the determined second feature information. This step can include: obtaining the input information of a plurality of network levels of the first decoding network after integer fixed-point processing according to the output information of the plurality of network levels of the first decoding network after integer fixed-point processing and the maximum value and the minimum value in the operation result, the output information of the plurality of network levels including the processing result of the plurality of network levels of the first decoding network after integer fixed-point processing on the first encoding side information; and obtaining the second feature information according to the input information of the network level and the convolution kernel parameter of the plurality of network levels after rounding.

[0095] In an example, the first encoded side information is input into the first decoding network after the integer quantization processing, and the neural network can be processed layer by layer. For example, the first network layer processes the first encoded side information to obtain the output information of the first network layer (i.e., the processing result of the first encoded side information by the first network layer), the input information of the second network layer is obtained after the rounding processing of the output information of the first network layer, and the output information of the second network layer (i.e., the processing result of the first encoded side information by the second network layer) is obtained after the processing of the second network layer. In this way, the input information of each network layer can be obtained and processed, and finally the second feature information can be obtained.

[0096] In an example, the output information of the last network layer of the convolution layer after the rounding processing can be rounded to obtain the input information of the convolution layer after the rounding processing, and the output information of the last network layer of other network layers (e.g., an activation layer, etc.) can be directly used as the input information without rounding processing. Alternatively, the output information of the last network layer of each network layer can be rounded to obtain the input information of the network layer, which is not limited in the present disclosure.

[0097] In an example, when rounding the output information, the rounding processing can be performed based on the above-mentioned parameters (i.e., the maximum value max x and the minimum value min x). For example, the numerical value of each element of the output information of the last network layer can be divided by the integer closest to , or the numerical value of each element of the output information of the last network layer can be multiplied by and then rounded to convert the numerical value of each element to an 8-bit integer in the numerical value range of [0, 255] to obtain the input information of each network layer, so that data overflow does not occur during calculation.

[0098] In an example, the input information of each network layer after rounding can be processed by the first decoding network after the integer quantization processing, and the above-mentioned second feature information can be output. In an example, the second feature information is integer feature information having a determined corresponding relationship with the first encoded side information.

[0099] In this way, the integer quantization processing can reduce the uncertainty of floating-point operations, provide an application basis for cross-platform applications, and does not change the structure of the neural network, ensures the performance of the neural network, and does not increase the training difficulty and design difficulty, so that the compression rate of the encoding does not decrease.

[0100] In a possible implementation, after the second feature information is obtained in the manner described above, the second feature information can be further processed. For example, the second feature information can be subjected to parameter prediction processing based on the first integer feature information to obtain an entropy coding parameter. The entropy coding parameter is a parameter required in the process of entropy coding and is also an important basis for determining a probability accumulation parameter in the process of image coding. The second feature information can be subjected to parameter prediction processing based on the first integer feature information, which can improve the prediction accuracy and further improve the accuracy of the probability accumulation parameter. Of course, the second feature information can also be directly subjected to parameter prediction processing. For example, the second feature information can be directly input into a parameter prediction network for processing to obtain the entropy coding parameter. The present disclosure does not limit this.

[0101] In a possible implementation, to improve the accuracy of the entropy coding parameter, the entropy coding parameter can be obtained according to the second feature information and the first integer feature information. This step can include: obtaining first transcendental feature information and second transcendental feature information according to the second feature information and the first integer feature information, the first transcendental feature information and the second transcendental feature information corresponding to part of the first integer feature information respectively; obtaining first entropy coding estimation parameters and second entropy coding estimation parameters according to the first transcendental feature information, the second transcendental feature information, and the parameter prediction network subjected to integer fixed-point processing; and obtaining the entropy coding parameter according to the first entropy coding estimation parameters and the second entropy coding estimation parameters.

[0102] In this way, the second feature information can be subjected to parameter prediction processing based on the first integer feature information, which can improve the accuracy of the obtained entropy coding parameter.

[0103] In a possible implementation, the obtaining of the first transcendental feature information and the second transcendental feature information according to the second feature information and the first integer feature information includes: decomposing the first integer feature information to obtain second integer feature information and third integer feature information; obtaining the first transcendental feature information according to the second integer feature information and the second feature information; and obtaining the second transcendental feature information according to the third integer feature information and the second feature information.

[0104] In one possible implementation, the first integer feature information can be feature information in tensor form (e.g., comprising multiple matrices), and the value of each element in this feature information is an integer. The decomposition of the first integer feature information can include various methods; for example, each matrix can be symmetrically decomposed to obtain two diagonal matrices, or symmetrically decomposed along the central row or central column of the matrix to obtain two matrices. To facilitate determining the relationship between elements, the decomposition can be performed in a chessboard pattern. This step may include: performing a chessboard decomposition on the first integer feature information to obtain the second integer feature information and the third integer feature information.

[0105] Figure 2 A schematic diagram illustrating the decomposition of a chessboard according to an embodiment of the present disclosure is shown. (As follows) Figure 2 As shown, each cell can represent an element in the matrix described above. The elements at shaded cell positions and the elements at blank cell positions can be separated to obtain two matrices. One matrix corresponds to the elements at the shaded cell positions, where the elements at the shaded cell positions have the same values ​​as the elements in the original matrix, and the elements at the blank cell positions can have values ​​of 0, 1, or random values; this disclosure does not impose any restrictions on these values. Similarly, the other matrix corresponds to the elements at the blank cell positions, where the elements at the blank cell positions have the same values ​​as the elements in the original matrix, and the elements at the shaded cell positions can have values ​​of 0, 1, or random values; this disclosure does not impose any restrictions on these values.

[0106] In this way, each of the two integer feature information pieces (the second integer feature information and the third integer feature information) has an element adjacent to the other feature information, which makes it easier to obtain the relationship between the elements in the two integer feature information pieces and to obtain semantic information.

[0107] In one possible implementation, the second feature information is also in tensor form and can be decomposed in the same way, for example, it can be decomposed in a symmetrical or chessboard manner. Taking chessboard decomposition as an example, the second feature information can also obtain the feature information corresponding to the shaded square positions and the feature information corresponding to the blank square positions (first transcendental feature information and second transcendental feature information). For example, the second integer feature information and the first transcendental feature information are the feature information corresponding to the shaded square positions, and the third integer feature information and the second transcendental feature information are the feature information corresponding to the blank square positions. This disclosure does not limit the feature information corresponding to the shaded square positions and the blank square positions.

[0108] In a possible implementation, the first transcendental feature information and the second transcendental feature information can also be determined in the second feature information according to the correspondence. For example, the first integer feature information is obtained by rounding the first feature information, the first encoded side information is obtained by encoding and rounding the first feature information, and then the second feature information is obtained by the first decoding network. Therefore, the second feature information and the first feature information have a correspondence, and the first integer feature information also has a correspondence. The second integer feature information is the feature information corresponding to the element at a specific position in the first integer feature information. Therefore, the feature information in the second feature information corresponding to the element at the specific position in the first integer feature information (the second integer feature information) can be determined as the first transcendental feature information, and similarly, the feature information in the second feature information corresponding to the element at the specific position in the first integer feature information (the third integer feature information) can be determined as the second transcendental feature information.

[0109] In this way, the first transcendental feature information and the second transcendental feature information can be obtained through the correspondence of the elements in the first integer feature information after the chessboard decomposition, which facilitates obtaining the relationship between the elements of the first transcendental feature information and the second transcendental feature information, and facilitates obtaining semantic information.

[0110] In a possible implementation, the first transcendental feature information and the second transcendental feature information can be processed respectively to obtain a first entropy coding estimation parameter and a second entropy coding estimation parameter. The first entropy coding estimation parameter and the second entropy coding estimation parameter are parameters used to estimate the entropy coding parameter. For example, the first entropy coding estimation parameter is a parameter corresponding to the first transcendental feature information, and the second entropy coding estimation parameter is a parameter corresponding to the second transcendental feature information. That is, the first entropy coding estimation parameter and the second entropy coding estimation parameter are parameters corresponding to the first integer feature information after the chessboard decomposition, which can be combined in the form of a chessboard to obtain the entropy coding parameter.

[0111] In an example, the first entropy coding estimation parameter and the second entropy coding estimation parameter can be directly input into the parameter prediction network after the integer quantization processing. The parameter prediction network can be a deep learning neural network, which is not limited in the present disclosure. The integer quantization processing performed on the parameter prediction network is consistent with the integer quantization processing performed on the first decoding network, which will not be described here. The first entropy coding estimation parameter and the second entropy coding estimation parameter can be parameters in the form of a tensor, which is not limited in the present disclosure.

[0112] In a possible implementation, as described above, the second integer feature information and the third integer feature information respectively have elements adjacent to the other feature information, so as to facilitate obtaining the relationship between the elements of the original feature information. Therefore, the accuracy of the two entropy coding estimation parameters can be improved based on the relationship between the elements. According to the first hyper-feature information, the second hyper-feature information, and the integer-quantized parameter prediction network, the first entropy coding estimation parameter and the second entropy coding estimation parameter are obtained, including: inputting the first hyper-feature information into the integer-quantized parameter prediction network to obtain the first entropy coding estimation parameter; inputting the second integer feature information into the integer-quantized semantic prediction network to obtain the first semantic feature information; and inputting the first semantic feature information and the second hyper-feature information into the integer-quantized parameter prediction network to obtain the second entropy coding estimation parameter.

[0113] In a possible implementation, one of the two hyper-feature information (for example, the first hyper-feature information) can be directly input into the integer-quantized parameter prediction network to obtain the corresponding entropy coding estimation parameter (the first entropy coding estimation parameter). The integer feature information (the second integer feature information) corresponding to the hyper-feature information (the first hyper-feature information) input into the parameter prediction network is input into the integer-quantized semantic prediction network to obtain the first semantic feature information. The first semantic feature information can represent the context feature information of the input integer feature information (the second integer feature information), for example, the feature information of the elements adjacent thereto (the elements in the third integer feature information). Further, the first semantic feature information and the other hyper-feature information (the second hyper-feature information corresponding to the third integer feature information) can be jointly input into the integer-quantized parameter prediction network to obtain the other entropy coding estimation parameter (the second entropy coding estimation parameter). The entropy coding estimation parameter obtained in this way can carry information representing the relationship between the adjacent elements, and the accuracy of the entropy coding estimation parameter can be improved. The disclosure does not limit the hyper-feature information directly input into the parameter prediction network.

[0114] In a possible implementation, the first entropy coding estimation parameter and the second entropy coding estimation parameter are parameters corresponding to the second integer feature information and the third integer feature information respectively. Based on the decomposition manner for decomposing the first integer feature information, the first entropy coding estimation parameter and the second entropy coding estimation parameter are combined, for example, the elements in the shadowed positions of the first entropy coding estimation parameter are combined with the elements in the blank positions of the second entropy coding estimation parameter to obtain the entropy coding parameter. The disclosure does not limit the manner of decomposition and combination.

[0115] In this way, the information about the relationship between adjacent elements can be considered when determining the entropy coding parameter, the first semantic feature information can be obtained, and the accuracy of the entropy coding parameter can be improved.

[0116] In a possible implementation, in step S13, the first encoding result of the first image can be obtained by performing encoding processing, for example, entropy encoding processing, on the entropy coding parameter first encoding side information and the first integer feature information. Step S13 can include: inputting the entropy coding parameter into a probability estimation network after probability quantization processing to obtain a first probability cumulative parameter, wherein the probability quantization processing makes the first probability cumulative parameter a determined parameter; performing entropy encoding processing according to the first probability cumulative parameter and the first integer feature information to obtain an encoded feature; and obtaining the first encoding result according to the encoded feature and the first encoding side information.

[0117] In a possible implementation, the first probability cumulative parameter can be obtained based on the entropy coding parameter, and the first probability cumulative parameter can be information in the form of a probability cumulative function. The form of the first probability cumulative parameter is not limited in the present disclosure. The first probability cumulative parameter is an important parameter for entropy encoding, and can be used to encode in the form of a probability cumulative function without losing information, to obtain encoded information conforming to a specific probability distribution. However, the probability cumulative parameter in the related art is not a determined parameter, and the results obtained in the use of different platforms can be inconsistent, and thus the encoding results obtained based on the parameter can also be inconsistent, resulting in distortion of the image obtained by decoding using the encoding result.

[0118] In a possible implementation, to solve this problem, the probability estimation network can be subjected to probability quantization processing, so that the relationship between the output information and the input information of the probability estimation network is determined, rather than an uncertain relationship based on probability. In an example, a distribution function (for example, a normal distribution function) of the probability distribution can be fitted by a deterministic function (for example, a quadratic function), to obtain a probability estimation network whose input information and output information based on the deterministic function have a determined relationship, and whose curve shape is close to the probability distribution function. The method further includes: performing quadratic fitting on the normal distribution function to obtain fitting coefficients; and obtaining the probability estimation network after the probability quantization processing according to the fitting coefficients.

[0119] In a possible implementation, the normal distribution function can be subjected to quadratic fitting by a quadratic function in the form of formula (3) as follows, to obtain the fitting coefficients:

[0120] p + (n)=ax 2 +bx (3)

[0121] where n is the value of the integer of the argument x (e.g., a quantity in the form of a vector, matrix, or tensor), p + (n) is the function value of the quadratic function of the argument n. a and b are the fitting coefficients.

[0122] In a possible implementation, the probability estimation network after the probability quantization processing can output the first probability accumulation parameter in the form of formula (4) as follows:

[0123] c(n) = Q HALF -sgn(n)p + (|n|) (4)

[0124] where sgn(n) is a function for obtaining the sign of n, Q HALF is a fitting coefficient, c(n) is the first probability accumulation parameter output by the probability estimation network after the probability quantization processing, and is also a value close to the function value of the normal distribution of the argument x.

[0125] In a possible implementation, the entropy coding parameter can be input into the probability estimation network after the probability quantization processing, and the first probability accumulation parameter having a determined correspondence with the entropy coding parameter can be obtained, so that the coding method can be used across platforms, and the possibility of image distortion after decoding is reduced.

[0126] In this way, the probability estimation network can output the determined first probability accumulation parameter through the probability quantization processing, so that the ability of cross-platform use is obtained, and the possibility of image distortion after decoding is reduced. Moreover, the structure of the neural network does not need to be changed, the training difficulty and design difficulty are not increased, and the probability estimation network does not need to be pruned, so that the compression rate of coding is not reduced.

[0127] In a possible implementation, the first probability accumulation parameter and the first integer feature information can be subjected to entropy coding processing to obtain coding features, for example, coding information conforming to the specific probability distribution. Further, the coding result of the first image, for example, a code stream, can be obtained according to the coding features and the first coding side information. For example, the coding features and the first coding information can be stored in the code stream to obtain the coding result of the first image.

[0128] In this way, the probability estimation network after the probability quantization processing can output the determined first probability accumulation parameter, and the coding is performed based on the first probability accumulation function, so that the coding result can have determinacy and a high compression rate, and cross-platform coding and decoding can be performed, and the probability of image distortion after decoding is reduced.

[0129] In a possible implementation, in the decoding process, whether the same platform as the encoding process or cross-platform decoding, the decoding result obtained can be consistent, so that the coding method can be applied cross-platform, expanding the application range. The method further includes: obtaining second encoding side information from the second encoding result to be decoded; performing decoding processing according to the second encoding side information to obtain fourth integer feature information; and performing decoding processing on the fourth integer feature information to obtain a second image.

[0130] In a possible implementation, taking decoding of an arbitrary encoding result (for example, the second encoding result) as an example, the second encoding side information corresponding to the second image to be decoded can be obtained from the code stream of the second encoding result. The second image is the image corresponding to the second encoding result, that is, the second encoding result is the encoding result of the second image.

[0131] In a possible implementation, further, the decoding processing can be performed based on the second encoding side information to obtain fourth integer feature information, that is, the integer feature information corresponding to the second image. And the second image can be obtained by decoding.

[0132] In a possible implementation, the decoding processing according to the second encoding side information to obtain the fourth integer feature information includes: inputting the second encoding side information into the first decoding network after integer fixed-point processing to obtain fourth feature information; obtaining fifth integer feature information according to the fourth feature information, where the fifth integer feature information corresponds to part of the fourth feature information; and obtaining the fourth integer feature information according to the fifth integer feature information and the fourth feature information.

[0133] In this way, the fourth feature information can be obtained based on the first decoding network after integer fixed-point processing, so that the fourth feature information is determined information, reducing the probability of decoding distortion.

[0134] In a possible implementation, the fourth feature information can be obtained by the first decoding network after integer fixed-point processing as described above. That is, the second encoding side information is input into the first decoding network after integer fixed-point processing to obtain the fourth feature information.

[0135] In a possible implementation, the fourth feature information can be decoded. For example, decoding can be performed in a manner opposite to the above encoding process. The fourth feature information is decomposed to obtain third transcendental feature information. The third transcendental feature information is input into the parameter prediction network after integer quantization processing to obtain third entropy coding estimation parameters. The third entropy coding estimation parameters are input into the probability estimation network after probability quantization processing to obtain second probability cumulative parameters. The second encoding result is decoded based on the second probability cumulative parameters to obtain fifth integer feature information.

[0136] In a possible implementation, the fourth feature information can be decomposed, for example, in the same manner as the above chessboard decomposition, to obtain third transcendental feature information and fourth transcendental feature information.

[0137] In a possible implementation, one of the transcendental feature information can be directly decoded. For example, the third transcendental feature information is directly input into the parameter prediction network after integer quantization processing to obtain third entropy coding estimation parameters. The third entropy coding estimation parameters are input into the probability estimation network after probability quantization processing to obtain second probability cumulative parameters corresponding to the third transcendental feature information.

[0138] In a possible implementation, the second encoding result can be decoded based on the second probability cumulative parameters. For example, the second encoding result can be decoded based on the second probability cumulative parameters by using a decoding network to obtain fifth integer feature information corresponding to the third transcendental feature information, which is also part of the entire integer feature information.

[0139] In this way, the fifth integer feature information corresponding to part of the entire integer feature information can be obtained by using the parameter prediction network after integer quantization processing and the probability estimation network after probability quantization processing, thereby improving the certainty of the fifth integer feature information.

[0140] In a possible implementation, the fourth integer feature information can be obtained based on the fifth integer feature information and the fourth feature information in the following manner. The fourth feature information is decomposed to obtain fourth transcendental feature information, which is part of the fourth feature information. Fourth entropy coding estimation parameters are obtained based on the fifth integer feature information and the fourth transcendental feature information. Sixth integer feature information is obtained based on the fourth entropy coding estimation parameters. The fourth integer feature information is obtained based on the fifth integer feature information and the sixth integer feature information.

[0141] In one possible implementation, the sixth integer feature information, i.e., another part of all integer feature information, can be obtained based on the fourth transcendental feature information obtained from the above decomposition, thereby obtaining the complete integer feature information, i.e., the fourth integer feature information. The sixth integer feature information can be obtained in the same way as the fifth integer feature information. In another example, since the third and fourth transcendental feature information carry information about the relationship between elements during encoding, this information can be used to improve the accuracy of decoding.

[0142] In one possible implementation, obtaining the fourth entropy coding estimation parameters based on the fifth integer feature information and the fourth transcendental feature information includes: inputting the fifth integer feature information into a semantic prediction network after integer fixed-point processing to obtain second semantic feature information; and inputting the second semantic feature information and the fourth transcendental feature information into a parameter prediction network after integer fixed-point processing to obtain the fourth entropy coding estimation parameters.

[0143] In this way, second semantic feature information, that is, information representing the relationship between adjacent elements, can be obtained, thereby improving the accuracy of the fourth entropy coding estimation parameters.

[0144] In one possible implementation, the fifth integer feature information can be input into the semantic prediction network described above to obtain the second semantic feature information, i.e., information representing the relationship between adjacent elements. The second semantic feature information and the fourth transcendental feature information can be input into the parameter prediction network after integer fixed-point processing as described above to obtain the fourth entropy coding estimation parameters. Based on the fourth entropy coding estimation parameters, the sixth integer feature information, i.e., another part of all integer feature information, can be obtained in the same way as the fifth integer feature information was obtained.

[0145] In this way, the sixth integer feature information can be obtained based on the information about the relationship between adjacent elements, and then the fourth integer feature information can be obtained, which can improve the accuracy of the fourth integer feature information.

[0146] In one possible implementation, the fifth and sixth integer feature information can be merged based on the decomposition method of the fourth feature information (e.g., chessboard decomposition) to obtain all integer feature information, i.e., the fourth integer feature information.

[0147] In a possible implementation, the fourth integer characteristic information can be decoded, for example, up-sampling can be performed, and the second image can be obtained, that is, the decoding of the second image is completed. In the above decoding process, the information in the decoding process can be obtained through the neural network after the integer fixed-point processing or the probability fixed-point processing, and then the decoded result, that is, the second image, is obtained, so that the decoding process has universality, and the inconsistency of the decoding result in cross-platform application caused by the uncertainty of floating-point calculation or the uncertainty of probability calculation is avoided, thereby the image distortion can be reduced, and the success rate of decoding can be improved.

[0148] In a possible implementation, the neural networks used above, for example, the first encoding network, the first decoding network after the integer fixed-point processing, the probability estimation network after the probability fixed-point processing, the second encoding network, the parameter prediction network after the integer fixed-point processing, the semantic prediction network after the integer fixed-point processing, the first decoding network, etc., can be trained together, and the integer fixed-point processing or the probability fixed-point processing is performed on the neural networks, for example, the first decoding network and the probability estimation network, in the training, so that the trained neural network has the ability to obtain consistent coding and decoding results through the integer fixed-point processing or the probability fixed-point processing.

[0149] In an example, the above encoding process can be performed on a sample image through the neural network, the encoding result is obtained, and the above decoding process can be performed on the encoding result through the neural network, the decoded image is obtained, and then the network loss of the neural network can be determined based on the difference between the decoded image and the sample image, and the network parameters of the neural network are adjusted through back propagation. After a plurality of training processes, when the network loss meets the error requirement, the training is completed, the trained neural network is obtained, and the trained neural network can be used in the encoding and decoding processes.

[0150] According to the image processing method of the embodiments of the present disclosure, the integer fixed-point processing and the probability fixed-point processing are used to make the floating-point number and the probability and other parameters output by the neural network become determined values, so that the obtained parameters can be consistent when the neural network is used for encoding across platforms, thereby the cross-platform encoding capability is obtained. In addition, the integer fixed-point processing and the probability fixed-point processing do not need to change the structure of the neural network, do not need to change the neural network into a full-integer operation network, guarantee the performance of the neural network, and do not increase the training difficulty and the design difficulty, and do not need to delete the probability estimation network, so that the compression rate of the encoding is not reduced. Further, the information of the relationship between adjacent elements can also be considered, and the accuracy of the coding and decoding process is further improved.

[0151] Figure 3A and Figure 3B An application schematic diagram of the image processing method according to the embodiments of the present disclosure is shown. As shown in FIG. 1, the image processing method according to the embodiments of the present disclosure can be applied to a video coding system. Figure 3AAs shown, in the process of encoding the image, the image can be input into the first encoding network ga to obtain the feature information y (first feature information), the y is rounded to obtain the integer feature information Y (first integer feature information), and the feature information y is input into the second encoding network ha to obtain the third feature information, and the third feature information is rounded to obtain the encoding side information Z (first encoding side information).

[0152] In a possible implementation, the encoding side information Z is input into the first decoding network hs after the integer fixed-point processing to obtain the feature information Ψ (second feature information). The integer feature information Y can be chessboard decomposed to obtain the integer feature information Y1 (second integer feature information) and the integer feature information Y2 (third integer feature information), the feature information corresponding to Y1 in the feature information Ψ is the transcendental feature information Ψ1 (first transcendental feature information), and the feature information corresponding to Y2 is the transcendental feature information Ψ2 (second transcendental feature information). The Ψ1 can be input into the parameter prediction network after the integer fixed-point processing to obtain the entropy coding estimation parameter θ1 (first entropy coding estimation parameter), the integer feature information Y1 is input into the semantic prediction network after the integer fixed-point processing to obtain the semantic feature information Φ (first semantic feature information), the semantic feature information Φ and the transcendental feature information Ψ2 (second transcendental feature information) are input into the parameter prediction network after the integer fixed-point processing to obtain the entropy coding estimation parameter θ2 (second entropy coding estimation parameter), and the θ1 and the θ2 are combined, for example, combined in the chessboard manner, to obtain the entropy coding parameter θ.

[0153] In a possible implementation, the entropy coding parameter θ is input into the probability estimation network after the probability fixed-point processing to obtain the first probability cumulative parameter, and then the first probability cumulative parameter and the integer feature information Y can be subjected to entropy coding processing to obtain the encoded feature, and the encoded feature and the first encoding information Z are stored in the code stream to obtain the encoding result.

[0154] In a possible implementation, as shown, Figure 3B In the decoding process, the encoding side information Z (second encoding information) can be obtained from the code stream, input into the first decoding network hs after the integer fixed-point processing to obtain the feature information Ψ (fourth feature information), and the feature information Ψ can be decomposed by the chessboard decomposition manner to obtain the transcendental feature information Ψ1 (third transcendental feature information) and Ψ2 (fourth transcendental feature information). The Ψ1 can be input into the parameter prediction network after the integer fixed-point processing to obtain the entropy coding estimation parameter θ1 (third entropy coding estimation parameter), the θ1 is input into the probability estimation network after the probability fixed-point processing to obtain the second probability cumulative parameter. The encoding result is decoded based on the second probability cumulative parameter through the decoding network, and the integer feature information Y1 (fifth integer feature information) can be obtained.

[0155] In a possible implementation, the integer feature information Y1 can be input into the integer fixed-point processed semantic prediction network to obtain semantic feature information Φ (second semantic feature information), and Φ and the transcendental feature information Ψ2 can be input into the integer fixed-point processed parameter prediction network to obtain an entropy coding estimation parameter θ2 (fourth entropy coding estimation parameter). In the same way, the integer feature information Y2 (sixth integer feature information) can be obtained.

[0156] In a possible implementation, the integer feature information Y2 and Y1 can be merged in a chessboard manner to obtain integer feature information Y (fourth integer feature information), which is input into the decoding network gs, and the image can be decoded to obtain the image, that is, the decoding is successful.

[0157] In a possible implementation, the image processing method can be used in scenarios such as data centers, cloud storage, streaming on-demand services, and the like, in which a large number of images or videos need to be transmitted between computing platforms of different architectures and successfully decoded on the heterogeneous platforms after transmission. The image processing method of the present disclosure can enable the deep learning-based coding and decoding method to be used across platforms, allow users to access any computing platform in the data center, and also allow users to directly distribute the coded visual data code stream to downstream mobile devices for decoding and reading, and has high coding and decoding performance. The application field of the image processing method of the present disclosure is not limited.

[0158] It can be understood that the above-mentioned various method embodiments mentioned in the present disclosure can be combined with each other to form combined embodiments without violating the principle logic. Limited by the length of the present disclosure, the above-mentioned method embodiments in the specific embodiments will not be described again. Those skilled in the art can understand that the specific execution order of each step in the above-mentioned method should be determined according to its function and possible internal logic.

[0159] In addition, the present disclosure also provides an image processing apparatus, an electronic device, a computer readable storage medium, and a program, all of which can be used to implement any one of the image processing methods provided by the present disclosure. The corresponding technical solutions and descriptions are described in the method part and will not be described again.

[0160] Figure 4 A block diagram of an image processing apparatus according to an embodiment of the present disclosure is shown as follows, Figure 4As shown, the apparatus comprises: a first information acquisition module 11, acquiring first integer feature information and first encoding side information of a first image to be encoded; an entropy coding parameter acquisition module 12, used for obtaining an entropy coding parameter according to the first integer feature information and the first encoding side information, the entropy coding parameter being encoding information obtained by a neural network after integer fixed-point processing, the integer fixed-point processing being used for making the output information of the neural network be integer information; and an encoding module 13, used for performing encoding processing according to the entropy coding parameter, the first encoding side information and the first integer feature information, to obtain a first encoding result of the first image.

[0161] In a possible implementation, the first information acquisition module is further used for: inputting the first image into a first encoding network to acquire first feature information; inputting the first feature information into a second encoding network to obtain third feature information; performing integer processing on the third feature information to obtain the first encoding side information; and performing integer processing on the first feature information to obtain the first integer feature information.

[0162] In a possible implementation, the entropy coding parameter acquisition module is further used for: inputting the first encoding side information into a first decoding network after integer fixed-point processing to obtain second feature information; and obtaining the entropy coding parameter according to the second feature information and the first integer feature information.

[0163] In a possible implementation, the entropy coding parameter acquisition module is further used for: the obtaining of the entropy coding parameter according to the second feature information and the first integer feature information comprises: obtaining first transcendental feature information and second transcendental feature information according to the second feature information and the first integer feature information, the first transcendental feature information and the second transcendental feature information respectively corresponding to part of the first integer feature information; obtaining first entropy coding estimation parameter and second entropy coding estimation parameter according to the first transcendental feature information, the second transcendental feature information and a parameter prediction network after the integer fixed-point processing; and obtaining the entropy coding parameter according to the first entropy coding estimation parameter and the second entropy coding estimation parameter.

[0164] In a possible implementation, the entropy coding parameter acquisition module is further used for: decomposing the first integer feature information to obtain second integer feature information and third integer feature information; obtaining the first transcendental feature information according to the second integer feature information and the second feature information; and obtaining the second transcendental feature information according to the third integer feature information and the second feature information.

[0165] In a possible implementation, the entropy coding parameter obtaining module is further configured to: perform chessboard decomposition on the first integer feature information to obtain the second integer feature information and the third integer feature information.

[0166] In a possible implementation, the entropy coding parameter obtaining module is further configured to: input the first transcendental feature information into the parameter prediction network after integer quantization processing to obtain first entropy coding estimation parameters; input the second integer feature information into the semantic prediction network after integer quantization processing to obtain first semantic feature information; and input the first semantic feature information and the second transcendental feature information into the parameter prediction network after integer quantization processing to obtain second entropy coding estimation parameters.

[0167] In a possible implementation, the encoding module is further configured to: input the entropy coding parameters into the probability estimation network after probability quantization processing to obtain first probability accumulation parameters, where the probability quantization processing makes the first probability accumulation parameters deterministic parameters; perform entropy coding processing according to the first probability accumulation parameters and the first integer feature information to obtain encoded features; and obtain the first encoding result according to the encoded features and the first encoding side information.

[0168] In a possible implementation, the apparatus further includes a decoding module configured to: obtain second encoding side information from a second encoding result to be decoded; perform decoding processing according to the second encoding side information to obtain fourth integer feature information; and perform decoding processing on the fourth integer feature information to obtain a second image.

[0169] In a possible implementation, the decoding module is further configured to: input the second encoding side information into the first decoding network after integer quantization processing to obtain fourth feature information; obtain fifth integer feature information according to the fourth feature information, where the fifth integer feature information corresponds to part of the fourth feature information; and obtain the fourth integer feature information according to the fifth integer feature information and the fourth feature information.

[0170] In a possible implementation, the decoding module is further configured to: perform decomposition on the fourth feature information to obtain third transcendental feature information, where the third transcendental feature information is part of the fourth feature information; input the third transcendental feature information into the parameter prediction network after integer quantization processing to obtain third entropy coding estimation parameters; input the third entropy coding estimation parameters into the probability estimation network after probability quantization processing to obtain second probability accumulation parameters; and perform decoding processing on the second encoding result according to the second probability accumulation parameters to obtain fifth integer feature information.

[0171] In a possible implementation, the decoding module is further configured to: decompose the fourth feature information to obtain fourth transcendental feature information, where the fourth transcendental feature information is part of the fourth feature information; obtain fourth entropy coding estimation parameters according to the fifth integer feature information and the fourth transcendental feature information; obtain sixth integer feature information according to the fourth entropy coding estimation parameters; and obtain the fourth integer feature information according to the fifth integer feature information and the sixth integer feature information.

[0172] In a possible implementation, the decoding module is further configured to: input the fifth integer feature information into the semantic prediction network after integer quantization processing to obtain second semantic feature information; and input the second semantic feature information and the fourth transcendental feature information into the parameter prediction network after integer quantization processing to obtain the fourth entropy coding estimation parameters.

[0173] In a possible implementation, the apparatus further includes an integer quantization module configured to perform rounding processing on the convolution kernel parameters of the multiple network levels of the first decoding network according to the maximum value and the minimum value of the convolution kernel parameters, to obtain the convolution kernel parameters of the multiple network levels after rounding processing; and the entropy coding parameter obtaining module is further configured to: determine the multiple network levels of the first decoding network respectively, and obtain input information of the multiple network levels according to the maximum value and the minimum value in the operation result of the preset calibration data and the output information of the multiple network levels of the first decoding network after integer quantization processing, where the output information of the multiple network levels includes processing results of the multiple network levels of the first decoding network after integer quantization processing on the first encoded side information; and obtain the second feature information according to the input information of the network levels and the convolution kernel parameters of the multiple network levels after rounding processing.

[0174] In a possible implementation, the apparatus further includes a probability quantization module configured to perform quadratic fitting on a normal distribution function to obtain fitting coefficients, and obtain the probability estimation network after probability quantization processing according to the fitting coefficients.

[0175] In some embodiments, the apparatus provided by the embodiments of the present disclosure has functions or includes modules that can be used to execute the methods described in the above method embodiments, and the specific implementation can be referred to the description of the above method embodiments. For briefness, details are not described herein.

[0176] The embodiment of the present disclosure further provides a computer readable storage medium, having stored thereon computer program instructions, the computer program instructions being executable by a processor to implement the method described above. The computer readable storage medium can be a non-volatile computer readable storage medium.

[0177] The embodiment of the present disclosure further provides an electronic device, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to invoke the instructions stored in the memory to execute the method described above.

[0178] The embodiment of the present disclosure further provides a computer program product, comprising computer readable code, when the computer readable code is run on a device, a processor in the device executes instructions for implementing the image processing method provided by any of the above embodiments.

[0179] The embodiment of the present disclosure further provides another computer program product for storing computer readable instructions, the instructions being executed to cause a computer to perform the operations of the image processing method provided by any of the above embodiments.

[0180] The electronic device can be provided as a terminal, a server or other forms of devices.

[0181] Figure 5 A block diagram of an electronic device 800 according to an embodiment of the present disclosure is shown. The electronic device 800 can be, for example, a terminal such as a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, and the like.

[0182] Referring to Figure 5 The electronic device 800 can include one or more of the following components: a processing component 802, a memory 804, a power supply component 806, a multimedia component 808, an audio component 810, an input / output (I / O) interface 812, a sensor component 814, and a communication component 816.

[0183] The processing component 802 usually controls overall operations of the electronic device 800, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 802 can include one or more processors 820 to execute instructions to complete all or part of steps of the methods described above. In addition, the processing component 802 can include one or more modules to facilitate the interaction between the processing component 802 and other components. For example, the processing component 802 can include a multimedia module to facilitate the interaction between the multimedia component 808 and the processing component 802.

[0184] The memory 804 is configured to store various types of data to support the operation of the electronic device 800. Examples of such data include instructions for any application or method operating on the electronic device 800, contact data, phonebook data, messages, pictures, videos, etc. The memory 804 can be implemented by any type of volatile or nonvolatile memory, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disc, or optical disc.

[0185] The power supply component 806 supplies power for various components of the electronic device 800. The power supply component 806 can include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the electronic device 800.

[0186] The multimedia component 808 includes a screen providing an output interface between the electronic device 800 and a user. In some embodiments, the screen can include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense a touch, a slide, and a gesture on the touch panel. The touch sensor can not only sense a position of a touch or a slide, but also detect a duration and a pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front camera and / or a back camera. The front camera and / or the back camera can receive external multimedia data when the electronic device 800 is in an operation mode, such as a photographing mode or a video mode. Each of the front camera and the back camera can be a fixed optical lens system or have a focal length and optical zoom capability.

[0187] The audio component 810 is configured to output and / or input an audio signal. For example, the audio component 810 includes a microphone (MIC) configured to receive an external audio signal when the electronic device 800 is in an operation mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal can be further stored in the memory 804 or transmitted via the communication component 816. In some embodiments, the audio component 810 also includes a speaker for outputting an audio signal.

[0188] The I / O interface 812 provides an interface between the processing component 802 and peripheral interface modules, which can be a keypad, a click wheel, buttons, etc. The buttons can include, but are not limited to, a home button, a volume button, a start button, and a lock button.

[0189] The sensor component 814 includes one or more sensors for providing status assessments for various aspects of the electronic device 800. For example, the sensor component 814 can detect an open / closed position of the electronic device 800, relative positioning of components of the electronic device 800, such as a display and a keypad of the electronic device 800, a change in position of the electronic device 800 or a component of the electronic device 800, presence or absence of user contact with the electronic device 800, orientation or acceleration / deceleration / g-force and temperature changes of the electronic device 800. The sensor component 814 can include an optical sensor for detecting ambient light, a proximity sensor configured to detect proximity of an object, a motion sensor, a temperature sensor, a magnetic sensor, an acceleration sensor, a gyroscope sensor, or a pressure sensor.

[0190] The communication component 816 is configured to facilitate wired or wireless communication between the electronic device 800 and other devices. The electronic device 800 can access a wireless network based on a corresponding communication standard, such as WiFi, 2G, or 3G, or a combination thereof. In an example embodiment, the communication component 816 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In an example embodiment, the communication component 816 further includes a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on Radio Frequency Identification (RFID) techniques, infrared data association (IrDA) techniques, ultra-wideband (UWB) techniques, Bluetooth (BT) techniques, and other techniques.

[0191] In an example embodiment, the electronic device 800 can be implemented using one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, micro-controllers, microprocessors, or other electronic elements, to perform the above-described methods.

[0192] In an example embodiment, a non-transitory computer-readable storage medium, such as the memory 804 including computer program instructions, is also provided, which can be executed by the processor 820 of the electronic device 800 to complete the above-described methods.

[0193] Figure 6 A block diagram of an electronic device 1900 according to an embodiment of the present disclosure is shown. The electronic device 1900 can be provided as a server, for example. Referring to FIG. 19, the electronic device 1900 includes one or more processors 1910, memory 1920, and at least one communication interface 1940. The processor 1910 can include a central processing unit (CPU), a microprocessor, or other hardware components that interpret and execute instructions. The memory 1920 can include a random access memory (RAM), a read-only memory (ROM), or another type of dynamic or static storage device (e.g., a flash memory or a magnetic tape). The memory 1920 can store data and / or instructions for the processor 1910. The communication interface 1940 can include a wired interface or a wireless interface configured to facilitate communication between the electronic device 1900 and other devices. Figure 6The electronic device 1900 includes a processing component 1922, which further includes one or more processors, and a memory resource represented by the memory 1932 for storing instructions, such as an application program, executable by the processing component 1922. The application program stored in the memory 1932 can include one or more than one module each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute the instructions to perform the above method.

[0194] The electronic device 1900 can further include a power supply component 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and an input / output (I / O) interface 1958. The electronic device 1900 can operate based on an operating system stored in the memory 1932, such as Windows Server TM , Mac OS X TM , Unix TM , Linux TM , FreeBSD TM or the like.

[0195] In an exemplary embodiment, a non-transitory computer readable storage medium, such as the memory 1932 including computer program instructions, is also provided, which can be executed by the processing component 1922 of the electronic device 1900 to complete the above method.

[0196] The present disclosure can be a system, a method, and / or a computer program product. The computer program product can include a computer readable storage medium (or media) having computer readable program instructions thereon for causing a processor to carry out aspects of the present disclosure.

[0197] Computer readable storage media can be tangible storage media which can retain and store instructions for use by an instruction execution device. Computer readable storage media can be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of computer readable storage media include the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as punch-cards or raised structures in a groove having instructions recorded thereon, and any suitable combination of the foregoing. A computer readable storage medium, as used herein, is not to be construed as being transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission media (e.g., light pulses passing through a fiber-optic cable), or electrical signals transmitted through a wire.

[0198] Computer readable program instructions described herein can be downloaded to respective computing / processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and / or a wireless network. The network can comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and / or edge servers. A network adapter card or network interface in each computing / processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium within the respective computing / processing device.

[0199] Computer readable program instructions for carrying out operations of the present disclosure can be assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state-setting data, or either source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer readable program instructions can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate array (FPGA), or programmable logic array (PLA) can execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present disclosure.

[0200] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0201] These computer readable program instructions can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks. These computer readable program instructions can also be stored in a computer readable storage medium that can include random access memory (RAM), read only memory (ROM), electrically erasable programmable read only memory (EEPROM), flash memory or other data storage device. When the computer readable program instructions are loaded into the computer and other programmable data processing apparatus, a series of operational steps are implemented that provide processes such that the instructions which operate on the computer or other programmable data processing apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0202] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus or other device to produce a computer implemented process such that the instructions which execute on the computer, other programmable data processing apparatus, or other device implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0203] The flow diagrams and the block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flow diagrams and the block diagrams can represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logic functions. In some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks can sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and / or flow diagrams, and combinations thereof, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and

[0204] The computer program product can be embodied by a hardware, software or a combination thereof. In an optional embodiment, the computer program product is embodied by a computer storage medium. In another optional embodiment, the computer program product is embodied by a software product, such as a software development kit (SDK) or the like.

[0205] The above description has described various embodiments of the present disclosure. The above description is exemplary and not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes are obvious to those skilled in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles, practical application or improvement of technology in the market of the embodiments, or to enable other ordinary skilled in the art to understand the embodiments disclosed herein.

Claims

1. An image processing method, characterized by, The method comprises: obtaining first integer feature information and first encoding side information of a first image to be encoded; obtaining an entropy encoding parameter according to the first integer feature information and the first encoding side information, the entropy encoding parameter being encoding information obtained by a neural network after integer fixed-point processing, the integer fixed-point processing being used to make the output information of the neural network be integer information; performing encoding processing according to the entropy encoding parameter, the first encoding side information and the first integer feature information to obtain a first encoding result of the first image; wherein the encoding processing according to the entropy encoding parameter, the first encoding side information and the first integer feature information to obtain the first encoding result of the first image comprises: inputting the entropy encoding parameter into a probability estimation network after probability fixed-point processing to obtain a first probability cumulative parameter, wherein the probability fixed-point processing makes the first probability cumulative parameter be a determined parameter; performing entropy encoding processing according to the first probability cumulative parameter and the first integer feature information to obtain an encoding feature; obtaining the first encoding result according to the encoding feature and the first encoding side information.

2. The method of claim 1, wherein, The method comprises: inputting the first image into a first encoding network to obtain first feature information; inputting the first feature information into a second encoding network to obtain third feature information; performing integer processing on the third feature information to obtain the first encoding side information; performing integer processing on the first feature information to obtain the first integer feature information.

3. The method according to claim 1 or 2, characterized in that, The method comprises: inputting the first encoding side information into a first decoding network after integer fixed-point processing to obtain second feature information; obtaining the entropy encoding parameter according to the second feature information and the first integer feature information.

4. The method of claim 3, wherein, The method comprises: obtaining first transcendental feature information and second transcendental feature information according to the second feature information and the first integer feature information, the first transcendental feature information and the second transcendental feature information respectively corresponding to part of the first integer feature information; obtaining first entropy encoding estimation parameter and second entropy encoding estimation parameter according to the first transcendental feature information, the second transcendental feature information and a parameter prediction network after the integer fixed-point processing; obtaining the entropy encoding parameter according to the first entropy encoding estimation parameter and the second entropy encoding estimation parameter.

5. The method of claim 4, wherein, The method comprises: decomposing the first integer feature information to obtain second integer feature information and third integer feature information; obtaining the first transcendental feature information according to the second integer feature information and the second feature information; obtaining the second transcendental feature information according to the third integer feature information and the second feature information.

6. The method of claim 5, wherein, The first integer characteristic information is decomposed to obtain second integer characteristic information and third integer characteristic information. The first integer characteristic information is decomposed to obtain second integer characteristic information and third integer characteristic information.

7. The method according to claim 5 or 6, characterized in that, According to the first transcendental characteristic information, the second transcendental characteristic information and the parameter prediction network after the integer fixed-point processing, first entropy coding estimation parameters and second entropy coding estimation parameters are obtained, including: The first transcendental characteristic information is input into the parameter prediction network after the integer fixed-point processing to obtain first entropy coding estimation parameters; The second integer characteristic information is input into the semantic prediction network after the integer fixed-point processing to obtain first semantic characteristic information; The first semantic characteristic information and the second transcendental characteristic information are input into the parameter prediction network after the integer fixed-point processing to obtain second entropy coding estimation parameters.

8. The method of claim 1 or 2, wherein, The method further includes: Second encoding side information is obtained from the second encoding result to be decoded; According to the second encoding side information, fourth integer characteristic information is obtained through decoding processing; The fourth integer characteristic information is decoded to obtain a second image.

9. The method of claim 8, wherein, According to the second encoding side information, fourth integer characteristic information is obtained through decoding processing, including: The second encoding side information is input into the first decoding network after the integer fixed-point processing to obtain fourth characteristic information; According to the fourth characteristic information, fifth integer characteristic information is obtained, wherein the fifth integer characteristic information corresponds to part of the fourth characteristic information; According to the fifth integer characteristic information and the fourth characteristic information, the fourth integer characteristic information is obtained.

10. The method of claim 9, wherein, According to the fourth characteristic information, fifth integer characteristic information is obtained, including: The fourth characteristic information is decomposed to obtain third transcendental characteristic information, wherein the third transcendental characteristic information is part of the fourth characteristic information; The third transcendental characteristic information is input into the parameter prediction network after the integer fixed-point processing to obtain third entropy coding estimation parameters; The third entropy coding estimation parameters are input into the probability estimation network after the probability fixed-point processing to obtain second probability cumulative parameters; According to the second probability cumulative parameters, the fifth integer characteristic information is obtained through decoding processing of the second encoding result.

11. The method according to claim 9 or 10, characterized in that, According to the fourth characteristic information, fifth integer characteristic information is obtained, including: The fourth characteristic information is decomposed to obtain fourth transcendental characteristic information, wherein the fourth transcendental characteristic information is part of the fourth characteristic information; According to the fifth integer characteristic information and the fourth transcendental characteristic information, fourth entropy coding estimation parameters are obtained; According to the fourth entropy coding estimation parameters, sixth integer characteristic information is obtained; According to the fifth integer characteristic information and the sixth integer characteristic information, the fourth integer characteristic information is obtained.

12. The method of claim 11, wherein, According to the fourth characteristic information, fifth integer characteristic information is obtained, including: The fifth integer characteristic information is input into the semantic prediction network after the integer fixed-point processing to obtain second semantic characteristic information; The second semantic feature information and the fourth transcendental feature information are input into the parameter prediction network after the integer quantization processing to obtain the fourth entropy coding estimation parameter.

13. The method of claim 3, wherein, The method further comprises: performing rounding processing on the convolution kernel parameters of the plurality of network levels of the first decoding network according to the maximum value and the minimum value of the convolution kernel parameters, to obtain the convolution kernel parameters of the plurality of network levels after the rounding processing; determining the plurality of network levels of the first decoding network respectively, and obtaining the maximum value and the minimum value in the operation result of the preset calibration data, wherein the first encoding side information is input into the first decoding network after the integer quantization processing to obtain second feature information, comprising: obtaining the input information of the plurality of network levels according to the output information of the plurality of network levels of the first decoding network after the integer quantization processing and the maximum value and the minimum value in the operation result, wherein the output information of the plurality of network levels comprises the processing result of the plurality of network levels of the first decoding network after the integer quantization processing on the first encoding side information; obtaining the second feature information according to the input information of the network level and the convolution kernel parameters of the plurality of network levels after the rounding processing.

14. The method of claim 1, wherein, The method further comprises: performing quadratic fitting on the normal distribution function to obtain a fitting coefficient; obtaining the probability estimation network after the probability quantization processing according to the fitting coefficient.

15. An image processing apparatus characterized by comprising: comprising: a first information acquisition module, configured to acquire first integer feature information and first encoding side information of a first image to be encoded; an entropy coding parameter acquisition module, configured to obtain an entropy coding parameter according to the first integer feature information and the first encoding side information, wherein the entropy coding parameter is encoding information obtained by a neural network after integer quantization processing, and the integer quantization processing is used to make the output information of the neural network be integer information; an encoding module, configured to perform encoding processing according to the entropy coding parameter, the first encoding side information and the first integer feature information to obtain a first encoding result of the first image; wherein the encoding module is further configured to: input the entropy coding parameter into a probability estimation network after probability quantization processing to obtain first probability cumulative parameters, wherein the probability quantization processing makes the first probability cumulative parameters be determined parameters; perform entropy coding processing according to the first probability cumulative parameters and the first integer feature information to obtain encoding features; and obtain the first encoding result according to the encoding features and the first encoding side information.

16. An electronic device, comprising: comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to invoke the instructions stored in the memory to execute the method in any one of claims 1 to 14.

17. A computer-readable storage medium having stored thereon computer program instructions, wherein, The computer program instructions are executed by the processor to implement the method in any one of claims 1 to 14. The computer program instructions are executed by the processor to implement the method in any one of claims 1 to 14.

Images