Image Processing Method, Apparatus, Electronic Device, and Storage Medium

Abstract

Embodiments of the present disclosure disclose an image processing method, apparatus, electronic device, and storage medium. The method includes: obtaining processed features in an image to be processed; determining similarity information of currently to-be-processed features in the image to be processed according to the processed features, where the similarity information includes similar features; and determining reference information of processed features when processing the currently to-be-processed features based on the similarity information. This technical solution can determine encoding and decoding reference information based on the global information of the currently to-be-processed features in the image features, so that the encoding process of the currently to-be-processed features can refer to the global information in the image features, rather than being limited to the local information of the currently to-be-processed features. For images with more repetitive textures, it can reduce the redundancy of global information in the encoded image and improve the compression ratio of the image.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technologies, and particularly to an image processing method, apparatus, electronic device, and storage medium. Background Art

[0002] In traditional image compression methods and deep learning-based image compression methods, when compressing a to-be-compressed image block or a to-be-compressed image feature, local information around the target image block or target feature is usually utilized. When there are relatively repetitive textures at different positions in an image, theoretically, better compression effects can be achieved for the repetitive textures. However, the inventors of the present disclosure have found that since the local-information-based image compression method only involves the local information of the to-be-compressed image block or feature and does not consider the repetitive textures outside the local information, there is a situation of repetitive compression of the repetitive textures. Therefore, how to reduce the repetitive compression of repetitive textures is one of the technical problems currently to be solved in this field. Summary of the Invention

[0003] Embodiments of the present disclosure provide an image processing method, apparatus, electronic device, and computer-readable storage medium.

[0004] In a first aspect, an image processing method provided in embodiments of the present disclosure includes:

[0005] Obtaining processed features in an image to be processed;

[0006] Determining similarity information of a currently to-be-processed feature in the image to be processed according to the processed features; wherein the similarity information includes similar features;

[0007] Determining reference information for processing the currently to-be-processed feature based on the similarity information.

[0008] Further, in the encoding process of the currently to-be-processed feature, the processed features include encoded features in the image to be processed; in the decoding process of the currently to-be-processed feature, the processed features include decoded features in the image to be processed.

[0009] Further, the method further includes:

[0010] Obtaining the image to be processed;

[0011] Compressing the image to be processed by using an encoding network in a first autoencoder model to obtain compressed features;

[0012] Quantizing the compressed features to obtain image features corresponding to the image to be processed.

[0013] Further, the similarity information further includes the similarity between the currently to-be-processed feature and the similar feature.

[0014] Further, determining the similarity information of the currently to-be-processed feature in the to-be-processed image according to the processed features includes:

[0015] Searching for the similar feature from the processed features by using the local information of the currently to-be-processed feature;

[0016] Determining the similarity between the currently to-be-processed feature and the similar feature.

[0017] Further, based on the similarity information, determining the reference information when processing the currently to-be-processed feature includes:

[0018] Obtaining first auxiliary information for determining the reference information according to the similar feature and the local information of the similar feature;

[0019] Determining the reference information after performing a weighting operation on the first auxiliary information by using the similarity.

[0020] Further, the method further includes:

[0021] Obtaining second auxiliary information for determining the reference information according to the local information of the currently to-be-processed feature;

[0022] Obtaining a first probability estimate of the currently to-be-processed feature in the image features corresponding to the to-be-processed image according to the second auxiliary information;

[0023] Obtaining the confidence of the currently to-be-processed feature according to the first probability estimate.

[0024] Further, determining the reference information after performing a weighting operation on the first auxiliary information by using the similarity includes:

[0025] Combining the first auxiliary information and the second auxiliary information to obtain a first combination result;

[0026] Filtering the first combination result by using the similarity and the confidence to obtain a filtering result;

[0027] Obtaining a second probability estimate of the currently to-be-processed feature according to the filtering result and the first probability estimate.

[0028] Further, the method further includes:

[0029] Obtaining third auxiliary information for determining the reference information from the image features corresponding to the to-be-processed image by using a second autoencoder model;

[0030] Combine the first auxiliary information, the second auxiliary information, and the third auxiliary information to obtain a second combination result;

[0031] Use the second combination result and the second probability estimate to obtain a third probability estimate of the currently to-be-processed feature.

[0032] Further, the method further includes:

[0033] Obtain a second encoded bitstream corresponding to the to-be-processed image;

[0034] Use a second autoencoder model to obtain third auxiliary information for determining the reference information from the second encoded bitstream;

[0035] Combine the first auxiliary information, the second auxiliary information, and the third auxiliary information to obtain a second combination result;

[0036] Use the second combination result and the second probability estimate to obtain a third probability estimate of the currently to-be-processed feature.

[0037] Further, after determining the reference information by weighting the first auxiliary information using the similarity, it further includes:

[0038] Determine the reference information according to the third probability estimate.

[0039] In a second aspect, an image encoding method provided in an embodiment of the present disclosure includes:

[0040] Obtain an image to be encoded;

[0041] Use the method described in the first aspect to obtain reference information for encoding each feature in the image to be encoded;

[0042] Encode the image to be encoded using the reference information.

[0043] In a third aspect, an image decoding method provided in an embodiment of the present disclosure includes:

[0044] Obtain an encoded bitstream of an image to be decoded;

[0045] Use the method described in the first aspect to obtain reference information for decoding each feature in the image to be decoded;

[0046] Decode the encoded bitstream using the reference information.

[0047] In a fourth aspect, an image processing method provided in an embodiment of the present disclosure includes:

[0048] Obtain encoded features in an image to be encoded;

[0049] Invoke a preset service interface so that the preset service interface determines similarity information of the currently to-be-processed feature in the to-be-encoded image according to the encoded feature, and determines reference information for encoding the currently to-be-processed feature based on the similarity information; wherein, the similarity information includes similar features;

[0050] Output the reference information.

[0051] In a fifth aspect, an image processing method provided in an embodiment of the present disclosure includes:

[0052] Obtain the decoded feature in the to-be-decoded image;

[0053] Invoke a preset service interface so that the preset service interface determines similarity information of the currently to-be-processed feature in the to-be-decoded image according to the decoded feature, and determines reference information for decoding the currently to-be-processed feature based on the similarity information; wherein, the similarity information includes similar features;

[0054] Output the reference information.

[0055] In a sixth aspect, an image processing method provided in an embodiment of the present disclosure includes:

[0056] Obtain the to-be-encoded image;

[0057] Invoke a preset service interface so that the preset service interface determines similarity information of the currently to-be-processed feature in the to-be-encoded image according to the encoded feature in the to-be-encoded image, and determines reference information for encoding the currently to-be-processed feature based on the similarity information, and encodes the currently to-be-processed feature by using the reference information; wherein, the similarity information includes similar features;

[0058] Output the encoding result.

[0059] In a seventh aspect, an image processing method provided in an embodiment of the present disclosure includes:

[0060] Obtain the encoded bitstream of the to-be-decoded image;

[0061] Invoke a preset service interface so that the preset service interface determines similarity information of the currently to-be-processed feature in the to-be-decoded image according to the decoded feature obtained from the encoded bitstream, and determines reference information for decoding the currently to-be-processed feature based on the similarity information, and decodes the currently to-be-processed feature based on the reference information; the similarity information includes similar features;

[0062] Output the decoding result.

[0063] In an eighth aspect, an image processing apparatus provided in an embodiment of the present disclosure includes:

[0064] A feature acquisition module configured to acquire processed features in an image to be processed;

[0065] A first determination module configured to determine similarity information of a currently to-be-processed feature according to the processed features; wherein the similarity information includes similar features;

[0066] A second determination module configured to determine reference information for processing the currently to-be-processed feature based on the similarity information.

[0067] In a ninth aspect, an image encoding apparatus provided in an embodiment of the present disclosure includes:

[0068] A tenth acquisition module configured to acquire an image to be encoded;

[0069] An eleventh acquisition module configured to acquire reference information for encoding each feature in the image to be encoded by using the apparatus described in the sixth aspect;

[0070] An encoding module configured to encode the image to be encoded by using the reference information.

[0071] In a tenth aspect, an image decoding apparatus provided in an embodiment of the present disclosure includes:

[0072] A twelfth acquisition module configured to acquire an encoded bitstream of an image to be decoded;

[0073] A thirteenth acquisition module configured to acquire reference information for decoding each feature in the image to be decoded by using the apparatus described in the sixth aspect;

[0074] A decoding module configured to decode the encoded bitstream by using the reference information.

[0075] In an eleventh aspect, an image processing apparatus provided in an embodiment of the present disclosure includes:

[0076] A fourteenth acquisition module configured to acquire encoded features in an image to be encoded;

[0077] A first call module configured to call a preset service interface, so that the preset service interface determines similarity information of a currently to-be-processed feature in the image to be encoded according to the encoded features, and determines reference information for encoding the currently to-be-processed feature based on the similarity information; wherein the similarity information includes similar features;

[0078] A first output module configured to output the reference information.

[0079] In a twelfth aspect, an image processing apparatus provided in an embodiment of the present disclosure includes:

[0080] A fifteenth acquisition module, configured to acquire decoded features in an image to be decoded;

[0081] A second call module, configured to call a preset service interface, so that the preset service interface determines similarity information of a currently to-be-processed feature in the image to be decoded according to the decoded features, and determines reference information for decoding the currently to-be-processed feature based on the similarity information; wherein the similarity information includes similar features;

[0082] A second output module, configured to output the reference information.

[0083] In a thirteenth aspect, an image processing apparatus provided in an embodiment of the present disclosure includes:

[0084] A sixteenth acquisition module, configured to acquire an image to be encoded;

[0085] A third call module, configured to call a preset service interface, so that the preset service interface determines similarity information of a currently to-be-processed feature in the image to be encoded according to the encoded features in the image to be encoded, and determines reference information for encoding the currently to-be-processed feature based on the similarity information, and encodes the currently to-be-processed feature by using the reference information; wherein the similarity information includes similar features;

[0086] A third output module, configured to output an encoding result.

[0087] In a fourteenth aspect, an image processing apparatus provided in an embodiment of the present disclosure includes:

[0088] A seventeenth acquisition module, configured to acquire an encoded bitstream of an image to be decoded;

[0089] A fourth call module, configured to call a preset service interface, so that the preset service interface determines similarity information of a currently to-be-processed feature in the image to be decoded according to the decoded features obtained from the encoded bitstream, and determines reference information for decoding the currently to-be-processed feature based on the similarity information, and decodes the currently to-be-processed feature based on the reference information; the similarity information includes similar features;

[0090] A fourth output module, configured to output a decoding result.

[0091] The functions may be implemented by hardware, or may be implemented by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions.

[0092] In a possible design, the structure of the above device includes a memory and a processor. The memory is used to store one or more computer instructions that support the above device to execute the corresponding method, and the processor is configured to execute the computer instructions stored in the memory. The above device may further include a communication interface for the device to communicate with other devices or communication networks.

[0093] In a fifteenth aspect, an embodiment of the present disclosure provides an electronic device, including a memory and a processor; wherein, the memory is used to store one or more computer instructions, and the one or more computer instructions are executed by the processor to implement the method described in any of the above aspects.

[0094] In a sixteenth aspect, an embodiment of the present disclosure provides a computer-readable storage medium for storing the computer instructions used by any of the above devices, which includes the computer instructions involved in executing the method described in any of the above aspects.

[0095] The technical solutions provided by the embodiments of the present disclosure may include the following beneficial effects:

[0096] In the process of determining the reference information in the embodiments of the present disclosure, by determining the similarity information of the current feature to be processed from the decoded features or the encoded features, the similarity information includes the similar features of the current feature to be processed, and then determining the reference information of the current feature to be processed according to the similarity information. In this way, the reference information can be determined based on the global information of the current feature to be processed in the image features, so that the encoding process of the current feature to be processed can refer to the global information in the image features, rather than being limited to the local information of the current feature to be processed. For images with a large number of repeated textures, it is possible to reduce the redundancy of the global information in the encoded image and improve the compression ratio of the image.

[0097] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0098] In combination with the drawings, through the following detailed description of non-limiting embodiments, other features, objectives, and advantages of the present disclosure will become more obvious. In the drawings:

[0099] Figure 1 A flowchart showing an image processing method according to an embodiment of the present disclosure;

[0100] Figure 2 A schematic diagram showing the search and matching process of similar features according to an embodiment of the present disclosure;

[0101] Figure 3 A schematic diagram showing an application framework in the process of image encoding or decoding according to an embodiment of the present disclosure;

[0102] Figure 4 Shows Figure 3 A schematic diagram of an implementation of the Local Context, Global Reference, and Hyperprior in the middle;

[0103] Figure 5 Shows a flowchart of an image encoding method according to an embodiment of the present disclosure;

[0104] Figure 6 Shows a flowchart of an image decoding method according to another embodiment of the present disclosure;

[0105] Figure 7 Shows a flowchart of an image processing method according to another embodiment of the present disclosure;

[0106] Figure 8 Shows a flowchart of an image processing method according to another embodiment of the present disclosure;

[0107] Figure 9 Is a schematic structural diagram of an electronic device suitable for implementing an image processing method, an image encoding method, and / or an image decoding method according to an embodiment of the present disclosure. Detailed implementation manners

[0108] Hereinafter, exemplary embodiments of the present disclosure will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily implement them. In addition, for clarity, parts unrelated to the description of the exemplary embodiments are omitted in the drawings.

[0109] In the present disclosure, it should be understood that terms such as "including" or "having" are intended to indicate the presence of features, numbers, steps, actions, components, parts, or combinations thereof disclosed in this specification, and are not intended to exclude the possibility of the presence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0110] It should be further noted that, without conflict, the embodiments in the present disclosure and the features in the embodiments can be combined with each other. The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with the embodiments.

[0111] The details of the embodiments of the present disclosure will be introduced in detail below through specific examples.

[0112] Figure 1 Shows a flowchart of an image processing method according to an embodiment of the present disclosure. As Figure 1 shown, the image processing method includes the following steps:

[0113] In step S101, obtain the processed features in the image to be processed;

[0114] In step S102, according to the processed features, determine the similarity information of the currently to-be-processed feature in the image to be processed; wherein, the similarity information includes similar features;

[0115] In step S103, based on the similarity information, determine the reference information for processing the currently to-be-processed feature.

[0116] In this embodiment, this image processing method is applicable to determining the reference information for encoding the currently to-be-processed feature or decoding the currently to-be-processed feature during the image encoding process or the image decoding process. Both the processed features and the currently to-be-processed feature are image features in the image to be processed. During the image encoding process, the image to be processed is the image to be encoded, and the image features can be features extracted from the image to be encoded. The processed features are the image features that have been encoded before the currently to-be-processed feature, and the currently to-be-processed feature can be the image feature to be encoded currently; during the image decoding process, the image to be processed is the image to be decoded, and the image features can be features obtained by decoding the encoding bitstream corresponding to the image to be decoded. The processed features are the image features that have been decoded before the currently to-be-processed feature, and the currently to-be-processed feature is the image feature to be decoded currently; when decoding the currently to-be-processed feature in the image features, the decoded features in the image features are known features, while the undecoded features are unknown features and need to be decoded from the encoding bitstream through subsequent decoding processes. The embodiments of the present disclosure are applicable to image encoding and decoding methods based on deep learning, and the network model based on deep learning can adopt an autoencoder model.

[0117] In some embodiments, during the image encoding process, the encoding network in the autoencoder model based on deep learning can be used to perform downsampling processing on the image to be encoded, and the result after downsampling processing can be quantized to obtain image features. In other embodiments, in the image decoding method, the encoding bitstream can be decoded by using the autoencoder model to obtain the image features corresponding to the image to be decoded. During the decoding process, decoding can be performed in a point-by-point manner, and when decoding the currently to-be-processed feature in the image features, the reference information of the currently to-be-processed feature can be determined by the image processing method disclosed in the embodiments of the present disclosure, and then the currently to-be-processed feature can be decoded according to the reference information. After that, the next feature can be decoded in the same way.

[0118] During the image encoding process, the processed features in the image features may include the encoded features encoded before the currently to-be-processed feature; while during the image decoding process, the processed features in the image features may include the decoded features decoded before the currently to-be-processed feature. The similarity information may at least include the similar features of the currently to-be-processed feature. In some embodiments, the similar features may be the features in the processed features that are most similar to the currently to-be-processed feature.

[0119] After obtaining the similarity information of the currently to-be-processed feature, the reference information of the currently to-be-processed feature may be determined according to the similarity information. The reference information may be the information referred to when encoding or decoding the currently to-be-processed feature. It can be understood that the similar features of the currently to-be-processed feature are obtained by performing a global search on the encoded features or decoded features in the image features, and then the reference information of the currently to-be-processed feature is determined according to the similar features, so that the encoding process of the currently to-be-processed feature refers to the global information of the currently to-be-processed feature in the image features, that is, in this way, the image can be encoded and decoded with reference to the global information.

[0120] In some embodiments, taking entropy coding as an example, the reference information may be the entropy coding information used to determine the codeword length adopted by the currently to-be-processed feature. For example, it may be the information representing the occurrence probability of the currently to-be-processed feature in the entire image features. In some cases, the entropy coding information may be the Gaussian distribution parameters (which may include the expected value μ and the standard deviation σ) of the currently to-be-processed feature in the image features.

[0121] It can be understood that when determining the reference information for processing the currently to-be-processed feature in the embodiments of the present disclosure, the similarity information of the currently to-be-processed feature is used as one of the factors, so that the reference information for encoding the image reflects the similarity information of the currently to-be-processed feature in the entire image features, that is, the global information. During the image encoding process, the encoding process for the currently to-be-processed feature refers to its similar features, rather than being limited to the local information of the currently to-be-processed feature (the local information may include the surrounding features of the currently to-be-processed feature, while the similar features include other features in the image features except the surrounding features, and the similar features reflect the global information).

[0122] After determining the reference information, encoding or decoding may be performed on the currently to-be-processed feature. During the encoding process, the codeword length of the currently to-be-processed feature may be determined based on the reference information, and then the currently to-be-processed feature may be encoded according to the codeword length to obtain the corresponding code stream. While during the decoding process, the codeword length of the currently to-be-processed feature may be determined based on the reference information, and then the code stream corresponding to the currently to-be-processed feature may be decoded according to the codeword length to obtain the currently to-be-processed feature.

[0123] In the process of determining the reference information in the embodiments of the present disclosure, by determining the similarity information of the currently to-be-processed feature from the decoded features or encoded features of the to-be-processed image, the similarity information includes the similar features of the currently to-be-processed feature, and then determining the reference information when processing the currently to-be-processed feature according to the similarity information. In this way, the reference information can be determined based on the global information of the currently to-be-processed feature in the image features, so that the encoding process of the currently to-be-processed feature can refer to the global information in the image features, rather than being limited to the local information of the currently to-be-processed feature. For images with a large number of repeated textures, it is possible to reduce the redundancy of the global information in the encoded image and improve the compression ratio of the image.

[0124] In an alternative implementation of this embodiment, the method further includes the following steps:

[0125] Obtain the to-be-processed image;

[0126] Use the encoding network in the first autoencoder model to compress the to-be-processed image to obtain compressed features;

[0127] Quantize the compression result to obtain the image features corresponding to the to-be-processed image.

[0128] In this alternative implementation, in the image compression scheme based on deep learning, the autoencoder model is often used for image encoding or decoding. The autoencoder model usually includes an encoding network and a decoding network, and both the encoding network and the decoding network can be convolutional neural network models based on deep learning. In the encoding process, the processing process of the autoencoder model for the to-be-processed image, that is, the to-be-encoded image, usually includes: using the encoding network to perform downsampling processing on the input to-be-encoded image, that is, the to-be-encoded image can be compressed and encoded to obtain compressed features. After quantization and other processing of the compressed features, the above-mentioned image features can be obtained. The image features can be further input into the encoder for encoding to obtain an encoded bitstream; the encoded bitstream can be decoded by the decoder corresponding to the encoder to restore the above-mentioned image features, and after upsampling processing by the decoding network in the first autoencoder model, the decoded image can be obtained.

[0129] Therefore, during the encoding process, the encoding network in the first autoencoder model can be used to compress the image to be encoded to obtain compressed features, and then the compressed features can be quantized and other processed to obtain image features. During the decoding process, point-by-point decoding can be performed on the encoded bitstream corresponding to the image to be decoded. Therefore, when decoding the currently to-be-processed feature, the decoded features in the image features can be obtained first, and then the features similar to the currently to-be-processed feature can be obtained from the decoded features. During the process of obtaining the similar features of the currently to-be-processed feature, since the currently to-be-processed feature has not been decoded yet, the decoded features in the image features can be searched based on the local information of the currently to-be-processed feature, that is, the surrounding features (the surrounding features only include the decoded features), and then the similar features of the currently to-be-processed feature can be determined.

[0130] In an alternative implementation of this embodiment, the similarity information further includes the similarity between the currently to-be-processed feature and the similar feature.

[0131] In an alternative implementation of this embodiment, step S102, that is, the step of determining the similarity information of the currently to-be-processed feature in the to-be-processed image according to the processed feature, further includes the following steps:

[0132] Search for the similar feature from the processed features in the image features by using the local information of the currently to-be-processed feature;

[0133] Determine the similarity between the currently to-be-processed feature and the similar feature.

[0134] In this alternative implementation, the similar feature of the currently to-be-processed feature can be obtained by performing point-by-point search on the encoded features in the image features. The similar feature can be the feature in the encoded features that is most similar to the currently to-be-processed feature. During the search process, the similarity can be determined by matching the local information of the currently to-be-processed feature with the local information of the encoded features in the image features, and the one with the maximum similarity is determined as the similar feature of the currently to-be-processed feature. During the actual processing, the position of the similar feature in the image features and the corresponding similarity are output by this step.

[0135] It should be noted that when searching for and matching similar features, the local information used is the features that have been encoded or decoded before the currently to-be-processed feature. For example, when the local information of the currently to-be-processed feature is 9 features of 3×3 around it, the local information used includes the features before the currently to-be-processed feature, that is, 4 features starting from the upper left corner (3 features in the first row and the first feature in the second row), and the other 5 features are the currently to-be-processed feature and the features after the currently to-be-processed feature. Since they have not been encoded or decoded, they are not used to match similar features.

[0136] In an alternative implementation of this embodiment, step S103, that is, the step of determining reference information when processing the current feature to be processed based on the similarity information, further includes the following steps:

[0137] Obtain first auxiliary information for determining the reference information according to the similar feature and the local information of the similar feature;

[0138] Determine the reference information after performing a weighting operation on the first auxiliary information using the similarity.

[0139] In this alternative implementation, the first auxiliary information for determining the reference information can be obtained by using the similar feature and the local information of the similar feature, and the reference information is used to encode or decode the current feature to be processed. The first auxiliary information can be a feature extracted from the similar feature and its local information. In some embodiments, the first auxiliary information can be obtained by multiplying a matrix composed of the similar feature and the local information of the similar feature (for example, a 3×3 feature matrix, with the similar feature in the middle) by a layer of mask convolution. The mask convolution can extract information from the 3×3 feature matrix by masking the features after the similar feature, thereby obtaining the first auxiliary information. For example, the local information of the similar feature can be 9 feature information around the similar feature, and the similar feature is located in the center. The features masked by the mask convolution layer include 4 features after the similar feature, that is, the last feature in the second row and the three features in the third row.

[0140] Figure 2 Shows a schematic diagram of the search and matching process of similar features in an embodiment of the present disclosure. As Figure 2 shown, for the current feature to be processed in the image feature by using the local information of the current feature to be processed to search in the encoded features, the position H of the similar feature and the similarity S can be obtained. After performing a mask convolution operation on the similar feature at the position H and the local information of the similar feature, the first auxiliary information can be obtained

[0141] The first auxiliary information can be used to represent the reference information for encoding the currently to-be-processed feature, which reflects the global information referred to during the encoding or decoding process. When the similarity between the similar feature and the currently to-be-processed feature is low, it can be considered that there is less texture in the image feature that repeats the currently to-be-processed feature. Therefore, when encoding the currently to-be-processed feature, the global information may not be referred to or referred to as little as possible. Thus, the method of determining the reference information of the currently to-be-processed feature after weighting the first auxiliary information by similarity can make it such that when there is more texture repeating the currently to-be-processed feature, a larger similarity-based weighting operation is performed on the first auxiliary information, and the reference information can consider the global information more. When there is less texture repeating the currently to-be-processed feature, a smaller similarity-based weighting operation is performed on the first auxiliary, and the reference information can refer to the global information less. The ultimate effect that can be achieved is that for images with more repeating texture, the image compression rate can be increased, and for images without repeating texture, the image compression rate will not be decreased.

[0142] In an alternative implementation of this embodiment, the method further includes the following steps:

[0143] Obtain second auxiliary information for determining the reference information according to the local information of the currently to-be-processed feature;

[0144] Obtain a first probability estimate of the currently to-be-processed feature in the image feature corresponding to the to-be-processed image according to the second auxiliary information;

[0145] Obtain the confidence level of the currently to-be-processed feature according to the first probability estimate.

[0146] In this alternative implementation, the second auxiliary information for determining the reference information can also be obtained using the local information of the currently to-be-processed feature, and this reference information is used for encoding or decoding the currently to-be-processed feature. The local information of the currently to-be-processed feature can be the image features around the currently to-be-processed feature. For example, it can be 9 features around the currently to-be-processed feature, which can form a 3×3 feature matrix with the currently to-be-processed feature in the middle. Since the second auxiliary information is extracted from the local information of the currently to-be-processed feature, it can enable the encoding process or the decoding process to refer to the local information to encode or decode the currently to-be-processed feature.

[0147] In some embodiments, a second auxiliary information can be obtained by multiplying a matrix formed by local information of a currently to-be-processed feature (for example, it can be a 3×3 feature matrix, and the middle one is the currently to-be-processed feature) by a layer of mask convolution. This second auxiliary information can be used to represent another reference information when encoding the currently to-be-processed feature. The mask convolution can extract information from the 3×3 feature matrix by masking the currently to-be-processed feature and the features after the currently to-be-processed feature, and then obtain the second auxiliary information. For example, the local information of the currently to-be-processed feature can be 3×3 = 9 feature information around the currently to-be-processed feature, and the currently to-be-processed feature is located at the centermost position. The features masked by the mask convolution layer include the currently to-be-processed feature and 5 features after the currently to-be-processed feature, that is, the last feature in the second row and the three features in the third row.

[0148] Through this second auxiliary information, a first probability estimate of the currently to-be-processed feature in the image features can be obtained. This first probability estimate can represent the occurrence probability of the currently to-be-processed feature in the image features from the perspective of local information. The first probability estimate can include the expected value μ and the standard deviation σ of the Gaussian distribution. In some embodiments, the reference information includes this first probability estimate.

[0149] Using this first probability estimate, the confidence of the currently to-be-processed feature can be obtained. This confidence can be used to represent the probability value of the currently to-be-processed feature appearing in the entire image features. The more features similar to the currently to-be-processed feature, the larger this probability value, and the fewer features similar to the currently to-be-processed feature, the smaller this probability value. During the encoding process, for features with a higher probability value, a bitstream with a shorter codeword can be assigned, while for features with a lower probability value, a bitstream with a longer codeword can be assigned. In this way, the total length of the encoded bitstream of the image will be relatively reduced, which can save the storage space and transmission bandwidth of the image.

[0150] In an alternative implementation manner of this embodiment, the step of determining the reference information by weighting the first auxiliary information with the similarity further includes the following steps:

[0151] Combine the first auxiliary information and the second auxiliary information to obtain a first combination result;

[0152] Filter the first combination result with the similarity and the confidence to obtain a filtered result;

[0153] Obtain a second probability estimate of the currently to-be-processed feature according to the filtered result and the first probability estimate.

[0154] In this optional implementation, when encoding the current feature to be processed, in order to jointly refer to local information and global information, the first auxiliary information obtained using global information and the second auxiliary information obtained using local information can be combined to obtain a first combination result. The combination operation can be, for example, a concat (character concatenation) operation, that is, concatenating the first auxiliary information and the second auxiliary information together.

[0155] After obtaining the first combination result by connection, the first combination result can be weighted using similarity and confidence, that is, in the case where the similarity and confidence are large, the first combination result can play a more important role in the process of determining the reference information, so that the current feature to be processed can refer more to local information and global information during the encoding process, thereby improving the image compression rate; while in the case where the similarity and confidence are small, the first combination result can be ignored or play a minor role in the process of determining the reference information, thereby reducing the encoding impact of the first auxiliary information and the second auxiliary information on the current feature to be processed.

[0156] According to the filtering result and the first probability estimate, a second probability estimate can be obtained, which can represent the occurrence probability of the current feature to be processed in the image features from the perspectives of global information and local information. The second probability estimate can include the expected value μ and the standard deviation σ of the Gaussian distribution. In some embodiments, the reference information includes the second probability estimate.

[0157] In an optional implementation of this embodiment, the method further includes the following steps:

[0158] Using a second autoencoder model to obtain third auxiliary information for determining the reference information from the image features corresponding to the image to be processed;

[0159] Combining the third auxiliary information with the first auxiliary information to obtain a second combination result;

[0160] Using the second combination result and the second probability estimate to obtain a third probability estimate of the current feature to be processed.

[0161] In this optional implementation, during the encoding process, third auxiliary information for determining reference information can also be obtained by using the entire image features. In this step, the second autoencoder model can be used to obtain the third auxiliary information from the entire image features. The autoencoder model includes an encoding network and a decoding network. After the image features are input into the encoding network, compressed intermediate features are obtained. After the intermediate features are encoded by the lossless encoder, an encoded bitstream (the encoded bitstream, as the compressed bitstream of the image features, will be output to the image decoding end) is obtained. After the encoded bitstream is decoded by the lossless decoder corresponding to the lossless encoder, the above intermediate features can be restored. The intermediate features can obtain the third auxiliary information through the decoding network in the second autoencoder model. The third auxiliary information can be understood as the reference information of the currently to-be-processed features predicted from the entire image features by using the autoencoder model.

[0162] This third auxiliary information can be combined with the above first auxiliary information and second auxiliary information. For example, the second combination result can be obtained through a concat (character connection) operation. Using this second combination result and the second probability estimate, a third probability estimate can be obtained. This third probability estimate is ultimately used to determine the reference information of the currently to-be-processed features. This third probability estimate is the occurrence probability of the currently to-be-processed features in the image features obtained by comprehensively considering the entire image features, the global information and local information of the currently to-be-processed features. The third probability estimate can include the expected value μ and standard deviation σ of the Gaussian distribution. In some embodiments, the reference information includes this third probability estimate.

[0163] In an optional implementation of this embodiment, the method further includes the following steps:

[0164] Obtain the second encoded bitstream corresponding to the to-be-processed image;

[0165] Use the second autoencoder model to obtain the third auxiliary information for determining the reference information from the second encoded bitstream;

[0166] Combine the first auxiliary information, the second auxiliary information, and the third auxiliary information to obtain a second combination result;

[0167] Use the second combination result and the second probability estimate to obtain the third probability estimate of the currently to-be-processed features.

[0168] In this optional implementation, as described above, during the encoding stage, when using the second autoencoder model to obtain the third auxiliary information from the image features corresponding to the image to be processed, an encoded bitstream corresponding to the image features will be generated, and this encoded bitstream is obtained by the image decoding end. It can be understood that during the image encoding process, two parts of bitstreams will be generated. One part is the first encoded bitstream generated by encoding each feature in the image features using the reference information, and the other part is the second encoded bitstream generated by encoding the image features in order to obtain the third auxiliary information for determining the reference information. During the decoding process, the third auxiliary information can be obtained by decoding the second encoded bitstream; that is, after the second encoded bitstream is input into the lossless decoder, intermediate features can be decoded, and when these intermediate features are input into the decoding network of the second autoencoder model, the third auxiliary information can be obtained. The third auxiliary information can be understood as the reference information of the current feature to be processed predicted from the entire image features using the autoencoder model.

[0169] This third auxiliary information can be combined with the above-mentioned first auxiliary information and second auxiliary information. For example, the second combination result can be obtained through a concat (character connection) operation. Using this second combination result and the second probability estimate, a third probability estimate can be obtained, and this third probability estimate is ultimately used to determine the reference information of the current feature to be processed. This third probability estimate is the occurrence probability of the current feature to be processed in the image features obtained by comprehensively considering the entire image features, the global information and local information of the current feature to be processed. This third probability estimate can include the expected value μ and standard deviation σ of the Gaussian distribution.

[0170] In an optional implementation of this embodiment, the step of determining the reference information after weighting the first auxiliary information using the similarity further includes the following steps:

[0171] Determine the reference information according to the third probability estimate.

[0172] In this optional implementation, the reference information of the current feature to be processed can ultimately be determined using the third probability estimate, and this reference information can be used to determine the codeword length of the current feature to be processed. During the encoding process, taking entropy coding as an example, according to the probability distribution given by the third probability estimate, when it is determined that the current feature to be processed appears more frequently in the image features, a bitstream with a shorter codeword can be assigned to the current feature to be processed, and when it is determined that the current processed feature appears less frequently in the image features, a bitstream with a longer codeword can be assigned to the current feature to be processed.

[0173] The implementation details of the embodiments of the present disclosure are illustrated below by way of examples.

[0174] Figure 3Shows a schematic diagram of an application framework in the process of image encoding or decoding. As Figure 3 shown, the three dashed boxes in this framework respectively represent structures for determining (i.e., the first auxiliary information mentioned above), (i.e., the second auxiliary information mentioned above), and (i.e., the third auxiliary information mentioned above); among them, the Global Reference part is used to determine the first auxiliary information the LocalContext part is used to determine the second auxiliary information the Hyperprior part is used to determine the third auxiliary information Outside the dashed boxes, that is, in the left half of the figure, the encoding and decoding process framework for encoding and decoding the image is shown.

[0175] Next, the Global Reference part, the LocalContext part, the Hyperprior part, and the encoding and decoding process framework will be described in detail respectively.

[0176] The encoding and decoding process framework includes an autoencoder model, and this autoencoder model includes an encoding network E and a decoding network D; during the encoding process, the input image X is compressed by the encoding network E to obtain a compressed feature y, and after the compressed feature y is quantized by a quantizer Q, an image feature is obtained Image feature After passing through a lossless encoder, an encoded bitstream can be obtained, and this encoded bitstream can be sent to the image decoding end for decoding; during the decoding process, the encoded bitstream can be restored to the image feature after passing through a lossless decoder Image feature Then, through the decoding network in the autoencoder model, the decoded image can be obtained

[0177] From Figure 3 it can be seen that during the process of lossless encoding or decoding of the image feature , the third probability estimate (μ 3 , σ 3 ) output by the Hyperprior part is used as reference information for the image feature, and is input to the lossless encoder or the lossless decoder to encode or decode the image.

[0178] The third probability estimate is based on the third auxiliary information and the first probability estimate (μ 1 , σ 1) is obtained, and the first probability estimate is obtained based on the first auxiliary information and the second probability estimate (μ 2 , σ 2 ), and the second probability estimate is obtained based on the second auxiliary information .

[0179] The Local Context is a structure that predicts and obtains the second auxiliary information by using the context information of the current feature to be processed, that is, the local information. represents the local information of the current feature to be processed in the image feature (as Figure 3 shown in, which are the features encoded or decoded before the current feature to be processed among the 3×3 = 9 features around the current feature to be processed i). In the Local Context, the local information of the current feature to be processed is used as the input, and after being processed by the Context Model, the second auxiliary information The second auxiliary information after being processed by a network model h le (·) outputs the second probability estimate (μ 2 , σ 2 ).

[0180] The Global Reference is a structure that predicts and obtains the first auxiliary information by using the similarity information of the current feature to be processed, that is, the global information. represents the local information of the current feature to be processed in the image feature (as Figure 3 shown in, which are the features encoded or decoded before the current feature to be processed among the 3×3 = 9 features around the current feature to be processed i). In the Global Reference, the local information of the current feature to be processed is used as the input, and after being processed by the Reference Model, the first auxiliary information The first auxiliary information and the second auxiliary information After being merged, the merged result is processed by a network model h ge (·), and after combining the processing result with the second probability estimate, the first probability estimate (μ 1 , σ 1 ) is output.

[0181] The Hyperprior also includes an autoencoder model, which consists of an encoding network HE and a decoding network HD. During the encoding process, the compressed feature y obtained through the encoding network E in the encoding and decoding process framework serves as the input to the encoding network HE in the Hyperprior. After further compression by the encoding network HE, the compressed feature z can be obtained. The compressed feature z is quantized by the quantizer Q to obtain the intermediate feature. Intermediate feature After being encoded by the lossless encoder AE, the encoded bitstream can be obtained, and this encoded bitstream can be sent to the image decoding end for decoding. During the decoding process, the encoded bitstream can be restored to the intermediate feature after being decoded by the lossless decoder AD. Intermediate feature Then, through the decoding network HD in the autoencoder model, the third auxiliary information can be obtained. The Hyperprior can utilize the Factorized Entropy Model during the encoding and decoding process. The third auxiliary information The first auxiliary information and the second auxiliary information After being merged, the merged result is processed by a network model h se (·), and after combining the processing result with the first probability estimate, the third probability estimate (μ 3 , σ 3 ) is output.

[0182] Figure 4 Show Figure 3 A schematic diagram of an implementation of the Local Context, Global Reference, and Hyperprior in it.

[0183] Such as Figure 4 As shown, for the current feature to be processed, the Local Context receives the local information of the current feature to be processed This local information is input into a masked convolutional layer Mask Conv for processing. The masked convolutional layer Mask Conv extracts the second auxiliary information by masking the current feature to be processed and the feature information after encoding or decoding of the current feature to be processed in the local information Then, after being processed by a convolutional layer with a convolution kernel of 1×1, the second probability estimate (μ The second auxiliary information , σ 2 , σ 2 )(What is shown in the figure is logσ 2 ) is obtained.

[0184] For the current feature to be processed, the local context part receives local information of the current feature to be processed. This local information After passing through the reference module, similar features can be searched and matched from the previously encoded or decoded features of the current feature to be processed, and the positions and similarity S of the similar features are output. The similar features and their local information are extracted according to the positions where the similar features are located and input into a Mask Conv layer for processing. The Mask Conv layer extracts the first auxiliary information by masking the feature information encoded or decoded after the similar features in the local information. The first auxiliary information The first auxiliary information After being merged with the second auxiliary information through a Concat operation, the merged result is further processed by a convolutional layer with a kernel size of 1×1. The result after multiplying the processed result with the similarity and confidence (the confidence is obtained according to the second probability estimate) is combined with the second probability estimate (μ 2 , σ 2 ) to obtain the first probability estimate (μ 1 , σ 1 ) (logσ 1 is shown in the figure).

[0185] For the current feature to be processed, during the encoding process, the compressed feature y is used as the input of the hyperprior part and undergoes a series of processes described above to obtain an intermediate feature. The intermediate feature After being input into the decoding network of the hyperprior part, the third auxiliary information is obtained. This third auxiliary information After being merged with the first auxiliary information the second auxiliary information through a Contact operation, the merged result is further processed by a convolutional layer with a kernel size of 1×1. The result after multiplying the processed result with the second probability estimate (μ 2 , σ 2 ) is combined to obtain the third probability estimate (μ 3 , σ 3 ) (logσ 1 is shown in the figure).

[0186] Figure 5 The flowchart showing an image encoding method according to an embodiment of the present disclosure is shown. As Figure 5 shown, the image processing method includes the following steps:

[0187] In step S501, an image to be encoded is obtained.

[0188] In step S502, the reference information for encoding each feature in the image to be encoded is obtained by using the above image processing method.

[0189] In step S503, the image to be encoded is encoded by using the reference information.

[0190] In this embodiment, during the image encoding process, for the obtained image to be encoded, the reference information for encoding each feature in the image to be encoded can be obtained point by point, and after the reference information of all features is determined, all features in the image to be encoded can be encoded according to the determined reference information. Of course, it can be understood that encoding can also be performed point by point during the encoding process, that is, the reference information corresponding to the currently to-be-processed feature in the image to be encoded is obtained point by point, and the currently to-be-processed feature is encoded according to the reference information, and the above process is repeated until all features in the image to be encoded are encoded.

[0191] For the acquisition of the reference information of the currently to-be-processed feature, reference can be made to the descriptions of the above Figure 1 illustrated embodiments and related embodiments, which will not be elaborated here.

[0192] During the process of encoding an image by using the image processing method in the embodiments of the present disclosure, since the information of similar features of the image features is referred to, that is, the global information is referred to, when encoding an image with more repetitive textures, the redundancy in the global information can be reduced, thereby improving the compression ratio of the image.

[0193] Figure 6 The flowchart showing an image decoding method according to another embodiment of the present disclosure is shown. As Figure 6 shown, the image decoding method includes the following steps:

[0194] In step S601, an encoded bitstream of the image to be decoded is obtained.

[0195] In step S602, the reference information for decoding each feature in the image to be decoded is obtained by using the above image processing method.

[0196] In step S603, the encoded bitstream is decoded by using the reference information.

[0197] In this embodiment, during the image decoding process, for the encoded bitstream of the to-be-decoded image obtained, the reference information for decoding each feature in the to-be-decoded image can be obtained point by point. After determining the reference information for the current to-be-processed feature, the bitstream corresponding to the current to-be-processed feature in the encoded bitstream can be decoded according to the reference information to obtain the current to-be-processed feature. The above process is repeated until all features in the to-be-decoded image are obtained.

[0198] For the acquisition of the reference information of the current to-be-processed feature, reference can be made to the descriptions of the above-described embodiments and related embodiments shown, which will not be elaborated here. Figure 1 shown in the embodiments and related embodiments, which will not be elaborated here.

[0199] Figure 7 The flowchart showing an image processing method according to another embodiment of the present disclosure is shown. As Figure 7 shown, the image processing method includes the following steps:

[0200] In step S701, the encoded features in the to-be-encoded image are obtained;

[0201] In step S702, a preset service interface is called so that the preset service interface determines the similarity information of the current to-be-processed feature in the to-be-encoded image according to the encoded features, and based on the similarity information, determines the reference information for encoding the current to-be-processed feature; wherein, the similarity information includes similar features;

[0202] In step S703, the reference information is output.

[0203] In this embodiment, the image processing method can be executed in the cloud. The preset service interface can be pre-deployed in the cloud. The preset service interface can be a Saas (Software-as-a-service) interface. The requester can obtain the usage right of the preset service interface in advance and can process the to-be-encoded image by calling the preset service interface when needed, so as to obtain the reference information of each feature in the to-be-encoded image.

[0204] This image processing method is applicable to determining the reference information for encoding the current to-be-processed feature during the image encoding process. During the image encoding process, the encoded features and the current to-be-processed feature can be image features extracted from the to-be-encoded image. The encoded features can be the features that have been encoded currently, while the current to-be-processed feature can be the feature to be encoded currently. The embodiments of the present disclosure are applicable to image encoding methods based on deep learning, and the network model based on deep learning can adopt an autoencoder model.

[0205] In some embodiments, during the image encoding process, the encoding network in the deep learning-based autoencoder model can be used to perform downsampling on the image to be encoded, and the result of the downsampling process can be quantized to obtain image features.

[0206] During the image encoding process, the processed features can include the encoded features encoded before the current feature to be processed. The similarity information can at least include the similar features of the current feature to be processed. In some embodiments, the similar features can be the features in the processed features that are most similar to the current feature to be processed.

[0207] After obtaining the similarity information of the current feature to be processed, the reference information of the current feature to be processed can be determined according to the similarity information. The reference information can be the information referred to when encoding the current feature to be processed. It can be understood that by globally searching the encoded features to obtain the similar features of the current feature to be processed, and then determining the reference information of the current feature to be processed according to the similar features, the encoding process of the current feature to be processed refers to the global information of the current feature to be processed in the image features, that is, in this way, the image can be encoded with reference to the global information.

[0208] In some embodiments, taking entropy encoding as an example, the reference information can be the entropy encoding information used to determine the codeword length adopted by the current feature to be processed. For example, it can be the information representing the occurrence probability of the current feature to be processed in the entire image features. In some cases, the entropy encoding information can be the Gaussian distribution parameters (which can include the expected value μ and the standard deviation σ) of the current feature to be processed in the image features.

[0209] It can be understood that in the embodiments of the present disclosure, when determining the reference information for encoding the current feature to be processed, the similarity information of the current feature to be processed is used as one of the factors, so that the reference information for encoding the image reflects the similarity information of the current feature to be processed in the entire image features, that is, the global information. During the image encoding process, the encoding process for the current feature to be processed refers to its similar features, rather than being limited to the local information of the current feature to be processed (the local information can include the surrounding features of the current feature to be processed, while the similar features include other features in the image features except the surrounding features, and the similar features reflect the global information).

[0210] After determining the reference information, encoding can be performed on the current feature to be processed. During the encoding process, the codeword length of the current feature to be processed can be determined based on the reference information, and then the current feature to be processed can be encoded according to the codeword length to obtain the corresponding bitstream.

[0211] In the process of determining reference information in the embodiments of the present disclosure, similar information of the currently to-be-processed feature is determined from the decoded features or the encoded features, and the similar information includes the similar features of the currently to-be-processed feature. Then, the reference information of the currently to-be-processed feature is determined according to the similar information. In this way, the reference information can be determined based on the global information of the currently to-be-processed feature in the image features corresponding to the image to be encoded, so that the encoding process of the currently to-be-processed feature can refer to the global information in the image features, rather than being limited to the local information of the currently to-be-processed feature. For images with a large number of repeated textures, the redundancy of the global information in the encoded image can be reduced, and the compression rate of the image can be improved.

[0212] Figure 8 The flowchart of the image processing method according to another embodiment of the present disclosure is shown. As Figure 8 shown, the image processing method includes the following steps:

[0213] In step S801, the decoded features in the image to be decoded are obtained;

[0214] In step S802, a preset service interface is called so that the preset service interface determines the similar information of the currently to-be-processed feature in the image to be decoded according to the decoded features, and determines the reference information for decoding the currently to-be-processed feature based on the similar information; wherein, the similar information includes similar features;

[0215] In step S803, the reference information is output.

[0216] In this embodiment, the image processing method can be executed in the cloud. The preset service interface can be pre-deployed in the cloud. The preset service interface can be a Saas (Software-as-a-service) interface. The demander can obtain the right to use the preset service interface in advance and can call the preset service interface to process the encoded bitstream corresponding to the image to be decoded when needed, so as to obtain the reference information of each feature in the image to be decoded.

[0217] This image processing method is applicable to determining the reference information for decoding the currently to-be-processed feature during the image decoding process. During the image decoding process, the decoded features can be the image features that have been decoded in the image features corresponding to the image to be decoded, and the currently to-be-processed feature can be the feature to be decoded currently. When decoding the currently to-be-processed feature, the decoded features in the image features are known features, and the undecoded features are unknown features, which need to be decoded from the encoded bitstream through subsequent decoding processes. The embodiments of the present disclosure are applicable to image decoding methods based on deep learning, and the network model based on deep learning can adopt an autoencoder model.

[0218] In some embodiments, in the method for decoding an image, an autoencoder model can be used to decode an encoded bitstream to obtain image features corresponding to the image to be decoded. During the decoding process, decoding can be performed in a point-by-point manner. When decoding the currently to-be-processed feature in the image features, the reference information for the currently to-be-processed feature can be determined by the image processing method disclosed in the embodiments of the present disclosure. Then, the currently to-be-processed feature can be decoded based on the reference information, and then the next feature can be decoded in the same way.

[0219] During the image decoding process, the processed features in the image features can include the decoded features decoded before the currently to-be-processed feature. The similarity information can at least include the similar features of the currently to-be-processed feature. In some embodiments, the similar features can be the features in the processed features that are most similar to the currently to-be-processed feature.

[0220] After obtaining the similarity information of the currently to-be-processed feature, the reference information for the currently to-be-processed feature can be determined based on the similarity information. The reference information can be the information referred to when decoding the currently to-be-processed feature. It can be understood that by globally searching the decoded features to obtain the similar features of the currently to-be-processed feature, and then determining the reference information for the currently to-be-processed feature based on the similar features, when the encoding of the currently to-be-processed feature refers to the global information of the currently to-be-processed feature in the image features, the currently to-be-processed feature can be decoded from the encoded bitstream through the reference information.

[0221] In some embodiments, taking entropy coding as an example, the reference information can be the entropy coding information used to determine the codeword length adopted by the currently to-be-processed feature. For example, it can be the information representing the occurrence probability of the currently to-be-processed feature in the entire image features. In some cases, the entropy coding information can be the Gaussian distribution parameters (which can include the expected value μ and the standard deviation σ) of the currently to-be-processed feature in the image features.

[0222] It can be understood that when the embodiments of the present disclosure determine the reference information for decoding the currently to-be-processed feature, the similarity information of the currently to-be-processed feature is used as one of the factors, so that the reference information for encoding the image reflects the similarity information, that is, the global information of the currently to-be-processed feature in the entire image features. Since during the image encoding process, the encoding process for the currently to-be-processed feature refers to its similar features, rather than being limited to the local information of the currently to-be-processed feature (the local information can include the surrounding features of the currently to-be-processed feature, while the similar features include other features in the image features except the surrounding features, and the similar features reflect the global information).

[0223] After determining the reference information, the current feature to be processed can be decoded. During the decoding process, the codeword length of the current feature to be processed can be determined based on the reference information, and then the bitstream corresponding to the current feature to be processed can be decoded according to the codeword length to obtain the current feature to be processed.

[0224] In the process of determining the reference information in the embodiments of the present disclosure, the similarity information of the current feature to be processed is determined by obtaining the similarity information of the current feature to be processed from the decoded features or the encoded features. The similarity information includes the similar features of the current feature to be processed, and then the reference information of the current feature to be processed is determined according to the similarity information. In this way, the reference information can be determined based on the global information of the current feature to be processed in the image features, so that the encoding process of the current feature to be processed can refer to the global information in the image features, rather than being limited to the local information of the current feature to be processed. For images with a large number of repetitive textures, the redundancy of the global information in the encoded image can be reduced, and the compression ratio of the image can be improved.

[0225] According to another embodiment of the present disclosure, an image processing method, the data processing method includes the following steps:

[0226] Obtain the image to be encoded;

[0227] Call a preset service interface, so that the preset service interface determines the similarity information of the current feature to be processed in the image to be encoded according to the encoded features in the image to be encoded, and based on the similarity information, determines the reference information for encoding the current feature to be processed, and encodes the current feature to be processed using the reference information; wherein, the similarity information includes similar features;

[0228] Output the encoding result.

[0229] In this embodiment, the image processing method can be executed in the cloud. The preset service interface can be pre-deployed in the cloud. The preset service interface can be a Saas (Software-as-a-service) interface. The requester can obtain the usage right of the preset service interface in advance and can encode the image to be encoded by calling the preset service interface when needed, so as to obtain the encoded bitstream corresponding to the image to be encoded.

[0230] The image processing method is applicable to encoding the image to be encoded during the image encoding process. During the image encoding process, the encoded features and the current feature to be processed can be image features extracted from the image to be encoded. The encoded features can be the features that have been encoded currently, and the current feature to be processed can be the feature to be encoded currently. The embodiments of the present disclosure are applicable to image encoding methods based on deep learning, and the network model based on deep learning can adopt an autoencoder model.

[0231] In some embodiments, during the image encoding process, the encoding network in the deep learning-based autoencoder model can be used to perform downsampling processing on the image to be encoded, and the result of the downsampling processing can be quantized to obtain image features.

[0232] During the image encoding process, the processed features can include the encoded features encoded before the current feature to be processed. The similar information can at least include the similar features of the current feature to be processed. In some embodiments, the similar feature can be the feature in the processed features that is most similar to the current feature to be processed.

[0233] After obtaining the similar information of the current feature to be processed, the reference information of the current feature to be processed can be determined according to the similar information. The reference information can be the information referred to when encoding the current feature to be processed. It can be understood that by globally searching the encoded features to obtain the similar features of the current feature to be processed, and then determining the reference information of the current feature to be processed according to the similar features, the encoding process of the current feature to be processed refers to the global information of the current feature in the image features, that is, in this way, the image can be encoded with reference to the global information.

[0234] In some embodiments, taking entropy encoding as an example, the reference information can be the entropy encoding information used to determine the codeword length adopted by the current feature to be processed. For example, it can be the information representing the occurrence probability of the current feature to be processed in the entire image features. In some cases, the entropy encoding information can be the Gaussian distribution parameters (which can include the expected value μ and the standard deviation σ) of the current feature to be processed in the image features.

[0235] It can be understood that in the embodiments of the present disclosure, when determining the reference information for encoding the current feature to be processed, the similar information of the current feature to be processed is used as one of the factors, so that the reference information for encoding the image reflects the similar information of the current feature to be processed in the entire image features, that is, the global information. During the image encoding process, the encoding process for the current feature to be processed refers to its similar features, rather than being limited to the local information of the current feature to be processed (the local information can include the surrounding features of the current feature to be processed, and the similar features include other features in the image features except the surrounding features, and the similar features reflect the global information).

[0236] After determining the reference information, the current feature to be processed can be encoded. During the encoding process, the codeword length of the current feature to be processed can be determined based on the reference information, and then the current feature to be processed can be encoded according to the codeword length to obtain the corresponding code stream. After encoding all the features in the image to be encoded, the encoded code stream of the image to be encoded can be obtained, and the encoded code stream can be output to the requester.

[0237] In the process of determining the reference information in the embodiments of the present disclosure, the similarity information of the currently to-be-processed feature is determined from the decoded features or the encoded features, and the similarity information includes the similar features of the currently to-be-processed feature. Then, the reference information of the currently to-be-processed feature is determined according to the similarity information. In this way, the reference information can be determined based on the global information of the currently to-be-processed feature in the image features corresponding to the image to be encoded, so that the encoding process of the currently to-be-processed feature can refer to the global information in the image features, rather than being limited to the local information of the currently to-be-processed feature. For images with a large number of repetitive textures, the redundancy of the global information in the encoded image can be reduced, and the compression ratio of the image can be improved.

[0238] According to another embodiment of the present disclosure, an image processing method includes the following steps:

[0239] Obtain the encoded bitstream of the image to be decoded;

[0240] Call a preset service interface, so that the preset service interface determines the similarity information of the currently to-be-processed feature in the image to be decoded according to the decoded features obtained from the encoded bitstream, and based on the similarity information, determines the reference information for decoding the currently to-be-processed feature, and decodes the currently to-be-processed feature based on the reference information; the similarity information includes similar features;

[0241] Output the decoding result.

[0242] In this embodiment, the image processing method can be executed in the cloud. The preset service interface can be pre-deployed in the cloud. The preset service interface can be a Saas (Software-as-a-service) interface. The demander can obtain the usage right of the preset service interface in advance and can decode the encoded bitstream corresponding to the image to be decoded by calling the preset service interface when needed, so as to obtain the image features of the image to be decoded.

[0243] The image processing method is applicable to determining the image features of the image to be decoded during the image decoding process. During the image decoding process, the decoded features can be the image features that have been decoded in the image features corresponding to the image to be decoded, and the currently to-be-processed feature can be the feature to be decoded currently. When decoding the currently to-be-processed feature, the decoded features in the image features are known features, and the undecoded features are unknown features, which need to be decoded from the encoded bitstream through subsequent decoding processes. The embodiments of the present disclosure are applicable to image decoding methods based on deep learning, and the network model based on deep learning can adopt an autoencoder model.

[0244] In some embodiments, in the method for decoding an image, an autoencoder model may be used to decode an encoded bitstream to obtain image features corresponding to the image to be decoded. During the decoding process, decoding may be performed in a point-by-point manner. When decoding the currently to-be-processed feature in the image features, the reference information for the currently to-be-processed feature may be determined by the image processing method disclosed in the embodiments of the present disclosure. Then, the currently to-be-processed feature may be decoded based on the reference information, and then the next feature may be decoded in the same manner.

[0245] During the image decoding process, the processed features in the image features may include the decoded features decoded before the currently to-be-processed feature. The similarity information may at least include the similar features of the currently to-be-processed feature. In some embodiments, the similar features may be the features in the processed features that are most similar to the currently to-be-processed feature.

[0246] After obtaining the similarity information of the currently to-be-processed feature, the reference information for the currently to-be-processed feature may be determined based on the similarity information. The reference information may be the information referred to when decoding the currently to-be-processed feature. It can be understood that by globally searching the decoded features to obtain the similar features of the currently to-be-processed feature, and then determining the reference information for the currently to-be-processed feature based on the similar features, when the encoding of the currently to-be-processed feature refers to the global information of the currently to-be-processed feature in the image features, the currently to-be-processed feature can be decoded from the encoded bitstream through the reference information.

[0247] In some embodiments, taking entropy coding as an example, the reference information may be the entropy coding information used to determine the codeword length adopted by the currently to-be-processed feature. For example, it may be the information representing the occurrence probability of the currently to-be-processed feature in the entire image features. In some cases, the entropy coding information may be the Gaussian distribution parameters (which may include the expected value μ and the standard deviation σ) of the currently to-be-processed feature in the image features.

[0248] It can be understood that when the embodiments of the present disclosure determine the reference information for decoding the currently to-be-processed feature, the similarity information of the currently to-be-processed feature is used as one of the factors, so that the reference information for encoding the image reflects the similarity information, that is, the global information, of the currently to-be-processed feature in the entire image features. Since during the image encoding process, the encoding process for the currently to-be-processed feature refers to its similar features, rather than being limited to the local information of the currently to-be-processed feature (the local information may include the surrounding features of the currently to-be-processed feature, while the similar features include other features in the image features except the surrounding features, and the similar features reflect the global information).

[0249] After determining the reference information, the current feature to be processed can be decoded. During the decoding process, the codeword length of the current feature to be processed can be determined based on the reference information, and then the code stream corresponding to the current feature to be processed can be decoded according to the codeword length to obtain the current feature to be processed. After obtaining all the image features in the image to be decoded, the image features can be output to the requester.

[0250] In the process of determining the reference information in the embodiments of the present disclosure, by determining the similarity information of the current feature to be processed from the decoded features or the encoded features, the similarity information includes the similar features of the current feature to be processed, and then the reference information of the current feature to be processed is determined according to the similarity information. In this way, the reference information can be determined based on the global information of the current feature to be processed in the image features, so that the encoding process of the current feature to be processed can refer to the global information in the image features, rather than being limited to the local information of the current feature to be processed. For images with more repetitive textures, the redundancy of the global information in the encoded image can be reduced, and the compression ratio of the image can be improved.

[0251] The following are the device embodiments of the present disclosure, which can be used to execute the method embodiments of the present disclosure.

[0252] An image processing device according to an embodiment of the present disclosure, the device can be implemented as part or all of an electronic device through software, hardware, or a combination of both. The image processing device includes:

[0253] A feature acquisition module, configured to acquire the processed features in the image to be processed;

[0254] A first determination module, configured to determine the similarity information of the current feature to be processed according to the processed features; wherein, the similarity information includes similar features;

[0255] A second determination module, configured to determine the reference information when processing the current feature to be processed based on the similarity information.

[0256] In an optional implementation manner of this embodiment, in the encoding process of the current feature to be processed, the processed features include the encoded features in the image to be processed; in the decoding process of the current feature to be processed, the processed features include the decoded features in the image to be processed.

[0257] In an optional implementation manner of this embodiment, the method further includes:

[0258] A first acquisition module, configured to acquire the image to be processed;

[0259] A compression module, configured to compress the image to be processed by using the encoding network in the first autoencoder model to obtain compressed features;

[0260] A quantization module, configured to quantize the compressed features to obtain image features corresponding to the image to be processed.

[0261] In an alternative implementation of this embodiment, the similarity information further includes the similarity between the currently to-be-processed feature and the similar feature.

[0262] In an alternative implementation of this embodiment, the first determination module includes:

[0263] A search sub-module, configured to search for the similar feature from the processed features by using the local information of the currently to-be-processed feature;

[0264] A first determination sub-module, configured to determine the similarity between the currently to-be-processed feature and the similar feature.

[0265] In an alternative implementation of this embodiment, the second determination module includes:

[0266] A first acquisition sub-module, configured to obtain first auxiliary information for determining the reference information according to the similar feature and the local information of the similar feature;

[0267] A second determination sub-module, configured to determine the reference information after performing a weighting operation on the first auxiliary information by using the similarity.

[0268] In an alternative implementation of this embodiment, the apparatus further includes:

[0269] A second acquisition module, configured to obtain second auxiliary information for determining the reference information according to the local information of the currently to-be-processed feature;

[0270] A third acquisition module, configured to obtain a first probability estimate of the currently to-be-processed feature in the image to be processed according to the second auxiliary information;

[0271] A fourth acquisition module, configured to obtain the confidence of the currently to-be-processed feature according to the first probability estimate.

[0272] In an alternative implementation of this embodiment, the second determination sub-module includes:

[0273] A combination sub-module, configured to combine the first auxiliary information and the second auxiliary information to obtain a first combination result;

[0274] A filtering sub-module, configured to filter the first combination result by using the similarity and the confidence to obtain a filtering result;

[0275] A second obtaining sub-module, configured to obtain a second probability estimate of the currently to-be-processed feature according to the filtering result and the first probability estimate.

[0276] In an alternative implementation of this embodiment, the apparatus further includes:

[0277] A fifth obtaining module, configured to obtain third auxiliary information for determining the reference information from the image features corresponding to the to-be-processed image by using a second autoencoder model;

[0278] A first combining module, configured to combine the first auxiliary information, the second auxiliary information, and the third auxiliary information to obtain a second combining result;

[0279] A sixth obtaining module, configured to obtain a third probability estimate of the currently to-be-processed feature by using the second combining result and the second probability estimate.

[0280] In an alternative implementation of this embodiment, the apparatus further includes:

[0281] A seventh obtaining module, configured to obtain a second encoded bitstream corresponding to the to-be-processed image;

[0282] An eighth obtaining module, configured to obtain third auxiliary information for determining the reference information from the second encoded bitstream by using a second autoencoder model;

[0283] A second combining module, configured to combine the first auxiliary information, the second auxiliary information, and the third auxiliary information to obtain a second combining result;

[0284] A ninth obtaining module, configured to obtain a third probability estimate of the currently to-be-processed feature by using the second combining result and the second probability estimate.

[0285] In an alternative implementation of this embodiment, the second determining sub-module further includes:

[0286] A third determining sub-module, configured to determine the reference information according to the third probability estimate.

[0287] The image processing apparatus in this embodiment corresponds to Figure 1 the image processing apparatus in the illustrated embodiment and related embodiments, and specific details can be referred to the description of Figure 1 the illustrated embodiment and related embodiments above, and will not be elaborated here.

[0288] An image encoding apparatus according to an embodiment of the present disclosure, the apparatus can be implemented as part or all of an electronic device through software, hardware, or a combination of both. The image encoding apparatus includes:

[0289] A tenth acquisition module, configured to acquire an image to be processed;

[0290] An eleventh acquisition module, configured to acquire, by using the above image processing method, reference information for encoding each feature in the image to be encoded;

[0291] An encoding module, configured to encode the image to be encoded by using the reference information.

[0292] The image processing device in this embodiment corresponds to the image encoding device in the Figure 5 illustrated embodiment and related embodiments. For specific details, reference may be made to the description of the Figure 5 illustrated embodiment and related embodiments above, which will not be elaborated herein.

[0293] An image decoding device according to an embodiment of the present disclosure. The device may be implemented as part or all of an electronic device through software, hardware, or a combination of both. The image decoding device includes:

[0294] A twelfth acquisition module, configured to acquire an encoded bitstream of an image to be decoded;

[0295] A thirteenth acquisition module, configured to acquire, by using the above image processing method, reference information for decoding each feature in the image to be decoded;

[0296] A decoding module, configured to decode the encoded bitstream by using the reference information.

[0297] The image processing device in this embodiment corresponds to the image processing device in the Figure 6 illustrated embodiment and related embodiments. For specific details, reference may be made to the description of the Figure 6 illustrated embodiment and related embodiments above, which will not be elaborated herein.

[0298] An image processing device according to another embodiment of the present disclosure. The device may be implemented as part or all of an electronic device through software, hardware, or a combination of both. The image processing device includes:

[0299] A fourteenth acquisition module, configured to acquire an encoded feature in the image to be encoded;

[0300] A first calling module, configured to call a preset service interface, so that the preset service interface determines similarity information of a currently to-be-processed feature in the image to be encoded according to the encoded feature, and determines, based on the similarity information, reference information for encoding the currently to-be-processed feature; wherein the similarity information includes similar features;

[0301] A first output module, configured to output the reference information.

[0302] The image processing device in this embodiment corresponds to the image processing devices in the Figure 7 illustrated embodiment and related embodiments. For specific details, reference can be made to the descriptions of the Figure 7 illustrated embodiment and related embodiments above, which will not be elaborated here.

[0303] According to another embodiment of the present disclosure, the image processing device can be implemented as part or all of an electronic device through software, hardware, or a combination of both. The image processing device includes:

[0304] A fifteenth acquisition module configured to acquire the decoded features in the image to be decoded;

[0305] A second call module configured to call a preset service interface so that the preset service interface determines similarity information of the currently to-be-processed feature in the image to be decoded according to the decoded features, and determines reference information for decoding the currently to-be-processed feature based on the similarity information; wherein the similarity information includes similar features;

[0306] A second output module configured to output the reference information.

[0307] The image processing device in this embodiment corresponds to the image processing devices in the Figure 8 illustrated embodiment and related embodiments. For specific details, reference can be made to the descriptions of the Figure 8 illustrated embodiment and related embodiments above, which will not be elaborated here.

[0308] According to another embodiment of the present disclosure, the image processing device can be implemented as part or all of an electronic device through software, hardware, or a combination of both. The image processing device includes:

[0309] A sixteenth acquisition module configured to acquire the image to be encoded;

[0310] A third call module configured to call a preset service interface so that the preset service interface determines similarity information of the currently to-be-processed feature in the image to be encoded according to the encoded features in the image to be encoded, and determines reference information for encoding the currently to-be-processed feature based on the similarity information, and encodes the currently to-be-processed feature using the reference information; wherein the similarity information includes similar features;

[0311] A third output module configured to output the encoding result.

[0312] The image processing apparatus in this embodiment corresponds to the image processing apparatus in the above related embodiments. For specific details, reference can be made to the description of the above related embodiments, which will not be elaborated here.

[0313] An image processing apparatus according to another embodiment of the present disclosure can be implemented as part or all of an electronic device through software, hardware, or a combination of both. The image processing apparatus includes:

[0314] A seventeenth acquisition module configured to acquire an encoded bitstream of an image to be decoded;

[0315] A fourth call module configured to call a preset service interface, so that the preset service interface determines similarity information of a currently to-be-processed feature in the image to be decoded based on the decoded features obtained from the encoded bitstream, and based on the similarity information, determines reference information for decoding the currently to-be-processed feature, and decodes the currently to-be-processed feature based on the reference information; the similarity information includes similar features;

[0316] A fourth output module configured to output a decoding result.

[0317] The image processing apparatus in this embodiment corresponds to the image processing apparatus in the above related embodiments. For specific details, reference can be made to the description of the above related embodiments, which will not be elaborated here.

[0318] Figure 9 It is a schematic structural diagram of an electronic device suitable for implementing an image processing method, an image encoding method, and / or an image decoding method according to an embodiment of the present disclosure.

[0319] As Figure 9 shown, the electronic device 900 includes a processing unit 901, which can be implemented as a processing unit such as a CPU, GPU, FPGA, NPU, etc. The processing unit 901 can execute various processes in the implementation manners of any of the above methods of the present disclosure according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage section 908 into a random access memory (RAM) 903. In the RAM 903, various programs and data required for the operation of the electronic device 900 are also stored. The processing unit 901, the ROM 902, and the RAM 903 are connected to each other through a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0320] The following components are connected to the I / O interface 905: an input section 906 including a keyboard, a mouse, etc.; an output section 907 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc. and a speaker, etc.; a storage section 908 including a hard disk, etc.; and a communication section 909 including a network interface card such as a LAN card, a modem, etc. The communication section 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to the I / O interface 905 as required. A removable medium 911, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 910 as required so that a computer program read from it can be installed into the storage section 908 as required.

[0321] Specifically, according to an embodiment of the present disclosure, any of the methods described above with reference to the embodiments of the present disclosure can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product that includes a computer program tangibly embodied on a machine-readable medium, the computer program including program code for performing any of the methods in the embodiments of the present disclosure. In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 909, and / or installed from the removable medium 911.

[0322] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code, and the module, program segment, or part of code includes one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0323] The units or modules involved in the embodiments described in the present disclosure can be implemented in software or in hardware. The described units or modules can also be provided in a processor, and the names of these units or modules do not constitute a limitation to the units or modules themselves in some cases.

[0324] As another aspect, the present disclosure also provides a computer-readable storage medium, which may be the computer-readable storage medium included in the device in the above-described embodiments; or it may exist separately and be a computer-readable storage medium not assembled into the device. The computer-readable storage medium stores one or more programs, and the one or more programs are used by one or more processors to execute the methods described in the present disclosure.

[0325] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles applied. Those skilled in the art should understand that the scope of the invention involved in the present disclosure is not limited to the technical solutions formed by the specific combination of the above technical features, and should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the inventive concept. For example, the technical solutions formed by mutually replacing the above features with the technical features (but not limited to) having similar functions disclosed in the present disclosure.

Claims

1. An image processing method, wherein, it includes: obtaining the processed features in the image to be processed; performing a global search on the processed features to obtain similarity information of the currently to-be-processed feature in the image to be processed; wherein, the similarity information includes similar features; in the encoding process of the currently to-be-processed feature, the processed features include the encoded features in the image to be processed; in the decoding process of the currently to-be-processed feature, the processed features include the decoded features in the image to be processed; determining reference information for processing the currently to-be-processed feature based on the similarity information; determining reference information for processing the currently to-be-processed feature based on the similarity information includes: obtaining auxiliary information for determining the reference information according to the similar features, the currently to-be-processed feature, and an autoencoder model, wherein the auxiliary information at least includes first auxiliary information, second auxiliary information, and third auxiliary information for determining the reference information; determining a probability estimate of the currently to-be-processed feature according to the auxiliary information, and determining the reference information based on the probability estimate.

2. The method according to claim 1, wherein, the method further includes: obtaining the image to be processed; compressing the image to be processed by using an encoding network in a first autoencoder model to obtain compressed features; quantizing the compressed features to obtain image features corresponding to the image to be processed.

3. The method according to claim 1, wherein, the similarity information further includes the similarity between the currently to-be-processed feature and the similar feature.

4. The method according to claim 3, wherein, performing a global search on the processed features to obtain similarity information of the currently to-be-processed feature in the image to be processed includes: searching for the similar features from the processed features by using local information of the currently to-be-processed feature; determining the similarity between the currently to-be-processed feature and the similar feature.

5. The method according to claim 4, wherein, determining reference information for processing the currently to-be-processed feature based on the similarity information includes: obtaining first auxiliary information for determining the reference information according to the similar features and local information of the similar features; determining the reference information after performing a weighting operation on the first auxiliary information by using the similarity.

6. The method according to claim 5, wherein, the method further includes: obtaining second auxiliary information for determining the reference information according to local information of the currently to-be-processed feature; obtaining a first probability estimate of the currently to-be-processed feature in the image features corresponding to the image to be processed according to the second auxiliary information; obtaining the confidence of the currently to-be-processed feature according to the first probability estimate.

7. The method according to claim 6, wherein, determining the reference information after performing a weighting operation on the first auxiliary information by using the similarity includes: combining the first auxiliary information and the second auxiliary information to obtain a first combination result; Filter the first combination result using the similarity and the confidence level to obtain a filtered result; Obtain a second probability estimate for the currently to-be-processed feature according to the filtered result and the first probability estimate.

8. The method according to claim 7, wherein, the method further comprises: Using a second autoencoder model to obtain third auxiliary information for determining the reference information from the image features corresponding to the to-be-processed image; Combine the first auxiliary information, the second auxiliary information, and the third auxiliary information to obtain a second combination result; Use the second combination result and the second probability estimate to obtain a third probability estimate for the currently to-be-processed feature.

9. The method according to claim 7, wherein, the method further comprises: Obtain a second encoded bitstream corresponding to the to-be-processed image; Use a second autoencoder model to obtain third auxiliary information for determining the reference information from the second encoded bitstream; Combine the first auxiliary information, the second auxiliary information, and the third auxiliary information to obtain a second combination result; Use the second combination result and the second probability estimate to obtain a third probability estimate for the currently to-be-processed feature.

10. The method according to claim 8 or 9, wherein, After performing a weighting operation on the first auxiliary information using the similarity to determine the reference information, it further comprises: Determine the reference information according to the third probability estimate.

11. An image encoding method, comprising: Obtain an image to be encoded; Use the method according to any one of claims 1, 2 - 8, 10 to obtain reference information for encoding each feature in the image to be encoded; Encode the image to be encoded using the reference information.

12. An image decoding method, comprising: Obtain the encoded bitstream of the image to be decoded; Use the method according to any one of claims 1, 2 - 7, 9 - 10 to obtain reference information for decoding each feature in the image to be decoded; Decode the encoded bitstream using the reference information.

13. An image processing method, comprising: Obtain the encoded features in the image to be encoded; Call a preset service interface so that the preset service interface performs a global search on the encoded features to obtain similarity information of the currently to-be-processed feature in the image to be encoded, and determine reference information for encoding the currently to-be-processed feature based on the similarity information; wherein the similarity information includes similar features; Output the reference information; Determining reference information for encoding the currently to-be-processed feature based on the similarity information includes: obtaining auxiliary information for determining the reference information according to the similar features, the currently to-be-processed feature, and an autoencoder model, where the auxiliary information at least includes first auxiliary information, second auxiliary information, and third auxiliary information for determining the reference information; determining a probability estimate of the currently to-be-processed feature according to the auxiliary information, and determining the reference information based on the probability estimate.

14. An image processing method, comprising: Obtain the decoded features in the image to be decoded; Invoke a preset service interface to globally search for the decoded feature through the preset service interface, obtain similarity information of the currently to-be-processed feature in the to-be-decoded image, and determine reference information for decoding the currently to-be-processed feature based on the similarity information; wherein the similarity information includes similar features; Output the reference information; Determining reference information for decoding the currently to-be-processed feature based on the similarity information includes: obtaining auxiliary information for determining the reference information according to the similar features, the currently to-be-processed feature, and an autoencoder model, where the auxiliary information at least includes first auxiliary information, second auxiliary information, and third auxiliary information for determining the reference information; determining a probability estimate of the currently to-be-processed feature according to the auxiliary information, and determining the reference information based on the probability estimate.

15. An image processing method, including: Obtain an image to be encoded; Invoke a preset service interface to globally search for the encoded feature in the image to be encoded through the preset service interface, obtain similarity information of the currently to-be-processed feature in the image to be encoded, and determine reference information for encoding the currently to-be-processed feature based on the similarity information, and encode the currently to-be-processed feature using the reference information; wherein the similarity information includes similar features; Output an encoding result; Determining reference information for encoding the currently to-be-processed feature based on the similarity information includes: obtaining auxiliary information for determining the reference information according to the similar features, the currently to-be-processed feature, and an autoencoder model, where the auxiliary information at least includes first auxiliary information, second auxiliary information, and third auxiliary information for determining the reference information; determining a probability estimate of the currently to-be-processed feature according to the auxiliary information, and determining the reference information based on the probability estimate.

16. An image processing method, including: Obtain the encoded bitstream of the image to be decoded; Invoke a preset service interface to globally search for the decoded feature obtained from the encoded bitstream through the preset service interface, obtain similarity information of the currently to-be-processed feature in the image to be decoded, and determine reference information for decoding the currently to-be-processed feature based on the similarity information, and decode the currently to-be-processed feature based on the reference information; The similarity information includes similar features; Output a decoding result; Determining reference information for decoding the currently to-be-processed feature based on the similarity information includes: obtaining auxiliary information for determining the reference information according to the similar features, the currently to-be-processed feature, and an autoencoder model, where the auxiliary information at least includes first auxiliary information, second auxiliary information, and third auxiliary information for determining the reference information; determining a probability estimate of the currently to-be-processed feature according to the auxiliary information, and determining the reference information based on the probability estimate.

17. An image processing apparatus, wherein, including: A feature acquisition module configured to acquire the processed feature in the image to be processed; A first determination module, configured to perform a global search on the processed features to obtain similarity information of a currently to-be-processed feature; wherein, the similarity information includes similar features; in the encoding process of the currently to-be-processed feature, the processed features include the encoded features in the to-be-processed image; in the decoding process of the currently to-be-processed feature, the processed features include the decoded features in the to-be-processed image; A second determination module, configured to determine reference information for processing the currently to-be-processed feature based on the similarity information, including: obtaining auxiliary information for determining the reference information according to the similar features, the currently to-be-processed feature, and an autoencoder model, wherein the auxiliary information at least includes a first auxiliary information, a second auxiliary information, and a third auxiliary information for determining the reference information; determining a probability estimate of the currently to-be-processed feature according to the auxiliary information, and determining the reference information based on the probability estimate.

18. An image encoding device, comprising: A tenth acquisition module, configured to acquire a to-be-encoded image; An eleventh acquisition module, configured to acquire reference information for encoding each feature in the to-be-encoded image by using the method according to claim 11; An encoding module, configured to encode the to-be-encoded image by using the reference information.

19. An image decoding device, comprising: A twelfth acquisition module, configured to acquire an encoded bitstream of a to-be-decoded image; A thirteenth acquisition module, configured to acquire reference information for decoding each feature in the to-be-decoded image by using the method according to claim 12; A decoding module, configured to decode the encoded bitstream by using the reference information.

20. An image processing device, comprising: A fourteenth acquisition module, configured to acquire encoded features in a to-be-encoded image; A first invocation module, configured to invoke a preset service interface, so that the preset service interface performs a global search on the encoded features to obtain similarity information of a currently to-be-processed feature in the to-be-encoded image, and determine reference information for encoding the currently to-be-processed feature based on the similarity information; wherein, the similarity information includes similar features; determining reference information for encoding the currently to-be-processed feature based on the similarity information includes: obtaining auxiliary information for determining the reference information according to the similar features, the currently to-be-processed feature, and an autoencoder model, wherein the auxiliary information at least includes a first auxiliary information, a second auxiliary information, and a third auxiliary information for determining the reference information; determining a probability estimate of the currently to-be-processed feature according to the auxiliary information, and determining the reference information based on the probability estimate; A first output module, configured to output the reference information.

21. An image processing device, comprising: A fifteenth acquisition module, configured to acquire decoded features in a to-be-decoded image; A second calling module, configured to call a preset service interface, so that the preset service interface globally searches for the decoded features to obtain similarity information of the currently to-be-processed feature in the to-be-decoded image, and based on the similarity information, determines reference information for decoding the currently to-be-processed feature; wherein the similarity information includes similar features; based on the similarity information, determining reference information for decoding the currently to-be-processed feature includes: obtaining auxiliary information for determining the reference information according to the similar features, the currently to-be-processed feature, and an autoencoder model, wherein the auxiliary information at least includes first auxiliary information, second auxiliary information, and third auxiliary information for determining the reference information; determining a probability estimate of the currently to-be-processed feature according to the auxiliary information, and determining the reference information based on the probability estimate; A second output module, configured to output the reference information.

22. An image processing apparatus, comprising: A sixteenth obtaining module, configured to obtain an image to be encoded; A third calling module, configured to call a preset service interface, so that the preset service interface globally searches for the encoded features in the image to be encoded to obtain similarity information of the currently to-be-processed feature in the image to be encoded, and based on the similarity information, determines reference information for encoding the currently to-be-processed feature, and encodes the currently to-be-processed feature by using the reference information; wherein the similarity information includes similar features; based on the similarity information, determining reference information for encoding the currently to-be-processed feature includes: obtaining auxiliary information for determining the reference information according to the similar features, the currently to-be-processed feature, and an autoencoder model, wherein the auxiliary information at least includes first auxiliary information, second auxiliary information, and third auxiliary information for determining the reference information; determining a probability estimate of the currently to-be-processed feature according to the auxiliary information, and determining the reference information based on the probability estimate; A third output module, configured to output an encoding result.

23. An image processing apparatus, comprising: A seventeenth obtaining module, configured to obtain an encoded bitstream of an image to be decoded; A fourth calling module, configured to call a preset service interface, so that the preset service interface globally searches for the decoded features obtained from the encoded bitstream to obtain similarity information of the currently to-be-processed feature in the image to be decoded, and based on the similarity information, determines reference information for decoding the currently to-be-processed feature, and decodes the currently to-be-processed feature based on the reference information; The similarity information includes similar features; Determine reference information for decoding the currently to-be-processed feature based on the similarity information, including: obtaining auxiliary information for determining the reference information according to the similarity feature, the currently to-be-processed feature, and the autoencoder model, where the auxiliary information at least includes first auxiliary information, second auxiliary information, and third auxiliary information for determining the reference information; determining a probability estimate of the currently to-be-processed feature according to the auxiliary information, and determining the reference information based on the probability estimate; A fourth output module, configured to output a decoding result.

24. An electronic device, wherein, including a memory and a processor; wherein, the memory is used to store one or more computer instructions, and the one or more computer instructions are executed by the processor to implement the method according to any one of claims 1-16.

25. A computer-readable storage medium, on which computer instructions are stored, wherein, when the computer instructions are executed by a processor, the method according to any one of claims 1-16 is implemented.

Images