Data processing method and device, equipment and storage medium

Abstract

The embodiment of the invention discloses a data processing method and device, equipment and a storage medium. The method comprises the steps of obtaining coding information of a coding unit in an image; determining correlation characteristics between the coding unit and candidate compensation modes in a candidate compensation mode set according to the coding information; selecting a candidate compensation mode matched with the coding unit from the candidate compensation mode set according to the correlation characteristics; performing compensation processing on the reconstructed pixel value of the coding unit by adopting the selected candidate compensation mode; the reconstructed pixel value of the coding unit is obtained by performing coding reconstruction processing on the original pixel value of the coding unit. According to the invention, the compensation processing complexity of the image can be reduced, and the coding efficiency of the image is improved.

Description

Technical Field

[0001] The present application relates to fields such as Internet technologies, and in particular to a data processing method, device, equipment and storage medium. Background Art

[0002] In the process of multimedia data processing, the encoding device encodes, transforms, and quantizes the original image to obtain an encoded image. Then, the encoded image is dequantized, detransformed, and predicted and compensated to obtain a reconstructed image. Compared with the original image, due to the influence of quantization, some information of the reconstructed image is different from that of the original image, resulting in distortion of the reconstructed image; therefore, compensation processing is required to reduce the degree of distortion of the reconstructed image. In practice, it is found that the current compensation processing method for reconstructing images is relatively complex, resulting in relatively low image coding efficiency. Summary of the invention

[0003] The embodiments of the present application provide a data processing method, apparatus, device and storage medium to reduce the complexity of image compensation processing and improve image coding efficiency.

[0004] An embodiment of the present application provides a data processing method, including:

[0005] Obtaining encoding information of a coding unit in an image;

[0006] Determine, according to the coding information, a correlation feature between the coding unit and a candidate compensation mode in a candidate compensation mode set;

[0007] According to the correlation feature, selecting a candidate compensation mode matching the coding unit from the candidate compensation mode set;

[0008] The selected candidate compensation mode is used to perform compensation processing on the reconstructed pixel values ​​of the coding unit; the reconstructed pixel values ​​of the coding unit are obtained by performing encoding and reconstruction processing on the original pixel values ​​of the coding unit.

[0009] An embodiment of the present application provides a data processing device, including:

[0010] An acquisition module, used for acquiring encoding information of encoding units in an image;

[0011] A determination module, configured to determine, according to the encoding information, a correlation feature between the encoding unit and a candidate compensation mode in a candidate compensation mode set;

[0012] A selection module, configured to select a candidate compensation mode matching the coding unit from the candidate compensation mode set according to the correlation feature;

[0013] The compensation module is used to adopt the selected candidate compensation mode to perform compensation processing on the reconstructed pixel value of the above coding unit; the reconstructed pixel value of the above coding unit is obtained by encoding and reconstructing the original pixel value of the above coding unit.

[0014] Optionally, the candidate compensation modes in the above candidate compensation mode set include a boundary compensation mode;

[0015] The selection and determination module determines the correlation characteristics between the coding unit and the candidate compensation mode in the candidate compensation mode set according to the coding information, including:

[0016] Obtaining, from the encoding information of the encoding unit of the image, the encoding modes corresponding to the N encoding sub-units of the encoding unit, respectively; N is a positive integer greater than 1;

[0017] From the N coding sub-units, K coding sub-units whose coding mode is a target coding mode are selected; K is a natural number; the target coding mode is a coding mode for encoding a motion vector corresponding to the image and a reference image of the image using an index;

[0018] According to the K coding sub-units, a correlation feature between the boundary compensation mode and the coding unit is determined.

[0019] Optionally, the determining module determines, according to the K coding sub-units, a correlation feature between the boundary compensation mode and the coding unit, including:

[0020] Obtain the sizes corresponding to the K encoding sub-units and the size of the encoding unit;

[0021] The sizes corresponding to the K encoding sub-units are accumulated to obtain a total size;

[0022] According to the total size and the size of the coding unit, a correlation characteristic between the boundary compensation mode and the coding unit is determined.

[0023] Optionally, the determining module determines the correlation feature between the boundary compensation mode and the coding unit according to the total size and the size of the coding unit, including:

[0024] Determining a size ratio between the total size and the size of the coding unit;

[0025] If the size ratio is less than or equal to the size ratio threshold, generating a correlation feature indicating that there is a correlation between the coding unit and the boundary compensation mode;

[0026] If the size ratio is greater than the size ratio threshold, a correlation feature is generated indicating that there is no correlation between the coding unit and the boundary compensation mode.

[0027] Optionally, the candidate compensation mode in the above candidate compensation mode set includes a sideband compensation mode;

[0028] The determination module determines, according to the encoding information, a correlation feature between the encoding unit and a candidate compensation mode in the candidate compensation mode set, including:

[0029] Obtaining S reconstructed pixel values ​​of the coding unit from the coding information; S is an integer greater than 1;

[0030] Determine the pixel value intervals to which the S reconstructed pixel values ​​respectively belong;

[0031] According to the pixel value intervals to which the S reconstructed pixel values ​​respectively belong, identifying the P stripes to which the S reconstructed pixel values ​​belong; one stripe corresponds to one pixel value interval, and P is a positive integer;

[0032] According to the P stripes to which the S reconstructed pixel values ​​belong, a correlation feature between the coding unit and the sideband compensation mode is determined.

[0033] Optionally, the determination module determines the correlation characteristics between the coding unit and the sideband compensation mode according to the P stripes to which the S reconstructed pixel values ​​belong, including:

[0034] Counting the number of pixel values ​​corresponding to the reconstructed pixel values ​​belonging to the stripe i among the S reconstructed pixel values ​​and the total number of pixel values ​​of the reconstructed pixel values ​​in the coding unit; i is a positive integer less than or equal to P;

[0035] The P strips are sorted in descending order according to the number of pixel values ​​corresponding to the P strips, so as to obtain an arrangement order of the P strips;

[0036] From the P strips, determine the Q strips that are in the first Q order, and accumulate the numbers of pixel values ​​corresponding to the Q strips to obtain an accumulated number of pixel values;

[0037] According to the total number of pixel values ​​and the accumulated number of pixel values, a correlation characteristic between the encoding unit and the sideband compensation mode is determined.

[0038] Optionally, the determination module determines the correlation feature between the coding unit and the side compensation mode according to the total number of pixel values ​​and the accumulated number of pixel values, including:

[0039] Determine the ratio between the accumulated pixel value number and the total pixel value number;

[0040] If the quantity ratio is greater than a quantity ratio threshold, generating a correlation feature indicating that there is a correlation between the coding unit and the sideband compensation mode;

[0041] If the quantity ratio is less than or equal to the quantity ratio threshold, a correlation feature is generated indicating that there is no correlation between the coding unit and the side compensation mode.

[0042] Optionally, the candidate compensation modes in the above candidate compensation mode set include a boundary compensation mode, a sideband compensation mode and a non-compensation mode;

[0043] The selection module selects a candidate compensation mode matching the coding unit from the candidate compensation mode set according to the correlation feature, including:

[0044] If the correlation feature indicates that there is a correlation between the coding unit and the boundary compensation mode, determining the boundary compensation mode as a candidate compensation mode matching the coding unit;

[0045] If the correlation feature indicates that there is a correlation between the coding unit and the sideband compensation mode, determining the sideband compensation mode as a candidate compensation mode matching the coding unit;

[0046] If the correlation feature indicates that the coding unit is correlated with both the sideband compensation mode and the boundary compensation mode, determining a candidate compensation mode matching the coding unit from the boundary compensation mode and the boundary compensation mode;

[0047] If the correlation feature indicates that there is no correlation between the coding unit and either the sideband compensation mode or the boundary compensation mode, the non-compensation mode is determined as a candidate compensation mode matching the coding unit.

[0048] Optionally, the selection module determines a candidate compensation mode matching the coding unit from the boundary compensation mode and the boundary compensation mode, including:

[0049] determining a first rate-distortion performance of the coding unit when the coding unit is compensated using the boundary compensation mode;

[0050] Determining a second rate-distortion performance of the coding unit when the sideband compensation mode is used to perform compensation processing on the coding unit;

[0051] If the first rate-distortion performance is less than the second rate-distortion performance, determining the boundary compensation mode as a candidate compensation mode matching the coding unit;

[0052] If the first rate-distortion performance is greater than the second rate-distortion performance, determining the sideband compensation mode as a candidate compensation mode matching the coding unit;

[0053] If the first rate-distortion performance is equal to the second rate-distortion performance, the sideband compensation mode or the boundary compensation mode is determined as a candidate compensation mode matching the coding unit.

[0054] Optionally, the compensation module adopts the selected candidate compensation mode to perform compensation processing on the reconstructed pixel value of the coding unit, including:

[0055] If the selected compensation mode is the boundary compensation mode, then the adjacent reconstructed pixel values ​​of the S reconstructed pixel values ​​of the coding unit in M ​​directions are obtained; S and M are both integers greater than 1;

[0056] Compare the reconstructed pixel value j with the adjacent reconstructed pixel values ​​of the reconstructed pixel value j in the M directions to obtain a comparison result; j is a positive integer less than or equal to S;

[0057] According to the comparison result, determining the pixel category of the reconstructed pixel value j;

[0058] According to the pixel categories to which the above S reconstructed pixel values ​​respectively belong, the pixel compensation values ​​corresponding to the above S reconstructed pixel values ​​are determined, and the pixel compensation values ​​corresponding to the above S reconstructed pixel values ​​are sent to the decoding device; one pixel category corresponds to one pixel compensation value.

[0059] Optionally, the compensation module adopts the selected candidate compensation mode to perform compensation processing on the reconstructed pixel value of the coding unit, including:

[0060] If the selected compensation mode is the sideband compensation mode, then obtain the P stripes to which the S reconstructed pixel values ​​of the coding unit belong; S and P are both integers greater than 1;

[0061] According to the pixel value intervals corresponding to the P strips, the P strips are divided into F strip groups;

[0062] From the F stripe groups, select a target stripe group whose rate-distortion performance is the minimum rate-distortion performance;

[0063] Obtain pixel compensation values ​​corresponding to the strips in the target strip group;

[0064] The pixel compensation value corresponding to the strip in the target strip group and the starting position of the target strip group are sent to a decoding device.

[0065] On one hand, an embodiment of the present application provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0066] On one hand, an embodiment of the present application provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the above method are implemented.

[0067] On the one hand, an embodiment of the present application provides a computer program product, including a computer program, which implements the steps of the above method when executed by a processor.

[0068] In the present application, based on the coding information of the coding unit, the correlation characteristics between the coding unit and the candidate compensation mode are analyzed, and based on the correlation characteristics, the candidate compensation mode matching the coding unit is quickly decided; there is no need to traverse the compensation results of each candidate compensation mode for the coding unit in turn, and the candidate compensation mode with the lowest rate-distortion performance is selected based on the coding result as the candidate compensation mode matching the coding unit, thereby reducing the complexity of the compensation processing of the image and improving the coding efficiency of the image. At the same time, the coding information of the coding units with different image contents is inconsistent, that is, the correlation characteristics reflect the correlation between the image content of the coding unit and the candidate compensation mode to a certain extent, that is, the correlation characteristics between the image content of the coding unit and the candidate compensation mode are fully considered, and the candidate compensation mode matching the coding unit is adaptively selected based on the correlation characteristics, thereby improving the accuracy of selecting the candidate compensation mode, and the coding units based on different image contents all have stable compensation effects, that is, the present application has good robustness. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0070] Figure 1 is a flow chart of video processing provided by this application;

[0071] Figure 2 It is a flowchart of a method for processing multimedia data provided by the present application;

[0072] Figure 3is a schematic diagram of a coding unit provided by the present application;

[0073] Figure 4 It is a flowchart of a data processing method provided by this application;

[0074] Figure 5 This is a schematic diagram of a scenario for determining a correlation feature between a boundary compensation mode and a coding unit provided by the present application;

[0075] Figure 6 This is a schematic diagram of a scenario for determining a correlation characteristic between a sideband compensation mode and a coding unit provided by the present application;

[0076] Figure 7 It is a flowchart of a data processing method provided by this application;

[0077] Figure 8 It is a flowchart of a data processing method provided by this application;

[0078] Fig. 9 is a structural schematic diagram of a data processing device provided in an embodiment of the present application;

[0079] Fig.10 It is a structural diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0080] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0081] The embodiments of the present application relate to multimedia data processing technology. Among them, multimedia data (or media data) refers to composite data formed by media data such as text, graphics, images, sounds, animations, moving images, etc. that are interrelated in content, and the multimedia data mentioned in the embodiments of the present application mainly includes image data composed of images, or video data composed of images and sounds. The processing process of multimedia data involved in the embodiments of the present application mainly includes media data acquisition, media data encoding, media data file encapsulation, media data file transmission, media data decoding and final data presentation. When the multimedia data is video data, then the complete processing process for the video data can be as follows. Figure 1 As shown, it may specifically include: video acquisition, video encoding, video file encapsulation, video transmission, video file decapsulation, video decoding and final video presentation.

[0082] Video capture is used to convert analog video into digital video and save it in the format of digital video files. That is to say, video capture can convert video signals into binary digital information, where the binary information converted from the video signal is a binary data stream, which can also be called the code stream or bit stream (Bitstream) of the video signal. Video encoding converts the original video format file into another video format file through compression technology. The generation of video media content mentioned in the embodiments of the present application includes real scenes captured by cameras and screen content scenes generated by computers. From the perspective of the method of obtaining video signals, video signals can be divided into two methods: those captured by cameras and those generated by computers. Due to different statistical characteristics, the corresponding compression encoding methods may also be different. Modern mainstream video coding technology, taking the international video coding standard HEVC (High Efficiency Video Coding, international video coding standard HEVC / H.265), VVC (versatile video coding, international video coding standard VVC / H.266), and video coding standard AVS (Audio Video Coding Standard, AVS), or AVS3 (the third-generation video coding standard launched by the AVS standards group) as examples, adopts a hybrid coding framework and performs the following series of operations and processing on the input original video signal, which can be specifically as follows: Figure 2 As shown: ① Block partition structure: The input multimedia data frame (such as a video frame in the video data) is divided into several non-overlapping processing units according to the size of a, and each processing unit will perform similar compression operations. In one embodiment, this processing unit is called CTU (Coding Tree Unit) or LCU (Largest Coding Unit). Among them, CTU can be further divided into more refined units to obtain one or more basic coding units, which are called CU (Coding Unit). Each CU is the most basic element in a coding link. In another embodiment, this processing unit is also called a coding tile (Tile, a rectangular area of ​​a multimedia data frame that can be independently decoded and encoded). Among them, the coding tile can be further divided into more refined areas to obtain one or more maximum coding blocks (SB: Superblock, the starting point of block division, which can be further divided into multiple sub-blocks), and then the maximum coding block is further divided to obtain one or more image blocks (block, B). Each image block is the most basic element in a coding link.

[0083] Among them, the relationship between LCU (or CTU) and CU can be as follows Figure 3 As shown, from Figure 3 It can be seen that a frame image in multimedia data includes one or more maximum coding units, and a maximum coding unit includes at least two coding units. It is understandable that in the process of encoding multimedia data, the encoding device performs block processing on the image frame in the multimedia data, then a coding unit in this application may refer to a maximum coding unit (LCU) or coding tree unit (CTU) in a frame image of multimedia data, and a coding sub-unit may refer to a coding unit (CU).

[0084] ② Predictive Coding: It includes intra-frame prediction and inter-frame prediction. The original video signal is predicted by the selected coded video signal to obtain the residual video signal. The encoder needs to select the most suitable one among many possible predictive coding modes for the current coding unit and inform the decoder.

[0085] a. Intra (picture) Prediction: The predicted signal comes from the area that has been encoded and reconstructed in the same image.

[0086] b. Inter-frame prediction (Inter (picture) Prediction: Inter-frame prediction): The predicted signal comes from other images that have already been encoded and are different from the current image (called reference images).

[0087] ③ Transform coding and quantization (Transform & Quantization): The residual video signal is transformed into the transform domain through DFT (Discrete Fourier Transform), DCT (Discrete Cosine Transform, a subset of DFT) and other transformation operations. The signal is called transform coefficient. The signal in the transform domain is further subjected to lossy quantization operations, which loses certain information, making the quantized signal conducive to compression expression.

[0088] In some video coding standards, there may be more than one transform to choose from. Therefore, the encoder also needs to select one of the transforms for the current coding unit and inform the decoder. The degree of quantization is usually determined by the quantization parameter (QP). A larger QP value means that coefficients with a larger value range will be quantized to the same output, which usually results in greater distortion and a lower bit rate. On the contrary, a smaller QP value means that coefficients with a smaller value range will be quantized to the same output, which usually results in less distortion and a higher bit rate.

[0089] ④ Entropy Coding or Statistical Coding: The quantized transform domain signal will be statistically compressed and encoded according to the frequency of each value, and finally output as a binary (0 or 1) compressed bit stream. At the same time, the encoding generates other information, such as the selected mode, motion vector, etc., which also needs to be entropy encoded to reduce the bit rate.

[0090] Statistical coding is a lossless coding method that can effectively reduce the bit rate required to express the same signal. Common statistical coding methods include variable length coding (VLC) or context-based binary arithmetic coding (CABAC).

[0091] ⑤ Loop Filtering: The encoded image (i.e. multimedia data frame) can be reconstructed into a decoded image after inverse quantization, inverse transformation and prediction compensation (the reverse operation of ② to ④ above). Compared with the original image, the reconstructed image has some information different from the original image due to the influence of quantization, resulting in distortion. Filtering operations such as deblocking, SAO (Sample Adaptive Offset) or ALF (Adaptive Loop Filter) on the reconstructed image can effectively reduce the degree of distortion caused by quantization. Since these filtered reconstructed images will be used as a reference for subsequent encoded images and used to predict future signals, the above filtering operations are also called loop filtering and filtering operations within the encoding loop.

[0092] Figure 2 The basic flow of a video encoder is shown in Figure 2 The kth CU (marked as S k [x, y]) is used as an example, where k is a positive integer greater than or equal to 1 and less than or equal to the number of CUs in the current input image, S k [x, y] represents the pixel with coordinates [x, y] in the kth CU, where x represents the horizontal coordinate of the pixel and y represents the median coordinate of the pixel. k [x, y] is obtained through a better processing such as motion compensation or intra-frame prediction to obtain a prediction signal S k [x, y] and Subtract the residual signal U k [x, y], and then the residual signal U k[x, y] is transformed and quantized. The quantized output data has two different destinations: one is to send it to an entropy encoder for entropy encoding. The encoded bit stream is output to a buffer for storage and waiting to be transmitted; the other application is to obtain the signal U' after inverse quantization and inverse transformation. k [x, y]. Transform the signal U' k [x, y] and Add together to get the new prediction signal S* k [x, y], and S* k [x, y] is sent to the current image buffer for storage. S* k [x, y] is obtained through intra-image prediction f(S* k [x, y]), S* k [x, y] is obtained after loop filtering S' k [x, y], and S' k [x, y] is sent to the decoded image buffer for storage to generate the reconstructed video. S' k [x, y] is obtained after motion-compensated prediction S' r [x+m x , y+m y ], S' r [x+m x , y+m y ] represents the reference block, m x and m y Represent the horizontal and vertical components of the motion vector respectively.

[0093] After encoding the multimedia data, the encoded data stream needs to be encapsulated and transmitted to the user. Video file encapsulation refers to storing the encoded and compressed video and audio in a file in a certain format according to a packaging format (or container, or file container). Common packaging formats include AVI format (Audio Video Interleaved) or ISOBMFF (ISO Based Media File Format, a media file format based on ISO (International Standard Organization) standards), among which ISOBMFF is the packaging standard for media files. The most typical ISOBMFF file is the MP4 (Moving Picture Experts Group 4) file.

[0094] The encapsulated file will be transmitted to the decoding device (i.e., the user terminal) through the video transmission, and the decoding device can present the final video content in the decoding device after performing inverse operations such as decapsulation and decoding. Among them, the encapsulated file can be sent to the decoding device through a transmission protocol, and the transmission protocol can be, for example, DASH (dynamic adaptive streaming over HTTP, dynamic adaptive streaming over HTTP is an adaptive bitrate streaming technology). Using DASH for transmission allows high-quality streaming media to be transmitted over the Internet through traditional HTTP network servers. In DASH, MPD (media presentation description, media presentation description signaling in DASH) is used to describe media segment information, and in DASH, a combination of one or more media components, such as a video file of a certain resolution, can be regarded as a Representation, and the multiple Representations contained can be regarded as an Adaptation Set (a set of video streams), and a DASH can contain one or more Adaptation Sets.

[0095] It can be understood that the file decapsulation process of the decoding device is the opposite of the above-mentioned file encapsulation process. The decoding device can decapsulate the encapsulated file according to the file format requirements at the time of encapsulation to obtain audio code streams and video code streams. The decoding process of the decoding device is also the opposite of the encoding process. The decoding device can decode the audio code stream and restore the audio content. According to the above encoding process, it can be seen that at the decoding end, for each CU, after the decoder obtains the compressed code stream, it first performs entropy decoding to obtain various mode information and quantized transform coefficients. Each coefficient is dequantized and inversely transformed to obtain a residual signal. On the other hand, based on the known encoding mode information, the prediction signal corresponding to the CU can be obtained. After adding the two, the reconstructed signal can be obtained. Finally, the reconstructed value of the decoded image needs to undergo a loop filtering operation to generate the final output signal.

[0096] The present application mainly involves the filtering operation based on SAO in the above step ⑤. SAO is a video coding technology. This technology adds appropriate pixel compensation values ​​to the reconstructed pixel values ​​after inverse quantization and inverse transformation, so that the reconstructed pixel values ​​of the coding unit are closer to the original pixel values, reducing compression distortion. In the current mainstream generations of coding standards, including H.265 and H.266, SAO is supported. As a part of the loop filtering process, the basic principle of SAO is to modify the reconstructed pixel values ​​after inverse quantization and inverse transformation. SAO is performed in units of CTU. A CTU contains a lot of reconstructed pixel values, and it is not practical to encode a pixel compensation value for each reconstructed pixel. SAO will classify the reconstructed pixel values ​​in a CTU according to the distribution of the reconstructed pixel values ​​or the size of the reconstructed pixel values, and then encode a shared pixel compensation value for each type of reconstructed pixel value. Depending on the classification method of the reconstructed pixel values, SAO can have a boundary compensation mode (Edge Offset, EO) and a sideband compensation mode (Band Offset, BO). In particular, the current CTU can also choose not to perform SAO. Therefore, in the SAO mode, a CTU can have three options: EO, BO and no compensation mode.

[0097] In the actual encoding process, the encoding device does not know which SAO mode is most suitable, and will traverse the three modes of EO, BO and non-compensation mode in turn, perform compensation processing on the encoding unit respectively, obtain the compensation result, and determine the compensation mode with the best rate-distortion performance of each compensation mode based on the compensation result as the final compensation mode. The process of determining the final compensation mode has a high computational complexity, because there are also many settings inside EO and BO that need to be determined in a similar traversal manner. For example, when statistically reconstructing the distribution of pixel values, EO has four directions to choose from, and the specific direction to be used must also be traversed in turn, and then selected according to the rate-distortion performance. High encoding complexity means that encoding requires more computing resources and longer encoding time, which is very unfriendly to high transcoding load services such as video on demand. For low-latency services such as live broadcasts, high encoding complexity will directly reduce the user's service experience. Therefore, it is very meaningful to make some quick judgments on SAO and reduce its computational complexity ratio.

[0098] Based on this, in this application, a data processing method is proposed, in which the correlation characteristics between the coding unit and the candidate compensation mode are analyzed based on the coding information of the coding unit, and the candidate compensation mode matching the coding unit is quickly decided based on the correlation characteristics; there is no need to traverse the compensation results of each candidate compensation mode for the coding unit in turn, and the candidate compensation mode with the lowest rate-distortion performance is selected based on the coding result as the candidate compensation mode matching the coding unit, thereby reducing the complexity of the compensation processing for the image, as well as reducing the encoding time for the image and the computing resources required for encoding, thereby improving the encoding efficiency of the image. In particular, it can improve the image compensation effect in high transcoding load services such as video on demand and low-latency services such as live broadcast, and improve the user's service experience.

[0099] For further information, see Figure 4 , is a flow chart of a data processing method provided in an embodiment of the present application. Figure 4 As shown, the method may be performed by an encoding device, which may include at least one of a terminal and a server. The method may include the following steps:

[0100] S101: Obtain coding information of a coding unit in an image.

[0101] In the present application, the encoding device may obtain the encoding information of the encoding unit in the image, and the encoding information may refer to the encoding mode of the encoding subunit in the encoding unit, and one or more of the reconstructed pixel values ​​of the encoding unit, and the encoding mode refers to the prediction encoding mode mentioned above. The encoding mode of the encoding subunit in the encoding unit is determined based on the image content of the encoding unit, such as the encoding mode of the encoding subunit may be determined based on the similarity between the image content of the encoding unit and the image content of the adjacent encoding unit, or the encoding mode of the encoding subunit may be determined based on the similarity between the image content of the image to which the encoding unit belongs and the image content of the adjacent frame. The reconstructed pixel value of the encoding unit is obtained by encoding and reconstructing the original pixel value of the encoding unit, and the original pixel value of the encoding unit can reflect the image content of the encoding unit. Therefore, it can be said that the encoding information of the encoding unit is determined based on the image content of the encoding unit, and encoding units with different image contents have different encoding information.

[0102] Among them, the prediction coding mode can include intra-frame prediction and inter-frame prediction; intra-frame prediction means that the prediction residual information and the intra-frame motion vector corresponding to the coding unit need to be encoded; inter-frame prediction mainly includes Inter mode, Merge mode and Skip mode. Inter mode requires inter-frame motion search for the coding unit to obtain the motion vector corresponding to the best matching coding unit, and then encode the motion vector and prediction residual information; Merge mode does not require inter-frame motion search, directly uses the motion vector of the adjacent coding unit that has been encoded in the spatial domain and time domain, and only needs to encode the usage index and prediction residual information corresponding to the motion vector; Skip mode only needs to encode the usage index of the motion vector of the adjacent coding unit, does not need to encode the prediction residual information, and directly uses the predicted pixel value as the reconstructed pixel value during decoding. As can be seen from the above, intra-frame prediction, Inter mode, and Merge mode all need to encode the prediction residual information of the coding unit, and the encoded prediction residual information needs to be transformed, quantized, and dequantized, which will cause some information of the coding unit (such as high-frequency information) to be lost. The Skip mode does not need to encode the prediction residual information of the coding unit, and therefore, will not cause the information of the coding unit (such as high-frequency information) to be lost.

[0103] S102: Determine, according to the encoding information, a correlation feature between the encoding unit and a candidate compensation mode in a candidate compensation mode set.

[0104] In the present application, the correlation feature is used to reflect whether the coding unit is correlated with the candidate compensation mode. If the coding unit is correlated with the candidate compensation mode, it indicates that the coding unit is suitable for compensation processing with the corresponding candidate compensation mode; if the coding unit is not correlated with the candidate compensation mode, it indicates that the coding unit is not suitable for compensation processing with the corresponding candidate compensation mode. The candidate compensation mode set includes multiple candidate compensation modes, and the candidate compensation modes may include but are not limited to: boundary compensation mode, sideband compensation mode and non-compensation mode.

[0105] Optionally, the encoding device may sequentially traverse the correlation features between the boundary compensation mode and the sideband compensation mode in the candidate compensation mode set and the coding unit. The correlation feature may include a first correlation feature, a second correlation feature, and a third correlation feature. The first correlation feature reflects whether the coding unit has a correlation with the boundary compensation mode, the second correlation feature reflects whether the coding unit has a correlation with the sideband compensation mode, and the third correlation feature reflects whether the coding unit has a correlation with the non-compensation mode. Specifically, the encoding device may first traverse the correlation feature between the coding unit and the boundary compensation mode according to the encoding information, and mark it as the first correlation feature. The encoding device may traverse the correlation feature between the coding unit and the sideband compensation mode according to the encoding information, and mark it as the second correlation feature. If the second correlation feature indicates that there is a correlation between the coding unit and the sideband compensation mode, and / or if the first correlation feature indicates that there is a correlation between the coding unit and the boundary compensation mode, the traversal is terminated; if the second correlation feature indicates that there is no correlation between the coding unit and the sideband compensation mode, and the first correlation feature indicates that there is no correlation between the coding unit and the boundary compensation mode, the encoding device may generate a third correlation feature indicating that the coding unit has a correlation with the non-compensation mode. The present application does not need to try to use each candidate compensation mode to compensate the coding unit in turn, and selects the final candidate compensation mode based on the compensation result. It only needs to identify the correlation between the coding unit and the candidate compensation mode, reduce the complexity and time consumption of selecting the candidate compensation mode, and then reduce the complexity of the compensation processing of the image, and reduce the encoding time and computing resources required for encoding, thereby improving the encoding efficiency of the image. At the same time, the encoding information of the coding unit is determined according to the image content of the coding unit, that is, the correlation feature between the coding unit and the candidate compensation mode can be said to be determined according to the correlation between the image content of the coding unit and the candidate compensation mode, which can improve the accuracy of obtaining the candidate compensation mode that matches the coding unit, and then indicate the compensation effect of the image.

[0106] Optionally, the encoding device may first traverse the second correlation feature between the encoding unit and the sideband compensation mode according to the encoding information, and the encoding device may traverse the first correlation feature between the encoding unit and the boundary compensation mode according to the encoding information. If the second correlation feature indicates that there is a correlation between the encoding unit and the sideband compensation mode, and / or if the first correlation feature indicates that there is a correlation between the encoding unit and the boundary compensation mode, the traversal is terminated; if the second correlation feature indicates that there is no correlation between the encoding unit and the sideband compensation mode, and the first correlation feature indicates that there is no correlation between the encoding unit and the boundary compensation mode, the encoding device may generate a third correlation feature indicating that the encoding unit is correlated with the non-compensation mode. The present application does not need to try to use each candidate compensation mode to compensate the encoding unit in turn, and select the final candidate compensation mode based on the compensation result. It only needs to identify the correlation between the encoding unit and the candidate compensation mode, reduce the complexity and time consumption of selecting the candidate compensation mode, and then reduce the complexity of the compensation processing of the image, and reduce the encoding time and computing resources required for encoding, thereby improving the encoding efficiency of the image. At the same time, the encoding information of the encoding unit is determined according to the image content of the encoding unit, that is, the correlation feature between the encoding unit and the candidate compensation mode is determined according to the correlation between the image content of the encoding unit and the candidate compensation mode. This can improve the accuracy of obtaining the candidate compensation mode that matches the encoding unit, and then prompt the compensation effect of the image.

[0107] It should be noted that the boundary compensation mode can be used to reduce the ringing effect of the coding unit. Since the coding unit obtains the prediction residual information after intra-frame prediction or inter-frame prediction, the prediction residual information will lose some high-frequency information of the coding unit after transformation, quantization, inverse quantization and other operations. For example, the more high-frequency classifications of the prediction residual information or the larger the quantization parameter of the coding unit, the more high-frequency information the coding unit loses. The effect of the high-frequency information loss of the coding unit on the decoded image is the ringing effect. Therefore, the boundary compensation mode has limited effect on the scene where the prediction residual information does not have obvious high-frequency information loss. From the above analysis, it can be seen that the coding mode of the coding unit determines whether the high-frequency information of the coding unit is lost. Then, the coding device can determine the correlation characteristics between the coding unit and the boundary compensation mode based on the coding mode in the coding information.

[0108] Optionally, the candidate compensation modes in the candidate compensation mode set include boundary compensation modes; the above-mentioned determining the correlation characteristics between the coding unit and the candidate compensation modes in the candidate compensation mode set based on the coding information includes: the coding device can obtain the coding modes corresponding to the N coding sub-units of the coding unit from the coding information of the coding unit of the image; N is a positive integer greater than 1; from the N coding sub-units, K coding sub-units whose coding modes are the target coding modes are screened out; K is a natural number; the target coding mode is a coding mode corresponding to the motion vector between the image and the reference image of the above-mentioned image, which is encoded using an index, such as the target coding mode can refer to the Skip mode mentioned above. Further, the coding device can determine the correlation characteristics between the boundary compensation mode and the coding unit based on the K coding sub-units. Since the Skip mode will not cause the high-frequency components of the coding unit to be lost, the K coding sub-units refer to the coding sub-units in which the high-frequency components of the coding unit will not be lost. If the number of K coding sub-units is larger, or the size corresponding to the K coding sub-units is larger, it indicates that there are fewer coding sub-units with lost high-frequency components in the coding unit. At this time, the coding unit is not suitable for compensation processing using the boundary compensation mode, which can quickly exclude the boundary compensation mode; if the number of K coding sub-units is smaller, or the size corresponding to the K coding sub-units is smaller, it indicates that there are more coding sub-units with lost high-frequency components in the coding unit. At this time, the coding unit is suitable for compensation processing using the boundary compensation mode, which is conducive to selecting a suitable compensation mode for the coding unit to avoid loss of high-frequency components of the coding unit.

[0109] Optionally, the above-mentioned determining the correlation characteristics between the boundary compensation mode and the coding unit according to the K coding sub-units includes: the coding device can obtain the sizes corresponding to the K coding sub-units and the size of the coding unit, and perform accumulation processing on the sizes corresponding to the K coding sub-units to obtain the total size; then, according to the total size and the size of the coding unit, determine the correlation characteristics between the boundary compensation mode and the coding unit. The size can refer to the area, such as the size of the coding sub-unit can refer to the area of ​​the coding sub-unit, and the size of the coding unit can refer to the area of ​​the coding unit.

[0110] Optionally, the above-mentioned determination of the correlation feature between the boundary compensation mode and the above-mentioned coding unit according to the total size and the size of the coding unit includes: the coding device can determine the size ratio between the total size and the size of the coding unit, that is, the total size divided by the size of the coding unit, to obtain the size ratio. If the size ratio is less than or equal to the size ratio threshold, it indicates that the Skip mode is used for encoding in a small number of areas in the coding unit, and there is no high-frequency component loss in a small number of areas in the coding unit, that is, it indicates that there is a high-frequency component loss in a large number of areas in the coding unit. At this time, the coding device can generate a correlation feature indicating that there is a correlation between the coding unit and the boundary compensation mode, which is conducive to the subsequent use of the boundary compensation mode to compensate the coding unit and reduce the high-frequency component loss of the coding unit. If the above-mentioned size ratio is greater than the above-mentioned size ratio threshold, it indicates that the Skip mode is used for encoding in a large number of areas in the coding unit, and there is no high-frequency component loss in a large number of areas in the coding unit, that is, it indicates that there is a high-frequency component loss in a small number of areas in the coding unit. At this time, the encoding device can generate a correlation feature indicating that there is no correlation between the encoding unit and the boundary compensation mode, which is conducive to quickly excluding the boundary compensation mode and improving the efficiency of selecting the candidate compensation mode matching the encoding unit.

[0111] For example, Figure 5 As shown, the coding unit 50 includes a coding subunit 51a, a coding subunit 521b, a coding subunit 522b, a coding subunit 523b, a coding subunit 524b, a coding subunit 53a, and a coding subunit 54a, wherein the coding subunit 521b, the coding subunit 522b, the coding subunit 523b, and the coding subunit 524b are also referred to as coding subunits of the coding subunit 52a. The coding modes of the coding subunit 51a, the coding subunit 524b, the coding subunit 53a, and the coding subunit 54a are all Skip modes, and the coding modes of the coding subunit 521b, the coding subunit 522b, the coding subunit 523b, and the coding subunit 524b are non-Skip modes. Therefore, the encoding device can perform cumulative processing on the sizes corresponding to the encoding subunit 51a, the encoding subunit 524b, the encoding subunit 53a, and the encoding subunit 54a, respectively, to obtain the total size, obtain the size of the encoding unit 50, and determine the correlation feature between the encoding unit and the boundary compensation mode according to the size of the encoding unit and the total size. For example, if the size is the area, the area ratio between the total area and the area of ​​the encoding unit is 0.8125, and thus, the encoding device can determine that there is no correlation between the encoding unit and the boundary compensation mode, that is, generate a correlation feature indicating that there is no correlation between the encoding unit and the boundary compensation mode, so as to quickly exclude the boundary compensation mode.

[0112] Optionally, the above-mentioned determining the correlation feature between the boundary compensation mode and the above-mentioned coding unit according to the total size and the size of the above-mentioned coding unit includes: the coding device can obtain the size difference between the size of the coding unit and the total size, that is, the size of the coding unit minus the total size to obtain the size difference. If the size difference is greater than the size difference threshold, it indicates that the Skip mode is used for encoding in a small number of areas in the coding unit, and there is no high-frequency component loss in a small number of areas in the coding unit, that is, it indicates that there is a high-frequency component loss in a large number of areas in the coding unit. At this time, the coding device can generate a correlation feature indicating that there is a correlation between the coding unit and the boundary compensation mode, which is conducive to the subsequent use of the boundary compensation mode to compensate the coding unit and reduce the high-frequency component loss of the coding unit. If the size difference is less than or equal to the above-mentioned size difference threshold, it indicates that the Skip mode is used for encoding in a large number of areas in the coding unit, and there is no high-frequency component loss in a large number of areas in the coding unit, that is, it indicates that there is a high-frequency component loss in a small number of areas in the coding unit. At this time, the coding device can generate a correlation feature indicating that there is no correlation between the coding unit and the boundary compensation mode, which is conducive to quickly excluding the boundary compensation mode and improving the efficiency of selecting the candidate compensation mode matching the coding unit.

[0113] Optionally, the above-mentioned determination of the correlation feature between the boundary compensation mode and the coding unit according to the K coding sub-units includes: the coding device can obtain the number of sub-units corresponding to the K coding sub-units (that is, the number of sub-units is K), and the total number of sub-units in the coding unit, and obtain the number ratio between the number of sub-units and the total number of sub-units. If the number ratio between the number of sub-units and the total number of sub-units is less than or equal to the preset number ratio, it indicates that a small number of coding sub-units in the coding unit are encoded using the Skip mode, and a small number of coding sub-units in the coding unit do not have high-frequency component loss, that is, it indicates that a large number of coding sub-units in the coding unit have high-frequency component loss. At this time, the coding device can generate a correlation feature indicating that there is a correlation between the coding unit and the boundary compensation mode, which is conducive to the subsequent use of the boundary compensation mode to compensate the coding unit and reduce the high-frequency component loss of the coding unit. If the number ratio between the number of sub-units and the total number of sub-units is greater than the preset number ratio, it indicates that a large number of coding sub-units in the coding unit are encoded using the Skip mode, and a large number of coding sub-units in the coding unit do not have high-frequency component loss, that is, it indicates that a small number of coding sub-units in the coding unit have high-frequency component loss. At this time, the encoding device can generate a correlation feature indicating that there is no correlation between the encoding unit and the boundary compensation mode, which is conducive to quickly excluding the boundary compensation mode and improving the efficiency of selecting the candidate compensation mode matching the encoding unit.

[0114] Optionally, the encoding device may also obtain the number of subunits corresponding to the K encoding subunits (i.e., the number of subunits is K), and the total number of subunits in the encoding unit, and subtract the number of subunits from the total number of subunits to obtain the difference in the number of subunits, and determine the correlation between the encoding unit and the boundary compensation mode according to the difference in the number of subunits. If the difference in the number of subunits is greater than the difference threshold, it indicates that a small number of encoding subunits in the encoding unit are encoded using the Skip mode, and a small number of encoding subunits in the encoding unit do not have high-frequency component loss, that is, it indicates that a large number of encoding subunits in the encoding unit have high-frequency component loss. The encoding device may generate a correlation feature indicating that there is a correlation between the encoding unit and the boundary compensation mode, which is conducive to the subsequent use of the boundary compensation mode to compensate the encoding unit and reduce the high-frequency component loss of the encoding unit. If the difference in the number of subunits is less than or equal to the difference threshold, it indicates that a large number of encoding subunits in the encoding unit are encoded using the Skip mode, and a large number of encoding subunits in the encoding unit do not have high-frequency component loss, that is, it indicates that a small number of encoding subunits in the encoding unit have high-frequency component loss. At this time, the encoding device can generate a correlation feature indicating that there is no correlation between the encoding unit and the boundary compensation mode, which is conducive to quickly excluding the boundary compensation mode and improving the efficiency of selecting the candidate compensation mode matching the encoding unit.

[0115] It should be noted that the sideband compensation mode is mainly applicable to compensating the reconstructed pixel values ​​within a certain pixel value interval, that is, to perform an overall offset on the reconstructed pixel values ​​within the pixel value interval to solve the problem of excessive quantization error of the low-frequency component of the coding unit. The sideband compensation mode is applicable to compensating the coding units with relatively concentrated colors, and the sideband compensation mode is not applicable to compensating the coding units with unconcentrated colors. The color concentration of the coding unit can be determined based on the reconstructed pixel value of the coding unit. Therefore, the coding device can determine the correlation characteristics between the coding unit and the sideband compensation mode based on the reconstructed pixel value of the coding unit.

[0116] Optionally, the candidate compensation modes in the above-mentioned candidate compensation mode set include a sideband compensation mode; step S102 includes: the encoding device can obtain S reconstructed pixel values ​​of the encoding unit from the encoding information; S is an integer greater than 1, and the pixel value intervals to which the S reconstructed pixel values ​​respectively belong are determined; according to the pixel value intervals to which the S reconstructed pixel values ​​respectively belong, the P strips to which the S reconstructed pixel values ​​belong are identified; one strip corresponds to one pixel value interval, and P is a positive integer; then, according to the P strips to which the S reconstructed pixel values ​​belong, the correlation characteristics between the encoding unit and the sideband compensation mode can be determined. The P strips to which the S reconstructed pixel values ​​belong reflect the color concentration of the coding unit. If the S reconstructed pixel values ​​belong to certain strips, it indicates that the color concentration of the coding unit is relatively high. At this time, the coding unit is more suitable for compensation processing using the sideband compensation mode, which is conducive to solving the problem of excessive quantization error of the low-frequency component of the coding unit; if the S reconstructed pixel values ​​are dispersed and belong to multiple strips, that is, the number of reconstructed pixel values ​​belonging to a single strip is relatively small; at this time, the coding unit is not suitable for compensation processing using the sideband compensation mode; this is conducive to quickly excluding the boundary compensation mode and improving the efficiency of selecting the candidate compensation mode that matches the coding unit.

[0117] Optionally, the above-mentioned determination of the correlation characteristics between the coding unit and the sideband compensation mode based on the P stripes to which the above-mentioned S reconstructed pixel values ​​belong includes: the coding device can count the number of pixel values ​​corresponding to the reconstructed pixel values ​​belonging to stripe i among the S reconstructed pixel values, and the total number of pixel values ​​of the reconstructed pixel values ​​in the coding unit; i is a positive integer less than or equal to P; the P stripes are sorted in order from large to small according to the number of pixel values ​​corresponding to the P stripes, so as to obtain the arrangement order of the P stripes. Further, from the P stripes, the Q stripes with the first Q arrangement order are determined, that is, the Q stripes with the most reconstructed pixel values ​​are selected from the P strips, and Q can be preset, such as 4; the number of pixel values ​​corresponding to the Q stripes are accumulated to obtain the accumulated number of pixel values. Further, based on the total number of pixel values ​​and the accumulated number of pixel values, the correlation characteristics between the coding unit and the sideband compensation mode are determined.

[0118] For example, Figure 6As shown, there are 32 pixel value intervals, and the pixel value interval can be expressed as [8c, 8c+7], where c belongs to [0, 31]. For example, the pixel value interval includes [0, 7], [8, 15], ..., [240, 247], [248, 255]. One pixel value interval corresponds to one stripe, such as [0, 7] corresponds to stripe 1, [8, 15] corresponds to stripe 2, ..., [240, 247] corresponds to stripe 31, and [248, 255] corresponds to stripe 32. The encoding device can obtain the pixel value interval to which the S reconstructed pixel values ​​of the encoding unit belong, and determine the stripes to which the S reconstructed pixel values ​​belong respectively according to the pixel value interval to which the S reconstructed pixel values ​​belong, and the S reconstructed pixel values ​​are respectively reconstructed pixel value 1, reconstructed pixel value 2, ..., reconstructed pixel value S-1, and reconstructed pixel value S. According to the strips to which the S reconstructed pixel values ​​belong, the number of pixel values ​​corresponding to the reconstructed pixel values ​​belonging to strips 1, strips 2, ..., strips 31, and strips 32 is counted, and the number of pixel values ​​corresponding to strips 1, strips 2, ..., strips 31, and strips 32 are obtained as pixel value number 1a, pixel value number 2a, ..., pixel value number 31a, and pixel value number 32a. The strips are sorted in order from high to low according to the number of pixel values, and the strips with the first 4 sorting orders are selected. The number of pixel values ​​corresponding to the strips with the first 4 sorting orders are accumulated to obtain the accumulated number of pixel values. For example, the strips with the first 4 sorting orders are strips 1, strips 10, strips 15, and strips 16, respectively. The number of pixel values ​​corresponding to strips 1a, the number of pixel values ​​corresponding to strips 10, the number of pixel values ​​corresponding to strips 15a, and the number of pixel values ​​corresponding to strips 16a are accumulated to obtain the accumulated number of pixel values. The number of pixel values ​​corresponding to strips 1 to 32 are accumulated to obtain the total number of pixel values, and the correlation characteristics between the side compensation mode and the coding unit are determined according to the accumulated number of pixel values ​​and the total number of pixel values.

[0119] Optionally, the above-mentioned determination of the correlation feature between the coding unit and the sideband compensation mode based on the total number of pixel values ​​and the accumulated number of pixel values ​​includes: the coding device can determine the number ratio between the accumulated number of pixel values ​​and the total number of pixel values; if the number ratio is greater than the number ratio threshold, a large number of reconstructed pixel values ​​in the coding unit are concentrated in the pixel value interval corresponding to the Q stripes, that is, the color concentration of the coding unit is relatively high, and the coding device can generate a correlation feature indicating that there is a correlation between the above coding unit and the sideband compensation mode, which is conducive to the subsequent use of the sideband compensation mode to compensate the coding unit and reduce the loss of high-frequency components of the coding unit. If the number ratio is less than or equal to the number ratio threshold, a small number of reconstructed pixel values ​​in the coding unit are concentrated in the pixel value interval corresponding to the Q stripes, that is, the color concentration of the coding unit is relatively low. At this time, the coding device can generate a correlation feature indicating that there is no correlation between the coding unit and the sideband compensation mode, which is conducive to quickly excluding the sideband compensation mode and improving the efficiency of selecting a candidate compensation mode that matches the coding unit.

[0120] S103: Select, according to the correlation feature, a candidate compensation mode that matches the coding unit from the candidate compensation mode set.

[0121] In the present application, the coding unit can select a candidate compensation mode that matches the coding unit from the candidate compensation mode set based on the correlation feature, that is, based on the correlation feature, select a candidate compensation mode that is correlated with the coding unit from the candidate compensation mode set, and determine the candidate compensation mode that is correlated with the coding unit as the candidate compensation mode that matches the coding unit; the candidate compensation mode that matches the coding unit may refer to the optimal compensation mode of the coding unit obtained from the candidate compensation mode set.

[0122] S104 , using the selected candidate compensation mode to perform compensation processing on the reconstructed pixel values ​​of the coding unit; the reconstructed pixel values ​​of the coding unit are obtained by performing encoding and reconstruction processing on the original pixel values ​​of the coding unit.

[0123] In the present application, the encoding device may adopt the selected candidate compensation mode to perform compensation processing on the reconstructed pixel value of the encoding unit. For example, when the selected candidate compensation mode is the boundary compensation mode, the boundary compensation mode may be adopted to determine the pixel compensation value corresponding to the reconstructed pixel value of the encoding unit, and the pixel compensation value is sent to the decoding device; or, when the selected candidate compensation mode is the sideband compensation mode, the sideband compensation mode may be adopted to determine the pixel compensation value corresponding to the reconstructed pixel value of the encoding unit, and the pixel compensation value is sent to the decoding device. When the selected candidate compensation mode is the non-compensation mode, the reconstructed pixel value of the encoding unit may not be compensated.

[0124] It should be noted that if both EO and BO are determined not to be the best compensation methods, that is, the coding unit has no correlation with the boundary compensation mode and the sideband compensation mode, the coding device may not perform SAO operation on the coding unit CTU. It should be pointed out that not performing SAO operation does not necessarily mean that the rate-distortion performance is worse than that of performing SAO operation. EO and BO themselves need to consume additional bit resources, such as encoding pixel compensation values. If the distortion of the coding unit is not significantly improved after compensation, their rate-distortion performance may be worse than that of not performing SAO operation.

[0125] In the present application, based on the coding information of the coding unit, the correlation characteristics between the coding unit and the candidate compensation mode are analyzed, and based on the correlation characteristics, the candidate compensation mode matching the coding unit is quickly decided; there is no need to traverse the compensation results of each candidate compensation mode for the coding unit in turn, and the candidate compensation mode with the lowest rate-distortion performance is selected based on the coding result as the candidate compensation mode matching the coding unit, thereby reducing the complexity of the compensation processing of the image and improving the coding efficiency of the image. At the same time, the coding information of the coding units with different image contents is inconsistent, that is, the correlation characteristics reflect the correlation between the image content of the coding unit and the candidate compensation mode to a certain extent, that is, the correlation characteristics between the image content of the coding unit and the candidate compensation mode are fully considered, and the candidate compensation mode matching the coding unit is adaptively selected based on the correlation characteristics, thereby improving the accuracy of selecting the candidate compensation mode, and the coding units based on different image contents all have stable compensation effects, that is, the present application has good robustness.

[0126] For further information, see Figure 7 , is a flow chart of a data processing method provided in an embodiment of the present application. Figure 7 As shown, the method may be performed by an encoding device, which may include at least one of a terminal and a server. The method may include the following steps:

[0127] S201. Obtain coding information of a coding unit in an image.

[0128] S202. Determine, based on the coding information, a correlation feature between the coding unit and a candidate compensation mode in a candidate compensation mode set; the candidate compensation modes in the candidate compensation mode set include a boundary compensation mode, a sideband compensation mode, and a non-compensation mode.

[0129] S203: If the correlation feature indicates that there is a correlation between the coding unit and the boundary compensation mode, determine the boundary compensation mode as a candidate compensation mode matching the coding unit.

[0130] S204: If the correlation feature indicates that there is a correlation between the coding unit and the sideband compensation mode, determine the sideband compensation mode as a candidate compensation mode matching the coding unit.

[0131] S205: If the correlation feature indicates that there is no correlation between the coding unit and either the sideband compensation mode or the boundary compensation mode, then the non-compensation mode is determined as a candidate compensation mode matching the coding unit.

[0132] S206: If the correlation feature indicates that the coding unit is correlated with both the sideband compensation mode and the boundary compensation mode, determine a candidate compensation mode matching the coding unit from the boundary compensation mode and the boundary compensation mode.

[0133] Optionally, steps S203 to S205 include: if the correlation feature indicates that there is a correlation between the coding unit and the boundary compensation mode, that is, the first correlation feature in the correlation feature indicates that there is a correlation between the coding unit and the boundary compensation mode, then the boundary compensation mode is determined as a candidate compensation mode that matches the coding unit; if the correlation feature indicates that there is a correlation between the above coding unit and the above sideband compensation mode, that is, the second correlation feature in the correlation feature indicates that there is a correlation between the above coding unit and the sideband compensation mode, then the above sideband compensation mode is determined as a candidate compensation mode that matches the above coding unit. If the correlation feature indicates that there is no correlation between the above coding unit and both the sideband compensation mode and the above boundary compensation mode, that is, the third correlation feature in the correlation feature indicates that there is a correlation between the above coding unit and the non-compensation mode, then the non-compensation mode is determined as a candidate compensation mode that matches the coding unit. If the correlation feature indicates that there is a correlation between the coding unit and both the sideband compensation mode and the boundary compensation mode, the first correlation feature in the correlation feature indicates that there is a correlation between the coding unit and the boundary compensation mode, and the second correlation feature in the correlation feature indicates that there is a correlation between the coding unit and the sideband compensation mode, then a candidate compensation mode matching the coding unit is determined from the boundary compensation mode and the boundary compensation mode.

[0134] Optionally, the determining of the candidate compensation mode matching the coding unit from the boundary compensation mode and the boundary compensation mode includes: the computer device can calculate, according to the rate-distortion performance formula, the first rate-distortion performance of the coding unit when the boundary compensation mode is used to compensate the coding unit, the first rate-distortion performance reflects the degree of distortion of the coding unit caused by the sampling boundary compensation mode when the coding unit is compensated, if the first rate-distortion performance is lower, it indicates that the degree of distortion of the coding unit caused by the sampling boundary compensation mode when the coding unit is compensated, and the coding unit is suitable for compensation by the boundary compensation mode. On the contrary, if the first rate-distortion performance is higher, it indicates that the degree of distortion of the coding unit caused by the sampling boundary compensation mode when the coding unit is compensated, and the coding unit is not suitable for compensation by the boundary compensation mode. The computer device can calculate the second rate-distortion performance of the coding unit when the sideband compensation mode is used to compensate the coding unit according to the rate-distortion performance formula; the second rate-distortion performance reflects the degree of distortion of the coding unit caused by the sampling sideband compensation mode when the coding unit is compensated. If the second rate-distortion performance is lower, it indicates that the degree of distortion of the coding unit caused by the sampling sideband compensation mode is lower, and the coding unit is suitable for the sideband compensation mode to be used for compensation. On the contrary, if the second rate-distortion performance is higher, it indicates that the degree of distortion of the coding unit caused by the sampling sideband compensation mode when the coding unit is compensated is higher, and the coding unit is not suitable for the sideband compensation mode to be used for compensation. Therefore, if the first rate-distortion performance is less than the second rate-distortion performance, the boundary compensation mode is determined as a candidate compensation mode that matches the coding unit; if the first rate-distortion performance is greater than the second rate-distortion performance, the sideband compensation mode is determined as a candidate compensation mode that matches the coding unit; if the first rate-distortion performance is equal to the second rate-distortion performance, the sideband compensation mode or the boundary compensation mode is determined as a candidate compensation mode that matches the coding unit. When the coding unit is correlated with both the boundary compensation mode and the sideband compensation mode, the candidate compensation mode with the lowest rate-distortion performance is selected through the rate-distortion performance as the candidate compensation mode that matches the coding unit, which can reduce the distortion degree of the coding unit and improve the compensation processing effect for the coding unit.

[0135] S207 . Use the selected candidate compensation mode to perform compensation processing on the reconstructed pixel values ​​of the coding unit; the reconstructed pixel values ​​of the coding unit are obtained by performing encoding and reconstruction processing on the original pixel values ​​of the coding unit.

[0136] Optionally, the above-mentioned use of the selected candidate compensation mode to compensate the reconstructed pixel values ​​of the above-mentioned coding unit includes: if the selected compensation mode is a boundary compensation mode, then the adjacent reconstructed pixel values ​​of the S reconstructed pixel values ​​of the coding unit in M ​​directions are obtained; S and M are both integers greater than 1; the M directions may include the horizontal direction (i.e. EO_0), vertical direction (EO_1), 135° direction (EO_2), and 45° direction (EO_3) of the reconstructed pixel value. Further, the reconstructed pixel value j is compared with the adjacent reconstructed pixel values ​​of the reconstructed pixel value j in the M directions to obtain a comparison result; j is a positive integer less than or equal to S; then, according to the comparison result, the pixel category of the reconstructed pixel value j is determined; according to the pixel categories to which the S reconstructed pixel values ​​respectively belong, the pixel compensation values ​​corresponding to the S reconstructed pixel values ​​are determined. That is, one direction corresponds to one comparison result, one type of comparison result corresponds to one pixel category, and one pixel category corresponds to one or more pixel compensation values. The encoding device can obtain the rate-distortion performance corresponding to the M directions when the compensation values ​​corresponding to the M directions are used to compensate the encoding unit, and the pixel compensation value corresponding to the direction with the lowest rate-distortion performance is used to determine the reconstructed pixel value with the same pixel category as the reconstructed pixel value j among the S reconstructed pixel values, and the pixel compensation value corresponding to the reconstructed pixel j. The pixel compensation values ​​corresponding to the S reconstructed pixel values ​​are sent to the decoding device; one pixel category corresponds to one pixel compensation value. By using the boundary compensation mode to compensate the encoding unit, it is beneficial to reduce the loss of high-frequency components of the encoding unit and improve the compensation effect of the image.

[0137] Optionally, the above-mentioned adopting the selected candidate compensation mode to compensate the reconstructed pixel values ​​of the coding unit includes: if the selected compensation mode is the sideband compensation mode, then obtaining the P stripes to which the S reconstructed pixel values ​​of the coding unit belong; S and P are both integers greater than 1; further, the P stripes can be divided into F stripe groups according to the pixel value intervals corresponding to the P stripes; a stripe group includes stripes corresponding to 4 continuous pixel value intervals, and a target stripe group with a minimum rate-distortion performance is selected from the F stripe groups, and the rate-distortion performance corresponding to the stripe group is the rate-distortion performance of the coding unit when the coding unit is compensated according to the pixel compensation value corresponding to the stripe group. Then, the pixel compensation value corresponding to the stripe in the target stripe group is obtained, and the pixel compensation value corresponding to the stripe in the target stripe group and the starting position of the target stripe group are sent to the decoding device.

[0138] Optionally, the above-mentioned obtaining of the pixel compensation value corresponding to the strip in the target strip group includes: the encoding device can obtain the pixel difference between the reconstructed pixel value of the strip in the target strip group and the corresponding original pixel value, and determine the pixel compensation value corresponding to the strip in the target strip group according to the pixel difference; for example, the pixel difference between the reconstructed pixel value and the original pixel value in the strip in the target strip group can be accumulated to obtain a total pixel difference, and the total pixel difference is determined as the pixel compensation value corresponding to the strip in the target strip group. By adopting the boundary compensation mode to perform compensation processing on the coding unit, it is beneficial to reduce the quantization error of the low-frequency component in the coding unit and improve the compensation effect of the image.

[0139] For example, in an implementation such as Figure 8 As shown, the data processing method in the present application may include steps s81 to s88; s81, the encoding device may calculate the coverage area ratio of the Skip mode, that is, the encoding device may calculate the area ratio (that is, the size ratio) between the area of ​​the encoding sub-unit whose encoding mode is the Skip mode in the encoding unit (that is, the total size mentioned above) and the area of ​​the encoding unit (that is, the size of the encoding unit mentioned above), and obtain the area ratio. s82, whether the area ratio is greater than T1, if so, execute step s83; if not, execute step s84; s83, determine whether there is a correlation with the boundary compensation mode, if the area ratio is greater than T1, determine that the encoding unit is correlated with the boundary compensation mode, and generate a first correlation feature indicating that the encoding unit is correlated with the boundary compensation mode, if the area ratio is less than or equal to T1, generate a first correlation feature indicating that the encoding unit is not correlated with the boundary compensation mode, skip the boundary compensation mode, that is, exclude the boundary compensation mode. s84, determine the stripe to which the reconstructed pixel value belongs, and refer to the description of the above embodiment. s85. Whether the total number of pixels of the top 4 strips accounts for more than T2; if so, execute step s86; if not, execute step s87; T2 is the quantity ratio threshold above, and the total number of pixels accounts for the ratio between the accumulated pixel value number and the total pixel value number.

[0140] Further, s86, correlation with the sideband compensation mode; if the total number of pixels is greater than T2, it is determined that the coding unit is correlated with the sideband compensation mode, and a second correlation feature indicating that the coding unit is correlated with the sideband compensation mode is generated; if the total number of pixels is less than or equal to T2, it is determined that the coding unit is not correlated with the sideband compensation mode, and a second correlation feature indicating that the coding unit is not correlated with the sideband compensation mode is generated. s87, whether the sideband compensation mode and the boundary compensation mode are not correlated, if so, execute step s88; if the first correlation feature indicates that the coding unit is not correlated with the boundary compensation mode, and the second correlation feature indicates that the coding unit is not correlated with the sideband compensation mode, execute s88 and do not perform compensation processing on the coding unit. If the first correlation feature indicates that the coding unit is correlated with the boundary compensation mode, and the second correlation feature indicates that the coding unit is not correlated with the sideband compensation mode, the boundary compensation mode is used to perform compensation processing on the coding unit. If the first correlation feature indicates that the coding unit is not correlated with the boundary compensation mode, and the second correlation feature indicates that the coding unit is correlated with the sideband compensation mode, the sideband compensation mode is used to perform compensation processing on the coding unit. If the first correlation feature indicates that the coding unit is correlated with the boundary compensation mode, and the second correlation feature indicates that the coding unit is correlated with the sideband compensation mode, a candidate compensation mode matching the coding unit is selected from the boundary compensation mode and the sideband compensation mode, and the coding unit is compensated using the selected candidate compensation mode.

[0141] In the present application, based on the coding information of the coding unit, the correlation characteristics between the coding unit and the candidate compensation mode are analyzed, and based on the correlation characteristics, the candidate compensation mode matching the coding unit is quickly decided; there is no need to traverse the compensation results of each candidate compensation mode for the coding unit in turn, and the candidate compensation mode with the lowest rate-distortion performance is selected based on the coding result as the candidate compensation mode matching the coding unit, thereby reducing the complexity of the compensation processing of the image and improving the coding efficiency of the image. At the same time, the coding information of the coding units with different image contents is inconsistent, that is, the correlation characteristics reflect the correlation between the image content of the coding unit and the candidate compensation mode to a certain extent, that is, the correlation characteristics between the image content of the coding unit and the candidate compensation mode are fully considered, and the candidate compensation mode matching the coding unit is adaptively selected based on the correlation characteristics, thereby improving the accuracy of selecting the candidate compensation mode, and the coding units based on different image contents all have stable compensation effects, that is, the present application has good robustness.

[0142] See also Fig. 9 , is a schematic diagram of the structure of a data processing device provided in an embodiment of the present application. Fig. 9 As shown, the data processing device may include:

[0143] An acquisition module 911 is used to acquire coding information of a coding unit in an image;

[0144] A determination module 912, configured to determine, according to the encoding information, a correlation feature between the encoding unit and a candidate compensation mode in a candidate compensation mode set;

[0145] A selection module 913, configured to select a candidate compensation mode matching the coding unit from the candidate compensation mode set according to the correlation feature;

[0146] The compensation module 914 is used to adopt the selected candidate compensation mode to perform compensation processing on the reconstructed pixel value of the above coding unit; the reconstructed pixel value of the above coding unit is obtained by encoding and reconstructing the original pixel value of the above coding unit.

[0147] Optionally, the candidate compensation modes in the above candidate compensation mode set include a boundary compensation mode;

[0148] The selection and determination module 912 determines the correlation characteristics between the coding unit and the candidate compensation mode in the candidate compensation mode set according to the coding information, including:

[0149] Obtaining, from the encoding information of the encoding unit of the image, the encoding modes corresponding to the N encoding sub-units of the encoding unit, respectively; N is a positive integer greater than 1;

[0150] From the N coding sub-units, K coding sub-units whose coding mode is a target coding mode are selected; K is a natural number; the target coding mode is a coding mode for encoding a motion vector corresponding to the image and a reference image of the image using an index;

[0151] According to the K coding sub-units, a correlation feature between the boundary compensation mode and the coding unit is determined.

[0152] Optionally, the determining module 912 determines the correlation feature between the boundary compensation mode and the coding unit according to the K coding sub-units, including:

[0153] Obtain the sizes corresponding to the K encoding sub-units and the size of the encoding unit;

[0154] The sizes corresponding to the K encoding sub-units are accumulated to obtain a total size;

[0155] According to the total size and the size of the coding unit, a correlation characteristic between the boundary compensation mode and the coding unit is determined.

[0156] Optionally, the determining module 912 determines the correlation feature between the boundary compensation mode and the coding unit according to the total size and the size of the coding unit, including:

[0157] Determining a size ratio between the total size and the size of the coding unit;

[0158] If the size ratio is less than or equal to the size ratio threshold, generating a correlation feature indicating that there is a correlation between the coding unit and the boundary compensation mode;

[0159] If the size ratio is greater than the size ratio threshold, a correlation feature is generated indicating that there is no correlation between the coding unit and the boundary compensation mode.

[0160] Optionally, the candidate compensation mode in the above candidate compensation mode set includes a sideband compensation mode;

[0161] The determination module 912 determines the correlation characteristics between the coding unit and the candidate compensation modes in the candidate compensation mode set according to the coding information, including:

[0162] Obtaining S reconstructed pixel values ​​of the coding unit from the coding information; S is an integer greater than 1;

[0163] Determine the pixel value intervals to which the S reconstructed pixel values ​​respectively belong;

[0164] According to the pixel value intervals to which the S reconstructed pixel values ​​respectively belong, identifying the P stripes to which the S reconstructed pixel values ​​belong; one stripe corresponds to one pixel value interval, and P is a positive integer;

[0165] According to the P stripes to which the S reconstructed pixel values ​​belong, a correlation feature between the coding unit and the sideband compensation mode is determined.

[0166] Optionally, the determination module 912 determines the correlation characteristics between the coding unit and the sideband compensation mode according to the P slices to which the S reconstructed pixel values ​​belong, including:

[0167] Counting the number of pixel values ​​corresponding to the reconstructed pixel values ​​belonging to the stripe i among the S reconstructed pixel values ​​and the total number of pixel values ​​of the reconstructed pixel values ​​in the coding unit; i is a positive integer less than or equal to P;

[0168] The P strips are sorted in descending order according to the number of pixel values ​​corresponding to the P strips, so as to obtain an arrangement order of the P strips;

[0169] From the P strips, determine the Q strips that are in the first Q order, and accumulate the numbers of pixel values ​​corresponding to the Q strips to obtain an accumulated number of pixel values;

[0170] According to the total number of pixel values ​​and the accumulated number of pixel values, a correlation characteristic between the encoding unit and the sideband compensation mode is determined.

[0171] Optionally, the determination module 912 determines the correlation feature between the coding unit and the side compensation mode according to the total number of pixel values ​​and the accumulated number of pixel values, including:

[0172] Determine the ratio between the accumulated pixel value number and the total pixel value number;

[0173] If the quantity ratio is greater than a quantity ratio threshold, generating a correlation feature indicating that there is a correlation between the coding unit and the sideband compensation mode;

[0174] If the quantity ratio is less than or equal to the quantity ratio threshold, a correlation feature is generated indicating that there is no correlation between the coding unit and the side compensation mode.

[0175] Optionally, the candidate compensation modes in the above candidate compensation mode set include a boundary compensation mode, a sideband compensation mode and a non-compensation mode;

[0176] The selection module 913 selects a candidate compensation mode matching the coding unit from the candidate compensation mode set according to the correlation feature, including:

[0177] If the correlation feature indicates that there is a correlation between the coding unit and the boundary compensation mode, determining the boundary compensation mode as a candidate compensation mode matching the coding unit;

[0178] If the correlation feature indicates that there is a correlation between the coding unit and the sideband compensation mode, determining the sideband compensation mode as a candidate compensation mode matching the coding unit;

[0179] If the correlation feature indicates that the coding unit is correlated with both the sideband compensation mode and the boundary compensation mode, determining a candidate compensation mode matching the coding unit from the boundary compensation mode and the boundary compensation mode;

[0180] If the correlation feature indicates that there is no correlation between the coding unit and either the sideband compensation mode or the boundary compensation mode, the non-compensation mode is determined as a candidate compensation mode matching the coding unit.

[0181] Optionally, the selection module 913 determines a candidate compensation mode matching the coding unit from the boundary compensation mode and the boundary compensation mode, including:

[0182] determining a first rate-distortion performance of the coding unit when the coding unit is compensated using the boundary compensation mode;

[0183] Determining a second rate-distortion performance of the coding unit when the sideband compensation mode is used to perform compensation processing on the coding unit;

[0184] If the first rate-distortion performance is less than the second rate-distortion performance, determining the boundary compensation mode as a candidate compensation mode matching the coding unit;

[0185] If the first rate-distortion performance is greater than the second rate-distortion performance, determining the sideband compensation mode as a candidate compensation mode matching the coding unit;

[0186] If the first rate-distortion performance is equal to the second rate-distortion performance, the sideband compensation mode or the boundary compensation mode is determined as a candidate compensation mode matching the coding unit.

[0187] Optionally, the adopting the selected candidate compensation mode to perform compensation processing on the reconstructed pixel value of the coding unit includes:

[0188] If the selected compensation mode is the boundary compensation mode, then the adjacent reconstructed pixel values ​​of the S reconstructed pixel values ​​of the coding unit in M ​​directions are obtained; S and M are both integers greater than 1;

[0189] Compare the reconstructed pixel value j with the adjacent reconstructed pixel values ​​of the reconstructed pixel value j in the M directions to obtain a comparison result; j is a positive integer less than or equal to S;

[0190] According to the comparison result, determining the pixel category of the reconstructed pixel value j;

[0191] According to the pixel categories to which the above S reconstructed pixel values ​​respectively belong, the pixel compensation values ​​corresponding to the above S reconstructed pixel values ​​are determined, and the pixel compensation values ​​corresponding to the above S reconstructed pixel values ​​are sent to the decoding device; one pixel category corresponds to one pixel compensation value.

[0192] Optionally, the compensation module 914 adopts the selected candidate compensation mode to perform compensation processing on the reconstructed pixel value of the coding unit, including:

[0193] If the selected compensation mode is the sideband compensation mode, then obtain the P stripes to which the S reconstructed pixel values ​​of the coding unit belong; S and P are both integers greater than 1;

[0194] According to the pixel value intervals corresponding to the P strips, the P strips are divided into F strip groups;

[0195] From the F stripe groups, select a target stripe group whose rate-distortion performance is the minimum rate-distortion performance;

[0196] Obtain pixel compensation values ​​corresponding to the strips in the target strip group;

[0197] The pixel compensation value corresponding to the strip in the target strip group and the starting position of the target strip group are sent to a decoding device.

[0198] When it is necessary to explain, the data processing device in the present application can be used to execute the description of the above-mentioned data processing method in the corresponding embodiment of the previous text. When the data processing device executes the description of the data processing method of the previous text, the beneficial effects brought about can refer to the beneficial effects described in the embodiment corresponding to the data processing method of the previous text, and will not be repeated here.

[0199] In the present application, based on the coding information of the coding unit, the correlation characteristics between the coding unit and the candidate compensation mode are analyzed, and based on the correlation characteristics, the candidate compensation mode matching the coding unit is quickly decided; there is no need to traverse the compensation results of each candidate compensation mode for the coding unit in turn, and the candidate compensation mode with the lowest rate-distortion performance is selected based on the coding result as the candidate compensation mode matching the coding unit, thereby reducing the complexity of the compensation processing of the image and improving the coding efficiency of the image. At the same time, the coding information of the coding units with different image contents is inconsistent, that is, the correlation characteristics reflect the correlation between the image content of the coding unit and the candidate compensation mode to a certain extent, that is, the correlation characteristics between the image content of the coding unit and the candidate compensation mode are fully considered, and the candidate compensation mode matching the coding unit is adaptively selected based on the correlation characteristics, thereby improving the accuracy of selecting the candidate compensation mode, and the coding units based on different image contents all have stable compensation effects, that is, the present application has good robustness.

[0200] See also Fig.10 , is a schematic diagram of the structure of a computer device provided in an embodiment of the present application. Fig.10As shown, the above-mentioned computer device 1000 may refer to a terminal or a server, including: a processor 1001, a network interface 1004 and a memory 1005. In addition, the above-mentioned computer device 1000 may also include: a user interface 1003, and at least one communication bus 1002. Among them, the communication bus 1002 is used to realize the connection and communication between these components. Among them, in some embodiments, the user interface 1003 may include a display screen (DiSPlay), a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory, or it may be a non-volatile memory (non-volatileMeMory), such as at least one disk storage. The memory 1005 may also be optionally at least one storage device away from the aforementioned processor 1001. As Fig.10 As shown, the memory 1005 as a computer-readable storage medium may include an operating system, a network communication module, a user interface module, and a computer program.

[0201] exist Fig.10 In the computer device 1000 shown, the network interface 1004 can provide a network communication function; the user interface 1003 is mainly used to provide an input interface; and the processor 1001 can be used to call a computer program stored in the memory 1005 to execute:

[0202] Obtaining encoding information of a coding unit in an image;

[0203] Determine, according to the coding information, a correlation feature between the coding unit and a candidate compensation mode in a candidate compensation mode set;

[0204] According to the correlation feature, selecting a candidate compensation mode matching the coding unit from the candidate compensation mode set;

[0205] The selected candidate compensation mode is used to perform compensation processing on the reconstructed pixel values ​​of the coding unit; the reconstructed pixel values ​​of the coding unit are obtained by performing encoding and reconstruction processing on the original pixel values ​​of the coding unit.

[0206] Optionally, the candidate compensation modes in the above candidate compensation mode set include a boundary compensation mode;

[0207] Optionally, the processor 1001 may be configured to call a computer program stored in the memory 1005 to execute, according to the coding information, determining a correlation feature between the coding list and a candidate compensation mode in the candidate compensation mode set, including:

[0208] Obtaining, from the encoding information of the encoding unit of the image, the encoding modes corresponding to the N encoding sub-units of the encoding unit, respectively; N is a positive integer greater than 1;

[0209] From the N coding sub-units, K coding sub-units whose coding mode is a target coding mode are selected; K is a natural number; the target coding mode is a coding mode for encoding a motion vector corresponding to the image and a reference image of the image using an index;

[0210] According to the K coding sub-units, a correlation feature between the boundary compensation mode and the coding unit is determined.

[0211] Optionally, the processor 1001 may be configured to call a computer program stored in the memory 1005 to execute, according to the K coding sub-units, determining a correlation feature between the boundary compensation mode and the coding unit, including:

[0212] Obtain the sizes corresponding to the K encoding sub-units and the size of the encoding unit;

[0213] The sizes corresponding to the K encoding sub-units are accumulated to obtain a total size;

[0214] According to the total size and the size of the coding unit, a correlation characteristic between the boundary compensation mode and the coding unit is determined.

[0215] Optionally, the processor 1001 may be configured to call a computer program stored in the memory 1005 to execute, according to the total size and the size of the coding unit, determining a correlation feature between the boundary compensation mode and the coding unit, including:

[0216] Determining a size ratio between the total size and the size of the coding unit;

[0217] If the size ratio is less than or equal to the size ratio threshold, generating a correlation feature indicating that there is a correlation between the coding unit and the boundary compensation mode;

[0218] If the size ratio is greater than the size ratio threshold, a correlation feature is generated indicating that there is no correlation between the coding unit and the boundary compensation mode.

[0219] Optionally, the candidate compensation mode in the above candidate compensation mode set includes a sideband compensation mode;

[0220] Optionally, the processor 1001 may be configured to call a computer program stored in the memory 1005 to execute, according to the encoding information, determining a correlation feature between the encoding unit and a candidate compensation mode in the candidate compensation mode set, including:

[0221] Obtaining S reconstructed pixel values ​​of the coding unit from the coding information; S is an integer greater than 1;

[0222] Determine the pixel value intervals to which the S reconstructed pixel values ​​respectively belong;

[0223] According to the pixel value intervals to which the S reconstructed pixel values ​​respectively belong, identifying the P stripes to which the S reconstructed pixel values ​​belong; one stripe corresponds to one pixel value interval, and P is a positive integer;

[0224] According to the P stripes to which the S reconstructed pixel values ​​belong, a correlation feature between the coding unit and the sideband compensation mode is determined.

[0225] Optionally, the processor 1001 may be configured to call a computer program stored in the memory 1005 to execute, according to the P strips to which the S reconstructed pixel values ​​belong, determining a correlation feature between the coding unit and the sideband compensation mode, including:

[0226] Counting the number of pixel values ​​corresponding to the reconstructed pixel values ​​belonging to the stripe i among the S reconstructed pixel values ​​and the total number of pixel values ​​of the reconstructed pixel values ​​in the coding unit; i is a positive integer less than or equal to P;

[0227] The P strips are sorted in descending order according to the number of pixel values ​​corresponding to the P strips, so as to obtain an arrangement order of the P strips;

[0228] From the P strips, determine the Q strips that are in the first Q order, and accumulate the numbers of pixel values ​​corresponding to the Q strips to obtain an accumulated number of pixel values;

[0229] According to the total number of pixel values ​​and the accumulated number of pixel values, a correlation characteristic between the encoding unit and the sideband compensation mode is determined.

[0230] Optionally, the processor 1001 may be configured to call a computer program stored in the memory 1005 to execute, according to the total number of pixel values ​​and the accumulated number of pixel values, a correlation feature between the coding unit and the sideband compensation mode, including:

[0231] Determine the ratio between the accumulated pixel value number and the total pixel value number;

[0232] If the quantity ratio is greater than a quantity ratio threshold, generating a correlation feature indicating that there is a correlation between the coding unit and the sideband compensation mode;

[0233] If the quantity ratio is less than or equal to the quantity ratio threshold, a correlation feature is generated indicating that there is no correlation between the coding unit and the side compensation mode.

[0234] Optionally, the candidate compensation modes in the above candidate compensation mode set include a boundary compensation mode, a sideband compensation mode and a non-compensation mode;

[0235] Optionally, the processor 1001 may be configured to call a computer program stored in the memory 1005 to execute, according to the correlation feature, selecting a candidate compensation mode matching the coding unit from the candidate compensation mode set, including:

[0236] If the correlation feature indicates that there is a correlation between the coding unit and the boundary compensation mode, determining the boundary compensation mode as a candidate compensation mode matching the coding unit;

[0237] If the correlation feature indicates that there is a correlation between the coding unit and the sideband compensation mode, determining the sideband compensation mode as a candidate compensation mode matching the coding unit;

[0238] If the correlation feature indicates that the coding unit is correlated with both the sideband compensation mode and the boundary compensation mode, determining a candidate compensation mode matching the coding unit from the boundary compensation mode and the boundary compensation mode;

[0239] If the correlation feature indicates that there is no correlation between the coding unit and either the sideband compensation mode or the boundary compensation mode, the non-compensation mode is determined as a candidate compensation mode matching the coding unit.

[0240] Optionally, the processor 1001 may be configured to call a computer program stored in the memory 1005 to execute, from the boundary compensation mode and the boundary compensation mode, determining a candidate compensation mode matching the coding unit, including:

[0241] determining a first rate-distortion performance of the coding unit when the coding unit is compensated using the boundary compensation mode;

[0242] Determining a second rate-distortion performance of the coding unit when the sideband compensation mode is used to perform compensation processing on the coding unit;

[0243] If the first rate-distortion performance is less than the second rate-distortion performance, determining the boundary compensation mode as a candidate compensation mode matching the coding unit;

[0244] If the first rate-distortion performance is greater than the second rate-distortion performance, determining the sideband compensation mode as a candidate compensation mode matching the coding unit;

[0245] If the first rate-distortion performance is equal to the second rate-distortion performance, the sideband compensation mode or the boundary compensation mode is determined as a candidate compensation mode matching the coding unit.

[0246] Optionally, the processor 1001 may be configured to call a computer program stored in the memory 1005 to execute compensation processing on the reconstructed pixel value of the coding unit by using the selected candidate compensation mode, including:

[0247] If the selected compensation mode is the boundary compensation mode, then the adjacent reconstructed pixel values ​​of the S reconstructed pixel values ​​of the coding unit in M ​​directions are obtained; S and M are both integers greater than 1;

[0248] Compare the reconstructed pixel value j with the adjacent reconstructed pixel values ​​of the reconstructed pixel value j in the M directions to obtain a comparison result; j is a positive integer less than or equal to S;

[0249] According to the comparison result, determining the pixel category of the reconstructed pixel value j;

[0250] According to the pixel categories to which the above S reconstructed pixel values ​​respectively belong, the pixel compensation values ​​corresponding to the above S reconstructed pixel values ​​are determined, and the pixel compensation values ​​corresponding to the above S reconstructed pixel values ​​are sent to the decoding device; one pixel category corresponds to one pixel compensation value.

[0251] Optionally, the processor 1001 may be configured to call a computer program stored in the memory 1005 to execute compensation processing on the reconstructed pixel value of the coding unit by using the selected candidate compensation mode, including:

[0252] If the selected compensation mode is the sideband compensation mode, then obtain the P stripes to which the S reconstructed pixel values ​​of the coding unit belong; S and P are both integers greater than 1;

[0253] According to the pixel value intervals corresponding to the P strips, the P strips are divided into F strip groups;

[0254] From the F stripe groups, select a target stripe group whose rate-distortion performance is the minimum rate-distortion performance;

[0255] Obtain pixel compensation values ​​corresponding to the strips in the target strip group;

[0256] The pixel compensation value corresponding to the strip in the target strip group and the starting position of the target strip group are sent to a decoding device.

[0257] In the present application, based on the coding information of the coding unit, the correlation characteristics between the coding unit and the candidate compensation mode are analyzed, and based on the correlation characteristics, the candidate compensation mode matching the coding unit is quickly decided; there is no need to traverse the compensation results of each candidate compensation mode for the coding unit in turn, and the candidate compensation mode with the lowest rate-distortion performance is selected based on the coding result as the candidate compensation mode matching the coding unit, thereby reducing the complexity of the compensation processing of the image and improving the coding efficiency of the image. At the same time, the coding information of the coding units with different image contents is inconsistent, that is, the correlation characteristics reflect the correlation between the image content of the coding unit and the candidate compensation mode to a certain extent, that is, the correlation characteristics between the image content of the coding unit and the candidate compensation mode are fully considered, and the candidate compensation mode matching the coding unit is adaptively selected based on the correlation characteristics, thereby improving the accuracy of selecting the candidate compensation mode, and the coding units based on different image contents all have stable compensation effects, that is, the present application has good robustness.

[0258] In addition, it should be pointed out here that: the embodiment of the present application also provides a computer-readable storage medium, and the computer-readable storage medium stores a computer program executed by the data processing device mentioned above, and the computer program includes program instructions. When the processor executes the program instructions, it can execute the description of the data processing method in the corresponding embodiment above, so it will not be repeated here. In addition, the description of the beneficial effects of the same method will not be repeated. For technical details not disclosed in the computer-readable storage medium embodiment involved in this application, please refer to the description of the method embodiment of this application.

[0259] As an example, the above program instructions may be deployed on a computer device for execution, or deployed on at least two computer devices at one location for execution, or executed on at least two computer devices distributed at at least two locations and interconnected through a communication network. At least two computer devices distributed at at least two locations and interconnected through a communication network may constitute a blockchain network.

[0260] The computer-readable storage medium may be the data processing device provided in any of the aforementioned embodiments or the central storage unit of the computer device, such as a hard disk or a memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart memory card (SMART Media card, SMC), a secure digital (Secure digital, SD) card, a flash memory card (flaSh card), etc. equipped on the computer device. Further, the computer-readable storage medium may also include both the central storage unit of the computer device and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by the computer device. The computer-readable storage medium may also be used to temporarily store data that has been output or is to be output.

[0261] The terms "first", "second", etc. in the description, claims, and drawings of the embodiments of the present application are used to distinguish between contents in different media, rather than to describe a specific order. In addition, the terms "including" and any of their variations are intended to cover non-exclusive inclusions. For example, a process, method, device, product, or equipment that includes a series of steps or units is not limited to the listed steps or modules, but may optionally include steps or modules that are not listed, or may optionally include other step units inherent to these processes, methods, devices, products, or equipment.

[0262] The data collection and processing in this application should be strictly in accordance with the requirements of relevant national laws and regulations when applied in practice, and the informed consent or separate consent of the personal information subject should be obtained. Subsequent data use and processing should be carried out within the scope of authorization of laws and regulations and the personal information subject.

[0263] The embodiment of the present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the description of the above-mentioned data processing method and decoding method in the corresponding embodiment above, so it will not be repeated here. In addition, the description of the beneficial effects of the same method will not be repeated. For technical details not disclosed in the embodiment of the computer program product involved in the present application, please refer to the description of the method embodiment of the present application.

[0264] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0265] The method and related apparatus provided by the embodiment of the present application are described with reference to the method flow chart and / or structural diagram provided by the embodiment of the present application. Specifically, each process and / or box in the method flow chart and / or structural diagram, as well as the combination of the processes and / or boxes in the flow chart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable network connection device to generate a machine, so that the instructions executed by the processor of the computer or other programmable network connection device generate instructions for implementing the process in the process. Figure 1 A process or multiple processes and / or structures Figure 1 These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable network-connected device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction device, which implements the functions specified in the process. Figure 1 A process or multiple processes and / or structures Figure 1 These computer program instructions can also be loaded onto a computer or other programmable network connected device, so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide for implementing the process. Figure 1 A flow or multiple flows and / or structures illustrate the steps of the functions specified in one block or multiple blocks.

[0266] The above disclosure is only the preferred embodiment of the present application, which certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

Claims

1. A data processing method, characterized in that: include: Obtaining coding information of coding units in an image; Determining, according to the encoding information, a correlation feature between the encoding unit and a candidate compensation mode in a candidate compensation mode set; Selecting, according to the correlation feature, a candidate compensation mode matching the coding unit from the candidate compensation mode set; The selected candidate compensation mode is adopted to perform compensation processing on the reconstructed pixel value of the coding unit; the reconstructed pixel value of the coding unit is obtained by encoding and reconstructing the original pixel value of the coding unit.

2. The method according to claim 1, characterized in that The candidate compensation modes in the candidate compensation mode set include a boundary compensation mode; The determining, according to the encoding information, a correlation feature between the encoding unit and a candidate compensation mode in a candidate compensation mode set includes: Obtaining, from the encoding information of the encoding unit of the image, the encoding modes corresponding to the N encoding sub-units of the encoding unit respectively; N is a positive integer greater than 1; From the N coding sub-units, K coding sub-units whose coding mode is a target coding mode are selected; K is a natural number; the target coding mode is a coding mode for encoding a motion vector corresponding to a reference image of the image using an index; According to the K coding sub-units, a correlation feature between the boundary compensation mode and the coding unit is determined.

3. The method according to claim 2, characterized in that The determining, according to the K coding sub-units, a correlation feature between the boundary compensation mode and the coding unit includes: Obtaining sizes corresponding to the K encoding sub-units and the size of the encoding unit; Accumulating the sizes corresponding to the K encoding sub-units to obtain a total size; A correlation characteristic between the boundary compensation mode and the coding unit is determined according to the total size and the size of the coding unit.

4. The method according to claim 3, characterized in that The determining, according to the total size and the size of the coding unit, a correlation feature between the boundary compensation mode and the coding unit comprises: Determining a size ratio between the total size and the size of the coding unit; If the size ratio is less than or equal to a size ratio threshold, generating a correlation feature indicating that there is a correlation between the coding unit and the boundary compensation mode; If the size ratio is greater than the size ratio threshold, a correlation feature is generated indicating that there is no correlation between the coding unit and the boundary compensation mode.

5. The method according to claim 1, characterized in that The candidate compensation modes in the candidate compensation mode set include a sideband compensation mode; The determining, according to the encoding information, a correlation feature between the encoding unit and a candidate compensation mode in a candidate compensation mode set includes: Obtaining S reconstructed pixel values ​​of the coding unit from the coding information; S is an integer greater than 1; Determine the pixel value intervals to which the S reconstructed pixel values ​​respectively belong; According to the pixel value intervals to which the S reconstructed pixel values ​​respectively belong, identifying P stripes to which the S reconstructed pixel values ​​belong; one stripe corresponds to one pixel value interval, and P is a positive integer; According to the P slices to which the S reconstructed pixel values ​​belong, a correlation characteristic between the coding unit and the sideband compensation mode is determined.

6. The method according to claim 5, characterized in that The determining, according to the P slices to which the S reconstructed pixel values ​​belong, a correlation characteristic between the coding unit and the sideband compensation mode includes: Counting the number of pixel values ​​corresponding to the reconstructed pixel values ​​belonging to the stripe i among the S reconstructed pixel values, and the total number of pixel values ​​of the reconstructed pixel values ​​in the coding unit; i is a positive integer less than or equal to P; Sorting the P strips in descending order of the number of pixel values ​​corresponding to the P strips to obtain an arrangement order of the P strips; From the P strips, determine Q strips that are arranged in the first Q order, and accumulate the numbers of pixel values ​​corresponding to the Q strips to obtain an accumulated number of pixel values; A correlation characteristic between the coding unit and the side compensation mode is determined according to the total number of pixel values ​​and the accumulated number of pixel values.

7. The method according to claim 6, characterized in that The determining, according to the total number of pixel values ​​and the accumulated number of pixel values, a correlation characteristic between the coding unit and the sideband compensation mode comprises: Determine a ratio between the number of accumulated pixel values ​​and the total number of pixel values; If the quantity ratio is greater than a quantity ratio threshold, generating a correlation feature indicating that there is a correlation between the coding unit and the sideband compensation mode; If the quantity ratio is less than or equal to the quantity ratio threshold, a correlation feature is generated indicating that there is no correlation between the coding unit and the side compensation mode.

8. The method according to claim 1, characterized in that The candidate compensation modes in the candidate compensation mode set include a boundary compensation mode, a sideband compensation mode and a non-compensation mode; The selecting, according to the correlation feature, a candidate compensation mode matching the coding unit from the candidate compensation mode set comprises: If the correlation feature indicates that there is a correlation between the coding unit and the boundary compensation mode, determining the boundary compensation mode as a candidate compensation mode matching the coding unit; If the correlation feature indicates that there is a correlation between the coding unit and the sideband compensation mode, determining the sideband compensation mode as a candidate compensation mode matching the coding unit; If the correlation feature indicates that the coding unit is correlated with both the sideband compensation mode and the boundary compensation mode, determining a candidate compensation mode matching the coding unit from the boundary compensation mode and the boundary compensation mode; If the correlation feature indicates that there is no correlation between the coding unit and either the sideband compensation mode or the boundary compensation mode, the non-compensation mode is determined as a candidate compensation mode matching the coding unit.

9. The method according to claim 8, characterized in that The step of determining a candidate compensation mode matching the coding unit from the boundary compensation mode and the boundary compensation mode includes: determining a first rate-distortion performance of the coding unit when the coding unit is compensated using the boundary compensation mode; determining a second rate-distortion performance of the coding unit when the coding unit is compensated using the sideband compensation mode; If the first rate-distortion performance is less than the second rate-distortion performance, determining the boundary compensation mode as a candidate compensation mode matching the coding unit; If the first rate-distortion performance is greater than the second rate-distortion performance, determining the sideband compensation mode as a candidate compensation mode matching the coding unit; If the first rate-distortion performance is equal to the second rate-distortion performance, the sideband compensation mode or the boundary compensation mode is determined as a candidate compensation mode matching the coding unit.

10. The method according to claim 1, characterized in that The adopting the selected candidate compensation mode to perform compensation processing on the reconstructed pixel value of the coding unit includes: If the selected compensation mode is the boundary compensation mode, then the adjacent reconstructed pixel values ​​of the S reconstructed pixel values ​​of the coding unit in M ​​directions are obtained; S and M are both integers greater than 1; Compare the reconstructed pixel value j with the adjacent reconstructed pixel values ​​of the reconstructed pixel value j in the M directions to obtain a comparison result; j is a positive integer less than or equal to S; Determining the pixel category of the reconstructed pixel value j according to the comparison result; According to the pixel categories to which the S reconstructed pixel values ​​respectively belong, pixel compensation values ​​corresponding to the S reconstructed pixel values ​​are determined, and the pixel compensation values ​​corresponding to the S reconstructed pixel values ​​are sent to a decoding device; one pixel category corresponds to one pixel compensation value.

11. The method according to claim 1, characterized in that: The adopting the selected candidate compensation mode to perform compensation processing on the reconstructed pixel value of the coding unit includes: If the selected compensation mode is the sideband compensation mode, then obtaining the P stripes to which the S reconstructed pixel values ​​of the coding unit belong; S and P are both integers greater than 1; Dividing the P strips into F strip groups according to the pixel value intervals respectively corresponding to the P strips; Selecting a target slice group whose rate-distortion performance is the minimum rate-distortion performance from the F slice groups; Obtaining pixel compensation values ​​corresponding to strips in the target strip group; The pixel compensation value corresponding to the stripe in the target stripe group and the starting position of the target stripe group are sent to a decoding device.

12. A data processing device, characterized in that: include: An acquisition module, used for acquiring encoding information of encoding units in an image; A determination module, configured to determine, according to the encoding information, a correlation feature between the encoding unit and a candidate compensation mode in a candidate compensation mode set; A selection module, configured to select a candidate compensation mode matching the coding unit from the candidate compensation mode set according to the correlation feature; The compensation module is used to adopt the selected candidate compensation mode to perform compensation processing on the reconstructed pixel value of the coding unit; the reconstructed pixel value of the coding unit is obtained by encoding and reconstructing the original pixel value of the coding unit.

13. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 11 are implemented.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 11 are implemented.

15. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 11 are implemented.

Images