Video processing method and related equipment

Abstract

The invention provides a video processing method and related equipment. The method comprises the following steps: determining a current chrominance block in a video code stream, wherein the current chrominance block adopts a chrominance block copy intra-frame prediction mode; obtaining a target displacement vector of a brightness collocation block corresponding to the current chrominance block; determining a predicted value of the current chroma block based on the target displacement vector; the prediction value is used for reconstructing a decoded image corresponding to the current chrominance block. The prediction value of the chrominance block can be determined by utilizing the displacement vector of the luminance co-located block, so that the coding performance is improved.

Description

Technical Field

[0001] The present application relates to the field of audio and video technology, specifically to the field of video coding technology, and in particular to a video processing method, a video processing device, a computer device, a computer-readable storage medium, and a computer program product. Background Art

[0002] In the existing video coding technology, a block-based hybrid coding framework is used to divide the original video data into a series of coding units (CUs), and video coding methods such as prediction, transform and entropy coding are combined to achieve video data compression. At present, mainstream video coding standards such as AVS3 (Audio Video Coding Standard 3) allow the luminance block and chrominance block to be encoded separately, but it is found in practice that if the chrominance block and the luminance block of the current coding unit adopt inconsistent division structures, the chrominance block cannot use intra-frame motion compensation technology, and the coding efficiency is low. Summary of the invention

[0003] The embodiments of the present application provide a video processing method and related devices, which can use the displacement vector of the luminance co-located block to determine the prediction value of the chrominance block, thereby improving the encoding performance.

[0004] On the one hand, an embodiment of the present application provides a video processing method, the method comprising:

[0005] Determine a current chroma block in a video bitstream, where the current chroma block adopts a chroma block copy intra prediction mode;

[0006] Get the target displacement vector of the luminance co-located block corresponding to the current chrominance block;

[0007] Based on the target displacement vector, a prediction value of the current chrominance block is determined; the prediction value is used to reconstruct a decoded image corresponding to the current chrominance block.

[0008] On the one hand, an embodiment of the present application provides a video processing method, the method comprising:

[0009] Determine a current chroma block in the video, the current chroma block adopts a chroma block copy intra prediction mode;

[0010] Get the target displacement vector of the luminance co-located block corresponding to the current chrominance block;

[0011] Determining a prediction value of the current chroma block based on the target displacement vector; the prediction value is used to reconstruct a decoded image corresponding to the current chroma block;

[0012] The video is encoded based on the predicted value of the current chrominance block to generate a video bitstream.

[0013] On the one hand, an embodiment of the present application provides a video processing device, the device comprising:

[0014] A determination unit, used to determine a current chroma block in a video bitstream, the current chroma block adopting a chroma block copy intra prediction mode;

[0015] A processing unit, used to obtain a target displacement vector of a luminance co-located block corresponding to a current chrominance block;

[0016] The processing unit is further used to determine a prediction value of the current chroma block based on the target displacement vector; the prediction value is used to reconstruct a decoded image corresponding to the current chroma block.

[0017] In a possible implementation, the chroma block copy intra prediction mode is a prediction mode independent of the normal intra prediction mode; or, the chroma block copy intra prediction mode is a sub-mode of the normal intra prediction mode;

[0018] The common intra prediction mode includes at least one of the following: an intra prediction DC coefficient mode, an angle-based intra prediction mode, a cross-component two-step prediction mode, and a prediction mode for deriving prediction values ​​based on intra reference pixels.

[0019] In a possible implementation, after determining the current chrominance block in the video code stream, the processing unit 902 is further configured to perform the following operations:

[0020] Divide the current chroma block into P×Q chroma sub-blocks, where P and Q are both integers;

[0021] Obtain the block vector of the luminance co-located sub-block corresponding to each chrominance sub-block;

[0022] Based on the obtained P×Q block vectors, a prediction value of the current chrominance block is determined.

[0023] In a possible implementation, the processing unit is further configured to perform the following operations:

[0024] Decoding the video bitstream to obtain mode indication information, where the mode indication information is used to indicate whether the current chroma block adopts a chroma block copy intra prediction mode;

[0025] The mode indication information includes at least one of the following: a sequence header flag, a picture header flag, a slice header flag, a coding unit flag, implicit information, and chroma indication information.

[0026] In a possible implementation, if the mode indication information is a sequence header flag, the mode indication information is set in a sequence header of a current sequence to which the current chroma block belongs; when the sequence header flag is a first preset value, it indicates that the current chroma block allows the use of a chroma block copy intra-frame prediction mode; and when the sequence header flag is a second preset value, it indicates that the current chroma block does not use a chroma block copy intra-frame prediction mode;

[0027] If the mode indication information is an image header flag, the mode indication information is set in an image header of a current image to which the current chroma block belongs; when the image header flag is a first preset value, it indicates that the current chroma block allows the use of a chroma block copy intra-frame prediction mode; and when the image header flag is a second preset value, it indicates that the current chroma block does not use a chroma block copy intra-frame prediction mode;

[0028] If the mode indication information is a slice header flag, the mode indication information is set in the slice header of the current slice to which the current chroma block belongs; when the slice header flag is a first preset value, it indicates that the current chroma block allows the use of the chroma block copy intra prediction mode, and when the slice header flag is a second preset value, it indicates that the current chroma block does not use the chroma block copy intra prediction mode;

[0029] If the mode indication information is a coding unit flag, the mode indication information is set in the current coding unit; when the coding unit flag is a first preset value, it indicates that the current chroma block adopts the chroma block copy intra prediction mode, and when the coding unit flag is a second preset value, it indicates that the current chroma block does not adopt the chroma block copy intra prediction mode;

[0030] If the mode indication information is implicit information, then when the implicit information indicates that the luminance sample at a preset position in the luminance co-located block corresponding to the current chrominance block meets a preset condition, the current chrominance block is indicated to adopt the chrominance block copy intra prediction mode;

[0031] If the mode indication information is chroma indication information, then when the chroma indication information indicates that the current chroma block adopts a preset intra-frame prediction mode, and the luminance sample at a preset position in the luminance co-located block meets a preset condition, it indicates that the current chroma block adopts a chroma block copy intra-frame prediction mode; the preset intra-frame prediction mode includes an inheritance mode.

[0032] In a possible implementation, the processing unit is further configured to perform the following operations:

[0033] Check whether the video code stream meets the decoding conditions;

[0034] If the video code stream meets the decoding condition, the step of decoding the video code stream to obtain mode indication information is performed.

[0035] In a possible implementation, the decoding condition includes: the color format adopted by the video code stream is a format including a chrominance component;

[0036] When the mode indication information is set in the current coding unit to which the current chroma block belongs, the decoding condition includes any one or more of the following:

[0037] The current coding unit contains only chroma components;

[0038] The luminance sample point at a preset position in the luminance co-location block corresponding to the current chrominance block meets a preset condition;

[0039] If there are multiple preset positions, the number of brightness sample points that meet the preset conditions is at least one, or is greater than or equal to a preset threshold.

[0040] In a possible implementation, the preset position is determined in any of the following ways:

[0041] Determine a preset chroma sample point in the current chroma block, and determine a preset position of a corresponding luminance sample point in a luminance co-located block according to the position of the preset chroma sample point;

[0042] Determining a preset position from the brightness co-located blocks;

[0043] According to the division method of the luminance co-located block, a preset position is determined in each sub-block obtained by dividing the luminance co-located block;

[0044] The luminance co-located block is divided into M×N sub-blocks, and a specified position in the M×N sub-blocks is used as a preset position, where the specified position includes: any position of the center, upper left corner, upper right corner, lower left corner, and lower right corner of the sub-block, and M and N are both positive integers.

[0045] In a possible implementation, the preset condition includes any one or more of the following:

[0046] The luminance sample adopts any mode in a specific mode set, which includes: intra-frame block copy prediction mode, intra-frame string copy prediction ordinary string sub-mode, intra-frame string copy prediction non-ordinary string sub-mode, or any one or more prediction modes in other prediction modes with intra-frame displacement vectors.

[0047] Luminance samples have valid displacement vectors;

[0048] The intra prediction history motion information table of the current chroma block is not empty.

[0049] In a possible implementation, the processing unit obtains a target displacement vector of a luma co-located block corresponding to the current chroma block, and performs the following operations:

[0050] Construct a list of candidate displacement vectors of the luma co-located block corresponding to the current chroma block;

[0051] A target displacement vector is determined from the list of candidate displacement vectors.

[0052] In a possible implementation, the processing unit constructs a candidate displacement vector list of the luma co-located block corresponding to the current chroma block, including any one or more of the following methods:

[0053] Add the displacement vectors of the luminance samples in the luminance co-located block that satisfy the specified mode to the candidate displacement vector list, where the specified mode includes: any one of the intra-frame block copy prediction mode, the intra-frame string copy prediction normal string sub-mode, the string copy intra-frame prediction non-normal string sub-mode, the intra-frame template matching mode, the inter-frame prediction mode, the affine motion mode, the inter-frame intra-frame composite mode, and the angle weighted prediction mode;

[0054] Add the displacement vector of the spatial adjacent block corresponding to the luminance co-located block to the candidate displacement vector list;

[0055] Adding the displacement vector in the historical motion information table or the intra-frame prediction historical motion information table to the candidate displacement vector list;

[0056] Adding the candidate displacement vectors to the candidate displacement vector list after offset processing, the candidate displacement vectors include: any one or more of the displacement vectors of the luminance samples satisfying the specified mode in the luminance co-located block, the displacement vectors of the spatial neighboring blocks corresponding to the luminance co-located block, and the displacement vectors of the spatial neighboring blocks corresponding to the luminance co-located block;

[0057] If there is no legal displacement vector in the luminance co-located block, a default displacement vector is added to the candidate displacement vector list; or, the displacement vector in the candidate displacement vector list is set to an invalid value.

[0058] In a possible implementation, the conditions that the displacement vectors in the candidate displacement vector list must satisfy include at least one of the following:

[0059] The reference block of the luma co-located block satisfies the reference range restriction of the intra block copy prediction mode;

[0060] The reference block of the luminance co-located block and the displacement vector all meet the reference range restriction of the intra-frame string copy prediction normal string sub-mode;

[0061] The displacement vector does not overlap with other displacement vectors except those in the candidate displacement vector list;

[0062] The displacement vector does not overlap with the first N or last N displacement vectors in the candidate displacement vector list, where N is an integer.

[0063] In a possible implementation, the ordering of the displacement vectors in the candidate displacement vector list includes any of the following:

[0064] Sort by predetermined order;

[0065] Sort by cost of each displacement vector, the cost includes at least: rate distortion cost, absolute error sum, transformed absolute error sum, difference square sum, mean-removed absolute error sum, mean-removed absolute square sum;

[0066] Sorting by type of displacement vector, the types of displacement vector at least include: block vector, string vector, motion vector, default vector;

[0067] Comprehensive sorting is performed according to the type of displacement vector and the cost of the displacement vector.

[0068] In a possible implementation manner, the processing unit determines the target displacement vector in the candidate displacement vector list, including any of the following manners:

[0069] The Kth displacement vector in the candidate displacement vector list is used as the target displacement vector, where K is a positive integer;

[0070] Determine a target displacement vector from a list of candidate displacement vectors according to the displacement vector index;

[0071] Obtain a candidate displacement vector from the candidate displacement vector list, scale the candidate displacement vector according to the color format sampling ratio, and retain the integer pixel accuracy of the scaled displacement vector to obtain a target displacement vector;

[0072] A candidate displacement vector is obtained from the candidate displacement vector list, and the candidate displacement vector is used as a target displacement vector.

[0073] In a possible implementation, the processing unit determines a prediction value of the current chroma block based on the target displacement vector, and performs the following operations:

[0074] If the target displacement vector has integer pixel accuracy, motion compensation is performed on the target displacement vector to derive a prediction value of the current chroma block; or,

[0075] If the target displacement vector has sub-pixel accuracy, the target displacement vector is interpolated, and motion compensation is performed on the interpolated target displacement vector to derive a predicted value of the current chroma block.

[0076] In a possible implementation, the processing unit is further configured to perform the following operations:

[0077] If the target displacement vector does not exist in the luminance co-located block corresponding to the current chrominance block, the intra-frame prediction mode is used to derive the prediction value of the current chrominance block;

[0078] Among them, the intra-frame prediction mode includes: cross-component intra-frame prediction mode, or specified intra-frame prediction mode; the cross-component intra-frame prediction mode includes: cross-component two-step prediction mode, cross-component linear model prediction mode; the specified intra-frame prediction mode includes: intra-frame prediction DC coefficient mode, intra-frame prediction bilinear prediction mode, intra-frame prediction planar prediction mode, horizontal direction intra-frame prediction mode, vertical direction intra-frame prediction mode, any one or more.

[0079] In a possible implementation, the processing unit is further configured to perform the following operations:

[0080] If the current chroma block adopts the chroma block copy intra prediction mode, determine whether the current chroma block needs to be residual decoded according to the information of the target luminance sample in the luminance co-located block corresponding to the current chroma block, and the target luminance sample is the luminance sample corresponding to the target displacement vector;

[0081] If it is determined that the current chroma block needs to be residually decoded, residual decoding is performed on the current chroma block to obtain a residual value;

[0082] Based on the residual value and prediction value of the current chrominance block, the decoded image corresponding to the current chrominance block is reconstructed.

[0083] In a possible implementation, the luminance co-located block corresponding to the current chrominance block includes multiple luminance blocks; and the judgment condition that the current chrominance block does not need to be subjected to residual decoding includes any one of the following:

[0084] If the target luminance sample adopts the intra-frame string copy prediction normal string mode, and the luminance block where the target luminance sample is located does not need to be residually encoded;

[0085] If the target luminance sample adopts the intra-frame string copy prediction mode, and the luminance block where the target luminance sample is located does not need to be residually encoded;

[0086] If the target luma sample adopts intra block copy prediction mode, and the luma block where the target luma sample is located does not need to be residual coded;

[0087] If the target luma sample adopts the intra-frame string copy prediction normal string sub-mode or the intra-frame block copy prediction mode, and the luma block where the target luma sample is located does not need to be residually encoded;

[0088] If the target luma sample adopts the intra-frame string copy prediction mode or the intra-frame block copy prediction mode, and the luma block where the target luma sample is located does not need to be residually encoded;

[0089] The judgment condition of whether the luminance block where the target luminance sample point is located needs to be residually encoded is determined according to: the coefficient flag of the luminance block, or the prediction mode of the current chrominance block.

[0090] On the one hand, an embodiment of the present application provides a video processing device, the device comprising:

[0091] A determination unit, used to determine a current chroma block in the video, the current chroma block adopting a chroma block copy intra prediction mode;

[0092] A processing unit, used to obtain a target displacement vector of a luminance co-located block corresponding to a current chrominance block;

[0093] The processing unit is further used to determine a prediction value of the current chroma block based on the target displacement vector; the prediction value is used to reconstruct a decoded image corresponding to the current chroma block;

[0094] The processing unit is further used to encode the video based on the predicted value of the current chrominance block to generate a video code stream.

[0095] On the one hand, an embodiment of the present application provides a computer device, which includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the above-mentioned video processing method.

[0096] On the one hand, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is read and executed by a processor of a computer device, the computer device executes the above-mentioned video processing method.

[0097] On the one hand, an embodiment of the present application provides a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the above-mentioned video processing method.

[0098] In an embodiment of the present application, a current chroma block in a video bitstream can be determined, and the current chroma block adopts a chroma block copy intra-frame prediction mode; a target displacement vector of a luminance co-located block corresponding to the current chroma block is obtained; based on the target displacement vector, a prediction value of the current chroma block is determined; and the prediction value is used to reconstruct a decoded image corresponding to the current chroma block. As can be seen from the above, in the decoding process of the video bitstream, when it is determined that the current chroma block adopts a chroma block copy intra-frame prediction mode, the displacement vector information of the luminance co-located block can be used to determine the prediction value of the current chroma block, and there is no need to perform intra-frame prediction processing on the current chroma block, which can improve the encoding and decoding efficiency of the current chroma block, thereby effectively improving the encoding performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0099] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0100] Figure 1 This is a basic working flow diagram of a video encoder provided by an embodiment of the present application;

[0101] Figure 2 is a schematic diagram of an inter-frame prediction provided by an embodiment of the present application;

[0102] Figure 3a It is a schematic diagram of establishing an airspace list provided in an embodiment of the present application;

[0103] Figure 3b is a schematic diagram of establishing a time domain list provided in an embodiment of the present application;

[0104] Figure 4a It is a schematic diagram of a flow chart of intra-frame block copy prediction provided by an embodiment of the present application;

[0105] Figure 4b It is a schematic diagram of a process of intra-frame string duplication prediction provided by an embodiment of the present application;

[0106] Figure 4c It is a schematic diagram of a non-ordinary string sub-mode for intra-frame string duplication prediction provided by an embodiment of the present application;

[0107] Figure 4d is a schematic diagram of a common brightness intra-frame prediction mode provided in an embodiment of the present application;

[0108] Figure 4e It is a schematic diagram of a template matching process provided in an embodiment of the present application;

[0109] Figure 4f is a schematic diagram of a video sampling method provided in an embodiment of the present application;

[0110] Figure 4g It is a structural diagram of a basic block division provided in an embodiment of the present application;

[0111] Figure 4h This is a schematic diagram of the position of a chroma block copy intra-frame prediction provided by an embodiment of the present application;

[0112] Figure 4i is a schematic diagram of the position of a sub-pixel sample point provided in an embodiment of the present application;

[0113] Figure 5is a structural diagram of a video processing system provided in an embodiment of the present application;

[0114] Figure 6 It is a flowchart of a video processing method provided by an embodiment of the present application;

[0115] Figure 7a It is a schematic diagram of a process of determining a brightness co-located block provided by an embodiment of the present application;

[0116] Figure 7b is a schematic diagram of sorting displacement vectors provided in an embodiment of the present application;

[0117] Figure 8 is a flowchart of another video processing method provided in an embodiment of the present application;

[0118] Fig. 9 is a structural schematic diagram of a video processing device provided in an embodiment of the present application;

[0119] Fig.10 is a structural schematic diagram of another video processing device provided in an embodiment of the present application;

[0120] Fig.11 It is a structural diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0121] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0122] The following is an introduction to the technical terms involved in this application:

[0123] 1. Video Encoding

[0124] A video can be composed of one or more video frames, each of which contains part of the video signal of the video. The video signal can be obtained by camera or computer generated. Due to the different statistical characteristics corresponding to different acquisition methods, the compression encoding method of the video may also be different.

[0125] In mainstream video coding technologies, such as HEVC (High Efficiency Video Coding, international video coding standard HEVC / H.265), VVC (versatile video coding, international video coding standard VVC / H.266), and AVS3 (Audio Video Coding Standard 3, audio and video source coding standard), a hybrid coding framework is adopted, under which the following series of operations and processing of videos are allowed:

[0126] 1) Block partition structure: According to the size of the current input frame (i.e., the video frame being encoded or decoded), the current frame can be divided into several non-overlapping processing units, and each processing unit will perform similar compression operations. This processing unit is called CTU (coding tree unit) or LCU (Largest Coding Unit). CTU can be further divided into more refined units to obtain one or more basic coding units, called CU (coding unit or coding block). It should be understood that the block is a structure after the image is divided. The block structure may include: coding block, coding unit, coding tree unit, coding sub-block, super block in the encoding process; similarly, the block structure may include: decoding block, decoding unit, decoding tree unit, decoding sub-block, super block in the decoding process. Each CU is the most basic element in the encoding link. The subsequent embodiments of this application describe various encoding and decoding processing flows that may be used for each CU.

[0127] 2) Prediction coding: including intra-frame prediction and inter-frame prediction modes. The original video signal contained in the current CU in the current frame (i.e., the CU being encoded or decoded in the current frame) is predicted by the reconstructed video signal in the selected reference CU to obtain a residual video signal. Here, the current CU can also be called the current block, the video frame where the current block is located is called the current frame, the reference CU used for predicting the current block can also be called the reference block of the current block, and the video frame where the reference block is located is called the reference frame. Among them, the encoder needs to decide for the current CU to select the most appropriate prediction coding mode from many possible prediction coding modes and tell the decoder. Among them, the prediction coding mode may include:

[0128] a. Intra-frame prediction (Intra (picture) Prediction): The reconstructed video signal used for prediction comes from the area that has been encoded and reconstructed in the same video frame, that is, the current block and the reference block are in the same video frame. Among them, the basic idea of ​​intra-frame prediction is to use the correlation of adjacent pixels in the same video frame to remove spatial redundancy. In video coding, adjacent pixels refer to the reconstructed pixels of the encoded CU around the current CU in the same video frame.

[0129] b. Inter-picture prediction: The reconstructed video signal used for prediction comes from other video frames that have been encoded and are different from the current frame, that is, the current block and the reference block are located in different video frames.

[0130] 3) Transform coding and quantization (Transform & Quantization): The residual video signal is transformed into the transform domain through DFT (Discrete Fourier Transform), DCT (Discrete Cosine Transform) and other transformation operations, which are called transform coefficients. The residual video signal in the transform domain is further subjected to lossy quantization operations, which loses certain information, making the quantized signal conducive to compression expression.

[0131] In some video coding standards, there may be more than one transform mode to choose from. Therefore, the encoder also needs to select one of the transforms for the current CU and inform the decoder. The degree of quantization is usually determined by QP (Quantization Parameters). When the QP value is large, the transform coefficients representing a larger value range will be quantized to the same output, which usually results in greater distortion and lower bitrates. On the contrary, when the QP value is small, the transform coefficients representing a smaller value range will be quantized to the same output, which usually results in less distortion and a higher bitrate.

[0132] 4) Entropy Coding or Statistical Coding: The quantized transform domain signal will be statistically compressed and encoded according to the frequency of occurrence of each value, and finally output as a binary (0 or 1) video stream. At the same time, the encoding will generate other information, such as the selected prediction coding mode, motion vector, etc. These other information also need to be entropy encoded to reduce the bit rate. Among them, statistical coding is a lossless coding method that can effectively reduce the bit rate required to express the same signal. Common statistical coding methods include variable length coding (VLC) or context-based binary arithmetic coding (CABAC).

[0133] 5) Loop Filtering: The decoded image corresponding to the encoded CU can be reconstructed after inverse quantization, inverse transformation and prediction compensation (the reverse operation of 2) to 4) above. Compared with the original image, the reconstructed decoded image has some information different from the original image due to the influence of quantization, resulting in distortion. Therefore, a filter can be used to filter the reconstructed decoded image to effectively reduce the degree of distortion caused by quantization. The filter can be, for example, a deblocking filter, SAO (sample adaptive offset) or ALF (adaptive loop filter), etc. Since these reconstructed decoded images after filtering will be used as reference CUs for other CUs that need to be encoded later and applied to the prediction process of other CUs, the above filtering operation is also called loop filtering, and filtering operation within the encoding loop.

[0134] Based on the above description of steps 1)-5), the present application embodiment provides a basic workflow diagram of a video encoder. Figure 1 , Figure 1 This is a basic working flow diagram of a video encoder provided by an embodiment of the present application. Figure 1 The current block is the kth CU in the current frame (current image) (such as Figure 1 The shown k [x,y]) is used as an example, k is a positive integer, and k is less than or equal to the total number of CUs contained in the current frame. k [x, y] represents the pixel point (referred to as pixel) with coordinates [x, y] in the kth CU, where x represents the horizontal coordinate of the pixel and y represents the vertical coordinate of the pixel; k[x, y] can obtain the prediction signal after motion compensation or intra-frame prediction. and the original signal s k [x, y] is subtracted to obtain the residual video signal u k [x,y]; then the residual video signal u k [x,y] is transformed and quantized. The output data of the quantization process has two different destinations, A and B:

[0135] A: The data output by the quantization process can be sent to an entropy encoder for entropy encoding to obtain an encoded bit stream (i.e., a video bit stream), and the bit stream is output to a buffer for storage and waiting to be transmitted.

[0136] B. The data output by the quantization process can be dequantized and inversely transformed to obtain the residual video signal u after inverse transformation. ′ k [x, y]. Then, the inverse transformed residual video signal u ′ k [x,y] and prediction signal Add together to get a new prediction signal And the new prediction signal The new prediction signal can then be sent to the current image buffer for storage. After intra-frame prediction processing, we get The new prediction signal After loop filtering, the reconstructed signal s can be obtained ′ k [x,y], and will reconstruct the signal s ′ k [x, y] is sent to the decoded image buffer for storage to generate the reconstructed video. Reconstructed signal s ′ k [x,y] is obtained through motion compensation prediction processing in Can represent the reference block, m x and m y Represent the horizontal and vertical components of the motion vector of the reference block, respectively.

[0137] In some embodiments, mainstream video coding standards such as HEVC, VVC, and AVS3 all adopt a block-based hybrid coding framework, which can divide the original video data into a series of coding blocks, and combine video coding methods such as prediction, transformation, and entropy coding to achieve video data compression. Among them, motion compensation is a commonly used prediction method for video coding. Motion compensation is based on the redundant characteristics of video content in the time domain or spatial domain, and derives the prediction value of the current block from the encoded reference block. This type of prediction method includes: inter-frame prediction, intra-frame block copy prediction, intra-frame string copy prediction, etc. In specific coding implementations, these prediction methods may be used alone or in combination. For coding blocks using these prediction methods, it is usually necessary to explicitly or implicitly encode one or more two-dimensional displacement vectors in the video bitstream. The displacement vector is used to indicate the displacement of the current block (or the same-position block of the current block) relative to one or more reference blocks.

[0138] It should be noted that in different prediction modes and different implementations, the displacement vector may have different names. This article uniformly describes them in the following way: 1) The displacement vector in inter-frame prediction is called motion vector (MV); 2) The displacement vector in intra-frame block copy prediction is called block vector (BV); 3) The displacement vector in intra-frame string copy prediction is called string vector (SV).

[0139] Next, the relevant background technologies involved in this application are introduced:

[0140] (1) Inter-frame prediction

[0141] Inter-frame prediction refers to: using the correlation of the video time domain to use the adjacent coded image pixels to predict the pixels of the current image, so as to effectively remove the video time domain redundancy and effectively save the bits of the coded residual data. See 2, Figure 2 is a schematic diagram of an inter-frame prediction provided by an embodiment of the present application. Figure 2 In the above example, P is the current frame, Pr is the reference frame, B is the current block, and Br is the reference block of B. B' and B have the same coordinate position in the image (i.e., B' is the same block as B), and the coordinates of Br are (x r ,y r ), the coordinates of B' are (x, y). The displacement between the current block and its reference block is called the motion vector (MV), that is: MV = (x r -x,y r -y).

[0142] Among them, considering that the temporal or spatial adjacent blocks have a strong correlation, the MV prediction technology can be used to further reduce the bits required for encoding the MV. In H.265 / HEVC, inter-frame prediction includes two MV prediction modes: Merge and AMVP (Advanced Motion Vector Prediction). The Merge mode and AMVP mode are described in detail below.

[0143] 1) Merge mode: An MV candidate list will be established for the current PU (Prediction Unit), which contains 5 candidate MVs (and their corresponding reference images). These 5 candidate MVs are traversed, and the one with the lowest rate-distortion cost is selected as the optimal MV. If the codec establishes the candidate list in the same way, the encoder only needs to transmit the index of the optimal MV in the candidate list. It should be noted that HEVC's MV prediction technology also has a skip mode, which is a special case of the Merge mode. After finding the optimal MV in the Merge mode, if the current block is basically the same as the reference block, there is no need to transmit the residual data, only the MV index and a skip flag.

[0144] The MV candidate list established in Merge mode includes two cases: spatial domain and temporal domain. For BSlice, it also includes a combined list. The spatial domain list provides up to 4 candidate MVs. The establishment of the spatial domain list is as follows: Figure 3a As shown, Figure 3a : is a schematic diagram of establishing an airspace list provided in an embodiment of the present application. The airspace list is established in the order of A1->B1->B0->A0->(B2), where B2 is a substitute, that is, when one or more of A1, B1, B0, and A0 do not exist, the motion information of B2 needs to be used; the temporal list only provides at most one candidate MV, and the establishment of the temporal list is as follows Figure 3b As shown, Figure 3b : is a schematic diagram of establishing a time domain list provided in an embodiment of the present application. The MV of the current PU can be scaled according to the MV of the same PU according to the following formula:

[0145] curMV=td * colMV / tb formula (1)

[0146] In the above formula (1), curMV and colMV represent the MV of the current PU and the MV of the co-located PU respectively, and td and tb represent the distances between the current image and the co-located image and their reference images respectively. If the PU at position D0 on the co-located block is not available, it is replaced by the co-located PU at position D1; for the PU in the B Slice, since there are two MVs, its MV candidate list also needs to provide two MVPs (Motion Vector Prediction). HEVC generates a combination list for B Slice by combining the first four candidate MVs in the MV candidate list in pairs.

[0147] 2) AMVP mode: Use the MV correlation of neighboring blocks in the spatial and temporal domains to establish a MV candidate list for the current PU. Unlike the Merge mode, the AMVP mode selects the optimal predicted MV from the established MV candidate list, and performs differential encoding with the optimal MV obtained through motion search of the current block to be encoded, that is, the encoded MVD = MV-MVP; the decoding end establishes the same MV candidate list, and only needs the sequence number (or index) of MVD and MVP in the MV candidate list to calculate the MV of the current decoding block. The AMVP candidate MV list also includes two cases, spatial domain and temporal domain. The difference is that the length of the AMVP list is 2.

[0148] HMVP is a new MV prediction technology adopted in H.266 / VVC. HMVP is a motion vector prediction method based on historical information. The motion vector MV of the historical coding block is stored in the HMVP list and used as the MVP of the current CU. H.266 / VVC adds HMVP to the MV candidate list of Merge mode, and its order is after the spatial and temporal MVPs. The HMVP technology stores the motion vector MV of the historical coding block in a first-in-first-out queue (FIFO). If the stored candidate MV is the same as the MV just encoded, the duplicate candidate MV will be removed, and then all candidate MVs in the HMVP will be moved forward, and the MV of the current coding unit CU will be added to the end of the FIFO. If the MV of the current coding unit is not the same as any candidate MV in the FIFO, the latest MV is added to the end of the FIFO. When adding a new MV to the HMVP list, if the current HMVP list has reached the maximum length, the first candidate MV in the FIFO is removed, and the latest MV is added to the end of the FIFO. The HMVP list will be reset (cleared) when a new CTU row is encountered. In H.266 / VVC, the HMVP list size S is set to 6. In order to reduce the number of redundant check operations, the following simplifications are introduced:

[0149] 1) Set the number of candidates in the HMVP list used for Merge list generation to (N<=4)?M:(8-N) where N represents the number of existing candidate MVs in the Merge list, and M represents the number of available HMVP candidate MVs in the table.

[0150] 2) Once the length of the available Merge list reaches the maximum allowed length minus 1, the HMVP merge candidate list construction process is terminated.

[0151] (2) Intra-block copy prediction

[0152] 1) Intra Block Copy (IBC) is an intra-frame coding tool adopted in the HEVC Screen Content Coding (Screen Cotent Coding, SCC) extension, which significantly improves the coding efficiency of screen content. In AVS3 and VVC, IBC technology is also adopted to improve the performance of screen content coding. This technology is called intra-frame block copy prediction technology in AVS3. IBC uses the spatial correlation of screen content video and uses the pixel values ​​of the encoded coding block in the current image to predict the pixel values ​​of the current coding block. There is no need to encode the block vector residual value (Block Vector Difference, BVD) of the current coding block, which can effectively save the bits required for encoding pixels. Please refer to Figure 4a , Figure 4a FIG. 1 is a flow chart of intra-frame block copy prediction provided by an embodiment of the present application. Figure 4a As shown in the figure, the displacement between the current block and its reference block in IBC (i.e., the displacement between the upper left corner position coordinates of the current block and the upper left corner position coordinates of the reference block) is called a block vector (BV for short), where the reference block refers to a coded block that has been coded, so the reference block is located in the coded area. H.266 / VVC uses a BV prediction technology similar to inter-frame prediction to further save the bits required for encoding BV.

[0153] 2) Judgment of the legitimacy of intra-frame block copy prediction

[0154] In the AVS3 standard, the reference block pointed to by the block vector decoded from the video code stream that conforms to the AVS3 standard should be limited to the current maximum coding unit or the range of N (N is a positive integer) maximum coding units to the left. All reference samples of the reference block should belong to the same slice as the samples of the current block and have been completely reconstructed. The value of N is as follows:

[0155]

[0156] Let the coordinates of the upper left corner of the current block be (xCur, yCur), the coordinates of the upper left corner of the reference block luminance component be (xRefTL, yRefTL), and the coordinates of the lower right corner of the reference block luminance component be (xRefBR, yRefBR). The coordinates (xRefTL, yRefTL) and (xRefBR, yRefBR) should satisfy the following conditions:

[0157]

[0158] When the upper left corner of the reference block pointed to by the block vector is in the left adjacent maximum coding unit, and the size of the maximum coding unit is 128×128, the upper left corner of the 64×64 area where the upper left corner of the reference block is shifted 128 pixels to the right should not have been reconstructed, and the coordinates (xCur, yCur) should meet the following conditions:

[0159]

[0160] (3) Intra-frame string duplication prediction

[0161] 1) Intra String Copy (ISC) is a technology that divides a coding block into a series of pixel strings or unmatched pixels according to a certain scanning order (raster scanning, round-trip scanning, and Zig-Zag scanning, etc.). In AVS3, ISC prediction modes can specifically include: string copy intra prediction normal string sub-mode, string copy intra prediction non-normal string sub-mode. Similar to IBC, each string searches for a reference string of the same shape in the encoded area of ​​the current image, and then derives the prediction value of the current string based on the reference string. By encoding the string vector difference (SVD) between the current string pixel value and the prediction value instead of directly encoding the string pixel value, it can effectively save bits. See Figure 4b , Figure 4b is a flow chart of intra-frame string duplication prediction provided by an embodiment of the present application, such as Figure 4b As shown, the string containing 14 pixels is the current string 1 to be encoded. In the encoded area, a reference string 1 with the same shape as the current string 1 (i.e., also containing 14 pixels) can be determined. Therefore, the displacement between the current string 1 to be encoded and the reference string 1 can be called string vector 1. Similarly, the string containing 15 pixels is the current string 2 to be encoded. In the encoded area, a reference string 2 with the same shape as the current string 2 to be encoded (i.e., also containing 15 pixels) can be determined. Therefore, the displacement between the current string 2 to be encoded and the reference string 2 can be called string vector 2. In addition, a black pixel represents an unmatched pixel.

[0162] Among them, the intra-frame string copy prediction technology needs to encode the string vector (StringVector, referred to as SV), string length, and matching string flag corresponding to each string in the current block. Among them, the string vector (SV) represents the displacement between the current string and its reference string; the string length represents the number of pixels contained in the current string; the matching string flag is used to indicate whether there is a matching coding string for the current string. If the value of the matching string flag is 1, the current string has a matching coding string. If the value of the matching string flag is 0, the current string does not have a matching coding string.

[0163] 2) Reference range limitation for intra-frame copy prediction

[0164] In the AVS3 standard, any reference pixel in the reference string pointed to by the string vector or the common position pointed to by the point vector decoded from the video code stream that conforms to the AVS3 standard should be limited to the current maximum coding unit or the range of the N maximum coding units to the left. The value of N is as follows:

[0165]

[0166] IscSubtypeFlag indicates the type of the current string. A value of 0 indicates a normal string, and a value of 1 indicates a non-normal string. If the value of IscSubtypeFlag is equal to 0, it means that the current string is a normal string. Let the coordinates of the upper left corner of the current block be (xCur, yCur), and the coordinates of the position of any brightness component A and B of the reference string be (xRefA, yRefA) and (xRefB, yRefB) respectively. The coordinates (xRefA, yRefA) and (xRefB, yRefB) should meet the following conditions:

[0167]

[0168] If the value of IscSubtypeFlag is equal to 0, and when any luminance component A of the reference string falls on the left adjacent maximum coding unit, and the size of the maximum coding unit is 128×128, the upper left corner of the 64×64 area where the luminance component A is shifted 128 pixels to the right should not have been reconstructed, and the coordinates (xCur, yCur) should meet the following conditions:

[0169]

[0170] If the value of IscSubtypeFlag is equal to 0, all reference samples in the reference string of the current string should be in the same slice as the samples in the current string and have been fully reconstructed. If the value of the string vector Y component is greater than or equal to 0, any reference pixel in the reference string pointed to by the string vector should not be in the current string.

[0171] If the value of IscSubtypeFlag is equal to 1, let the coordinates of the upper left corner of the current block be (xCur, yCur), and the coordinates of the position of the brightness component A of the constant position be (xRef, yRef). The coordinates (xCur, yCur) and (xRef, yRef) should meet the following conditions:

[0172]

[0173] Let numOfP = IscNumOfNewPv + IscNumofReusedPv, if the value of IscSubtypeFlag = 1:

[0174] i. If the product of the width and height of the current coding unit is greater than 64, the value of numOfP should not be greater than 15;

[0175] ii. If the product of the width and height of the current coding unit is greater than 32 (i.e., 8×8, 4×16, or 16×4), numOfP≤10;

[0176] iii. If the product of the width and height of the current coding unit is greater than 16 (ie, 8×4 or 4×8), numOfP≤5;

[0177] iv. If the product of the width and the height of the current coding unit is 4×4, numOfP≤2.

[0178] The point vectors used by the current coding unit are stored in the point prediction information table PpInfoList

[15] [2]. The point vectors in PpInfoList

[15] [2] consist of two parts: one part comes from the historical point prediction information table PrevPpInfoList

[28] [2], where the number of point vectors in the historical point prediction information table is ≤28; the other part is the point vector that appears newly in the current coding unit.

[0179] If the current coding unit uses a normal string sub-mode, the sum of the number of matching strings, the number of incomplete matching strings containing at least one matching sample, the number of unmatched samples, and IscPartNumSplit should be less than or equal to one quarter of the number of samples in the current coding unit; otherwise, if the current coding unit uses a non-normal string sub-mode, the sum of the number of equivalent strings, the number of unit basis vector strings, the number of unmatched samples, and IscPartNumSplit should be less than or equal to one quarter of the number of samples in the current coding unit.

[0180] It should be noted that the reference range limit of the intra-block copy prediction mode and the reference range limit of the intra-string copy prediction mode may have different limit methods in different standards. In this application, the displacement vector is a legal vector, which means that the displacement vector should meet the reference range limit, and the specific limit method is not limited to the reference range limit of AVS3.

[0181] 3) Intra-frame string duplication prediction of non-ordinary string sub-mode

[0182] The intra-frame string copy prediction non-ordinary string mode is also called the equal value string and unit vector string mode. This method was adopted into the AVS3 standard in October 2020. See Figure 4c , Figure 4c is a schematic diagram of an intra-frame string duplication prediction non-ordinary string sub-mode provided by an embodiment of the present application, such as Figure 4c As shown, similar to the intra-frame string copy prediction mode, this mode divides a coding / decoding block into a series of pixel strings or unmatched pixels according to a certain scanning order. The type of pixel string can be an equal value string or a unit base vector string. The characteristic of an equal value string is that all pixels in the pixel string have the same prediction value. The characteristic of a unit vector string (also called a unit base vector string, a unit offset string, a copy above string, etc.) is that its displacement vector is (0, -1), and its implementation is relatively simple. Each pixel of the string uses the pixel above as the prediction value of the current pixel. The equal value string mode requires encoding the type, length and prediction value information of each string of the current coding block in the bitstream. Accordingly, the decoding end derives the prediction sample from the information decoded from the bitstream.

[0183] (4) Normal intra prediction mode

[0184] In AVS3, intra prediction modes may include: intra block copy prediction, intra string copy prediction, and normal intra prediction mode. Among them, normal intra prediction mode can be divided into: normal luma intra prediction mode, normal chroma intra prediction mode, and spatial angle weighted prediction luma intra prediction mode. In this application, chroma block copy intra prediction mode can also be used as a sub-mode of normal chroma intra prediction mode.

[0185] 1) Normal brightness intra prediction mode:

[0186] See also Figure 4d , Figure 4d is a schematic diagram of a common brightness intra-frame prediction mode provided by an embodiment of the present application. Figure 4dAs shown in the figure, the normal brightness intra prediction mode with the value of IntraLumaPredMode being 0 is the DC mode; the normal brightness intra prediction mode with the value of IntraLumaPredMode being 1 is the Planer mode; and the normal brightness intra prediction mode with the value of IntraLumaPredMode being 2 is the Bilinear mode. Similarly, according to different IntraLumaPredMode values, the normal brightness intra prediction mode can be divided into various modes as shown in Table 1 below.

[0187] Table 1. Common luma intra prediction modes

[0188] - IntraLumaPredMode Intra prediction mode 0 Intra_Luma_DC 1 Intra_Luma_Plane 2 Intra_Luma_Bilinear 3～11 Intra_Luma_Angular 12 Intra_Luma_Vertical 13～23 Intra_Luma_Angular 24 Intra_Luma_Horizontal 25～32 Intra_Luma_Angular 33 Intra_Luma_PCM 34～65 Intra_Luma_Angular

[0189] In Table 1 above, Intra_Luma_Horizontal, Intra_Luma_Vertical, and Intra_Luma_Angular are angle prediction modes, and the others are non-angle prediction modes.

[0190] 2) Normal chroma intra prediction mode:

[0191] Table 2. Chroma prediction block intra prediction modes

[0192]

[0193]

[0194] In Table 2 above, TSCPM, PMC, and EPMC are cross-component prediction modes, and the DM mode is an inheritance mode.

[0195] (5) Template matching

[0196] In order to further improve the compression efficiency of the VVC standard, a template matching (TM) is proposed to refine the motion on the decoding side. In TM mode, the motion is refined by building a template from the adjacent reconstructed samples on the left and above and finding the closest match between the template in the current image and the reference frame.

[0197] See also Figure 4e , Figure 4e is a schematic diagram of a template matching process provided in an embodiment of the present application. Figure 4eAs shown, a better MV is searched around the initial motion of the current CU in the [-8, +8] pixel search range. Template matching was previously proposed in JVET-J0021. In ECM, template matching also adopts some modifications: the search step size is determined by the AMVR mode, and TM can be cascaded with the bilateral matching process. In addition, template matching is also used to determine the displacement vector, the displacement vector offset, the merge index of the reordered merge candidate list and the AMVP candidate list, the reference line of the intra prediction mode, and so on.

[0198] (6) Intra-frame prediction motion vector prediction in AVS3

[0199] Intra-block copy (IBC) and intra-string copy (ISC) are two screen content coding tools in AVS3. They both use the current image as a reference and derive the prediction value of the coding unit through motion compensation. Considering that IBC and ISC have similar reference areas, the block vector (BV) and string vector (SV) have a high correlation, and the coding efficiency can be further improved by allowing prediction between the two. AVS3 uses an intra-prediction history motion information table (IntraHMVP) similar to HMVP to record the displacement vector information, size information and number of repetitions of these two types of coding blocks, and derives the prediction block vector (BVP) and prediction string vector (SVP) from IntraHMVP. In order to support parallel coding, if the current maximum coding unit is the first maximum coding unit of the current row in the current slice, the value of CntIntraHmvp in the intra-prediction history motion information table is initialized to 0.

[0200] 1) For IBC coded blocks, AVS3 uses a method called Class based Block Vector Prediction (CBVP) to derive BVP. This method first classifies the candidate BVs in IntraHMVP according to the following conditions:

[0201] Category 0: The area of ​​the historical coding block is greater than or equal to 64 pixels;

[0202] Category 1: The frequency of BV is greater than or equal to 2;

[0203] Category 2: The coordinates of the upper left corner of the historical coding block are located to the left of the coordinates of the upper left corner of the current block;

[0204] Category 3: The coordinates of the upper left corner of the historical coding block are above the coordinates of the upper left corner of the current block;

[0205] Category 4: The coordinates of the upper left corner of the historical coding block are located to the upper left of the coordinates of the upper left corner of the current block;

[0206] Category 5: The coordinates of the upper left corner of the historical coding block are located above and to the right of the coordinates of the upper left corner of the current block;

[0207] Category 6: The coordinates of the upper left corner of the historically encoded block are located to the lower left of the coordinates of the upper left corner of the current block.

[0208] In addition, a displacement vector index (can be referred to as index for short) can be encoded in the video bitstream, which represents the index of the category corresponding to the best candidate BV in the CBVP list. For example, if the index is "6", it means that the category corresponding to the best candidate BV in the CBVP list is category 6; subsequently, the decoding end can decode the corresponding BVP from the CBVP list according to the index.

[0209] 2) For ISC coding blocks, similarly, each string needs to encode an index, which is used to indicate the position of the SVP of the corresponding string in IntraHMVP. Similar to the skip mode in inter-frame prediction, the SV of the current string is equal to the SVP, so there is no need to encode the string vector residual value SVD between SV and SVP.

[0210] (7) Video image color format

[0211] YUV is a color encoding format that is mainly used to optimize the transmission of color video signals. Compared with RGB video signal transmission, its biggest advantage is that it only takes up very little bandwidth (RGB requires three independent video signals to be transmitted simultaneously). "Y" represents brightness or luminance (Luminance or Luma), which is the grayscale value; while "U" and "V" represent chrominance (Chrominance or Chroma), which is used to describe the color and saturation of the image and is used to specify the color of the pixel. Among them, "luminance" is established by the RGB input signal by superimposing specific parts of the RGB signal together; "chrominance" defines two aspects of color: hue and saturation, represented by Cr and Cb respectively, where Cr is used to reflect the difference between the red part of the RGB input signal and the luminance value of the RGB signal, and Cb reflects the difference between the blue part of the RGB input signal and the luminance value of the RGB signal. The importance of using the YUV color format is that its luminance signal Y and chrominance signals U and V are separated. If an image only includes the Y component (luminance component) but not the U and V components (chrominance components), then the image is a black and white grayscale image. In daily life, color televisions use the YUV format to describe images in order to solve the compatibility problem between color televisions and black-and-white televisions based on the brightness component Y, ensuring that black-and-white televisions can also receive color television signals.

[0212] The storage format of the YUV video code stream is actually closely related to its sampling method. Figure 4f , Figure 4f is a schematic diagram of a video sampling method provided in an embodiment of the present application, such as Figure 4fAs shown in the figure, the mainstream YUV sampling formats mainly include the following three: YUV4:4:4, YUV4:2:2, YUV4:2:0. Among them, the symbol "A:B:C" is used to describe the sampling frequency of U and V relative to Y.

[0213] YUV4:4:4 sampling means that the chroma channels are not downsampled, that is, each Y component corresponds to a set of UV components in this sampling mode.

[0214] YUV4:2:2 sampling means: horizontal downsampling is performed in 2:1 format, and there is no vertical downsampling. For each scan line, every two U or V samples contain four Y samples; that is, in this sampling mode, every two Y components share a set of UV components.

[0215] YUV4:2:2 sampling means: horizontal downsampling is performed in 2:1 format and vertical downsampling is performed in 2:1 format, that is, every four Y components share a set of UV components in this sampling mode.

[0216] (8) Block partition structure in AVS3

[0217] 1) Basic block division structure

[0218] AVS3 uses a basic block partitioning structure based on QT, BT and EQT. The previous generation AVS2 standard uses a quadtree (QT) partitioning structure, which divides a CU into four sub-CUs. In addition to supporting quadtree partitioning, AVS3 also supports binary tree (BT) and EQT partitioning. Among them, BT can divide a CU into two sub-CUs, left and right / upper and lower; EQT includes two I-shaped partitioning methods, horizontal and vertical, which divides a CU into 4 sub-CUs.

[0219] The representation of the basic block division structure of QT, BT and EQT in AVS3 in the video bitstream can be found in Figure 4g As shown, Figure 4g This is a schematic diagram of a basic block division structure provided by an embodiment of the present application. Figure 4g As shown, for any coding unit (CU), first determine whether to perform QT ​​division. If so, directly perform quadtree division to obtain QT; if not, it is necessary to further determine whether to perform non-split (Non-Split), and if not, end the determination; if division is required, it is necessary to further determine whether to perform EQT division or BT division. If EQT division is required, it is necessary to determine whether to perform horizontal division (Hor) or vertical division (Ver); similarly, if BT division is required, it is also necessary to determine whether to perform horizontal division (Hor) or vertical division (Ver). Finally, according to the above division method, the current coding unit can be divided into the basic block structure of QT, BT, or EQT.

[0220] 2) Chroma tree partitioning restrictions

[0221] In AVS3, the luminance component and chrominance component of the coding unit in the video code stream are usually divided and encoded using the same block division structure, that is, the luminance component of the coding unit is divided into multiple luminance blocks, and the chrominance component of the coding unit is divided into multiple chrominance blocks. In order to reduce complexity, AVS3 limits the minimum length or width of the luminance block and the chrominance block to at least 4 pixels. For the coding unit in the YUV420 format, 4 pixels of the chrominance component correspond to 8 pixels of the luminance component; when the coding unit is divided into blocks, if the block size after division is less than the minimum value, the division will not be performed. Since the chrominance block has a smaller size in the YUV420 format, there may be a situation where the chrominance block has stopped dividing and the luminance block continues to divide. In this case, one chrominance block may correspond to multiple luminance blocks. This situation where the luminance block and chrominance block division structure are inconsistent is also called Local Separate Tree (LST for short). AVS3 allows luminance and chrominance to be encoded separately. In particular, for the chrominance block, the encoding mode will be selected according to the information of the luminance sample corresponding to the chrominance sample position at the lower right corner. If the luma block uses the inter-frame prediction mode, the chroma block will use the reference frame and displacement vector information of the luma block to perform inter-frame motion compensation and derive the prediction value; otherwise, the chroma block will derive the prediction value from the available intra-frame prediction mode, where the chroma block allows the selection of an intra-frame prediction mode different from that of the luma block.

[0222] (9) Interpolation of chroma block copy intra prediction

[0223] 1) Chroma samples

[0224] See also Figure 4h , Figure 4h Schematic diagram of the position of a chroma block copy intra prediction provided by an embodiment of the present application. The sample position in the chroma sample matrix of the reference image is as follows: Figure 4h As shown, A, B, C, and D are adjacent integer pixel samples, dx and dy are the horizontal and vertical distances between the sub-pixel sample a (dx, dy) around the integer pixel sample A and A, dx is equal to fx & 1, dy is equal to fy & 1, and (fx, fy) is the coordinate of the sub-pixel sample in the chroma sample matrix with 1 / 2 precision. Further, Figure 4i is a schematic diagram of the position of a sub-pixel sample point provided in an embodiment of the present application. The specific positions of the integer pixel Ax, y and the surrounding three sub-pixel sample points ax, y (dx, dy) are detailed in Figure 4i .

[0225] 2) Chroma filter coefficient

[0226] Table 3. Chroma filter coefficients

[0227] Array Identification Filter coefficients C[0] {64,0} C[1] {32,32}

[0228] For sub-pixel points with dx=0 or dy=0, they can be directly interpolated based on the chrominance integer pixel points; for points where both dx and dy are not equal to 0, sub-pixel points on the integer pixel row (dy==0) are used for calculation:

[0229]

[0230] Where a'x,y(dx,0) is the temporary value of the sub-pixel on the integer pixel row, defined as:

[0231]

[0232] 2. Video Decoding

[0233] On the decoding side, for each CU, after obtaining the video bitstream, on the one hand, the video bitstream is first entropy decoded to obtain information on various prediction coding modes and quantized transform coefficients, and then each transform coefficient is dequantized and inversely transformed to obtain a residual video signal (also called a residual value). On the other hand, based on the known information on the prediction coding mode, the prediction signal (also called a prediction value) corresponding to the CU can be obtained, and the residual video signal and the prediction signal are added to obtain a reconstructed video signal, which can be used to reconstruct the decoded image corresponding to the CU. Finally, the reconstructed video signal needs to undergo a loop filtering operation to generate the final output signal.

[0234] Based on the above description, the embodiment of the present application provides a video processing solution, which is applicable to a video encoder or video compression product using motion compensation technology (such as inter-frame prediction, intra-frame block copy, and intra-frame string copy mode), which is beneficial to improving the compression performance of the video. Specifically, the general principle of the video processing solution is as follows:

[0235] At the encoding end: First, the current chroma block in the video can be determined, and the current chroma block adopts the chroma block copy intra-frame prediction mode; then, the target displacement vector of the luminance co-located block corresponding to the current chroma block can be obtained; next, based on the target displacement vector, the prediction value of the current chroma block is determined. Finally, the video can be encoded based on the prediction value of the current chroma block to generate a video stream, and the video stream is sent to the decoding end.

[0236] At the decoding end: after receiving the video bitstream sent by the encoding end, the current chroma block in the video bitstream can be determined, and the current chroma block adopts the chroma block copy intra-frame prediction mode; then, the target displacement vector of the luminance co-located block corresponding to the current chroma block can be obtained; next, based on the target displacement vector, the prediction value of the current chroma block is determined. Finally, the prediction value of the current chroma block can be used to reconstruct the decoded image corresponding to the chroma block.

[0237] As can be seen from the above, in the video encoding and decoding process, the present application proposes a new prediction mode for the current chroma block, that is, when it is determined that the current chroma block adopts the chroma block copy intra-frame prediction mode, the displacement vector information of the luminance co-located block can be used to determine the prediction value of the current chroma block. There is no need to perform intra-frame prediction processing on the current chroma block, which can improve the encoding and decoding efficiency of the current chroma block, thereby effectively improving the encoding performance.

[0238] Next, the video processing system provided in the embodiment of the present application is described.

[0239] See also Figure 5 , Figure 5 A schematic diagram of the architecture of a video processing system provided in an embodiment of the present application. The video processing system 50 may include an encoding device 501 and a decoding device 502. The encoding device 501 is located at the encoding end, and the decoding device is located at the decoding end. The encoding device 501 can be a terminal or a server; the decoding device 502 can be a terminal or a server, and a communication connection can be established between the encoding device 501 and the decoding device 502 in a wired or wireless manner. Among them, the terminal can be a mobile phone, a tablet computer, a laptop computer, a PDA, a mobile Internet device (MID, mobile internet device), a vehicle-mounted device, an aircraft, a wearable device (such as a smart watch, a smart bracelet, a pedometer, etc.), a virtual reality device (such as a VR (Virtual Reality) device, an AR (Augmented Reality, augmented reality) device), and the like. The server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), as well as big data and artificial intelligence platforms.

[0240] (1) For the encoding device 501:

[0241] The encoding device 501 can obtain a video to be encoded, which can be obtained by shooting with a camera or generated by a computer. The camera can be a hardware component provided in the encoding device 501, for example, the camera can be a common camera, a stereo camera, a light field camera, etc. provided in a terminal. The camera can also refer to a hardware device connected to the encoding device 501, such as a camera connected to a server.

[0242] Wherein, a video includes one or more video frames, and the encoding device 501 can divide each video into one or more CUs and encode each CU. When encoding any CU, first, the current chroma block being encoded can be determined, and the current chroma block adopts the chroma block copy intra-frame prediction mode; then, the target displacement vector of the luminance co-location block corresponding to the current chroma block is obtained; next, based on the target displacement vector, the prediction value of the current chroma block is determined; wherein, the prediction value of the current chroma block can be understood as the prediction signal corresponding to the current chroma block, and the prediction value of the current chroma block can be used to reconstruct the decoded image corresponding to the current chroma block; finally, the video is encoded based on the prediction value of the current chroma block to generate a video bitstream.

[0243] Then, the encoding device 501 performs transform coding, quantization, entropy coding and other operations on the video based on the predicted value of the chrominance block of the current CU to obtain a video code stream, and sends the video code stream to the decoding device 502 so that the decoding device 502 decodes the video code stream.

[0244] (2) For the decoding device 502:

[0245] After receiving the video code stream sent by the encoding device 501, the decoding device 502 can decode the video code stream and reconstruct the video corresponding to the video code stream. Specifically: when the decoding device 502 decodes any CU, first, it can determine the current chroma block of the CU being decoded (hereinafter referred to as the current coding unit), and the current chroma block adopts the chroma block replication intra-frame prediction mode; then, obtain the target displacement vector of the luminance co-location block corresponding to the current chroma block; next, based on the target displacement vector, determine the prediction value of the current chroma block; wherein the prediction value of the current chroma block can be understood as the prediction signal corresponding to the current chroma block, and the prediction value of the current chroma block can be used to reconstruct the decoded image corresponding to the current chroma block; finally, based on the prediction value of the current chroma block, encode the video to generate a video code stream.

[0246] Whether the current chroma block adopts the intra-frame copy prediction mode is indicated by the mode indication information obtained after decoding the video code stream. The mode indication information includes at least one of the following: sequence header flag, picture header flag, slice header flag, coding unit flag, implicit information, and context model indication information.

[0247] 1) If the mode indication information is a sequence header flag, the mode indication information is set in the sequence header of the current sequence to which the current chroma block belongs; when the sequence header flag is a first preset value, it indicates that the current chroma block allows the use of the chroma block copy intra-frame prediction mode, and when the sequence header flag is a second preset value, it indicates that the current chroma block does not allow the use of the chroma block copy intra-frame prediction mode; 2) If the mode indication information is an image header flag, the mode indication information is set in the image header of the current image to which the current chroma block belongs; when the image header flag is a first preset value, it indicates that the current chroma block allows the use of the chroma block copy intra-frame prediction mode, and when the image header flag is a second preset value, it indicates that the current chroma block does not allow the use of the chroma block copy intra-frame prediction mode; 3) If the mode indication information is a slice header flag, the mode indication information is set in the slice header of the current slice to which the current chroma block belongs; when the slice header flag is a first preset value, it indicates that the current chroma block allows the use of the chroma block copy intra-frame prediction mode, and when the slice header flag is a second preset value, it indicates that the current chroma block does not allow the use of the chroma block copy intra-frame prediction mode. 4) If the mode indication information is a coding unit flag, the mode indication information is set in the current coding unit; when the coding unit flag is a first preset value, it indicates that the current chroma block adopts the chroma block copy intra-frame prediction mode, and when the coding unit flag is a second preset value, it indicates that the current chroma block does not adopt the chroma block copy intra-frame prediction mode; 5) If the mode indication information is implicit information, when the implicit information indicates that the luminance sample at a preset position in the luminance co-location block corresponding to the current chroma block meets the preset condition, it indicates that the current chroma block adopts the chroma block copy intra-frame prediction mode; 6) If the mode indication information is context model indication information, when the context model indication information; 7) If the mode indication information is chroma indication information, when the chroma indication information indicates that the current chroma block adopts a preset intra-frame prediction mode (for example, DM mode), and the luminance sample at a preset position in the luminance co-location block meets the preset condition, it indicates that the current chroma block adopts the chroma block copy intra-frame prediction mode. It can be seen that the embodiments of the present application can obtain mode indication information for indicating the prediction mode of the current chroma block mode by decoding the video code stream, and when the current chroma block adopts the chroma block copy intra-frame prediction mode, the target displacement vector of the luminance co-located block corresponding to the current chroma block can be used to determine the prediction value of the current chroma block.

[0248] Furthermore, if it is determined that the current chroma block needs to be residually decoded, the decoding device 502 performs inverse quantization and inverse transformation on the quantized transform coefficients to obtain the residual value of the current chroma block, and superimposes the predicted value and the residual value of the current chroma block to obtain the reconstruction value of the current chroma block, and then reconstructs the decoded image corresponding to the current chroma block according to the reconstruction value. The decoded image can be used as a reference image used when decoding other CUs, and can also be used to reconstruct video.

[0249] In the video encoding and decoding process, the embodiment of the present application proposes a new prediction mode for the current chroma block, that is, when it is determined that the current chroma block adopts the chroma block copy intra-frame prediction mode, the displacement vector information of the luminance co-located block can be used to determine the prediction value of the current chroma block. There is no need to perform intra-frame prediction processing on the current chroma block, which can improve the encoding and decoding efficiency of the current chroma block, thereby effectively improving the encoding performance.

[0250] Next, the video processing method provided by the embodiment of the present application is described. Figure 6 , Figure 6 The following is a flow chart of a video processing method provided in an embodiment of the present application. The video processing method may be executed by a decoding device in the above-mentioned video processing system. The video processing method described in this embodiment may include the following steps S601-S603:

[0251] S601: Determine a current chroma block in a video bitstream, and the current chroma block adopts a chroma block copy intra prediction mode.

[0252] In this application, the current chroma block refers to the chroma block currently being decoded, and the chroma block refers to a coding block that only contains chroma components. In other words, the chroma block refers to the block of chroma components in the coding unit, and the luminance block refers to the block of luminance components in the coding unit; wherein the blocks here may include but are not limited to: coding unit, prediction unit, transform unit, prediction block, transform block.

[0253] In a possible implementation, a video bitstream is decoded to obtain mode indication information, where the mode indication information is used to indicate whether the current chroma block adopts an intra-frame copy prediction mode; wherein the mode indication information includes at least one of the following: a sequence header flag, an image header flag, a slice header flag, a coding unit flag, implicit information, and chroma indication information. The chroma block copy intra-frame prediction mode is a prediction mode independent of the normal intra-frame prediction mode; or, the chroma block copy intra-frame prediction mode is a sub-mode of the normal intra-frame prediction mode. The normal intra-frame prediction mode includes at least one of the following: an intra-frame prediction DC coefficient mode, an angle-based intra-frame prediction mode, a cross-component two-step prediction mode, and a prediction mode based on intra-frame reference pixels to derive prediction values.

[0254] 1) If the mode indication information is a sequence header flag seq_cibc_flag, the mode indication information is set in the sequence header of the current sequence to which the current chroma block belongs; when the sequence header flag seq_cibc_flag is a first preset value (such as 1), it indicates that the current chroma block allows the use of the chroma block copy intra-frame prediction mode, and when the sequence header flag seq_cibc_flag is a second preset value (such as 0), it indicates that the current chroma block does not allow the use of the chroma block copy intra-frame prediction mode. In this case, the chroma block copy intra-frame prediction mode is a prediction mode independent of the normal intra-frame prediction mode.

[0255] Optionally, whether the current chroma block copy intra prediction mode is used may be determined according to other sequence headers, including the following implementations:

[0256] a. There is a sequence header flag seq_ibc_flag in the video bitstream indicating whether IBC is used. When seq_ibc_flag is 1, seq_cibc_flag is decoded;

[0257] b. There is a sequence header flag seq_isc_flag in the video bitstream indicating whether ISC is used. When seq_isc_flag is 1, seq_cibc_flag is decoded;

[0258] c. There is a sequence header flag seq_isc_ordinary_flag in the video bitstream indicating whether the ISC ordinary string mode is used. When seq_isc_ordinary_flag is 1, seq_cibc_flag is decoded;

[0259] d. The video bitstream contains a sequence header flag seq_isc_non_ordinary_flag indicating whether the ISC non-ordinary string mode is used. When seq_isc_non_ordinary_flag is 1, seq_cibc_flag is decoded;

[0260] e. A combination of the above methods ad. For example, if seq_ibc_flag and seq_isc_flag exist in the video bitstream, seq_cibc_flag is decoded only when at least one of them has a value of 1.

[0261] f. The above methods ad may be implicit methods, that is, there is no need to decode seq_cibc_flag from the video bitstream, and the flag seq_cibc_flag is directly determined according to the values ​​of the above flags (seq_ibc_flag, seq_isc_flag, seq_isc_ordinary_flag, seq_isc_non_ordinary_flag). For example, if seq_ibc_flag and seq_isc_flag exist in the video bitstream, and at least one of them has a value of 1, then the value of seq_cibc_flag is set to 1; or if seq_ibc_flag exists in the video bitstream and its value is 1, then the value of seq_cibc_flag is set to 1; or if seq_isc_flag exists in the bitstream and its value is 1, then the value of seq_cibc_flag is set to 1.

[0262] 2) If the mode indication information is the picture header flag pic_cibc_flag, the mode indication information is set in the picture header of the current picture to which the current chroma block belongs; when the picture header flag pic_cibc_flag is a first preset value (such as 1), it indicates that the current chroma block allows the use of the chroma block copy intra-frame prediction mode, and when the picture header flag pic_cibc_flag is a second preset value (such as 0), it indicates that the current chroma block does not allow the use of the chroma block copy intra-frame prediction mode. In this case, the chroma block copy intra-frame prediction mode is a prediction mode independent of the ordinary intra-frame prediction mode.

[0263] Optionally, whether the current chroma block copy intra prediction mode is used may be determined according to other sequence headers, including the following implementations:

[0264] a. There is a picture header flag pic_ibc_flag in the video bitstream indicating whether IBC is used. When pic_ibc_flag is 1, pic_cibc_flag is decoded;

[0265] b. There is a picture header flag pic_isc_flag in the video bitstream indicating whether ISC is used. When pic_isc_flag is 1, pic_cibc_flag is decoded;

[0266] c. There is a picture header flag pic_isc_ordinary_flag in the video bitstream indicating whether the ISC ordinary string mode is used. When pic_isc_ordinary_flag is 1, pic_cibc_flag is decoded;

[0267] d. The video bitstream contains a picture header flag pic_isc_non_ordinary_flag indicating whether the ISC non-ordinary string mode is used. When pic_isc_non_ordinary_flag is 1, pic_cibc_flag is decoded;

[0268] e. A combination of the above methods ad. For example, if pic_ibc_flag and pic_isc_flag exist in the video bitstream, pic_cibc_flag is decoded only when at least one of them has a value of 1.

[0269] f. The above methods ad can be implicit methods, that is, there is no need to decode pic_cibc_flag from the video bitstream, and the flag pic_cibc_flag is directly determined according to the values ​​of the above flags (pic_ibc_flag, pic_isc_flag, pic_isc_ordinary_flag, pic_isc_non_ordinary_flag). For example, if pic_ibc_flag and pic_isc_flag exist in the video bitstream, and at least one of them has a value of 1, then the value of pic_cibc_flag is set to 1; or if pic_ibc_flag exists in the video bitstream and its value is 1, then the value of pic_cibc_flag is set to 1; or if pic_isc_flag exists in the bitstream and its value is 1, then the value of pic_cibc_flag is set to 1.

[0270] 3) If the mode indication information is a slice header flag patch_cibc_flag, the mode indication information is set in the slice header of the current slice to which the current chroma block belongs; when the slice header flag patch_cibc_flag is a first preset value (such as 1), it indicates that the current chroma block allows the use of the chroma block copy intra-frame prediction mode, and when the slice header flag patch_cibc_flag is a second preset value (such as 0), it indicates that the current chroma block does not allow the use of the chroma block copy intra-frame prediction mode. In this case, the chroma block copy intra-frame prediction mode is a prediction mode independent of the normal intra-frame prediction mode.

[0271] Optionally, whether the current chroma block copy intra prediction mode is used may be determined according to other sequence headers, including the following implementations:

[0272] a. The video bitstream contains a patch_ibc_flag header flag indicating whether IBC is used. When patch_ibc_flag is 1, patch_cibc_flag is decoded;

[0273] b. There is a patch_isc_flag in the video bitstream that indicates whether ISC is used. When patch_isc_flag is 1, patch_cibc_flag is decoded;

[0274] c. There is a patch_isc_ordinary_flag in the video stream that indicates whether the ISC ordinary string mode is used. When patch_isc_ordinary_flag is 1, patch_cibc_flag is decoded;

[0275] d. There is a patch_isc_non_ordinary_flag in the video bitstream to indicate whether the ISC non-ordinary string mode is used. When patch_isc_non_ordinary_flag is 1, patch_cibc_flag is decoded;

[0276] e. A combination of the above methods ad. For example, if patch_ibc_flag and patch_isc_flag exist in the video bitstream, patch_cibc_flag is decoded only when at least one of them has a value of 1.

[0277] f. The above methods ad can be implicit methods, that is, there is no need to decode patch_cibc_flag from the video bitstream, and the flag patch_cibc_flag is directly determined according to the values ​​of the above flags (patch_ibc_flag, patch_isc_flag, patch_isc_ordinary_flag, patch_isc_non_ordinary_flag). For example, if patch_ibc_flag and patch_isc_flag exist in the video bitstream, and at least one of them has a value of 1, then the value of patch_cibc_flag is set to 1; or if patch_ibc_flag exists in the video bitstream and its value is 1, then the value of patch_cibc_flag is set to 1; or if patch_isc_flag exists in the bitstream and its value is 1, then the value of patch_cibc_flag is set to 1.

[0278] 4) If the mode indication information is the coding unit flag cu_cibc_flag, the mode indication information is set in the current coding unit; when the coding unit flag cu_cibc_flag is a first preset value (such as 1), it indicates that the current chroma block adopts the chroma block copy intra-frame prediction mode, and when the coding unit flag cu_cibc_flag is a second preset value (such as 0), it indicates that the current chroma block does not adopt the chroma block copy intra-frame prediction mode.

[0279] 5) If the mode indication information is implicit information, when the implicit information indicates that the luminance sample at the preset position in the luminance co-located block corresponding to the current chrominance block meets the preset condition, the current chrominance block is indicated to adopt the chrominance block copy intra prediction mode.

[0280] 6) If the mode indication information is chroma indication information, then when the chroma indication information indicates that the current chroma block adopts the preset intra-frame prediction mode, for example, the preset intra-frame prediction mode is the DM mode (derived mode, a mode specific to chroma prediction), and the luminance samples at the preset position in the luminance co-located block meet the preset conditions, it is indicated that the current chroma block adopts the chroma block copy intra-frame prediction mode.

[0281] As can be seen from the above, the embodiment of the present application can indicate whether the prediction mode adopted by the current chroma block of the current coding unit in the video code stream is an intra-frame copy prediction mode by decoding the mode indication information set at different positions in the video code stream (such as the sequence header, the image header, the slice header, and the coding unit). It should be understood that the above-mentioned mode indication information 1)-6) for indicating whether the current chroma block adopts the chroma block copy intra-frame prediction mode can be used alone or in combination. For example, the mode indication information may include: a sequence header flag of the current sequence, and a coding unit flag of the current coding unit. When the sequence header flag is a first preset value and the coding unit flag is a first preset value, it indicates that the current chroma block adopts the chroma block copy intra-frame prediction mode; for another example, the mode indication information includes the coding unit flag of the current coding unit. When the coding unit flag is a first preset value, it indicates that the current chroma block adopts the chroma block copy intra-frame prediction mode, etc.; the present application does not make specific limitations on this.

[0282] In a possible implementation, during the decoding of a video stream, it is necessary to check whether the video stream meets the decoding conditions; if the video stream meets the decoding conditions, the step of decoding the video stream and obtaining the mode indication information is performed. Specifically, the decoding conditions include: the color format used by the video stream is a format containing a chroma component, and the format containing the chroma component includes: any one or more of the YUV420 format, the YUV422 format, and the YUV444 format. When the mode indication information is set in the current coding unit to which the current chroma block belongs, the decoding conditions include any one or more of the following:

[0283] i. The current coding unit contains only chrominance components; wherein the chrominance components can be determined by the current coding tree component type, and the current coding tree component type includes at least three types: brightness component, chrominance component, and bright chrominance component, wherein the coding tree refers to the division structure of the coding unit.

[0284] ii. The luminance samples at the preset positions in the luminance co-located block corresponding to the current chrominance block meet the preset conditions; wherein, if the number of preset positions is multiple (for example, 10), the number of luminance samples that meet the preset conditions includes: at least one, or greater than or equal to a preset threshold (T=6).

[0285] 1) The preset position may be determined by one or more of the following methods:

[0286] a. In one example, a preset chroma sample point can be first determined in the current chroma block, and then the preset position of the corresponding luminance sample point in the luminance co-located block is determined according to the position of the preset chroma sample point. The so-called chroma sample point refers to the sample point (i.e., pixel point) in the current chroma block. For example, the positions of the preset chrominance samples include: any one or more of the center point position (xC+wC / 2, yC+hC / 2), (xC+wC / 2-1, yC+hC / 2-1), (xC+wC / 2-1, yC+hC / 2), (xC+wC / 2, yC+hC / 2-1), the upper left corner position (xC+wC-1, yC), the upper right corner position (xC+wC, yC), the lower left corner position (xC, yC+hC-1), and the lower right corner position (xC+wC-1, yC+hC-1); then, the position of the corresponding luminance sample in the luminance co-located block may be calculated according to the following formula (2).

[0287] (xL, yL) = (xC * scaleX, yC * scaleY) Formula (2)

[0288] Wherein, in formula (2), (xC, yC) refers to the coordinates of the upper left corner of the current chrominance block, (xL, yL) refers to the coordinates of the upper left corner of the luminance co-located block corresponding to the current chrominance block; scaleX refers to the horizontal sampling ratio between the luminance component and the chrominance component in the currently adopted YUV color format, and scaleY refers to the vertical sampling ratio between the luminance component and the chrominance component in the currently adopted YUV color format. For example, if the currently adopted YUV color format is the YUV420 format, then scaleX=2(2:1), scaleY=2(2:1); for another example, if the currently adopted YUV color format is the YUV422 format, then scaleX=2(2:1), scaleY=1; for another example, if the currently adopted YUV color format is the YUV444 format, then scaleX=1, scaleY=1.

[0289] b. In another example, the preset position can be directly determined from the luminance co-located block. For example, it may include: (xL+wL / 2, yL+hL / 2), (xL+wL / 2-1, yL+hL / 2-1), (xL+wL / 2-1, yL+hL / 2), (xC+wC / 2, yC+hC / 2-1), the preset positions of the luminance samples corresponding to the upper left corner (xL, yL), the upper right corner (xL+wL-1, yL), the lower left corner ((xL, yL+hL-1), the lower right corner (xL+wL-1, yL+hL-1). It should be noted here that in the current video coding standard, motion information is stored according to a fixed block size (such as 4x4), and the information of each sample point within this size is the same, so the preset position obtained by this method is actually equivalent to the above step a.

[0290] c. In another example, according to the division method of the luminance co-located block, a preset position is determined in each sub-block obtained by dividing the luminance co-located block. For example, the preset position may be the upper left corner position of each sub-block; wherein the information of each sample point in the sub-block is the same.

[0291] d. In another example, the luminance co-located block is divided into M×N sub-blocks, and a specified position in the M×N sub-blocks is used as a preset position. The specified position includes: any position of the center, upper left corner, upper right corner, lower left corner, and lower right corner of the sub-block, and M and N are both positive integers. For example, the upper left corner of the 4x4 sub-block is selected as the preset position.

[0292] 2) The methods for determining the preset conditions include any one or more of the following:

[0293] a. The luminance sample adopts any one of the modes in the specific mode set, which includes: intra-frame block copy prediction mode, intra-frame string copy prediction ordinary string sub-mode, intra-frame string copy prediction non-ordinary string sub-mode, or any one or more prediction modes in other prediction modes with intra-frame displacement vectors.

[0294] b. The luma sample has a legal displacement vector. The so-called legal displacement vector means that the luma co-location block to which the luma sample belongs should meet the reference range limit of the intra-block copy prediction mode; or, the luma co-location block to which the luma sample belongs should meet the reference range limit of the intra-string copy prediction normal string sub-mode, and the reference block corresponding to the luma co-location block is allowed to overlap with the luma co-location block.

[0295] c. The intra prediction history motion information table (or intra block copy history motion information table) of the current chroma block is not empty.

[0296] It should be noted that, in addition to being expressed as a chroma block copy intra-frame prediction mode, the prediction mode adopted by the current chroma block in the embodiment of the present application may also have other different expressions, for example, it may also be expressed as: intra-frame copy mode, intra-frame block copy mode, chroma frame intra-copy mode, chroma string copy mode or other prediction mode names, etc. The present application does not limit the specific name of the prediction mode adopted by the current chroma block. Among them, the chroma block copy intra-frame prediction mode in the embodiment of the present application is characterized in that a prediction value is derived for the chroma block with the current frame as a reference frame, and in the process of deriving the prediction value, the chroma block may be used as a unit, or the chroma sub-block may be used as a unit.

[0297] S602: Obtain a target displacement vector of a luma co-located block corresponding to the current chroma block.

[0298] 1. Determine the luminance co-location block corresponding to the current chrominance block

[0299] In a possible implementation, the position and size of the corresponding luminance co-located block can be determined according to the position and size of the current chrominance block and the color format of the video image. Assume that the coordinates of the upper left corner of the current chrominance block are (xC, yC), the size is (wC, hC), and the horizontal sampling ratio and vertical sampling ratio between the luminance component and the chrominance component corresponding to the color format are (scaleX, scaleY), for example, if the color format is YUV420 format, then scaleX = 2, scaleY = 2. Then, the position (xL, yL) and size (wL, hL) of the luminance co-located block corresponding to the current chrominance block can be calculated according to the following formulas 3-1 and 3-2:

[0300] (xL, yL) = (xC * scaleX, yC * scaleY) (3-1)

[0301] (wL, hL) = (wC * scaleX, hC * scaleY) (3-2)

[0302] Where (xL, yL) refers to the horizontal coordinate xL and vertical coordinate yL of the upper left corner of the luma block, (wL, hL) refers to the length wL and height hL of the luma block; (xC, yC) refers to the horizontal coordinate xC and vertical coordinate yC of the upper left corner of the current chroma block, (wC, hC) refers to the length wC and height hC of the upper left corner of the current chroma block. See Figure 7a , Figure 7a is a schematic diagram of a flow chart of determining brightness co-located blocks provided in an embodiment of the present application, such as Figure 7aAs shown, assuming that the coordinates (xC, yC) of the upper left corner of the current chrominance block are (1, 2), the size (wC, hC) of the current chrominance block is (8, 4), and the color format is YUV420 format, then according to the above formulas 3-1 and 3-2, the position of the luminance co-located block corresponding to the current chrominance block can be determined as (xL, yL) = (1*2, 2*2) = (2, 4), and the size (wL, hL) = (8*2, 4*2) = (16, 8).

[0303] 2. Determine the target displacement vector from the brightness co-location block

[0304] In a possible implementation, obtaining a target displacement vector of a luma co-located block corresponding to a current chroma block includes: first, constructing a candidate displacement vector list of the luma co-located block corresponding to the current chroma block; and then determining the target displacement vector in the candidate displacement vector list. It should be noted that in actual applications, it is not necessary to allocate memory for constructing a candidate displacement vector list, and it can also be implemented in other ways, for example, each candidate displacement vector in the candidate displacement vector list can be checked in turn.

[0305] (1) First, the specific process of how to construct the candidate displacement vector list is described in detail.

[0306] 1) Construct a candidate displacement vector list of the luminance co-located block corresponding to the current chrominance block, including any one or more of the following methods:

[0307] i. Add the displacement vectors of the luminance samples in the luminance co-located block that satisfy the specified mode to the candidate displacement vector list. The specified mode includes: intra-frame block copy prediction mode, intra-frame string copy prediction ordinary string sub-mode, intra-frame string copy prediction non-ordinary string sub-mode, intra-frame template matching mode, inter-frame prediction mode, affine motion mode, inter-frame intra-frame composite mode, Angular Weighted Prediction (AWP) mode, Spatial Angular Weighted Prediction (SAWP) mode, and any one of the inter-frame geometric partitioning prediction mode (GPM) modes.

[0308] ii. Add the displacement vector of the spatial neighboring block corresponding to the luminance co-located block to the candidate displacement vector list. The spatial neighboring block refers to the coding block adjacent to the current coding block (also known as the coding unit) in the current frame of the video, the current frame refers to the video frame to which the current coding unit belongs; the current coding block refers to the coding block being decoded in the current frame.

[0309] iii. Add the displacement vector in the historical motion information table or the intra-frame prediction historical motion information table to the candidate displacement vector list.

[0310] iv. Add the candidate displacement vectors in the above steps i-iii to the candidate displacement vector list after offset processing. The candidate displacement vectors include: the displacement vectors of the luminance samples that meet the specified mode in the luminance co-located block, the displacement vectors of the spatial adjacent blocks corresponding to the luminance co-located block, and any one or more of the displacement vectors of the spatial adjacent blocks corresponding to the luminance co-located block. Specifically, assuming that there is a displacement vector (mvX, mvY), then the displacement vector after the displacement vector is offset can be expressed as (mvX+offsetX, mvY+offsetY), where the values ​​of offsetX and offsetY can be any integers, for example, offsetX=1, offsetY=2.

[0311] v. If there is no legal displacement vector, the default displacement vector can be used to fill the list. The default displacement vectors include: (-w, 0), (0, -h), (-w, -h), (-2*w, 0), (0, -2*h), where "w" refers to the width of the luminance co-located block and "h" refers to the height of the luminance co-located block; alternatively, the displacement vector in the candidate displacement vector list is set to an invalid value, such as (0, 0).

[0312] The conditions that the displacement vectors in the candidate displacement vector list must satisfy include at least one of the following:

[0313] i. The reference block of the luma co-located block meets the reference range restriction of the intra block copy prediction mode;

[0314] ii. The reference block of the luminance co-located block and the displacement vector both meet the reference range restrictions of the intra-frame string copy prediction normal string sub-mode;

[0315] iii. The displacement vector does not overlap with any displacement vector in other displacement vector lists except the candidate displacement vector list. In other words, the displacement vector added to the candidate displacement vector list is unique and does not exist in other displacement vector lists except the candidate displacement vector list.

[0316] 2) The sorting method of each displacement vector in the candidate displacement vector list includes any of the following:

[0317] i. Sort in a predetermined order. For example, arrange the displacement vectors in sequence from far to near (or from near to far) according to the acquisition time of each displacement vector. For example, the acquisition times of the displacement vectors MV1, MV2, and MV3 used to construct the candidate displacement vector list are t1, t2, and t3, respectively, and t1<t2<t3, then the three displacement vectors MV1, MV2, and MV3 are sorted in the candidate displacement vector list as follows: MV1, MV2, MV3. It should be understood that the displacement vectors in the candidate displacement vector list can be sorted in order from left to right or from top to bottom. For example, the displacement vectors in the candidate displacement vector list are sorted in order from top to bottom, and the sorted candidate displacement vector list is shown in Table 4-1 below:

[0318] Table 4-1. List of candidate displacement vectors

[0319] Displacement Vector Get time MV1 t1 MV2 t2 MV3 t3

[0320] Optional, see Figure 7b , Figure 7b is a schematic diagram of sorting displacement vectors provided in an embodiment of the present application. Figure 7b As shown, if the coding blocks corresponding to the displacement vectors in the candidate displacement vector list include: C0, C1, TL, TR, BL, BR respectively, then the predetermined order can be: C0, C1, TL, TR, BL, BR.

[0321] ii. Sort according to the cost of each displacement vector. For example, the cost may include but is not limited to: rate-distortion cost, Sum of Absolute Difference (SAD), Sum of Absolute Transformed Difference (SATD), Sum of Squared Error (SSE), Mean-removed SAD, Mean-removed SSE. The above costs can be calculated by using template matching. Taking the rate-distortion cost as an example, for instance, the displacement vectors used to construct the candidate displacement vector list include: MV1, MV2, MV3, MV4, MV5, and are calculated by using template matching: the rate-distortion cost of MV1 < the rate-distortion cost of MV2 < the rate-distortion cost of MV3 < the rate-distortion cost of MV4 < the rate-distortion cost of MV5. Then the sorting order of each displacement vector in the candidate displacement vector list is: MV1, MV2, MV3, MV4, MV5. It should be understood that each displacement vector in the candidate displacement vector list can be sorted in sequence from left to right or from top to bottom. For example, if the displacement vectors in the candidate displacement vector list are sorted in sequence from left to right, the sorted candidate displacement vector list is shown in Table 4-2 below:

[0322] Table 4-2. Candidate Displacement Vector List

[0323] MV1 MV2 MV3 MV4 MV5 ... Loss1 Loss2 Loss3 Loss4 Loss5 ...

[0324] Among them, Loss1 refers to the rate-distortion cost of MV1, Loss2 refers to the rate-distortion cost of MV2, and so on. In Table 4-2, each displacement vector is sorted in ascending order of the rate-distortion cost. It should be understood that in other feasible embodiments, each displacement vector can also be sorted in descending order of the rate-distortion cost, and the present application does not make specific limitations on this.

[0325] Optionally, the cost of each displacement vector (such as rate-distortion cost, SAD, SATD, SSE, Mean-removed SAD, Mean-removed SSE) can be weighted to obtain the total cost of each displacement vector; then sort according to the total cost of each displacement vector.

[0326] iii. Sort by displacement vector type, where the displacement vector types include at least: block vector, string vector, motion vector (motion vector for inter-frame motion), and default vector. The priority of the block vector is greater than the priority of the string vector, the priority of the string vector is greater than the priority of the motion vector, and the priority of the motion vector is greater than the priority of the default vector. That is, in the candidate displacement vector list, the block vector, string vector, motion vector, and default vector are sorted in this order. For example, the sorted candidate displacement vector list is shown in Table 4-3 below:

[0327] Table 4-3. List of candidate displacement vectors

[0328]

[0329]

[0330] In Table 4-3, displacement vectors MV1 and MV2 are both block vectors, MV3 and MV4 are both string vectors, MV5 is a motion vector for inter-frame motion, and MV6 is a default vector.

[0331] iv. Comprehensively sort according to the type of displacement vector and the rate-distortion cost of the displacement vector. Specifically, each displacement vector in the displacement vector list can be divided into different categories (block vector category, string vector category, default vector category), and then sorted according to the cost of the displacement vector in each category. For example, the sorted candidate displacement vector list is shown in Table 4-4 below:

[0332] Table 4-4. List of candidate displacement vectors

[0333]

[0334] In Table 4-4, displacement vectors MV1 and MV2 are both block vectors, and the cost of MV1 is smaller than that of MV2; MV3 and MV4 are both string vectors, and the cost of MV3 is smaller than that of MV4; MV5 and MV6 are both motion vectors, and the cost of MV5 is smaller than that of MV6; MV7 is the default vector.

[0335] (2) Then, the specific process of how to determine the target displacement vector is described in detail.

[0336] In a possible implementation manner, determining the target displacement vector in the candidate displacement vector list includes any of the following manners:

[0337] 1) The Kth displacement vector in the candidate displacement vector list is used as the target displacement vector, where K is a positive integer. For example, the first displacement vector in the candidate displacement vector list is used as the target displacement vector.

[0338] 2) Determine the target displacement vector from the candidate displacement vector list according to the displacement vector index. The displacement vector index is used to indicate the position of the displacement vector in the candidate displacement vector column, and the displacement vector index can be decoded from the video bitstream. For example, if the displacement vector index is m (m is a positive integer), the target displacement vector determined based on the displacement vector index refers to the displacement vector numbered m in the displacement vector list. Optionally, when the number of displacement vectors included in the displacement vector list is greater than 1, it is allowed to decode the displacement vector index from the video bitstream.

[0339] 3) Obtain a candidate displacement vector from the candidate displacement vector list, scale the candidate displacement vector according to the color format sampling ratio, and retain the integer pixel accuracy of the scaled displacement vector to obtain a target displacement vector. Specifically, if the obtained candidate displacement vector MV is represented as (5, 3), and the color format is YUV420 format, the sampling ratio under the YUV420 format is scaleX=2, scaleY=2, then the displacement vector MV' obtained after scaling the displacement vector represented as (5, 3) is (2.5, 1.5); further, after retaining the integer pixel accuracy of the MV', the target displacement vector obtained can be (2, 1). Wherein, when retaining the integer pixel accuracy of the displacement vector, it can be rounded up or rounded down, and this application does not limit this.

[0340] 4) Obtain a candidate displacement vector from the candidate displacement vector list, and use the displacement vector of the brightness component as the target displacement vector. For example, if the obtained candidate displacement vector MV is expressed as (5.1, 3.2), the accuracy of the displacement vector (whole pixel accuracy or sub-pixel accuracy) is retained, and the obtained displacement vector (5.1, 3.2) is directly used as the target displacement vector.

[0341] It should be noted that in the above steps 3) and 4), MV is usually stored with a certain precision. For example, AVS3 stores MV with 1 / 4 pixel precision. An MV with integer pixel precision is (2,1), which will be enlarged to (8,4) when stored. In this case, the integer pixel part of MV can be obtained by MV / 4, and the sub-pixel part can be obtained by MV%4, where " / " means integer division and rounding down, and "%" means remainder.

[0342] S603: Determine a prediction value of the current chroma block based on the target displacement vector, where the prediction value is used to reconstruct a decoded image corresponding to the current chroma block.

[0343] In one possible implementation, if the target displacement vector has integer pixel accuracy, motion compensation is performed on the target displacement vector to derive a predicted value of the current chroma block; or, if the target displacement vector has sub-pixel accuracy, the target displacement vector is interpolated and motion compensation is performed on the interpolated target displacement vector to derive a predicted value of the current chroma block.

[0344] In another possible implementation, if the target displacement vector does not exist in the luminance co-located block corresponding to the current chrominance block, the intra-frame prediction mode is used to derive the prediction value of the current chrominance block; wherein the intra-frame prediction mode includes: a cross-component intra-frame prediction mode, or a specified intra-frame prediction mode. 1) The cross-component intra-frame prediction mode includes: a two-step cross-component prediction mode (TSPCM) and a cross-component linear model prediction mode (CCLM) prediction mode; 2) The specified intra-frame prediction mode includes: any one or more of an intra-frame prediction direct current coefficient mode (DC), an intra-frame prediction bilinear prediction mode (Blinear prediction mode), an intra-frame prediction planar prediction mode (Planar / Plane prediction mode), a horizontal intra-frame prediction mode, and a vertical intra-frame prediction mode.

[0345] In a possible implementation, 1) if the current chroma block adopts the chroma block copy intra-frame prediction mode, then determine whether the current chroma block needs to be residually decoded based on the information of the target luminance sample in the luminance co-location block corresponding to the current chroma block, where the target luminance sample is the luminance sample corresponding to the target displacement vector; 2) if it is determined that the current chroma block needs to be residually decoded, then the current chroma block is residually decoded to obtain a residual value; 3) based on the residual value and prediction value of the current chroma block, reconstruct the decoded image corresponding to the current chroma block. Wherein, the luminance co-location block corresponding to the current chroma block contains multiple luminance blocks, and the judgment conditions that the current chroma block does not need to be residually decoded include any of the following:

[0346] i. If the target luminance sample adopts the intra-frame string copy prediction normal string mode, and the luminance block where the target luminance sample is located does not need to be residually encoded;

[0347] ii. If the target luminance sample adopts the intra-frame string copy prediction mode, and the luminance block where the target luminance sample is located does not need to be residually encoded;

[0348] iii. If the target luminance sample adopts the intra block copy prediction mode, and the luminance block where the target luminance sample is located does not need to be residually encoded;

[0349] iv. If the target luminance sample adopts the intra-frame string copy prediction normal string mode, or the intra-frame block copy prediction mode, and the luminance block where the target luminance sample is located does not need to be residually encoded;

[0350] v. If the target luminance sample adopts the intra-frame string copy prediction mode or the intra-frame block copy prediction mode, and the luminance block where the target luminance sample is located does not need to be residually encoded;

[0351] Among them, the judgment condition of whether the luminance block where the target luminance sample is located needs to be residually encoded is determined according to: the coefficient flag of the luminance block, or the prediction mode of the current chrominance block. For example, in video encoding and decoding, each coding block corresponds to a coefficient flag, and the coefficient flag is used to indicate whether the current coding block needs to encode the residual value (also called the residual coefficient). If the coefficient flag of the luminance block where the target luminance sample is located is 0, it can be deduced that the coefficient flag of the current chrominance block is also 0, which can indicate that the current chrominance block does not need to be residually decoded. For example, if the prediction mode of the current chrominance block is skip mode, the current chrominance block does not need to be residually decoded.

[0352] In a possible implementation, the application determines the prediction value of the current chroma block, and may also include the following scheme: determine the current chroma block in the video code stream; divide the current chroma block into P×Q chroma sub-blocks, where P and Q are both integers; obtain the block vector of the luminance co-located sub-block corresponding to each chroma sub-block; and determine the prediction value of the current chroma block based on the obtained P×Q block vectors. The specific process of obtaining the block vector of any chroma sub-block can be referred to the specific execution process of determining the target displacement vector in step S602 above, which will not be repeated here; similarly, the specific process of determining the prediction value of the current chroma block according to each block vector can be referred to the specific execution process of determining the prediction value in step S603 above, which will not be repeated here.

[0353] In an embodiment of the present application, a current chroma block in a video bitstream can be determined, and the current chroma block adopts a chroma block copy intra-frame prediction mode; a target displacement vector of a luminance co-located block corresponding to the current chroma block is obtained; based on the target displacement vector, a prediction value of the current chroma block is determined; and the prediction value is used to reconstruct a decoded image corresponding to the current chroma block. As can be seen from the above, in the decoding process of the video bitstream, when it is determined that the current chroma block adopts a chroma block copy intra-frame prediction mode, the displacement vector information of the luminance co-located block can be used to determine the prediction value of the current chroma block, and there is no need to perform intra-frame prediction processing on the current chroma block, which can improve the encoding and decoding efficiency of the current chroma block, thereby effectively improving the encoding performance.

[0354] See also Figure 8 , Figure 8This is a flow chart of another video processing method provided in an embodiment of the present application. The video processing method may be executed by an encoding device in a video processing system. The video processing method described in this embodiment may include the following steps S801-S804:

[0355] S801: Determine a current chroma block in a video, and the current chroma block adopts a chroma block copy intra prediction mode.

[0356] In a possible implementation, whether the current chroma block adopts the chroma block copy intra prediction mode is indicated by mode indication information, and the mode indication information includes at least one of the following: a sequence header flag, an image header flag, a slice header flag, a coding unit flag, implicit information, and chroma indication information. The chroma block copy intra prediction mode is a prediction mode independent of the normal intra prediction mode; or, the chroma block copy intra prediction mode is a sub-mode of the normal intra prediction mode. The normal intra prediction mode includes at least one of the following: an intra prediction DC coefficient mode, an angle-based intra prediction mode, a cross-component two-step prediction mode, and a prediction mode based on intra reference pixels to derive prediction values.

[0357] (1) The chroma block copy intra prediction mode is a prediction mode independent of the normal intra prediction mode.

[0358] 1) If the mode indication information is the sequence header flag seq_cibc_flag, when the sequence header flag seq_cibc_flag is a first preset value (such as 1), it indicates that the current chroma block is allowed to adopt the chroma block copy intra-frame prediction mode, and when the sequence header flag seq_cibc_flag is a second preset value (such as 0), it indicates that the current chroma block is not allowed to adopt the chroma block copy intra-frame prediction mode.

[0359] 2) If the mode indication information is the picture header flag pic_cibc_flag, when the picture header flag pic_cibc_flag is a first preset value (such as 1), it indicates that the current chroma block is allowed to adopt the chroma block copy intra-frame prediction mode; when the picture header flag pic_cibc_flag is a second preset value (such as 0), it indicates that the current chroma block is not allowed to adopt the chroma block copy intra-frame prediction mode.

[0360] 3) If the mode indication information is the slice header flag patch_cibc_flag, when the slice header flag patch_cibc_flag is a first preset value (such as 1), it indicates that the current chroma block is allowed to adopt the chroma block copy intra-frame prediction mode; when the slice header flag patch_cibc_flag is a second preset value (such as 0), it indicates that the current chroma block is not allowed to adopt the chroma block copy intra-frame prediction mode.

[0361] 4) If the mode indication information is the coding unit flag cu_cibc_flag, when the coding unit flag cu_cibc_flag is a first preset value (such as 1), it indicates that the current chroma block adopts the chroma block copy intra-frame prediction mode; when the coding unit flag cu_cibc_flag is a second preset value (such as 0), it indicates that the current chroma block does not adopt the chroma block copy intra-frame prediction mode.

[0362] It should be noted that the above steps 1)-4) can be used separately or in combination. For example, when the sequence header flag seq_cibc_flag=1, it can be determined that the current chroma block in the current sequence adopts the chroma block copy intra-frame prediction mode; for another example, when the sequence header flag seq_cibc_flag=1 and the coding unit flag cu_cibc_flag=1, it can be determined that the current chroma block adopts the chroma block copy intra-frame prediction mode; for another example, when seq_cibc_flag=1, pic_cibc_flag=1, patch_cibc_flag=1, and cu_cibc_flag=1, it can be determined that the current chroma block adopts the chroma block copy intra-frame prediction mode, and so on.

[0363] (2) The chroma block copy intra prediction mode is a sub-mode of the normal intra prediction mode. In this mode, the on or off of the chroma block copy intra prediction mode is associated with the intra prediction mode (eg, IBC or ISC).

[0364] 1) If it is determined that the current sequence turns on the IBC mode or the ISC mode, it can be determined that the current chroma block allows the chroma block copy intra-frame prediction mode; if it is determined that the current sequence turns off the IBC mode or the ISC mode, it can be determined that the current chroma block does not allow the chroma block copy intra-frame prediction mode.

[0365] 2) If it is determined that the current image turns on the IBC mode or the ISC mode, it can be determined that the current chroma block allows the use of the chroma block copy intra-frame prediction mode; if it is determined that the current image turns off the IBC mode or the ISC mode, it can be determined that the current chroma block does not allow the use of the chroma block copy intra-frame prediction mode.

[0366] 3) If it is determined that the current slice turns on the IBC mode or the ISC mode, it can be determined that the current chroma block allows the use of the chroma block copy intra-frame prediction mode; if it is determined that the current slice turns off the IBC mode or the ISC mode, it can be determined that the current chroma block does not allow the use of the chroma block copy intra-frame prediction mode.

[0367] 4) If it is determined that the current coding unit turns on the IBC mode or the ISC mode, it can be determined that the current chroma block allows the chroma block copy intra-frame prediction mode; if it is determined that the current coding unit turns off the IBC mode or the ISC mode, it can be determined that the current chroma block does not allow the chroma block copy intra-frame prediction mode.

[0368] It should be noted that, in addition to being expressed as a chroma block copy intra-frame prediction mode, the prediction mode adopted by the current chroma block in the embodiment of the present application may also have other different representation methods. For example, it may also be expressed as a sub-mode of a normal intra-frame prediction mode, or other prediction mode names, etc. The present application does not limit the specific name of the prediction mode adopted by the current chroma block.

[0369] S802: Obtain a target displacement vector of the luma co-located block corresponding to the current chroma block.

[0370] First, the luminance co-located block corresponding to the current chrominance block needs to be determined. In one possible implementation, the position and size of the corresponding luminance co-located block can be determined according to the position and size of the current chrominance block and the color format of the video image. Assume that the coordinates of the upper left corner of the current chrominance block are (xC, yC), the size is (wC, hC), and the horizontal sampling ratio and vertical sampling ratio between the luminance component and the chrominance component corresponding to the color format are (scaleX, scaleY), for example, if the color format is YUV420 format, then scaleX=2, scaleY=2.

[0371] Then, a target displacement vector is determined from the luma co-located block. In a possible implementation, obtaining a target displacement vector of a luma co-located block corresponding to a current chroma block includes: first, constructing a candidate displacement vector list of the luma co-located block corresponding to the current chroma block; and then determining the target displacement vector in the candidate displacement vector list.

[0372] (1) The displacement vectors used to construct the candidate displacement vector list include the following:

[0373] i. The displacement vector of the luma samples in the luma co-located block that satisfy the specified mode. The specified mode includes: intra-frame block copy prediction mode, intra-frame string copy prediction ordinary string sub-mode, intra-frame string copy prediction non-ordinary string sub-mode, intra-frame template matching mode, inter-frame prediction mode, affine motion mode, inter-frame intra-frame composite mode, Angular Weighted Prediction (AWP) mode, Spatial Angular Weighted Prediction (SAWP) mode, and any one of the inter-frame geometric partitioning prediction mode (GPM) modes.

[0374] ii. The displacement vector of the spatially adjacent block corresponding to the luminance co-located block. The spatially adjacent block refers to the coding block adjacent to the current coding block (also called coding unit) in the current frame of the video, the current frame refers to the video frame to which the current coding unit belongs; the current coding block refers to the coding block being decoded in the current frame.

[0375] iii. The displacement vector in the intra-prediction historical motion information table (eg HMVP, IntraHMVP) of the luma co-located block.

[0376] iv. The displacement vector obtained after the candidate displacement vectors in the above steps i-iii are offset.

[0377] v. If there is no legal displacement vector, the default displacement vector can be used to fill the list. The default displacement vectors include: (-w, 0), (0, -h), (-w, -h), (-2*w, 0), (0, -2*h); or, the displacement vector in the candidate displacement vector list is set to an invalid value, such as (0, 0).

[0378] The conditions that the displacement vectors in the candidate displacement vector list must satisfy include at least one of the following:

[0379] i. The reference block of the luma co-located block meets the reference range restriction of the intra block copy prediction mode;

[0380] ii. The reference block of the luminance co-located block and the displacement vector both meet the reference range restrictions of the intra-frame string copy prediction normal string sub-mode;

[0381] iii. The displacement vector does not overlap with any displacement vector in other displacement vector lists except the candidate displacement vector list. In other words, the displacement vector added to the candidate displacement vector list is unique and does not exist in other displacement vector lists except the candidate displacement vector list.

[0382] (2) Determining the target displacement vector in the candidate displacement vector list includes any of the following methods:

[0383] 1) The Kth displacement vector in the candidate displacement vector list is used as the target displacement vector, where K is a positive integer.

[0384] 2) Determine the target displacement vector from the candidate displacement vector list according to the displacement vector index.

[0385] 3) Obtain a candidate displacement vector from the candidate displacement vector list, scale the candidate displacement vector according to the color format sampling ratio, and retain the integer pixel accuracy of the scaled displacement vector to obtain the target displacement vector.

[0386] 4) Obtain a candidate displacement vector from the candidate displacement vector list, and use the displacement vector of the brightness component as the target displacement vector.

[0387] It should be noted that the specific process of constructing the candidate displacement vector list and the specific process of determining the target displacement vector can refer to the relevant steps in the above-mentioned embodiment, and the embodiments of the present application will not be repeated here.

[0388] S803: Determine a prediction value of the current chroma block based on the target displacement vector.

[0389] In one possible implementation, if the target displacement vector has integer pixel accuracy, motion compensation is performed on the target displacement vector to derive a predicted value of the current chroma block; or, if the target displacement vector has sub-pixel accuracy, the target displacement vector is interpolated and motion compensation is performed on the interpolated target displacement vector to derive a predicted value of the current chroma block.

[0390] In another possible implementation, if the target displacement vector does not exist in the luminance co-located block corresponding to the current chrominance block, the intra-frame prediction mode is used to derive the prediction value of the current chrominance block; wherein the intra-frame prediction mode includes: a cross-component intra-frame prediction mode, or a specified intra-frame prediction mode. The cross-component intra-frame prediction mode includes: a two-step cross-component prediction mode (TSPCM), a cross-component linear model prediction mode (CCLM) prediction mode; the specified intra-frame prediction mode includes: any one or more of an intra-frame prediction direct current coefficient mode (DC), an intra-frame prediction bilinear prediction mode (Blinear prediction mode), an intra-frame prediction planar prediction mode (Planar / Plane prediction mode), a horizontal intra-frame prediction mode, and a vertical intra-frame prediction mode.

[0391] S804: Encode the video based on the prediction value of the current chrominance block to generate a video bitstream.

[0392] In a possible implementation, before encoding the video based on the prediction value of the current chrominance block to generate a video bitstream, it is also necessary to check whether the current chrominance block needs to be residual encoded. Specifically, there are two situations:

[0393] (1) If the current chroma block does not need to be residually encoded, the current chroma block is directly encoded based on the predicted value of the current chroma block to obtain the encoded image corresponding to the current chroma block.

[0394] (2) If the current chroma block needs to be residually encoded, residual encoding is performed on the current chroma block to obtain a residual value; and based on the residual value and the predicted value of the current chroma block, the current chroma block is encoded to obtain an encoded image corresponding to the current chroma block.

[0395] The luminance co-located block corresponding to the current chrominance block includes multiple luminance blocks, and the judgment condition of whether the current chrominance block needs to be residually encoded includes any of the following:

[0396] i. If the target luminance sample adopts the intra-frame string copy prediction normal string mode, and the luminance block where the target luminance sample is located does not need to be residually encoded;

[0397] ii. If the target luminance sample adopts the intra-frame string copy prediction mode, and the luminance block where the target luminance sample is located does not need to be residually encoded;

[0398] iii. If the target luminance sample adopts the intra block copy prediction mode, and the luminance block where the target luminance sample is located does not need to be residually encoded;

[0399] iv. If the target luminance sample adopts the intra-frame string copy prediction normal string mode, or the intra-frame block copy prediction mode, and the luminance block where the target luminance sample is located does not need to be residually encoded;

[0400] v. If the target luma sample adopts the intra string copy prediction mode or the intra block copy prediction mode, and the luma block where the target luma sample is located does not need to be residually encoded.

[0401] In the embodiment of the present application, the current chroma block in the video can be determined, and the current chroma block adopts the chroma block copy intra-frame prediction mode; the target displacement vector of the luminance co-location block corresponding to the current chroma block is obtained; based on the target displacement vector, the prediction value of the current chroma block is determined; based on the prediction value of the current chroma block, the video is encoded to generate a video bitstream. In the video encoding process, when the current chroma block adopts the chroma block copy intra-frame prediction mode, the displacement vector information of the luminance co-location block can be used to determine the prediction value of the current chroma block, and there is no need to perform encoding prediction and other processing on the current chroma block, which can improve the efficiency of chroma block decoding, thereby effectively improving the encoding performance.

[0402] The following is combined with Figure 9-11 The relevant devices of the embodiments of the present application are introduced accordingly.

[0403] See also Fig. 9 , Fig. 9 Schematic diagram of the structure of a video processing device provided in an embodiment of the present application. Fig. 9As shown, the video processing device 900 can be applied to the computer device mentioned in the above embodiment, and the computer device can be the decoding device mentioned in the above method embodiment. Specifically, the video processing device 900 can be a computer program (including program code) running in a computer device, for example, the video processing device 900 is an application software; the video processing device 900 can be used to execute the corresponding steps in the video processing method provided in the embodiment of the present application. In specific implementation, the video processing device 900 can specifically include:

[0404] A determination unit 901 is used to determine a current chroma block in a video bitstream, where the current chroma block adopts a chroma block copy intra prediction mode;

[0405] Processing unit 902, used to obtain a target displacement vector of a luma co-located block corresponding to a current chroma block;

[0406] The processing unit 902 is further used to determine a prediction value of the current chroma block based on the target displacement vector; the prediction value is used to reconstruct a decoded image corresponding to the current chroma block.

[0407] In a possible implementation, the chroma block copy intra prediction mode is a prediction mode independent of the normal intra prediction mode; or, the chroma block copy intra prediction mode is a sub-mode of the normal intra prediction mode;

[0408] The common intra prediction mode includes at least one of the following: an intra prediction DC coefficient mode, an angle-based intra prediction mode, a cross-component two-step prediction mode, and a prediction mode for deriving prediction values ​​based on intra reference pixels.

[0409] In a possible implementation, after determining the current chrominance block in the video code stream, the processing unit 902 is further configured to perform the following operations:

[0410] Divide the current chroma block into P×Q chroma sub-blocks, where P and Q are both integers;

[0411] Obtain the block vector of the luminance co-located sub-block corresponding to each chrominance sub-block;

[0412] Based on the obtained P×Q block vectors, a prediction value of the current chrominance block is determined.

[0413] In a possible implementation, the processing unit 902 is further configured to perform the following operations:

[0414] Decoding the video bitstream to obtain mode indication information, where the mode indication information is used to indicate whether the current chroma block adopts a chroma block copy intra prediction mode;

[0415] The mode indication information includes at least one of the following: a sequence header flag, a picture header flag, a slice header flag, a coding unit flag, implicit information, and chroma indication information.

[0416] In a possible implementation, if the mode indication information is a sequence header flag, the mode indication information is set in a sequence header of a current sequence to which the current chroma block belongs; when the sequence header flag is a first preset value, it indicates that the current chroma block allows the use of a chroma block copy intra-frame prediction mode; and when the sequence header flag is a second preset value, it indicates that the current chroma block does not use a chroma block copy intra-frame prediction mode;

[0417] If the mode indication information is an image header flag, the mode indication information is set in an image header of a current image to which the current chroma block belongs; when the image header flag is a first preset value, it indicates that the current chroma block allows the use of a chroma block copy intra-frame prediction mode; and when the image header flag is a second preset value, it indicates that the current chroma block does not use a chroma block copy intra-frame prediction mode;

[0418] If the mode indication information is a slice header flag, the mode indication information is set in the slice header of the current slice to which the current chroma block belongs; when the slice header flag is a first preset value, it indicates that the current chroma block allows the use of the chroma block copy intra prediction mode, and when the slice header flag is a second preset value, it indicates that the current chroma block does not use the chroma block copy intra prediction mode;

[0419] If the mode indication information is a coding unit flag, the mode indication information is set in the current coding unit; when the coding unit flag is a first preset value, it indicates that the current chroma block adopts the chroma block copy intra prediction mode, and when the coding unit flag is a second preset value, it indicates that the current chroma block does not adopt the chroma block copy intra prediction mode;

[0420] If the mode indication information is implicit information, then when the implicit information indicates that the luminance sample at a preset position in the luminance co-located block corresponding to the current chrominance block meets a preset condition, the current chrominance block is indicated to adopt the chrominance block copy intra prediction mode;

[0421] If the mode indication information is chroma indication information, then when the chroma indication information indicates that the current chroma block adopts a preset intra-frame prediction mode, and the luminance sample at a preset position in the luminance co-located block meets a preset condition, it indicates that the current chroma block adopts a chroma block copy intra-frame prediction mode; the preset intra-frame prediction mode includes an inheritance mode.

[0422] In a possible implementation, the processing unit 902 is further configured to perform the following operations:

[0423] Check whether the video code stream meets the decoding conditions;

[0424] If the video code stream meets the decoding condition, the step of decoding the video code stream to obtain mode indication information is performed.

[0425] In a possible implementation, the decoding condition includes: the color format adopted by the video code stream is a format including a chrominance component;

[0426] When the mode indication information is set in the current coding unit to which the current chroma block belongs, the decoding condition includes any one or more of the following:

[0427] The current coding unit contains only chroma components;

[0428] The luminance sample point at a preset position in the luminance co-location block corresponding to the current chrominance block meets a preset condition;

[0429] If there are multiple preset positions, the number of brightness sample points that meet the preset conditions is at least one, or is greater than or equal to a preset threshold.

[0430] In a possible implementation, the preset position is determined in any of the following ways:

[0431] Determine a preset chroma sample point in the current chroma block, and determine a preset position of a corresponding luminance sample point in a luminance co-located block according to the position of the preset chroma sample point;

[0432] Determining a preset position from the brightness co-located blocks;

[0433] According to the division method of the luminance co-located block, a preset position is determined in each sub-block obtained by dividing the luminance co-located block;

[0434] The luminance co-located block is divided into M×N sub-blocks, and a specified position in the M×N sub-blocks is used as a preset position, where the specified position includes: any position of the center, upper left corner, upper right corner, lower left corner, and lower right corner of the sub-block, and M and N are both positive integers.

[0435] In a possible implementation, the preset condition includes any one or more of the following:

[0436] The luminance sample adopts any mode in a specific mode set, which includes: intra-frame block copy prediction mode, intra-frame string copy prediction ordinary string sub-mode, intra-frame string copy prediction non-ordinary string sub-mode, or any one or more prediction modes in other prediction modes with intra-frame displacement vectors.

[0437] Luminance samples have valid displacement vectors;

[0438] The intra prediction history motion information table of the current chroma block is not empty.

[0439] In a possible implementation, the processing unit 902 obtains a target displacement vector of a luma co-located block corresponding to the current chroma block, and performs the following operations:

[0440] Construct a list of candidate displacement vectors of the luma co-located block corresponding to the current chroma block;

[0441] A target displacement vector is determined from the list of candidate displacement vectors.

[0442] In a possible implementation, the processing unit 902 constructs a candidate displacement vector list of the luma co-located block corresponding to the current chroma block, including any one or more of the following methods:

[0443] Add the displacement vectors of the luminance samples in the luminance co-located block that satisfy the specified mode to the candidate displacement vector list, where the specified mode includes: any one of the intra-frame block copy prediction mode, the intra-frame string copy prediction normal string sub-mode, the string copy intra-frame prediction non-normal string sub-mode, the intra-frame template matching mode, the inter-frame prediction mode, the affine motion mode, the inter-frame intra-frame composite mode, and the angle weighted prediction mode;

[0444] Add the displacement vector of the spatial adjacent block corresponding to the luminance co-located block to the candidate displacement vector list;

[0445] Adding the displacement vector in the historical motion information table or the intra-frame prediction historical motion information table to the candidate displacement vector list;

[0446] Adding the candidate displacement vectors to the candidate displacement vector list after offset processing, the candidate displacement vectors include: any one or more of the displacement vectors of the luminance samples satisfying the specified mode in the luminance co-located block, the displacement vectors of the spatial neighboring blocks corresponding to the luminance co-located block, and the displacement vectors of the spatial neighboring blocks corresponding to the luminance co-located block;

[0447] If there is no legal displacement vector in the luminance co-located block, a default displacement vector is added to the candidate displacement vector list; or, the displacement vector in the candidate displacement vector list is set to an invalid value.

[0448] In a possible implementation, the conditions that the displacement vectors in the candidate displacement vector list must satisfy include at least one of the following:

[0449] The reference block of the luma co-located block satisfies the reference range restriction of the intra block copy prediction mode;

[0450] The reference block of the luminance co-located block and the displacement vector all meet the reference range restriction of the intra-frame string copy prediction normal string sub-mode;

[0451] The displacement vector does not overlap with other displacement vectors except those in the candidate displacement vector list;

[0452] The displacement vector does not overlap with the first N or last N displacement vectors in the candidate displacement vector list, where N is an integer.

[0453] In a possible implementation, the ordering of the displacement vectors in the candidate displacement vector list includes any of the following:

[0454] Sort by predetermined order;

[0455] Sort by cost of each displacement vector, the cost includes at least: rate distortion cost, absolute error sum, transformed absolute error sum, square sum of difference, absolute error sum without mean, and absolute square sum without mean;

[0456] Sorting by type of displacement vector, the types of displacement vector at least include: block vector, string vector, motion vector, default vector;

[0457] Comprehensive sorting is performed according to the type of displacement vector and the cost of the displacement vector.

[0458] In a possible implementation manner, the processing unit 902 determines the target displacement vector in the candidate displacement vector list, including any of the following manners:

[0459] The Kth displacement vector in the candidate displacement vector list is used as the target displacement vector, where K is a positive integer;

[0460] Determine a target displacement vector from a list of candidate displacement vectors according to the displacement vector index;

[0461] Obtain a candidate displacement vector from the candidate displacement vector list, scale the candidate displacement vector according to the color format sampling ratio, and retain the integer pixel accuracy of the scaled displacement vector to obtain a target displacement vector;

[0462] A candidate displacement vector is obtained from the candidate displacement vector list, and the candidate displacement vector is used as a target displacement vector.

[0463] In a possible implementation, the processing unit 902 determines a prediction value of the current chroma block based on the target displacement vector, and performs the following operations:

[0464] If the target displacement vector has integer pixel accuracy, motion compensation is performed on the target displacement vector to derive a prediction value of the current chroma block; or,

[0465] If the target displacement vector has sub-pixel accuracy, the target displacement vector is interpolated, and motion compensation is performed on the interpolated target displacement vector to derive a predicted value of the current chroma block.

[0466] In a possible implementation, the processing unit 902 is further configured to perform the following operations:

[0467] If the target displacement vector does not exist in the luminance co-located block corresponding to the current chrominance block, the intra-frame prediction mode is used to derive the prediction value of the current chrominance block;

[0468] Among them, the intra-frame prediction mode includes: cross-component intra-frame prediction mode, or specified intra-frame prediction mode; the cross-component intra-frame prediction mode includes: cross-component two-step prediction mode, cross-component linear model prediction mode; the specified intra-frame prediction mode includes: intra-frame prediction DC coefficient mode, intra-frame prediction bilinear prediction mode, intra-frame prediction planar prediction mode, horizontal direction intra-frame prediction mode, vertical direction intra-frame prediction mode, any one or more.

[0469] In a possible implementation, the processing unit 902 is further configured to perform the following operations:

[0470] If the current chroma block adopts the chroma block copy intra prediction mode, determine whether the current chroma block needs to be residual decoded according to the information of the target luminance sample in the luminance co-located block corresponding to the current chroma block, and the target luminance sample is the luminance sample corresponding to the target displacement vector;

[0471] If it is determined that the current chroma block needs to be residually decoded, residual decoding is performed on the current chroma block to obtain a residual value;

[0472] Based on the residual value and prediction value of the current chrominance block, the decoded image corresponding to the current chrominance block is reconstructed.

[0473] In a possible implementation, the luminance co-located block corresponding to the current chrominance block includes multiple luminance blocks; and the judgment condition that the current chrominance block does not need to be subjected to residual decoding includes any one of the following:

[0474] If the target luminance sample adopts the intra-frame string copy prediction normal string mode, and the luminance block where the target luminance sample is located does not need to be residually encoded;

[0475] If the target luminance sample adopts the intra-frame string copy prediction mode, and the luminance block where the target luminance sample is located does not need to be residually encoded;

[0476] If the target luminance sample adopts the intra block copy prediction mode, and the luminance block where the target luminance sample is located does not need to be residual coded;

[0477] If the target luma sample adopts the intra-frame string copy prediction normal string sub-mode or the intra-frame block copy prediction mode, and the luma block where the target luma sample is located does not need to be residually encoded;

[0478] If the target luma sample adopts the intra-frame string copy prediction mode or the intra-frame block copy prediction mode, and the luma block where the target luma sample is located does not need to be residually encoded;

[0479] The judgment condition of whether the luminance block where the target luminance sample point is located needs to be residually encoded is determined according to: the coefficient flag of the luminance block, or the prediction mode of the current chrominance block.

[0480] In an embodiment of the present application, a current chroma block in a video bitstream can be determined, and the current chroma block adopts a chroma block copy intra-frame prediction mode; a target displacement vector of a luminance co-located block corresponding to the current chroma block is obtained; based on the target displacement vector, a prediction value of the current chroma block is determined; and the prediction value is used to reconstruct a decoded image corresponding to the current chroma block. As can be seen from the above, in the decoding process of the video bitstream, when it is determined that the current chroma block adopts a chroma block copy intra-frame prediction mode, the displacement vector information of the luminance co-located block can be used to determine the prediction value of the current chroma block, and there is no need to perform intra-frame prediction processing on the current chroma block, which can improve the encoding and decoding efficiency of the current chroma block, thereby effectively improving the encoding performance.

[0481] See also Fig.10 , Fig.10 is a schematic diagram of the structure of another video processing device provided in an embodiment of the present application. Fig.10 As shown, the video processing device 1000 can be applied to the computer device mentioned in the above embodiment, and the computer device can be the encoding device mentioned in the above method embodiment. Specifically, the video processing device 1000 can be a computer program (including program code) running in a computer device, for example, the video processing device 1000 is an application software; the video processing device 1000 can be used to execute the corresponding steps in the video processing method provided in the embodiment of the present application. In specific implementation, the video processing device 1000 can specifically include:

[0482] A determining unit 1001 is used to determine a current chroma block in a video, where the current chroma block adopts a chroma block copy intra prediction mode;

[0483] Processing unit 1002, used to obtain a target displacement vector of a luma co-located block corresponding to a current chroma block;

[0484] The processing unit 1002 is further configured to determine a prediction value of the current chroma block based on the target displacement vector; the prediction value is used to reconstruct a decoded image corresponding to the current chroma block;

[0485] The processing unit 1002 is further configured to encode the video based on the predicted value of the current chrominance block to generate a video bit stream.

[0486] In an embodiment of the present application, a current chroma block in a video bitstream can be determined, the current chroma block refers to a current coding unit that does not contain a luminance component, and the current chroma block adopts a chroma block copy intra-frame prediction mode; a target displacement vector of a luminance co-location block corresponding to the current chroma block is obtained; based on the target displacement vector, a prediction value of the current chroma block is determined; and the prediction value is used to reconstruct a decoded image corresponding to the current chroma block. As can be seen from the above, in the decoding process of the video bitstream, when it is determined that the current chroma block adopts a chroma block copy intra-frame prediction mode, the displacement vector information of the luminance co-location block can be used to determine the prediction value of the current chroma block, and there is no need to perform intra-frame prediction processing on the current chroma block, which can improve the encoding and decoding efficiency of the current chroma block, thereby effectively improving the encoding performance.

[0487] See also Fig.11 , Fig.11 Schematic diagram of the structure of a computer device provided in an embodiment of the present application. Fig.11 As shown, the computer device 1100 may include: a processor 1101, an input device 1102, an output device 1103 and a memory 1104. The processor 1101, the input device 1102, the output device 1103 and the memory 1104 are connected via a bus 1105. The memory 1104 is used to store a computer program, the computer program includes program instructions, and the processor 1101 is used to execute the program instructions stored in the memory 1104.

[0488] Specifically, the processor 1101 executes the following operations by running the program instructions in the memory 1104:

[0489] In one embodiment, the computer device 1100 may be the above-mentioned decoding device. In this embodiment, the processor 1101 executes the following operations by running the program instructions in the memory 1104:

[0490] Determine a current chroma block in a video bitstream, where the current chroma block adopts a chroma block copy intra prediction mode;

[0491] Get the target displacement vector of the luminance co-located block corresponding to the current chrominance block;

[0492] Based on the target displacement vector, a prediction value of the current chrominance block is determined; the prediction value is used to reconstruct a decoded image corresponding to the current chrominance block.

[0493] In a possible implementation, the chroma block copy intra prediction mode is a prediction mode independent of the normal intra prediction mode; or, the chroma block copy intra prediction mode is a sub-mode of the normal intra prediction mode;

[0494] The common intra prediction mode includes at least one of the following: an intra prediction DC coefficient mode, an angle-based intra prediction mode, a cross-component two-step prediction mode, and a prediction mode for deriving prediction values ​​based on intra reference pixels.

[0495] In a possible implementation, after determining the current chrominance block in the video code stream, the processing unit 902 is further configured to perform the following operations:

[0496] Divide the current chroma block into P×Q chroma sub-blocks, where P and Q are both integers;

[0497] Obtain the block vector of the luminance co-located sub-block corresponding to each chrominance sub-block;

[0498] Based on the obtained P×Q block vectors, a prediction value of the current chrominance block is determined.

[0499] In a possible implementation manner, the processor 1101 is further configured to perform the following operations:

[0500] Decoding the video bitstream to obtain mode indication information, where the mode indication information is used to indicate whether the current chroma block adopts a chroma block copy intra prediction mode;

[0501] The mode indication information includes at least one of the following: a sequence header flag, a picture header flag, a slice header flag, a coding unit flag, implicit information, and chroma indication information.

[0502] In a possible implementation, if the mode indication information is a sequence header flag, the mode indication information is set in a sequence header of a current sequence to which the current chroma block belongs; when the sequence header flag is a first preset value, it indicates that the current chroma block allows the use of a chroma block copy intra-frame prediction mode; and when the sequence header flag is a second preset value, it indicates that the current chroma block does not use a chroma block copy intra-frame prediction mode;

[0503] If the mode indication information is an image header flag, the mode indication information is set in an image header of a current image to which the current chroma block belongs; when the image header flag is a first preset value, it indicates that the current chroma block allows the use of a chroma block copy intra-frame prediction mode; and when the image header flag is a second preset value, it indicates that the current chroma block does not use a chroma block copy intra-frame prediction mode;

[0504] If the mode indication information is a slice header flag, the mode indication information is set in the slice header of the current slice to which the current chroma block belongs; when the slice header flag is a first preset value, it indicates that the current chroma block allows the use of the chroma block copy intra prediction mode, and when the slice header flag is a second preset value, it indicates that the current chroma block does not use the chroma block copy intra prediction mode;

[0505] If the mode indication information is a coding unit flag, the mode indication information is set in the current coding unit; when the coding unit flag is a first preset value, it indicates that the current chroma block adopts the chroma block copy intra prediction mode, and when the coding unit flag is a second preset value, it indicates that the current chroma block does not adopt the chroma block copy intra prediction mode;

[0506] If the mode indication information is implicit information, then when the implicit information indicates that the luminance sample at a preset position in the luminance co-located block corresponding to the current chrominance block meets a preset condition, the current chrominance block is indicated to adopt the chrominance block copy intra prediction mode;

[0507] If the mode indication information is chroma indication information, then when the chroma indication information indicates that the current chroma block adopts a preset intra-frame prediction mode, and the luminance sample at a preset position in the luminance co-located block meets a preset condition, it indicates that the current chroma block adopts a chroma block copy intra-frame prediction mode; the preset intra-frame prediction mode includes an inheritance mode.

[0508] In a possible implementation manner, the processor 1101 is further configured to perform the following operations:

[0509] Check whether the video code stream meets the decoding conditions;

[0510] If the video code stream meets the decoding condition, the step of decoding the video code stream to obtain mode indication information is performed.

[0511] In a possible implementation, the decoding condition includes: the color format adopted by the video code stream is a format including a chrominance component;

[0512] When the mode indication information is set in the current coding unit to which the current chroma block belongs, the decoding condition includes any one or more of the following:

[0513] The current coding unit contains only chroma components;

[0514] The luminance sample point at a preset position in the luminance co-location block corresponding to the current chrominance block meets a preset condition;

[0515] If there are multiple preset positions, the number of brightness sample points that meet the preset conditions is at least one, or is greater than or equal to a preset threshold.

[0516] In a possible implementation, the preset position is determined in any of the following ways:

[0517] Determine a preset chroma sample point in the current chroma block, and determine a preset position of a corresponding luminance sample point in a luminance co-located block according to the position of the preset chroma sample point;

[0518] Determining a preset position from the brightness co-located blocks;

[0519] According to the division method of the luminance co-located block, a preset position is determined in each sub-block obtained by dividing the luminance co-located block;

[0520] The luminance co-located block is divided into M×N sub-blocks, and a specified position in the M×N sub-blocks is used as a preset position, where the specified position includes: any position of the center, upper left corner, upper right corner, lower left corner, and lower right corner of the sub-block, and M and N are both positive integers.

[0521] In a possible implementation, the preset condition includes any one or more of the following:

[0522] The luminance sample adopts any mode in a specific mode set, which includes: intra-frame block copy prediction mode, intra-frame string copy prediction ordinary string sub-mode, intra-frame string copy prediction non-ordinary string sub-mode, or any one or more prediction modes in other prediction modes with intra-frame displacement vectors.

[0523] Luminance samples have valid displacement vectors;

[0524] The intra prediction history motion information table of the current chroma block is not empty.

[0525] In a possible implementation, the processor 1101 obtains a target displacement vector of a luma co-located block corresponding to the current chroma block, and performs the following operations:

[0526] Construct a list of candidate displacement vectors of the luma co-located block corresponding to the current chroma block;

[0527] A target displacement vector is determined from the list of candidate displacement vectors.

[0528] In a possible implementation, the processor 1101 constructs a candidate displacement vector list of the luma co-located block corresponding to the current chroma block, including any one or more of the following methods:

[0529] Add the displacement vectors of the luminance samples in the luminance co-located block that satisfy the specified mode to the candidate displacement vector list, where the specified mode includes: any one of the intra-frame block copy prediction mode, the intra-frame string copy prediction normal string sub-mode, the string copy intra-frame prediction non-normal string sub-mode, the intra-frame template matching mode, the inter-frame prediction mode, the affine motion mode, the inter-frame intra-frame composite mode, and the angle weighted prediction mode;

[0530] Add the displacement vector of the spatial adjacent block corresponding to the luminance co-located block to the candidate displacement vector list;

[0531] Adding the displacement vector in the historical motion information table or the intra-frame prediction historical motion information table to the candidate displacement vector list;

[0532] Adding the candidate displacement vectors to the candidate displacement vector list after offset processing, the candidate displacement vectors include: any one or more of the displacement vectors of the luminance samples satisfying the specified mode in the luminance co-located block, the displacement vectors of the spatial neighboring blocks corresponding to the luminance co-located block, and the displacement vectors of the spatial neighboring blocks corresponding to the luminance co-located block;

[0533] If there is no legal displacement vector in the luminance co-located block, a default displacement vector is added to the candidate displacement vector list; or, the displacement vector in the candidate displacement vector list is set to an invalid value.

[0534] In a possible implementation, the conditions that the displacement vectors in the candidate displacement vector list must satisfy include at least one of the following:

[0535] The reference block of the luma co-located block satisfies the reference range restriction of the intra block copy prediction mode;

[0536] The reference block of the luminance co-located block and the displacement vector all meet the reference range restriction of the intra-frame string copy prediction normal string sub-mode;

[0537] The displacement vector does not overlap with other displacement vectors except those in the candidate displacement vector list;

[0538] The displacement vector does not overlap with the first N or last N displacement vectors in the candidate displacement vector list, where N is an integer.

[0539] In a possible implementation, the ordering of the displacement vectors in the candidate displacement vector list includes any of the following:

[0540] Sort by predetermined order;

[0541] Sort by cost of each displacement vector, the cost includes at least: rate distortion cost, absolute error sum, transformed absolute error sum, difference square sum, mean-removed absolute error sum, mean-removed absolute square sum;

[0542] Sorting by type of displacement vector, the types of displacement vector at least include: block vector, string vector, motion vector, default vector;

[0543] Comprehensive sorting is performed according to the type of displacement vector and the cost of the displacement vector.

[0544] In a possible implementation manner, the processor 1101 determines the target displacement vector in the candidate displacement vector list, including any of the following manners:

[0545] The Kth displacement vector in the candidate displacement vector list is used as the target displacement vector, where K is a positive integer;

[0546] Determine a target displacement vector from a list of candidate displacement vectors according to the displacement vector index;

[0547] Obtain a candidate displacement vector from the candidate displacement vector list, scale the candidate displacement vector according to the color format sampling ratio, and retain the integer pixel accuracy of the scaled displacement vector to obtain a target displacement vector;

[0548] A candidate displacement vector is obtained from the candidate displacement vector list, and the candidate displacement vector is used as a target displacement vector.

[0549] In a possible implementation, the processor 1101 determines a prediction value of the current chroma block based on the target displacement vector, and is configured to perform the following operations:

[0550] If the target displacement vector has integer pixel accuracy, motion compensation is performed on the target displacement vector to derive a prediction value of the current chroma block; or,

[0551] If the target displacement vector has sub-pixel accuracy, the target displacement vector is interpolated, and motion compensation is performed on the interpolated target displacement vector to derive a predicted value of the current chroma block.

[0552] In a possible implementation, the processing unit 902 is further configured to perform the following operations:

[0553] If the target displacement vector does not exist in the luminance co-located block corresponding to the current chrominance block, the intra-frame prediction mode is used to derive the prediction value of the current chrominance block;

[0554] Among them, the intra-frame prediction mode includes: cross-component intra-frame prediction mode, or specified intra-frame prediction mode; the cross-component intra-frame prediction mode includes: cross-component two-step prediction mode, cross-component linear model prediction mode; the specified intra-frame prediction mode includes: intra-frame prediction DC coefficient mode, intra-frame prediction bilinear prediction mode, intra-frame prediction planar prediction mode, horizontal direction intra-frame prediction mode, vertical direction intra-frame prediction mode, any one or more.

[0555] In a possible implementation manner, the processor 1101 is further configured to perform the following operations:

[0556] If the current chroma block adopts the chroma block copy intra prediction mode, determine whether the current chroma block needs to be residual decoded according to the information of the target luminance sample in the luminance co-located block corresponding to the current chroma block, and the target luminance sample is the luminance sample corresponding to the target displacement vector;

[0557] If it is determined that the current chroma block needs to be residually decoded, residual decoding is performed on the current chroma block to obtain a residual value;

[0558] Based on the residual value and prediction value of the current chrominance block, the decoded image corresponding to the current chrominance block is reconstructed.

[0559] In a possible implementation, the luminance co-located block corresponding to the current chrominance block includes multiple luminance blocks; and the judgment condition that the current chrominance block does not need to be subjected to residual decoding includes any one of the following:

[0560] If the target luminance sample adopts the intra-frame string copy prediction normal string mode, and the luminance block where the target luminance sample is located does not need to be residually encoded;

[0561] If the target luminance sample adopts the intra-frame string copy prediction mode, and the luminance block where the target luminance sample is located does not need to be residually encoded;

[0562] If the target luminance sample adopts the intra block copy prediction mode, and the luminance block where the target luminance sample is located does not need to be residual coded;

[0563] If the target luma sample adopts the intra-frame string copy prediction normal string sub-mode or the intra-frame block copy prediction mode, and the luma block where the target luma sample is located does not need to be residually encoded;

[0564] If the target luma sample adopts the intra-frame string copy prediction mode or the intra-frame block copy prediction mode, and the luma block where the target luma sample is located does not need to be residually encoded;

[0565] The judgment condition of whether the luminance block where the target luminance sample point is located needs to be residually encoded is determined according to: the coefficient flag of the luminance block, or the prediction mode of the current chrominance block.

[0566] In another embodiment, the computer device 1100 may be the above encoding device. In this embodiment, the processor 1101 executes the following operations by running the program instructions in the memory 1104:

[0567] Determine a current chroma block in the video, the current chroma block adopts a chroma block copy intra prediction mode;

[0568] Get the target displacement vector of the luminance co-located block corresponding to the current chrominance block;

[0569] Based on the target displacement vector, determining a prediction value of the current chrominance block; the prediction value is used to reconstruct a decoded image corresponding to the current chrominance block;

[0570] The video is encoded based on the predicted value of the current chrominance block to generate a video bitstream.

[0571] In an embodiment of the present application, the current coding block in the video is determined, and the coding information of the adjacent coding blocks of the current coding block is determined; then, the transformation information of the current coding block is determined based on the coding information of the adjacent coding blocks; based on the determined transformation information of the current coding block, the current coding block is encoded. In the video encoding process, the transformation information of the current coding block is determined directly based on the coding information of the adjacent coding blocks, so that only the cost of encoding the transformation information of the coding block needs to be reduced at the encoding end, thereby reducing the bit rate consumption in the entire encoding and decoding process, thereby improving the encoding efficiency.

[0572] In addition, it should be pointed out here that: the embodiment of the present application also provides a computer-readable storage medium, and the computer-readable storage medium stores a computer program, and the computer program includes program instructions. When the processor executes the above program instructions, it can execute the above Figure 5 and Fig. 9 The method in the corresponding embodiment, therefore, will not be repeated here. For technical details not disclosed in the computer-readable storage medium embodiment involved in this application, please refer to the description of the method embodiment of this application. As an example, the program instructions can be deployed on a computer device, or executed on multiple computer devices located in one location, or, executed on multiple computer devices distributed in multiple locations and interconnected by a communication network.

[0573] According to one aspect of the present application, a computer program product is provided, the computer program product comprising a computer program, the computer program being stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium, and the processor executes the computer program, so that the computer device can execute the method in the above embodiment, and therefore, it will not be described in detail here.

[0574] A person skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0575] The above disclosure is only the preferred embodiment of the present application, which certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

Claims

1. A video processing method, characterized in that: include: Determine a current chroma block in a video bitstream, where the current chroma block adopts a chroma block copy intra prediction mode; Obtain a target displacement vector of a luminance co-located block corresponding to the current chrominance block; Determining a prediction value of the current chroma block based on the target displacement vector; Based on the prediction value, an image corresponding to the current chrominance block is reconstructed.

2. The method according to claim 1, characterized in that The chroma block copy intra prediction mode is a prediction mode independent of the normal intra prediction mode; or, the chroma block copy intra prediction mode is a sub-mode of the normal intra prediction mode; The common intra-frame prediction mode includes at least one of the following: an intra-frame prediction DC coefficient mode, an angle-based intra-frame prediction mode, a cross-component two-step prediction mode, and a prediction mode for deriving prediction values ​​based on intra-frame reference pixels.

3. The method according to claim 1, characterized in that After determining the current chroma block in the video code stream, the method further includes: Divide the current chroma block into P×Q chroma sub-blocks, where P and Q are both integers; Obtain the block vector of the luminance co-located sub-block corresponding to each chrominance sub-block; Based on the acquired P×Q block vectors, a prediction value of the current chroma block is determined.

4. The method according to claim 1, characterized in that The method further comprises: Decoding the video code stream to obtain mode indication information, where the mode indication information is used to indicate whether the current chroma block adopts a chroma block copy intra prediction mode; The mode indication information includes at least one of the following: a sequence header flag, a picture header flag, a slice header flag, a coding unit flag, implicit information, and chroma indication information.

5. The method according to claim 4, characterized in that If the mode indication information is a sequence header flag, the mode indication information is set in the sequence header of the current sequence to which the current chroma block belongs; when the sequence header flag is a first preset value, it indicates that the current chroma block allows the use of a chroma block copy intra-frame prediction mode; when the sequence header flag is a second preset value, it indicates that the current chroma block does not use a chroma block copy intra-frame prediction mode; If the mode indication information is an image header flag, the mode indication information is set in an image header of a current image to which the current chroma block belongs; when the image header flag is a first preset value, it indicates that the current chroma block allows the use of a chroma block copy intra-frame prediction mode; and when the image header flag is a second preset value, it indicates that the current chroma block does not use a chroma block copy intra-frame prediction mode; If the mode indication information is a slice header flag, the mode indication information is set in the slice header of the current slice to which the current chroma block belongs; when the slice header flag is a first preset value, it indicates that the current chroma block allows the use of a chroma block copy intra-frame prediction mode; when the slice header flag is a second preset value, it indicates that the current chroma block does not use a chroma block copy intra-frame prediction mode; If the mode indication information is a coding unit flag, the mode indication information is set in the current coding unit; When the coding unit flag is a first preset value, it indicates that the current chroma block adopts a chroma block copy intra prediction mode, and when the coding unit flag is a second preset value, it indicates that the current chroma block does not adopt a chroma block copy intra prediction mode; If the mode indication information is implicit information, then when the implicit information indicates that the luma sample at a preset position in the luma co-located block corresponding to the current chroma block meets a preset condition, the current chroma block is indicated to adopt a chroma block copy intra prediction mode; If the mode indication information is chroma indication information, then when the chroma indication information indicates that the current chroma block adopts a preset intra-frame prediction mode, and the luma sample at a preset position in the luma co-located block meets a preset condition, instructing the current chroma block to adopt a chroma block copy intra-frame prediction mode; The preset intra-frame prediction mode includes an inheritance mode.

6. The method according to claim 4 or 5, characterized in that The method further comprises: Check whether the video code stream meets the decoding condition; If the video code stream meets the decoding condition, the step of decoding the video code stream to obtain mode indication information is performed.

7. The method according to claim 6, characterized in that The decoding conditions include: the color format adopted by the video code stream is a format containing chrominance components; When the mode indication information is set in the current coding unit to which the current chroma block belongs, the decoding condition includes any one or more of the following: The current coding unit only includes chrominance components; The luminance sample point at a preset position in the luminance co-located block corresponding to the current chrominance block meets a preset condition; If the number of the preset positions is multiple, the number of brightness sample points that meet the preset condition is at least one, or is greater than or equal to a preset threshold.

8. The method according to claim 7, characterized in that The preset position may be determined in any of the following ways: Determining a preset chroma sample in the current chroma block, and determining a preset position of a corresponding luminance sample in the luminance co-located block according to the position of the preset chroma sample; Determining a preset position from the brightness co-located blocks; According to the division method of the luminance co-located block, determining a preset position in each sub-block obtained by dividing the luminance co-located block; The luminance co-located block is divided into M×N sub-blocks, and a specified position in the M×N sub-blocks is used as a preset position, wherein the specified position includes: any one of the center, upper left corner, upper right corner, lower left corner, and lower right corner of the sub-block, and M and N are both positive integers.

9. The method according to claim 7, characterized in that The preset conditions include any one or more of the following: The luma sample adopts any one of the specific modes, wherein the specific mode set includes: intra block copy prediction mode, intra string copy prediction normal string sub-mode, intra string copy prediction non-normal string sub-mode, or any one or more prediction modes with intra displacement vector; The brightness sample has a legal displacement vector; The intra-frame prediction history motion information table of the current chroma block is not empty.

10. The method according to claim 1, characterized in that The obtaining of a target displacement vector of a luminance co-located block corresponding to the current chrominance block includes: Constructing a candidate displacement vector list of the luma co-located block corresponding to the current chroma block; A target displacement vector is determined in the candidate displacement vector list.

11. The method according to claim 10, characterized in that The constructing of a candidate displacement vector list of the luma co-located block corresponding to the current chroma block includes any one or more of the following methods: Add the displacement vectors of the luminance samples in the luminance co-located block that satisfy the specified mode to the candidate displacement vector list, wherein the specified mode includes: any one of the intra-frame block copy prediction mode, the intra-frame string copy prediction normal string sub-mode, the string copy intra-frame prediction non-normal string sub-mode, the intra-frame template matching mode, the inter-frame prediction mode, the affine motion mode, the inter-frame intra-frame composite mode, and the angle weighted prediction mode; Adding the displacement vector of the spatial domain adjacent block corresponding to the luminance co-located block to the candidate displacement vector list; Adding the displacement vector in the historical motion information table or the intra-frame prediction historical motion information table to the candidate displacement vector list; Adding candidate displacement vectors to a candidate displacement vector list after performing offset processing, wherein the candidate displacement vectors include: any one or more of the displacement vectors of luminance samples satisfying a specified mode in the luminance co-located block, the displacement vectors of spatial neighboring blocks corresponding to the luminance co-located block, and the displacement vectors of spatial neighboring blocks corresponding to the luminance co-located block; If no legal displacement vector exists in the luminance co-located block, a default displacement vector is added to the candidate displacement vector list; or, the displacement vector in the candidate displacement vector list is set to an invalid value.

12. The method according to claim 10 or 11, characterized in that The conditions that the displacement vectors in the candidate displacement vector list must satisfy include at least one of the following: The reference block of the luma co-located block satisfies the reference range restriction of the intra block copy prediction mode; The reference block of the luminance co-located block and the displacement vector both meet the reference range restriction of the intra-frame string copy prediction common string sub-mode; The displacement vector does not overlap with other displacement vectors except the candidate displacement vector list; The displacement vector does not overlap with the first N or last N displacement vectors in the candidate displacement vector list, where N is an integer.

13. The method according to claim 12, characterized in that The sorting method of the displacement vectors in the candidate displacement vector list includes any one of the following: Sort by predetermined order; Sort by cost of each displacement vector, the cost comprising at least: rate distortion cost, absolute error sum, transformed absolute error sum, square sum of difference, mean-removed absolute error sum, mean-removed absolute variance sum; Sorting by type of displacement vectors, wherein the types of displacement vectors include at least: block vectors, string vectors, motion vectors, and default vectors; Comprehensive sorting is performed according to the type of displacement vector and the cost of the displacement vector.

14. The method according to claim 10, characterized in that Determining the target displacement vector in the candidate displacement vector list includes any of the following methods: Taking the Kth displacement vector in the candidate displacement vector list as the target displacement vector, where K is a positive integer; Determine a target displacement vector from the candidate displacement vector list according to the displacement vector index; Obtaining a candidate displacement vector from the candidate displacement vector list, scaling the candidate displacement vector according to a color format sampling ratio, and retaining the integer pixel accuracy of the scaled displacement vector to obtain the target displacement vector; A candidate displacement vector is obtained from the candidate displacement vector list, and the candidate displacement vector is used as the target displacement vector.

15. The method according to claim 1, wherein: The method further comprises: If there is no target displacement vector in the luminance co-located block corresponding to the current chrominance block, the prediction value of the current chrominance block is derived using the intra-frame prediction mode; Among them, the intra-frame prediction mode includes: a cross-component intra-frame prediction mode, or a specified intra-frame prediction mode; the cross-component intra-frame prediction mode includes: a cross-component two-step prediction mode, a cross-component linear model prediction mode; the specified intra-frame prediction mode includes: any one or more of an intra-frame prediction DC coefficient mode, an intra-frame prediction bilinear prediction mode, an intra-frame prediction planar prediction mode, a horizontal direction intra-frame prediction mode, and a vertical direction intra-frame prediction mode.

16. The method according to claim 1, wherein: The method further comprises: If the current chroma block adopts the chroma block copy intra prediction mode, determining whether the current chroma block needs to be residually decoded according to information of a target luminance sample in a luminance co-located block corresponding to the current chroma block, the target luminance sample being a luminance sample corresponding to the target displacement vector; If it is determined that the current chroma block needs to be residually decoded, residually decode the current chroma block to obtain a residual value; Based on the residual value and the prediction value of the current chrominance block, a decoded image corresponding to the current chrominance block is reconstructed.

17. The method according to claim 16, characterized in that The luminance co-located block corresponding to the current chrominance block includes multiple luminance blocks; the judgment condition that the current chrominance block does not need to be subjected to residual decoding includes any one of the following: If the target luminance sample adopts the intra-frame string copy prediction common string mode, and the luminance block where the target luminance sample is located does not need to be residually encoded; If the target luminance sample adopts the intra-frame string copy prediction mode, and the luminance block where the target luminance sample is located does not need to be residually encoded; If the target luminance sample adopts the intra block copy prediction mode, and the luminance block where the target luminance sample is located does not need to be residually encoded; If the target luma sample adopts the intra-frame string copy prediction normal string sub-mode or the intra-frame block copy prediction mode, and the luma block where the target luma sample is located does not need to be residually encoded; If the target luma sample adopts the intra-frame string copy prediction mode or the intra-frame block copy prediction mode, and the luma block where the target luma sample is located does not need to be residually encoded; The judgment condition of whether the luminance block where the target luminance sample is located needs to be residually encoded is determined according to: the coefficient flag of the luminance block, or the prediction mode of the current chrominance block.

18. A video processing method, characterized in that: include: Determine a current chroma block in the video, wherein the current chroma block adopts a chroma block copy intra prediction mode; Obtain a target displacement vector of a luminance co-located block corresponding to the current chrominance block; Determining a prediction value of the current chroma block based on the target displacement vector; The video is encoded based on the predicted value of the current chrominance block to generate a video code stream.

19. A computer device, characterized in that: include: a processor adapted to execute a computer program; A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by the processor, the video processing method according to any one of claims 1 to 18 is executed.

20. A computer-readable storage medium, characterized in that: The computer storage medium stores a computer program, and when the computer program is executed by a processor, the video processing method according to any one of claims 1 to 18 is executed.

Images