Cross-component prediction method, video encoding method and apparatus, video decoding method and apparatus, medium, and device

Abstract

Embodiments of the present application provide a cross-component prediction method, a video encoding method and apparatus, a video decoding method and apparatus, a medium, and a device. The cross-component prediction method comprises: on the basis of a pixel value comprised in a template region adjacent to a current block, constructing a cross-component prediction model corresponding to the current block, the size of the template region being determined on the basis of video frame information corresponding to the current block, wherein the video frame information comprises at least one of the following: the position of the current block in a video frame, the correlation between the current block and an adjacent region of the current block, the size of the current block, and the resolution of the video frame; and on the basis of the cross-component prediction model and a luminance component reconstructed pixel of the current block, generating a predicted chrominance component value of the current block.

Description

Cross-component prediction method, video coding method, device, medium and equipment

[0001] The present application claims priority to the Chinese patent application No. 202410745178.2, filed on June 7, 2024, and entitled "Cross-component prediction method, video coding method, device, medium and equipment", the content of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0002] The present application relates to the field of computer and communication technology, in particular, to a cross-component prediction method, a video coding method, a device, a medium and equipment. BACKGROUND

[0003] With the wide application of video technology in the fields of digital media, real-time communication, education and entertainment, higher requirements are put forward for the efficiency and real-time performance of video coding. In the field of video coding, a prediction model (such as a nonlinear prediction model, a linear prediction model, etc.) can be used for prediction coding, and the prediction model usually needs to use the data in the template region to determine the template parameters.

[0004] TECHNICAL CONTENT

[0005] Embodiments of the present application provide a cross-component prediction method, a video coding method, a device, a medium and equipment, which can realize adaptive adjustment of the size of the template region, thereby improving the coding performance of the video and reducing the hardware cost.

[0006] Other characteristics and advantages of the present application will become apparent from the following detailed description, or will be learned by practice of the present application.

[0007] The embodiments of the present application provide a cross-component prediction method, comprising: constructing a cross-component prediction model corresponding to a current block according to pixel values contained in a template region adjacent to the current block, the size of the template region being determined based on video frame information corresponding to the current block; wherein the template region comprises a first region located on the left side of the current block and a second region located above the current block, and the size of the template region comprises at least one of the width of the first region and the height of the second region; the video frame information comprises at least one of the following: the position of the current block in the video frame; the correlation of the current block and its adjacent region; the size of the current block; and the resolution of the video frame; generating a chroma component prediction value of the current block according to the cross-component prediction model and the luminance component reconstruction pixel of the current block.

[0008] The embodiment of the present application provides a video decoding method, comprising: performing decoding processing on a video code stream to obtain pixel values contained in a template region adjacent to a current block; constructing a cross-component prediction model corresponding to the current block according to the pixel values contained in the template region, wherein the size of the template region is determined based on video frame information corresponding to the current block; wherein the template region comprises a first region located on the left side of the current block and a second region located above the current block, and the size of the template region comprises at least one of the width of the first region and the height of the second region; the video frame information comprises at least one of the following: the position of the current block in the video frame; the correlation between the current block and its adjacent region; the size of the current block; and the resolution of the video frame; generating a chroma component prediction value of the current block according to the cross-component prediction model and a luminance component reconstruction pixel of the current block; and performing decoding processing based on the chroma component prediction value of the current block.

[0009] The embodiment of the present application provides a video encoding method, comprising: obtaining pixel values contained in a template region adjacent to a current block; constructing a cross-component prediction model corresponding to the current block according to the pixel values contained in the template region, wherein the size of the template region is determined based on video frame information corresponding to the current block; wherein the template region comprises a first region located on the left side of the current block and a second region located above the current block, and the size of the template region comprises at least one of the width of the first region and the height of the second region; the video frame information comprises at least one of the following: the position of the current block in the video frame; the correlation between the current block and its adjacent region; the size of the current block; and the resolution of the video frame; generating a chroma component prediction value of the current block according to the cross-component prediction model and a luminance component reconstruction pixel of the current block; and performing encoding processing based on the chroma component prediction value of the current block to obtain an encoded video code stream.

[0010] The embodiment of the present application provides a cross-component prediction device, comprising: a processing unit, configured to construct a cross-component prediction model corresponding to a current block according to pixel values contained in a template region adjacent to the current block, wherein the size of the template region is determined based on video frame information corresponding to the current block; wherein the template region comprises a first region located on the left side of the current block and a second region located above the current block, and the size of the template region comprises at least one of the width of the first region and the height of the second region; the video frame information comprises at least one of the following: the position of the current block in the video frame; the correlation between the current block and its adjacent region; the size of the current block; and the resolution of the video frame; and a generating unit, configured to generate a chroma component prediction value of the current block according to the cross-component prediction model and a luminance component reconstructed pixel of the current block.

[0011] The embodiment of the present application provides a video decoding device, comprising: a decoding unit, configured to decode a video code stream to obtain pixel values contained in a template region adjacent to a current block; a processing unit, configured to construct a cross-component prediction model corresponding to the current block according to the pixel values contained in the template region, wherein the size of the template region is determined based on video frame information corresponding to the current block; wherein the template region comprises a first region located on the left side of the current block and a second region located above the current block, and the size of the template region comprises at least one of the width of the first region and the height of the second region; the video frame information comprises at least one of the following: the position of the current block in the video frame; the correlation between the current block and its adjacent region; the size of the current block; and the resolution of the video frame; and a generating unit, configured to generate a chroma component prediction value of the current block according to the cross-component prediction model and a luminance component reconstructed pixel of the current block, to perform decoding processing based on the chroma component prediction value of the current block.

[0012] The embodiment of the present application provides a video coding device, comprising: an acquisition unit configured to acquire pixel values contained in a template region adjacent to a current block; a processing unit configured to construct a cross-component prediction model corresponding to the current block according to the pixel values contained in the template region, wherein the size of the template region is determined based on video frame information corresponding to the current block; wherein the template region comprises a first region located on the left side of the current block and a second region located above the current block, and the size of the template region comprises at least one of the width of the first region and the height of the second region; the video frame information comprises at least one of the following: the position of the current block in the video frame; the correlation between the current block and its adjacent region; the size of the current block; and the resolution of the video frame; a generation unit configured to generate a chroma component prediction value of the current block according to the cross-component prediction model and the luminance component reconstruction pixel of the current block; and an encoding unit configured to perform encoding processing based on the chroma component prediction value of the current block to obtain an encoded video bitstream.

[0013] The embodiment of the present application provides a computer readable medium, which stores a computer program, and the computer program is executed by a processor to implement the cross-component prediction method, the video decoding method or the video coding method as described in the above embodiment.

[0014] The embodiment of the present application provides an electronic device, comprising: one or more processors; and a storage device configured to store one or more computer programs, which, when executed by the one or more processors, cause the electronic device to implement the cross-component prediction method, the video decoding method or the video coding method as described in the above embodiment.

[0015] The embodiment of the present application provides a computer program product, which comprises a computer program stored in a computer readable storage medium. A processor of an electronic device reads and executes the computer program from the computer readable storage medium, so that the electronic device executes the cross-component prediction method, the video decoding method or the video coding method provided in the various embodiments.

[0016] The embodiment of the present application also provides a non-volatile computer readable storage medium, which stores a code stream formed by a computer program, and the non-volatile computer readable storage medium stores one or more sets of instructions configured to be executed by a computing device having a control circuit and a memory, and when the one or more sets of instructions are executed, the video decoding method provided in the various embodiments of the present application is implemented.

[0017] The embodiment of the present application also provides a method for storing a video bitstream, wherein the video bitstream is generated according to the video coding method provided in the various embodiments of the present application.

[0018] The embodiments of the present application further provide a computer storage medium storing instructions executable by at least one processor to perform the video encoding method provided by the embodiments of the present application, to generate a bitstream and store.

[0019] It should be understood that the general description above and the detailed description below are only exemplary and explanatory and are not restrictive of the present application.

[0020] BRIEF DESCRIPTION OF DRAWINGS

[0021] FIG. 1 shows a schematic diagram of an exemplary system architecture to which the technical solutions of the embodiments of the present application can be applied;

[0022] FIG. 2 shows a schematic diagram of the placement of a video encoding device and a video decoding device in a streaming system;

[0023] FIG. 3 shows a basic flowchart of a video encoder;

[0024] FIG. 4 shows a flowchart of cross-component nonlinear prediction;

[0025] FIG. 5 shows a schematic diagram of samples used in the cross-component nonlinear prediction process;

[0026] FIG. 6 shows a schematic diagram of the position of a line buffer;

[0027] FIG. 7 shows a flowchart of a cross-component prediction method according to some embodiments of the present application;

[0028] FIG. 8 shows a schematic diagram of a template region according to some embodiments of the present application;

[0029] FIG. 9 shows a flowchart of a video decoding method according to some embodiments of the present application;

[0030] FIG. 10 shows a flowchart of a video encoding method according to some embodiments of the present application;

[0031] FIG. 11 shows a block diagram of a cross-component prediction device according to some embodiments of the present application;

[0032] FIG. 12 shows a block diagram of a video decoding device according to some embodiments of the present application;

[0033] FIG. 13 shows a block diagram of a video encoding device according to some embodiments of the present application;

[0034] FIG. 14 shows a structural schematic diagram of a computer system of an electronic device suitable for use to implement the embodiments of the present application. DETAILED DESCRIPTION

[0035] Example implementations are now described with reference to the drawings. Example implementations can, however, be implemented in various forms and should not be considered limited to the examples described and / or illustrated. Rather, the intent is to cover all modification, equivalents, and alternatives falling within the scope of the example implementations. Numerous specific details are described to provide a thorough understanding of example implementations.

[0036] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are recited to provide a thorough understanding of embodiments of the application. However, one skilled in the relevant art will recognize that the embodiments can be practiced without the specific details, that numerous options exist, that the described embodiments are but a few illustrative examples of the possible embodiments, and are not intended to be exhaustive or limiting.

[0037] In embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program with a predetermined function, and works together with other related parts to achieve a predetermined goal, and can be implemented entirely or partially using software, hardware (such as a processing circuit or a memory), or a combination thereof. Similarly, one processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be a part of an overall module or unit that includes the functions of the module or unit.

[0038] The block diagrams in the drawings show functional entities, which do not necessarily have to correspond to physically independent entities. That is, the functional entities can be implemented in software, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0039] The flowcharts shown in the drawings are merely exemplary illustrations, and do not necessarily include all contents and operations / steps, nor are they necessarily executed in the order described. For example, some operations / steps can be further broken down, and some operations / steps can be combined or partially combined, so the actual execution order can be changed according to the actual situation.

[0040] It should be noted that "multiple" referred to herein means two or more. The association relationship of "and / or" between associated objects is described, which means that there can be three relationships, for example, A and / or B can represent the three cases of A alone, A and B together, and B alone. The character " / " generally represents an "or" relationship between the associated objects before and after it.

[0041] In the field of video coding, a prediction model (such as a nonlinear prediction model, a linear prediction model, etc.) can be used for prediction coding, and the prediction model usually needs to use data in a template region to determine template parameters. If the template region is set too large, the hardware cost will increase due to caching data in the template region, and if the template region is set too small, the prediction model will be inaccurate, which will affect the coding effect of the video. Therefore, how to reasonably set the size of the template region is a technical problem to be solved.

[0042] In the technical solutions provided in some embodiments of the present application, the size of the template region of the current block is adaptively adjusted according to the video frame information corresponding to the current block, so as to construct a cross-component prediction model corresponding to the current block based on the pixel values contained in the template region adjacent to the current block, and generate the prediction value of the chroma component of the current block according to the cross-component prediction model and the luminance component reconstruction pixels of the current block, so that the hardware implementation cost caused by the too large template region can be avoided, and the prediction result caused by the too small template region can also be avoided, the adaptive adjustment of the size of the template region is realized, which is conducive to improving the coding performance of the video and reducing the hardware cost.

[0043] FIG. 1 shows a schematic diagram of an exemplary system architecture to which the technical solutions of the embodiments of the present application can be applied.

[0044] As shown in FIG. 1, the system architecture 100 includes a plurality of terminal devices that can communicate with each other via, for example, a network 150. For example, the system architecture 100 can include a first terminal device 110 and a second terminal device 120 interconnected via the network 150. In the embodiment of FIG. 1, the first terminal device 110 and the second terminal device 120 perform unidirectional data transmission.

[0045] For example, the first terminal device 110 can code video data (such as a stream of video pictures that are captured by the terminal device 110) for transmission to the second terminal device 120 via the network 150, the coded video data is in the form of one or more coded video bitstreams, the second terminal device 120 can receive the coded video data from the network 150, decode the coded video data to recover the video data, and display video pictures based on the recovered video data.

[0046] In some embodiments of the application, system architecture 100 can include third and fourth terminal devices 130 and 140 that perform bidirectional transmission of coded video data that can occur, for example, during videoconferencing. For bidirectional transmission of data, each terminal device 130 and 140 can code video data (e.g., a stream of video pictures that are captured by the terminal device) for transmission to the other terminal device 130 and 140 in the system. Each terminal device 130 and 140 also receives the coded video data transmitted by the other terminal device 130 and 140, and decodes the coded video data to recover the video pictures for use in, e.g., a video conference.

[0047] In the example of FIG. 1, first, second, third and fourth terminal devices 110, 120, 130 and 140 can be servers or terminals, although the principles of the present application can not be limited to this.

[0048] A server can be a standalone physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud service, cloud database, cloud computing, cloud function, cloud storage, network service, cloud communication, middleware service, domain name service, security service, content delivery network (CDN), and basic cloud computing services such as big data and artificial intelligence platform. A terminal can be a smartphone, tablet computer, notebook computer, desktop computer, smart speaker, smart voice interaction device, smart watch, smart home appliance, vehicle-mounted terminal, aircraft, etc., but is not limited thereto.

[0049] Network 150 shown in FIG. 1 represents any number of networks that convey coded video data between first, second, third and fourth terminal devices 110, 120, 130 and 140, including for example wire- and / or wireless communication networks. Communication network 150 can exchange data in circuit-switched and / or packet-switched channels. The network can include telecommunication networks, local area and / or wide area networks, and / or the Internet. For the purposes of the present application, the architecture and topology of the network 150 can be immaterial to the operation of the application unless otherwise explained herein below.

[0050] In some embodiments of the application, FIG. 2 shows the placement of video encoding and video decoding devices in a streaming environment. The disclosed subject matter can be equally applicable to other video enabled applications, including, for example, video conferencing, digital television (TV), storing of compressed video on digital media including CD, DVD, memory stick and the like, and so on.

[0051] The streaming system can include a capture subsystem 213 that can include a video source 201, for example a digital camera, that creates an uncompressed video picture stream 202. In an embodiment, the video picture stream 202 includes samples as they are captured by the digital camera. In contrast to encoded video data 204 (or encoded video bitstreams 204), the video picture stream 202 is depicted with a bold line to emphasize the high data volume of the video picture stream 202 that can be processed by the electronic device 220 that includes a video encoding device 203 coupled to the video source 201. The video encoding device 203 can include hardware, software, or a combination thereof to enable or implement aspects of the disclosed subject matter as described in greater detail below. In contrast to the video picture stream 202, the encoded video data 204 (or encoded video bitstream 204) is depicted with a thin line to emphasize the relatively lower data volume of the encoded video data 204 (or encoded video bitstream 204) that can be stored on a streaming server 205 for future use. One or more streaming client subsystems, such as client subsystem 206 and client subsystem 208 in FIG. 2, can access the streaming server 205 to retrieve copies 207 and 209 of the encoded video data 204. The client subsystem 206 can include a video decoding device 210, for example, in the electronic device 230. The video decoding device 210 decodes the incoming copy 207 of encoded video data and creates an outgoing video picture stream 211 that can be rendered on a display 212, such as a display screen, or other rendering device. In some streaming systems, the encoded video data 204, video data 207, and video data 209 (e.g., video bitstreams) can be encoded according to certain video encoding / compression standards.

[0052] It is noted that the electronic devices 220 and 230 can include other components not shown in the figures. For example, the electronic device 220 can include a video decoding device, and the electronic device 230 can also include a video encoding device.

[0053] In some embodiments of the present application, taking the High Efficiency Video Coding (HEVC) in the international video coding standard, Versatile Video Coding (VVC), and the Chinese national video coding standard AVS as examples, after an input video frame image is input, the video frame image is divided into a plurality of non-overlapping processing units according to a block size, and each processing unit will perform similar compression operations. This processing unit is called a Coding Tree Unit (CTU), or referred to as a Largest Coding Unit (LCU). The CTU can be further divided into one or more basic Coding Units (CUs) for more detailed division. The CU is the most basic element in the coding link.

[0054] In other embodiments, the processing unit can also be referred to as a coding tile (i.e., a tile), which is a rectangular area of a multimedia data frame that can be independently decoded and encoded. In the first generation video coding standard (Alliance for Open Media Video 1, AV1) standard formulated by the Alliance for Open Media, the coding tile can be further divided into one or more Superblocks (SBs) for more detailed division. The SB is the starting point of block division and can be further divided into a plurality of sub-blocks. Then the superblock is further divided into one or more blocks (B). Each block is the most basic element in the coding link. In some embodiments, one SB can contain a plurality of B.

[0055] The above division method of the video frame image can be referred to as a block partition structure. Some concepts in the coding process are introduced as follows:

[0056] Predictive coding (Predictive Coding): Predictive coding includes intra prediction and inter prediction, etc. After the original video signal is predicted by the selected reconstructed video signal, a residual video signal is obtained. The encoding end needs to determine which prediction coding mode is selected for the current coding unit (or coding block) and inform the decoding end. Intra prediction refers to that the predicted signal comes from the already coded and reconstructed area in the same image; inter prediction refers to that the predicted signal comes from the already coded other image (referred to as reference image) different from the current image.

[0057] Transform & Quantization: After the residual video signal is transformed by Discrete Fourier Transform (DFT), Discrete Cosine Transform (DCT) or other transform operations, the signal is converted into the transform domain, which is called transform coefficient. The transform coefficient is further subjected to a lossy quantization operation, which loses some information, so that the quantized signal is conducive to compressed expression. In some video coding standards, more than one transform method can be selected, so the encoding end also needs to select one of them for the current coding unit (or coding block) and inform the decoding end. The degree of quantization precision is usually determined by the quantization parameter (QP). If the QP value is large, the coefficients with larger value range will be quantized to the same output, so it usually brings larger distortion and lower code rate; on the contrary, if the QP value is small, the coefficients with smaller value range will be quantized to the same output, so it usually brings smaller distortion and higher code rate.

[0058] Entropy Coding or Statistical Coding: The quantized transform domain signal will be statistically compressed and coded according to the frequency of each value, and finally the binary (0 or 1) compressed code stream will be output. At the same time, other information such as the selected coding mode, motion vector data, etc. also needs to be entropy coded to reduce the code rate. Statistical coding is a lossless coding method that can effectively reduce the code rate required to express the same signal. Common statistical coding methods include variable length coding (VLC) or context-based binary arithmetic coding (CABAC).

[0059] The CABAC process mainly includes three steps: binarization, context modeling and binary arithmetic coding. After the input syntax elements are binarized, the binary data can be encoded by regular coding mode and bypass coding mode. The bypass coding mode does not need to assign a specific probability model to each binary bit, and the input binary bit bin value is directly encoded by a simple bypass encoder to speed up the entire encoding and decoding speed. Generally, different syntax elements are not completely independent, and the same syntax element also has certain memory. Therefore, according to the conditional entropy theory, the use of other coded syntax elements for conditional coding can further improve the coding performance compared with independent coding or non-memory coding. These coded symbol information used as conditions are called contexts. In the regular coding mode, the binary bits of the syntax elements enter the context modeler in order, and the encoder assigns an appropriate probability model to each input binary bit according to the value of the previously coded syntax element or binary bit. This process is called context modeling. The context model corresponding to the syntax element can be located through the context index increment (ctxIdxInc) and the context index start (ctxIdxStart). After the bin value and the assigned probability model are sent to the binary arithmetic encoder for coding, the context model needs to be updated according to the bin value, that is, the adaptive process in coding.

[0060] Loop filtering: The signal after transformation and quantization is obtained by inverse quantization, inverse transformation and prediction compensation to obtain the reconstructed image. Compared with the original image, the reconstructed image has some differences due to the influence of quantization, that is, the reconstructed image will produce distortion. Therefore, the reconstructed image can be filtered, for example, deblocking filter (DB), sample adaptive offset (SAO) or adaptive loop filter (ALF) filter, which can effectively reduce the distortion degree caused by quantization. Since these filtered reconstructed images will be used as references for subsequent encoded images to predict future image signals, the above filtering operation is also called loop filtering, that is, the filtering operation within the coding loop.

[0061] In some embodiments of the present application, FIG. 3 shows a basic flowchart of a video encoder, in which the intra prediction is taken as an example for illustration. In the flowchart, the original image signal s k[x, y] and the prediction image signal Difference operation is performed to obtain the residual signal u k [x, y], the residual signal u k After the transform and quantization processing, the quantized coefficients are obtained, which are encoded into bitstream by entropy coding on one hand, and are reconstructed into the reconstructed residual signal u' by inverse quantization and inverse transform processing on the other hand k [x, y], the prediction image signal and the reconstructed residual signal u' k [x, y] is superimposed to generate the image signal The image signal On one hand, it is input into the intra mode decision module and the intra prediction module for intra prediction processing, and on the other hand, it is output as the reconstructed image signal s' through loop filtering k [x, y], the reconstructed image signal s' k [x, y] can be used as the reference image for motion estimation and motion compensation prediction of the next frame. Then, based on the result s' of the motion compensation prediction r [x + m x , y + m y ] and the intra prediction result the prediction image signal of the next frame is obtained and the above process is continued to repeat until the encoding is completed.

[0062] Based on the above encoding process, at the decoding end, for each coding unit (or coding block), after the compressed code stream (i.e. bitstream) is obtained, entropy decoding is performed to obtain various mode information and quantized coefficients. Then, the quantized coefficients are processed by inverse quantization and inverse transform to obtain the residual signal. On the other hand, according to the known encoding mode information, the prediction signal corresponding to the coding unit (or coding block) can be obtained, and then the residual signal is added to the prediction signal to obtain the reconstructed signal. The reconstructed signal is further processed by loop filtering and other operations to generate the final output signal. In this series of encoding processes, the encoding framework is mainly decided based on rate-distortion optimization (RDO) to select the optimal encoding parameters.

[0063] Some prediction models used in the prediction process in video encoding are introduced as follows:

[0064] The cross component nonlinear intra prediction model (CCNPM) is a cross component prediction model based on the neighboring pixels of the current block to construct the luminance component and the chrominance component, and then uses the reconstructed luminance component pixels of the current block to predict the chrominance component. FIG. 4 shows the prediction process of the CCNPM, which mainly includes the following three steps:

[0065] ①Downsample the luminance component of the current block and its neighboring template region (i.e., the gray region shown in FIG. 4) to obtain the downsampled luminance component.

[0066] ②Construct a cross component nonlinear prediction model between the luminance component (Y) and the chrominance component (U / V) according to the neighboring template region. As shown in FIG. 5, the cross component nonlinear prediction model uses the target chrominance component sample to be predicted, the position C in the downsampled luminance component (Y) corresponding to the chrominance component sample, and the surrounding downsampled luminance component samples (W, N, S, E). Based on these samples obtained in the template region, the following nonlinear model structure can be constructed:

[0067] wherein C' represents the target chrominance component sample, p i (i.e., p0, p1, p2, p3, p4, p5, p6) represents the model parameters, and B represents the constant term. By solving multiple nonlinear model equations, the model parameters of the CCNPM can be obtained.

[0068] ③After obtaining the model parameters of the CCNPM, the downsampled luminance component samples corresponding to the current block can be input to the cross component prediction model, and then the chrominance component prediction value can be obtained.

[0069] wherein the size of the template region adjacent to the CCNPM is 6 rows of chrominance and 12 rows of luminance (in the YUV420 format as an example), i.e., the maximum line buffer required is 12 rows.

[0070] The CCNPM can be applied not only to intra prediction but also to inter prediction, so the CCNPM can also be referred to as a cross component nonlinear prediction model (i.e., Cross Component Nonlinear Prediction Model). Alternatively, the cross component nonlinear prediction model applied to inter prediction can be denoted as inter CCNPM, and its principle is consistent with the CCNPM applied to intra prediction.

[0071] A cross component linear prediction model (CCLM) is another prediction model. Similar to the CCNPM, the CCLM also uses the luminance component reference pixels and the chrominance component reference pixels in the neighboring template region to construct a cross component prediction model, and then calculates the chrominance component prediction value according to the luminance component reconstructed value of the current block. The difference is that the prediction model constructed by the CCLM is a linear model. The linear prediction model of the CCLM can be expressed as: pred C (i,j) = a rec'(i,j) + b L (i,j)

[0072] wherein pred C (i,j) represents the current chrominance component prediction point with the coordinates (i,j), rec'(i,j) represents the down-sampled current luminance component reconstructed pixel point, and a and b are model parameters. L

[0073] In addition, when performing prediction and filtering and the like, the video coding often needs to use the reconstructed pixels adjacent to the current block. In order to preserve the reconstructed pixels or data, the video coding sets a line buffer to buffer one or more rows of pixels of the image or video frame, so as to avoid the increase of bandwidth demand caused by repeated reading of the data. As shown in FIG. 6, the line buffer is arranged directly above each row of CTUs. The larger the line buffer is, the greater the hardware implementation cost is.

[0074] Therefore, although the prediction technologies such as the CCNPM bring considerable coding gain, the line buffer demand is also increased due to the need to buffer the data in the template region, thereby leading to the expansion of the hardware implementation cost. If the template region is set to be too small, the prediction model will be inaccurate, which will affect the coding and decoding effect of the video. In order to further improve the coding performance and reduce the hardware cost, embodiments of the present application propose to adaptively adjust the size of the template region used by the prediction model according to the video frame information, so as to further improve the coding performance and reduce the hardware cost.

[0075] The implementation details of the technical solutions of the embodiments of the present application are described in detail as follows:

[0076] FIG. 7 shows a flowchart of a cross component prediction method according to some embodiments of the present application, which can be performed by a video coding device or a video decoding device, which can be a terminal device or a server and the like. Referring to FIG. 7, the cross component prediction method at least includes S710 to S720, which are described in detail as follows:

[0077] ​In S710, a cross-component prediction model corresponding to the current block is constructed according to pixel values contained in a template region adjacent to the current block, a size of the template region being determined based on video frame information corresponding to the current block; wherein the template region includes a first region located on a left side of the current block and a second region located above the current block, the size of the template region including at least one of a width of the first region and a height of the second region; and the video frame information includes at least one of: a position of the current block in the video frame; a correlation of the current block with a neighboring region thereof; a size of the current block; and a resolution of the video frame.

[0078] In some embodiments, a video includes a sequence of video image frames, and each video image frame includes a series of images, each image can be further divided into slices, and each slice can be further divided into a series of LCUs (or CTUs), and each LCU includes a plurality of CUs. Video image frames are encoded in units of blocks. In some new video encoding standards, such as in the H.264 standard, there are macroblocks (MBs), and each macroblock can be further divided into a plurality of prediction blocks (PBs) for prediction encoding. In the HEVC standard, a plurality of block units are divided in terms of functions, and a new tree-based structure is used for description. For example, a CU can be divided into smaller CUs according to a quadtree, and the smaller CUs can be further divided, thereby forming a quadtree structure. The current block in the embodiments of the present application can be a CU, or a smaller block than a CU, such as a smaller block obtained by dividing a CU.

[0079] In some embodiments, the video frame information used when determining the size of the template region can include one or more of the following: a position of the current block in the video frame, a correlation of the current block with a neighboring region thereof, a size of the current block, and a resolution of the video frame. The following will be described in detail respectively.

[0080] In some embodiments, the video frame information can include a position of the current block in the video frame, and when determining the size of the template region, if the current block is located in a first row of coding tree units in the video frame, the size of the template region is determined according to a distance between a first row of pixels in the current block and an upper boundary of the video frame.

[0081] For example, when the current block is located in a first row of coding tree units in the video frame, if a distance between the first row of pixels in the current block and an upper boundary of the video frame is greater than or equal to a default size of the template region (e.g., a default height of the template region), the default size is used as the size of the template region (e.g., the default height is used as the height of the template region); if the distance between the first row of pixels in the current block and the upper boundary of the video frame is less than the default size, the size of the template region is determined according to the distance between the first row of pixels in the current block and the upper boundary of the video frame (e.g., the distance between the first row of pixels in the current block and the upper boundary of the video frame is used as the height of the template region). It should be noted that the size of the template region determined in this way can be the height of the template region located above the current block.

[0082] In some embodiments, the template region includes a first region located to the left of the current block and a second region located above the current block, and the size of the template region includes at least one of a width of the first region and a height of the second region. For example, if the current block is located in a first row of coding tree units in the video frame, the height of the second region is determined according to a distance between the first row of pixels in the current block and an upper boundary of the video frame; if the current block is not located in the first row of coding tree units in the video frame, the height of the second region is determined according to a position relationship between a template region of a default size and a line buffer corresponding to a target row of coding tree units; the target row of coding tree units is a row of coding tree units in which the current block is located.

[0083] In some embodiments, a width value of the template region (the first region) located to the left of the current block can be set to be the same as a height value of the template region (the second region) located above the current block, or a similar method can be used to determine the width of the template region (the first region) located to the left of the current block.

[0084] For example, when the current block is located in a first column of coding tree units in the video frame, if a distance between the first column of pixels in the current block and a left boundary of the video frame is greater than or equal to a default size of the template region (e.g., a default width of the first region), the default width is used as the width of the template region (the first region) located to the left of the current block; if the distance between the first column of pixels in the current block and the left boundary of the video frame is less than the default width, the distance between the first row of pixels in the current block and the left boundary of the video frame is used as the width of the template region (the first region) located to the left of the current block.

[0085] It should be noted that the default size of the template region in the embodiments of the present application can be a set template size or a template size specified in a video coding standard. The default template size can be used to represent at least one of a default width of the template region (the first region) on the left side of the current block and a default height of the template region (the second region) above the current block, and the default width of the template region (the first region) on the left side of the current block can be the same as or different from the default height of the template region (the second region) above the current block.

[0086] In some embodiments, if the current block is not in the first row of coding tree units in the video frame, the size of the template region (for example, the height of the second region) is determined according to the positional relationship between the template region of the default size and the line buffer corresponding to the target row coding tree unit, where the target row coding tree unit is the row coding tree unit in which the current block is located.

[0087] For example, when the current block is not in the first row of coding tree units in the video frame, if the template region of the default size does not exceed the area range of the line buffer, the default size is used as the size of the template region; if the template region of the default size exceeds the area range of the line buffer, the sum of the distance between the first row of pixels in the current block and the upper boundary of the target row coding tree unit and the size of the line buffer is used as the size of the template region. It should be noted that the size of the template region determined in this way can be the height of the template region above the current block, that is, if the template region of the default height does not exceed the area range of the line buffer, the default height is used as the height of the template region; if the template region of the default height exceeds the area range of the line buffer, the sum of the distance between the first row of pixels in the current block and the upper boundary of the target row coding tree unit and the size of the line buffer is used as the height of the template region. In some embodiments, the width of the template region on the left side of the current block can be set to be the same as the height of the template region above the current block, or the width of the template region on the left side of the current block can be determined separately.

[0088] In some embodiments, the video frame information can include the correlation of the current block and its adjacent region, which can be, for example, the correlation between pixel values, and when determining the size of the template region, the size of the template region can be determined according to the correlation between the luminance component reconstructed pixels of the current block and the luminance component pixel values in the adjacent region.

[0089] Specifically, the template region includes a first region on the left side of the current block and a second region above the current block, and the width of the first region and the height of the second region can be determined respectively, and specific examples are as follows:

[0090] In some embodiments, the process of determining the width of the first region left to the current block can be: fixing the right boundary of the first region, expanding the width of the first region from 1 to the left neighboring region, when the width of the first region is i, calculating the average horizontal gradient of the luminance component pixels contained in the first region in the horizontal direction, to obtain the average horizontal gradient when the width is i; if the absolute difference between the average horizontal gradient when the width is i and the average horizontal gradient when the width is i-1 is greater than or equal to a set threshold, the width of the first region is set to i-1; if the absolute difference between the average horizontal gradient when the width is i and the average horizontal gradient when the width is i-1 is less than the set threshold, the first region continues to expand to the left until the width of the first region reaches the default width of the first region; wherein i is a natural number greater than 1.

[0091] Specifically, as shown in FIG. 8, assuming that the current block is a 2x2 size block (white area in FIG. 8) and contains luminance component pixels P1, P2, P3 and P4, when the width of the first region is 1, the average horizontal gradient of the luminance component pixels P11 and P12 in the first region with a width of 1 directly adjacent to the current block is calculated, and the average horizontal gradient is obtained by calculating the pixel difference on both sides of P11 and P12, so the average horizontal gradient If the width of the first region is expanded to 2, the average horizontal gradient of the luminance component pixels P11, P12, P13 and P14 in the first region with a width of 2 directly adjacent to the current block is calculated, and the average horizontal gradient is If (a is a set threshold), it indicates that the luminance component pixels in the first region with a width of 2 and the luminance component pixels in the first region with a width of 1 have high correlation in the horizontal direction, so the first region can continue to expand to the left until the width of the first region reaches the default template width. Of course, if If

[0092] In some embodiments, the process of determining the height of the second region above the current block can be: fixing the lower boundary of the second region, extending the height of the second region upwards to the adjacent region, when the height of the second region is j, calculating the average gradient in the vertical direction of the luminance component pixels contained in the second region, to obtain the average vertical gradient when the height is j; if the absolute difference between the average vertical gradient when the height is j and the average vertical gradient when the height is j-1 is greater than or equal to a set threshold, the height of the second region is set to j-1; if the absolute difference between the average vertical gradient when the height is j and the average vertical gradient when the height is j-1 is less than the set threshold, the second region is continuously extended upwards until the height of the second region reaches the default height of the second region; wherein j is a natural number greater than 1.

[0093] Specifically, continuing to refer to FIG. 8, assuming that the current block is a 2x2 size block (the white area in FIG. 8) and contains luminance component pixels P1, P2, P3 and P4, if the height of the second region is 1, then the average vertical gradient If the height of the second region is extended to 2, then the average vertical gradient If (a is a set threshold), it indicates that the adjacent region with a height of 2 and the adjacent region with a height of 1 have a high pixel correlation in the vertical direction, so the second region can be continuously extended upwards until the height of the second region reaches the default height. Of course, if If

[0094] In some embodiments, when determining the size of the template region according to the correlation between the reconstructed pixel of the luminance component of the current block and the luminance component pixel value in the adjacent region, the template region can also be extended based on the current block, that is, the first region is extended to the left adjacent region, and the second region is extended to the upper adjacent region, when the size of the template region is l (i.e., the width of the first region and the height of the second region are l), the pixel value error between the luminance component pixels in the template region and the reconstructed pixel of the luminance component of the current block is calculated; if the corresponding pixel value error when the size of the template region is l is greater than or equal to a set threshold, the size of the template region (i.e., the width of the first region and the height of the second region) is set to l-1; if the corresponding pixel value error when the size of the template region is l is less than the set threshold, the template region is continuously extended until the size of the template region reaches the default size; wherein l is a natural number greater than 1.

[0095] Specifically, continuing to refer to FIG. 8, assuming that the current block is a 2x2 size block (the white area in FIG. 8) and contains luminance component pixels P1, P2, P3 and P4, if the size of the template region (i.e. the height of the template region located above the current block and the width of the template region located left to the current block) is expanded to 2, then the luminance component pixels contained in the template region are P5, P6, P7, P8, P11, P12, P13 and P14, and the pixel value error S between the luminance component pixels in the template region and the luminance component reconstructed pixels of the current block can be calculated. If S < a (a is a set threshold), it indicates that the correlation between the template region with size 2 and the luminance component reconstructed pixels of the current block is high, and then the size of the template region can be continuously expanded until the size of the template region reaches the default size. Of course, if S ≥ a, it indicates that the correlation between the template region with size 2 and the luminance component reconstructed pixels of the current block is low, and then the size of the template region can not be continuously expanded, and the size of the template region is set to 1, in other words, the size of the template region of the current block can be 1.

[0096] It should be noted that since the size of the template region is not the same as the size and shape of the current block, when calculating the pixel value error between the luminance component pixels in the template region and the luminance component reconstructed pixels of the current block, the luminance component pixels in the template region and the luminance component reconstructed pixels of the current block can be mapped to the same size space region, and then the pixel value error between the mapping results of the luminance component pixels and the mapping results of the luminance component reconstructed pixels is calculated.

[0097] In some embodiments, the pixel value error can be a sum of absolute difference (SAD), a sum of squared difference (SSD), a mean squared error (MSE), etc.

[0098] In some embodiments, the video frame information can include the size of the current block, and then the size of the template region can be determined according to the size of the current block, wherein the size of the template region is in a positive correlation with the size of the current block, and the size of the current block can be the larger one of the width and the height of the current block. For example, if the size of the current block is less than or equal to 8, the size of the template region is 3; if the size of the current block is between 8 and 16, the size of the template region is 4, etc.

[0099] In some embodiments, the video frame information can include a resolution of the video frame; then the size of the template region can be determined according to the resolution of the video frame, wherein the size of the template region is in a positive correlation with the resolution of the video frame. For example, if the resolution of the video frame is 4K, the size of the template region is 6; if the resolution of the video frame is 2K, the size of the template region is 4, and so on.

[0100] In some embodiments, the process of constructing the cross-component prediction model corresponding to the current block according to the pixel values contained in the template region adjacent to the current block can refer to the process shown in FIG. 4, and similar methods can be used for both linear prediction models and nonlinear prediction models.

[0101] Continuing to refer to FIG. 7, in S720, the chroma component prediction value of the current block is generated according to the cross-component prediction model and the luminance component reconstructed pixel of the current block.

[0102] In some embodiments, the process of generating the chroma component prediction value of the current block according to the cross-component prediction model and the luminance component reconstructed pixel of the current block can refer to the process shown in FIG. 4, and specifically, the luminance component reconstructed pixel of the current block can be input into the cross-component prediction model, and then the chroma component prediction value of the current block can be obtained.

[0103] In some embodiments, the cross-component prediction model in the embodiments of the present application can be: a cross-component nonlinear prediction model for intra prediction, a cross-component linear prediction model for intra prediction, a cross-component nonlinear prediction model for inter prediction, and a cross-component linear prediction model for inter prediction. In other words, the cross-component prediction model in the embodiments of the present application can not only be a nonlinear prediction model, but also a linear prediction model; and can not only be applied to intra prediction, but also to inter prediction.

[0104] Meanwhile, it should be noted that the scheme for determining the template region in the embodiments of the present application can not only be applied to cross-component prediction, but also to non-cross-component prediction. As long as the prediction model uses the template region, the size of the template region can be determined by the technical scheme of the embodiments of the present application, regardless of whether it is intra prediction or inter prediction, whether it is nonlinear prediction or linear prediction, or whether it is cross-component prediction.

[0105] On the basis of the cross-component prediction method shown in FIG. 7, the present application also proposes a video decoding method, which can be executed by a device with computing processing function, such as a terminal device or a server. The specific process is shown in FIG. 9, which includes S910 to S940, and the details are as follows:

[0106] In S910, the video code stream is decoded to obtain pixel values contained in a template region adjacent to the current block.

[0107] In S920, a cross-component prediction model corresponding to the current block is constructed according to the pixel values contained in the template region, and a size of the template region is determined based on video frame information corresponding to the current block.

[0108] In S930, a chroma component prediction value of the current block is generated according to the cross-component prediction model and a luminance component reconstructed pixel of the current block.

[0109] In S940, a decoding process is performed based on the chroma component prediction value of the current block.

[0110] Specific implementation details of S920 and S930 can refer to the description of S710 and S720 in the foregoing embodiments, that is, the video decoding device can determine the size of the template region based on the video frame information corresponding to the current block, so as to avoid that the template region is too large to increase the hardware implementation cost, or the template region is too small to affect the prediction result, to realize adaptive adjustment of the size of the template region, and thus to improve the coding performance of the video and reduce the hardware cost.

[0111] On the basis of the cross-component prediction method shown in FIG. 7, the embodiment of the present application further proposes a video coding method, which can be executed by a device with a computing processing function, such as a terminal device or a server. The specific process is shown in FIG. 10, including S1010 to S1040, and the details are as follows:

[0112] In S1010, pixel values contained in a template region adjacent to the current block are obtained.

[0113] In S1020, a cross-component prediction model corresponding to the current block is constructed according to the pixel values contained in the template region, and a size of the template region is determined based on video frame information corresponding to the current block.

[0114] In S1030, a chroma component prediction value of the current block is generated according to the cross-component prediction model and a luminance component reconstructed pixel of the current block.

[0115] In S1040, an encoding process is performed based on the chroma component prediction value of the current block to obtain an encoded video code stream.

[0116] In some embodiments, the flag included in the sequence header can be used to indicate that the coding blocks in the entire sequence all use a template region of a set size; the flag included in the picture header can be used to indicate that the coding blocks in the entire picture all use a template region of a set size; the flag included in the slice header can be used to indicate that the coding blocks in the entire slice all use a template region of a set size; the flag included in the CTU header can be used to indicate that the coding blocks in the entire CTU all use a template region of a set size; and the flag included in the coding block can be used to indicate that the current coding block uses a template region of a set size.

[0117] In some embodiments, the flag included in the sequence header can be used to indicate that the coding blocks in the entire sequence all use a template region of a set size; the flag included in the picture header can be used to indicate that the coding blocks in the entire picture all use a template region of a set size; the flag included in the slice header can be used to indicate that the coding blocks in the entire slice all use a template region of a set size; the flag included in the CTU header can be used to indicate that the coding blocks in the entire CTU all use a template region of a set size; and the flag included in the coding block can be used to indicate that the current coding block uses a template region of a set size.

[0118] In summary, the technical scheme of the embodiments of the present application mainly adaptively selects the size of the template region used by the cross-component prediction technology according to the information of the video frame, so as to further improve the coding performance and / or reduce the hardware cost. The technical scheme of the embodiments of the present application can be applied to the products of video encoders and decoders. In some embodiments, different template regions of different sizes can be used for different blocks in the same CTU or different CTUs.

[0119] In some embodiments, the size of the template region can be determined according to the line buffer size and the position of the template region. Specifically, assuming that the vertical coordinate of the first row of pixels of the current block in the entire image is y, the default size of the CTU is max_cu_height (which can be the size of the luminance block CTU in the YUV420 format, while the size of the chrominance block CTU is half of it), the line buffer size of the chrominance component is K, the default template size (height or width) of the cross-component prediction model is set to N, and the finally determined template region size (height or width) is M. K, N, and M are all non-negative integers.

[0120] If the current block is in the first row CTU of the video frame, i.e., y < (max cu height » 1), it is determined whether the distance between y and the top boundary of the whole video frame satisfies the default template height, and then the template region size of the prediction model is adaptively determined. Specifically, if the distance between the first row of pixels of the current block (i.e., the current chroma block to be predicted) and the top boundary of the video frame is greater than or equal to the default template height, the final template region height is set as the default template height. Otherwise, if the distance between the first row of pixels of the current block and the top boundary of the video frame is less than the default template height, the final template region height is set as the distance between the first row of pixels of the current block and the top boundary of the video frame. The formula is as follows: if y ≥ N, then M = N; otherwise, if y < N, then M = y.

[0121] If the current block is not in the first row CTU of the video frame, i.e., y ≥ (max cu height » 1), it is determined whether the template region position of the default height exceeds the limit of the line buffer, and then the template region height of the prediction model is adaptively determined. Specifically, if the default template region of the current block does not exceed the limit of the line buffer region of the CTU where the current block is located, the final template region height is set as the default template height. Otherwise, the final template region height is set as the distance between the first row of pixels of the current block and the top boundary of the CTU where the current block is located plus the size of the chroma component line buffer. The formula is as follows: if y % (max cu height » 1) ≥ (N-K), then M = N; otherwise, if y % (max cu height » 1) < (N-K), then M = (y % (max cu height » 1)) + K.

[0122] In some embodiments, the final template region size can be determined according to the content relevance of the current block and its neighboring region. This is because theoretically the higher the relevance of the template region and the pixels of the current block, the more suitable the cross-component prediction model and prediction parameters based on the template region are for the prediction of the current block. Therefore, the size of the template region can be adaptively selected according to the relevance of the neighboring region of the current block and the reconstructed pixels of the luminance component of the current block.

[0123] Specifically, the size of the template region can be adaptively selected using gradient changes. For example, the average horizontal gradient G x i and the average vertical gradient G y i(Wherein, x represents horizontal direction, y represents vertical direction, i represents corresponding template size). If the average gradient value changes little as the template region size increases from 1 to N, it indicates that the pixel correlation in the template region with size N is high, and the final template size M=N can be set. Otherwise, the size of the region with smaller average gradient value change is selected as the final template region size.

[0124] Again, taking the example shown in FIG. 8, when the template region width and height are 1, the average horizontal gradient Average vertical gradient When the template width and height are 2, the average horizontal gradient Average vertical gradient In turn, the average gradient value corresponding to different template sizes can be calculated. If the absolute value of the difference between the average horizontal gradient value of the template size 2 and the average horizontal gradient value of the template size 1 is less than a certain threshold, i.e. It is considered that the template region with width 2 and the template region with width 1 have high pixel correlation in the horizontal direction, and the final template width in the horizontal direction can be set to 2. In turn, the gradient values of different template sizes in different directions can be calculated, and the final template size can be obtained according to the change.

[0125] In some embodiments, the size of the template region can be adaptively selected based on the difference between the pixels of the template region luminance component and the reconstructed pixels of the current block. Specifically, the correlation between the pixels of the template region luminance component and the luminance pixels of the current block can be measured based on criteria such as SAD, SSD, and MSE, and the higher the correlation, the more suitable the cross-component prediction model and prediction parameters based on the template region are for the prediction of the current block. Generally, the template region is an inverted L shape with width / height N (as shown in FIG. 4), while the block to be predicted is a rectangle, and the two cannot be directly calculated by point-to-point SAD, SSD, or MSE. Therefore, the luminance pixels of the template region and the block to be predicted can be mapped to the same width and height space, and then the point-to-point SAD, SSD, or MSE calculation can be performed.

[0126] In some embodiments, the size of the template region can be determined according to the size of the block. This is because different sizes of blocks contain different amounts of information, and the size of the template region required is also different. A template region that is too large can introduce noise to the cross-component prediction model of the current block, and a template region that is too small can result in insufficient construction of the cross-component prediction model. Therefore, different sizes of template regions can be set according to different sizes of blocks, so as to achieve globally optimal adaptive template region size. An example is shown in Table 1 below:

[0127] Table 1

[0128] In Table 1, block width represents the width of the current block; block height represents the height of the current block; and max(block width, block height) represents the maximum of the width and the height of the current block.

[0129] In some embodiments, the size of the template region can be determined according to the resolution of the image. This is because different resolutions of images contain different amounts of information, and the size of the template region required is also different. A template region that is too large can introduce noise into the cross-component prediction model, and a template region that is too small can result in an insufficient construction of the cross-component prediction model. Therefore, different sizes of template regions can be set according to the resolution of the image, so as to achieve a globally optimal adaptive template region size. An example is shown in Table 2 below:

[0130] Table 2

[0131] In some embodiments, the video encoding end can select the optimal template region size for the current block, CTU, slice or current frame by rate-distortion optimization, and indicate the used template region size by transmitting a flag. The video decoding end can obtain the template region size used by the current block by parsing the block-level, CTU-level, slice-level or frame-level flag. An example is shown in Table 3 below:

[0132] Table 3

[0133] The various methods proposed in the above embodiments of the present application can be used alone or in combination, and can be applied to video coding technologies based on template regions, such as intra, inter, filtering, etc., and products of video encoders and decoders.

[0134] The device embodiments of the present application are described below, which can be used to execute the methods in the above embodiments of the present application. For details not disclosed in the device embodiments of the present application, please refer to the above embodiments of the methods of the present application.

[0135] FIG. 11 shows a block diagram of a cross-component prediction device according to some embodiments of the present application, which can be applied to a video encoding device or a video decoding device, which can be a terminal device or a server, etc.

[0136] Referring to FIG. 11, the cross-component prediction device 1100 according to some embodiments of the present application includes a processing unit 1102 and a generating unit 1104.

[0137] The processing unit 1102 is configured to construct a cross-component prediction model corresponding to the current block according to pixel values contained in a template region adjacent to the current block, the size of the template region being adaptively adjusted based on video frame information corresponding to the current block; and the generating unit 1104 is configured to generate a chroma component prediction value of the current block according to the cross-component prediction model and a luminance component reconstructed pixel of the current block.

[0138] FIG. 12 shows a block diagram of a video decoding apparatus according to some embodiments of the present application, which can be applied to a device with computing processing function, such as a terminal device or a server.

[0139] Referring to FIG. 12, a video decoding apparatus 1200 according to some embodiments of the present application includes a decoding unit 1202, a processing unit 1204, and a generating unit 1206.

[0140] The decoding unit 1202 is configured to decode a video bitstream to obtain pixel values contained in a template region adjacent to a current block; the processing unit 1204 is configured to construct a cross-component prediction model corresponding to the current block according to the pixel values contained in the template region, the size of the template region being adaptively adjusted based on video frame information corresponding to the current block; and the generating unit 1206 is configured to generate a chroma component prediction value of the current block according to the cross-component prediction model and a luminance component reconstructed pixel of the current block, to perform decoding processing based on the chroma component prediction value of the current block.

[0141] FIG. 13 shows a block diagram of a video encoding apparatus according to some embodiments of the present application, which can be applied to a device with computing processing function, such as a terminal device or a server.

[0142] Referring to FIG. 13, a video encoding apparatus 1300 according to some embodiments of the present application includes an obtaining unit 1302, a processing unit 1304, a generating unit 1306, and an encoding unit 1308.

[0143] The obtaining unit 1302 is configured to obtain pixel values contained in a template region adjacent to a current block; the processing unit 1304 is configured to construct a cross-component prediction model corresponding to the current block according to the pixel values contained in the template region, the size of the template region being adaptively adjusted based on video frame information corresponding to the current block; the generating unit 1306 is configured to generate a chroma component prediction value of the current block according to the cross-component prediction model and a luminance component reconstructed pixel of the current block; and the encoding unit 1308 is configured to perform encoding processing based on the chroma component prediction value of the current block to obtain an encoded video bitstream.

[0144] In some embodiments of the present application, based on the foregoing scheme, the video bitstream further comprises a flag for indicating the size of the template region; wherein the flag comprises one or more of the following flags: a flag included in a sequence header, a flag included in a picture header, a flag included in a slice header, a flag included in a CTU header, and a flag included in a coding block.

[0145] FIG. 14 shows a structural schematic diagram of a computer system of an electronic device suitable for implementing the embodiments of the present application, which can be the cross-component prediction apparatus, the video encoding device or the video decoding device in the foregoing embodiments.

[0146] It should be noted that the computer system 1400 of the electronic device shown in FIG. 14 is only an example, and should not impose any limitation on the functions and use range of the embodiments of the present application.

[0147] As shown in FIG. 14, the computer system 1400 can include a central processing unit (CPU) 1401, which can perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) 1402 or loaded into a random access memory (RAM) 1403 from a storage portion 1408, such as performing the methods described in the foregoing embodiments. Various programs and data required for system operation are also stored in the RAM 1403. The CPU 1401, the ROM 1402, and the RAM 1403 are connected to each other through a bus 1404. An input / output (I / O) interface 1405 is also connected to the bus 1404.

[0148] The following components can be connected to the I / O interface 1405: an input portion 1406 including a keyboard, a mouse, etc.; an output portion 1407 including, for example, a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 1408 including a hard disk, etc.; and a communication portion 1409 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication portion 1409 performs communication processing via a network such as the Internet. A drive 1410 is also connected to the I / O interface 1405 as necessary. A removable recording medium 1411 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, and the like is attached to the drive 1410 as necessary, so that a computer program read therefrom is installed in the storage portion 1408 as necessary.

[0149] In particular, according to embodiments of the present application, the processes described above with reference to the flowcharts can be implemented as a computer software program. For example, embodiments of the present application include a computer program product comprising a computer program for performing the methods illustrated by the flowcharts carried on a computer readable medium. In such embodiments, the computer program can be downloaded and installed from a network via the communication section 1409, and / or installed from the removable medium 1411. When the computer program is executed by the central processing unit (CPU) 1401, various functions defined in the system of the present application are performed.

[0150] It should be noted that the computer readable medium shown in the embodiments of the present application can be a computer readable signal medium or a computer readable storage medium or any combination of the two. The computer readable storage medium may, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or apparatus, or any combination of the above. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (Compact Disc Read-Only Memory, CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, the computer readable storage medium can be any tangible medium containing or storing a computer program that can be used by or in conjunction with an instruction execution system, device or apparatus. In the present application, the computer readable signal medium can include a data signal carried in a baseband or as a carrier wave part of a data signal, which carries a computer readable computer program. Such a propagated data signal can take many forms, including but not limited to an electromagnetic signal, an optical signal or any suitable combination of the above. The computer readable signal medium can also be any computer readable medium other than the computer readable storage medium, which can send, propagate or transmit a program for use by or in conjunction with an instruction execution system, device or apparatus. The computer program contained on the computer readable medium can be transmitted by any suitable medium, including but not limited to wireless, wired, or the like, or any suitable combination of the above.

[0151] The flow diagrams and the block diagrams in the drawings are illustrations of architectures, functionalities, and operations of possible implementations of systems, methods, and computer program products according to various embodiments disclosed in this application. It should be noted that each block in the flow diagrams or block diagrams can represent a module, a program segment, or a portion of code, which comprises one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or in the reverse order, depending on the functionality involved. It is also noted that each block of the block diagrams or flow diagrams and combinations of blocks in the block diagrams or flow diagrams can be implemented by dedicated hardware-based systems which perform the specified functions or operations, or by combinations of dedicated hardware and computer programs.

[0152] The units involved in the embodiments described in this application can be implemented in the form of software, or can be implemented in the form of hardware, and the described units can also be arranged in a processor. In some cases, the names of these units do not constitute a limitation on the units themselves.

[0153] As another aspect, the application also provides a computer readable medium, which can be included in the electronic device described in the above embodiments, or can exist separately without being assembled into the electronic device. The above computer readable medium carries one or more computer programs, which, when executed by the electronic device, enable the electronic device to implement the method described in the above embodiments.

[0154] It should be noted that although several modules or units of the device for action execution are mentioned in the above detailed description, such division is not mandatory. In fact, according to the embodiments of the application, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided into a plurality of modules or units.

[0155] From the above description of the embodiments, those skilled in the art can easily understand that the example embodiments described herein can be implemented by software, or by software in combination with necessary hardware. Therefore, the technical solutions according to the embodiments of the application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a U disk, a mobile hard disk, etc.) or a network, and includes a number of instructions to enable an electronic device to perform the method according to the embodiments of the application.

[0156] For example, the electronic device can be a cross-component prediction apparatus, and the cross-component prediction apparatus can perform the cross-component prediction method shown in FIG. 7. For another example, the electronic device can be a video decoding device, and the video decoding device can perform the video decoding method shown in FIG. 9. For another example, the electronic device can be a video encoding device, and the video encoding device can perform the video encoding method shown in FIG. 10.

[0157] The embodiments of the present application further provide a non-volatile computer readable storage medium, which stores a code stream formed by a computer program, and the non-volatile computer readable storage medium stores one or more sets of instructions configured to be executed by a computing device having a control circuit and a memory, and when the one or more sets of instructions are executed, the video decoding method shown in FIG. 9 is implemented.

[0158] The embodiments of the present application further provide a method for storing or transmitting a video code stream, wherein the video code stream is generated according to the video encoding method shown in FIG. 10, or the video code stream is decoded based on the video decoding method shown in FIG. 9.

[0159] The embodiments of the present application further provide a computer storage medium, which stores instructions executable by at least one processor for performing the video encoding method shown in FIG. 10 to generate a code stream and store.

[0160] Other embodiments of the present application will be apparent to those skilled in the art from consideration of the specification and practice of the embodiments disclosed herein. The present application is intended to cover any variations, uses, or adaptations of the application following, in general, the principles of the application and including such departures from the present disclosure as come within known or customary practice in the art to which the application pertains.

[0161] It should be understood that the application is not limited to the precise construction that has been described above and shown in the accompanying drawings, and that various modifications and changes can be made by those skilled in the art without departing from the scope of the application. The scope of the application is limited only by the appended claims.

Claims

1. A cross-component prediction method, performed by an electronic device, comprising: constructing a cross-component prediction model corresponding to a current block according to pixel values contained in a template region neighboring the current block, a size of the template region being determined based on video frame information corresponding to the current block, the template region including a first region located to the left of the current block and a second region located above the current block, the size of the template region including at least one of a width of the first region and a height of the second region, the video frame information including at least one of a position of the current block in the video frame, a correlation of the current block with neighboring regions thereof, a size of the current block, and a resolution of the video frame; generating a prediction value of a chroma component of the current block according to the cross-component prediction model and a luma component of the current block.

2. The cross-component prediction method of claim 1, wherein, when the video frame information includes the position of the current block in the video frame, the cross-component prediction method further comprises: if the current block is located in a first row coding tree unit in the video frame, determining the height of the second region according to a distance between a first row pixel in the current block and an upper boundary of the video frame; if the current block is not located in the first row coding tree unit in the video frame, determining the height of the second region according to a position relationship between a template region of a default size and a line buffer corresponding to a target row coding tree unit, the target row coding tree unit being a row coding tree unit in which the current block is located.

3. The cross-component prediction method of claim 2, wherein, determining the height of the second region according to the distance between the first row pixel in the current block and the upper boundary of the video frame, comprises: if the distance between the first row pixel in the current block and the upper boundary of the video frame is greater than or equal to a default height of the second region, taking the default height as the height of the second region; if the distance between the first row pixel in the current block and the upper boundary of the video frame is less than the default height of the second region, taking the distance between the first row pixel in the current block and the upper boundary of the video frame as the height of the second region.

4. The cross-component prediction method of claim 2 or 3, wherein, determining the height of the second region according to the position relationship between the template region of the default size and the line buffer corresponding to the target row coding tree unit, comprises: if the template region of the default size does not exceed a region range of the line buffer, taking a default height of the second region as the height of the second region; if the template region of the default size exceeds the region range of the line buffer, taking a sum of a distance between a first row pixel in the current block and an upper boundary of the target row coding tree unit and a size of the line buffer as the width of the first region.

5. The cross-component prediction method according to any of claims 2 to 4, further comprising: setting the width of the first region to be the same as the height of the second region. 6.The cross-component prediction method of any one of claims 2 to 4, further comprising: when the current block is located in a first column coding tree unit in the video frame, determining the width of the first region according to a distance between a first column pixel in the current block and a left boundary of the video frame.

7. The cross-component prediction method according to any of the claims 1 to 6, wherein, When the video frame information comprises the correlation of the current block and its neighboring region, the cross-component prediction method further comprises: determining the size of the template region according to the correlation between the luminance component reconstructed pixels of the current block and the luminance component pixel values in the neighboring region.

8. The cross-component prediction method according to claim 7, wherein, determining the size of the template region according to the correlation between the luminance component reconstructed pixels of the current block and the luminance component pixel values in the neighboring region comprises: fixing the right boundary of the first region, expanding the width of the first region to the left neighboring region from 1, and calculating the average horizontal gradient of each row of luminance component pixels contained in the first region when the width of the first region is i, to obtain the average horizontal gradient when the width is i; if the absolute difference between the average horizontal gradient when the width is i and the average horizontal gradient when the width is i-1 is greater than or equal to a set threshold, setting the width of the first region as i-1; if the absolute difference between the average horizontal gradient when the width is i and the average horizontal gradient when the width is i-1 is less than the set threshold, continuing to expand the first region to the left until the width of the first region reaches the default width of the first region; wherein i is a natural number greater than 1.

9. The cross-component prediction method according to claim 7 or 8, wherein, determining the size of the template region according to the correlation between the luminance component reconstructed pixels of the current block and the luminance component pixel values in the neighboring region further comprises: fixing the lower boundary of the second region, expanding the height of the second region to the upper neighboring region from 1, and calculating the average vertical gradient of each column of luminance component pixels contained in the second region when the height of the second region is j, to obtain the average vertical gradient when the height is j; if the absolute difference between the average vertical gradient when the height is j and the average vertical gradient when the height is j-1 is greater than or equal to a set threshold, setting the height of the second region as j-1; if the absolute difference between the average vertical gradient when the height is j and the average vertical gradient when the height is j-1 is less than the set threshold, continuing to expand the second region upwards until the height of the second region reaches the default height of the second region; wherein j is a natural number greater than 1.

10. The cross-component prediction method of claim 7, wherein, determining the size of the template region according to the correlation between the luminance component reconstructed pixels of the current block and the luminance component pixel values in the neighboring region comprises: fixing the right boundary of the first region, expanding the first region to the left neighboring region, fixing the lower boundary of the second region, and expanding the second region to the upper neighboring region, and calculating the pixel value error between the luminance component pixels in the template region and the luminance component reconstructed pixels of the current block when the width of the first region and the height of the second region are both l. if a pixel value error corresponding to a case where the width of the first region and the height of the second region are l is greater than or equal to a set threshold, setting the width of the first region and the height of the second region as l-1; if the pixel value error corresponding to the case where the width of the first region and the height of the second region are l is less than the set threshold, continuing to expand the first region and the second region until the width of the first region and the height of the second region reach respective default widths and default heights; wherein l is a natural number greater than 1.

11. The cross-component prediction method of claim 10, wherein, calculating a pixel value error between the luminance component pixels in the template region and the luminance component reconstructed pixels of the current block, including: mapping the luminance component pixels in the template region and the luminance component reconstructed pixels of the current block into a same size spatial region; calculating a pixel value error between the mapping results of the luminance component pixels and the mapping results of the luminance component reconstructed pixels.

12. The cross-component prediction method according to any of claims 1 to 11, wherein, When the video frame information includes the size of the current block, the cross-component prediction method further includes: determining the size of the template region according to the size of the current block, wherein the size of the template region is positively correlated with the size of the current block, and the size of the current block is the larger one of the width and the height of the current block.

13. The cross-component prediction method according to any of the claims 1 to 12, wherein, When the video frame information includes the resolution of the video frame, the cross-component prediction method further includes: determining the size of the template region according to the resolution of the video frame, wherein the size of the template region is positively correlated with the resolution of the video frame.

14. The cross-component prediction method according to any of the claims 1 to 13, wherein, The cross-component prediction model includes any one of: a cross-component nonlinear prediction model for intra prediction, a cross-component linear prediction model for intra prediction, a cross-component nonlinear prediction model for inter prediction, and a cross-component linear prediction model for inter prediction.

15. A video decoding method performed by an electronic device, comprising: decoding a video bitstream to obtain pixel values contained in a template region adjacent to a current block; constructing a cross-component prediction model corresponding to the current block according to the pixel values contained in the template region, wherein a size of the template region is determined based on video frame information corresponding to the current block; wherein the template region includes a first region located on a left side of the current block and a second region located above the current block, and the size of the template region includes at least one of a width of the first region and a height of the second region; and the video frame information includes at least one of: a position of the current block in the video frame; a correlation between the current block and a neighboring region thereof; a size of the current block; and a resolution of the video frame; generating a chroma component prediction value of the current block according to the cross-component prediction model and a luminance component reconstructed pixel of the current block; performing decoding processing based on the chroma component prediction value of the current block.

16. A video encoding method performed by an electronic device, comprising: obtaining pixel values contained in a template region adjacent to a current block; construct a cross-component prediction model corresponding to the current block according to pixel values contained in a template region, a size of the template region being determined based on video frame information corresponding to the current block; wherein the template region comprises a first region located left of the current block and a second region located above the current block, the size of the template region comprising at least one of a width of the first region and a height of the second region; the video frame information comprising at least one of: a position of the current block in the video frame; a correlation of the current block with neighboring regions thereof; a size of the current block; and a resolution of the video frame; generate a color component prediction value of the current block according to the cross-component prediction model and a reconstructed pixel of a luma component of the current block; perform encoding processing based on the color component prediction value of the current block to obtain an encoded video bitstream.

17. The video coding method of claim 15, wherein, The video bitstream further comprises a flag bit used to indicate the size of the template region. The flag bit comprises one or more of the following flag bits: a flag bit contained in a sequence header, a flag bit contained in a picture header, a flag bit contained in a slice header, a flag bit contained in a coding tree unit (CTU) header, and a flag bit contained in a coding block.

18. A cross-component prediction apparatus, comprising: a processing unit configured to construct a cross-component prediction model corresponding to a current block according to pixel values contained in a template region adjacent to the current block, a size of the template region being determined based on video frame information corresponding to the current block; wherein the template region comprises a first region located left of the current block and a second region located above the current block, the size of the template region comprising at least one of a width of the first region and a height of the second region; the video frame information comprising at least one of: a position of the current block in the video frame; a correlation of the current block with neighboring regions thereof; a size of the current block; and a resolution of the video frame; a generating unit configured to generate a color component prediction value of the current block according to the cross-component prediction model and a reconstructed pixel of a luma component of the current block.

19. A computer readable medium having stored thereon a computer program, characterized in that, The computer program, when executed by a processor, implements the method of any one of claims 1 to 17.

20. An electronic device, comprising: comprising: one or more processors; a memory for storing one or more computer programs that, when executed by the one or more processors, cause the electronic device to implement the method of any one of claims 1 to 17.

21. A computer program product, characterised in that, The computer program product comprises a computer program stored in a computer readable storage medium, and a processor of an electronic device reads and executes the computer program from the computer readable storage medium, so that the electronic device executes the method of any one of claims 1 to 17.

22. A non-transitory computer readable storage medium, comprising: A non-transitory computer readable storage medium having stored thereon a bitstream formed by a computer program, the non-transitory computer readable storage medium storing one or more sets of instructions configured for execution by a computing device having a control circuit and a memory, the one or more sets of instructions when executed, implementing the method of claim 15.

23. A method of storing a video bitstream, the method comprising: The video bitstream is generated according to the video encoding method of claim 16.

24. A computer storage medium, comprising, Storing instructions executable by at least one processor for performing the video encoding method of claim 16, generating a bitstream and storing.

Images