Brightness component prediction method and device, equipment, storage medium and product

Abstract

The embodiment of the invention provides a brightness component prediction method and device, equipment, a storage medium and a product. The method comprises the steps of obtaining code stream data of an image, decoding the code stream data to obtain a residual error of a target block and a reference block of the target block, predicting a brightness component of the target block according to the reference block and a first brightness coefficient to obtain a brightness predicted value of the target block, the first brightness coefficient being obtained by deriving based on pixels in the target block, and the second brightness coefficient being obtained by deriving based on pixels in the target block. And displaying an image based on the luminance prediction value of the target block and the residual error of the target block. Therefore, in the brightness component prediction process, the loss generated by quantization can be reduced by using the first brightness coefficient, so that the prediction accuracy of the brightness component is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of multimedia data processing in computer technology, and particularly relates to a luminance component prediction method, a luminance component prediction device, a computer device, a computer readable storage medium and a luminance component prediction product. BACKGROUND

[0002] With the progress of scientific research, computer technology and Internet technology are developing rapidly. There are a large amount of multimedia data in the Internet. Due to limited transmission resources (such as bandwidth), the multimedia data is usually encoded by a device before being transmitted (encoding), and compressed code stream data is obtained. The encoding process involves steps such as prediction encoding, transform encoding and quantization. Research shows that in the intra prediction mode, improving the prediction accuracy of the luminance component helps to improve the coding efficiency. How to improve the prediction accuracy of the luminance component has become a hot issue in current research. SUMMARY

[0003] The embodiments of the present application provide a luminance component prediction method, device, equipment, storage medium and product, which can improve the prediction accuracy of the luminance component.

[0004] In one aspect, the embodiments of the present application provide a luminance component prediction method, comprising:

[0005] obtaining code stream data of an image;

[0006] decoding the code stream data to obtain a residual of a target block and a reference block of the target block, the target block being any unreconstructed block in the image;

[0007] predicting a luminance component of the target block according to the reference block and a first luminance coefficient of the target block to obtain a luminance prediction value of the target block, the first luminance coefficient being derived based on pixels in the target block;

[0008] displaying the image based on the luminance prediction value of the target block and the residual of the target block.

[0009] In the embodiments of the present application, the code stream data of the image is obtained, the code stream data is decoded to obtain the residual of the target block and the reference block of the target block, the luminance component of the target block is predicted according to the reference block and the first luminance coefficient to obtain the luminance prediction value of the target block, the first luminance coefficient is derived based on the pixels in the target block, and the image is displayed based on the luminance prediction value of the target block and the residual of the target block. It can be seen that in the luminance component prediction process, the loss caused by quantization can be reduced by using the first luminance coefficient, so as to improve the prediction accuracy of the luminance component.

[0010] In one aspect, the embodiments of the present application provide a luminance component prediction method, comprising:

[0011] obtaining a reference block of the target block, the target block being any one of uncoded blocks in the image;

[0012] predicting a luminance component of the target block according to the reference block and a first luminance coefficient of the target block, to obtain a luminance prediction value of the target block, the first luminance coefficient being derived based on pixels in the target block;

[0013] determining a residual of the target block based on the luminance prediction value of the target block, the residual of the target block being used to generate the bitstream data of the image.

[0014] In the embodiments of the present application, the reference block of the target block is obtained, the luminance component of the target block is predicted according to the reference block and the first luminance coefficient of the target block, to obtain the luminance prediction value of the target block, the first luminance coefficient being derived based on the pixels in the target block, and the residual of the target block is determined based on the luminance prediction value of the target block, the residual of the target block being used to generate the bitstream data of the image. It can be seen that, in the luminance component prediction process, the first luminance coefficient is used, so that the loss caused by quantization can be reduced, thereby improving the prediction accuracy of the luminance component.

[0015] In one aspect, the embodiments of the present application provide a luminance component prediction device, comprising:

[0016] an obtaining unit, configured to obtain bitstream data of an image;

[0017] a processing unit, configured to decode the bitstream data to obtain a residual of a target block and a reference block of the target block, the target block being any one of unreconstructed blocks in the image;

[0018] and configured to predict a luminance component of the target block according to the reference block and a first luminance coefficient of the target block, to obtain a luminance prediction value of the target block, the first luminance coefficient being derived based on pixels in the target block;

[0019] and configured to display the image based on the luminance prediction value of the target block and the residual of the target block.

[0020] In one aspect, the embodiments of the present application provide a luminance component prediction device, comprising:

[0021] an obtaining unit, configured to obtain a reference block of the target block, the target block being any one of uncoded blocks in the image;

[0022] a processing unit, configured to predict a luminance component of the target block according to the reference block and a first luminance coefficient of the target block, to obtain a luminance prediction value of the target block, the first luminance coefficient being derived based on pixels in the target block;

[0023] And the residual of the target block is used to determine the target block's brightness prediction value, and the residual of the target block is used to generate the image's bitstream data.

[0024] On one hand, this application provides a computer device, which includes:

[0025] Memory, which stores computer programs;

[0026] The processor is used to load computer programs to implement the aforementioned luminance component prediction method.

[0027] On the one hand, this application provides a computer-readable storage medium storing a computer program adapted to be loaded by a processor and executed by the above-described luminance component prediction method.

[0028] On one hand, this application provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the aforementioned luminance component prediction method.

[0029] In this embodiment, both the encoding and decoding devices predict the luminance components of the target block based on the reference block and the first luminance coefficient during encoding and decoding, thereby obtaining the predicted luminance value of the target block. It is evident that by using the first luminance coefficient during luminance component prediction, the loss caused by quantization can be reduced, thereby improving the prediction accuracy of the luminance components and ultimately enhancing encoding and decoding efficiency. Attached Figure Description

[0030] Figure 1 This is a schematic diagram of the architecture of a luminance component prediction system provided in an embodiment of this application;

[0031] Figure 2 A schematic diagram of a video encoding process provided in an embodiment of this application;

[0032] Figure 3 A schematic diagram illustrating the derivation and prediction process of local illumination compensation parameters provided in this application embodiment;

[0033] Figure 4 A flowchart of a luminance component prediction method provided in an embodiment of this application;

[0034] Figure 5 A flowchart of another luminance component prediction method provided in the embodiments of this application;

[0035] Figure 6 This application provides a schematic diagram of a video live streaming scenario.

[0036] Figure 7 This application provides a schematic diagram of a video-on-demand scenario.

[0037] Figure 8 This is a schematic diagram of the structure of a luminance component prediction device provided in an embodiment of this application;

[0038] Figure 9 This is a schematic diagram of another luminance component prediction device provided in an embodiment of this application;

[0039] Figure 10 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0040] It should be noted in advance that, in order to enable those skilled in the art to better understand the technical solutions proposed in the embodiments of this application, the embodiments of this application will be described clearly and completely in conjunction with one or more accompanying drawings. Furthermore, the various drawings shown in the embodiments of this application are merely illustrative examples; for example, the execution order of each step in the drawings can be adaptively adjusted according to the actual application scenario. In addition, in the embodiments of this application, the block diagrams shown in the various drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, or in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0041] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0042] It should be noted that "multiple" in this article refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0043] This application relates to multimedia data processing technology, specifically the process of encoding and decoding video content. The video content mentioned in this application can refer to video content generated in real-time and requiring real-time transmission during live video streaming, video conferencing, etc., or video content that does not require real-time generation and transmission in scenarios such as video-on-demand and short video content. Video encoding refers to the technology of encoding video content to convert it into a smaller file size, obtaining the corresponding video bitstream. Video decoding refers to the technology of decoding the video bitstream, which is the reverse process of video encoding, restoring the video bitstream to uncompressed video content, obtaining the corresponding video content.

[0044] Please see Figure 1 , Figure 1 This is a schematic diagram of the architecture of a luminance component prediction system provided in an embodiment of this application, as shown below. Figure 1 As shown, the luminance component prediction system includes a video encoding device 101 and a video decoding device 102, both of which are video processing devices. The video encoding device 101 can be a video processing device that creates video content. After creating the video content, it can encode the video content to obtain a video stream. Then, the video encoding device 101 can transmit the video stream to the video decoding device 102. The video decoding device 102 can be a video processing device that receives the video stream sent by the video encoding device 101 and decodes it to obtain the corresponding video content. The video encoding device 101 can be an electronic device used by a provider of video content (such as a creator of video content). The video decoding device 102 can be an electronic device used by a user of video content (such as a user).

[0045] The video encoding device 101 and the video decoding device 102 can be connected directly or indirectly via wired or wireless means, for example... Figure 1 As shown, video encoding device 101 and video decoding device 102 can communicate via network 103. Video encoding device 101 and video decoding device 102 can perform one-way data transmission via network 103. For example, video encoding device 101 can encode video content and transmit it to video decoding device 102 via network 103; the encoded video content is transmitted in the form of a video stream. Video decoding device 102 can obtain the video stream via network 103, decode the video stream to recover the video content, and then display the recovered video content.

[0046] As another example, video encoding device 101 and video decoding device 102 can perform bidirectional data transmission over network 103. This bidirectional data transmission can occur during live video streaming, video conferencing, or similar processes. For bidirectional data transmission, video encoding device 101 can encode video content (e.g., a video image stream captured by video encoding device 101) in real time, and then transmit the encoded video stream to video decoding device 102 in real time via network 103. Video decoding device 102 can receive the video stream transmitted by video encoding device 101, decode the video stream to recover the video content, and then display the video images in the video image stream on an accessible display device according to the video content.

[0047] It should be noted that, Figure 1 The number and configuration of devices shown are for illustrative purposes only and do not constitute a limitation on the embodiments of this application. In actual applications, the video processing devices (such as video encoding device 101 and video decoding device 102) may be the same electronic device or different electronic devices, and this application does not limit this. The network 103 mentioned above may, for example, include wired and / or wireless communication networks. The communication network 150 may exchange data in circuit-switched and / or packet-switched channels. This network may include telecommunications networks, local area networks, wide area networks, and / or the Internet. For the purposes of this application, unless explained below, the architecture and topology of network 150 may be irrelevant to the operation disclosed in this application.

[0048] The aforementioned video processing equipment (such as video encoding device 101 and video decoding device 102) can be smartphones, tablets, laptops, desktop computers, smart speakers, smartwatches, smart voice interaction devices, smart home appliances, in-vehicle terminals, VR devices, aircraft, etc., but is not limited to these. The aforementioned video processing equipment (such as video encoding device 101 and video decoding device 102) can also be servers, for example, independent physical servers, server clusters or distributed systems composed of multiple physical servers, or cloud servers providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0049] The processing steps performed in the video encoding device 101 mainly include: (1) the acquisition and production of video content; and (2) the video encoding process. The processing steps performed in the video decoding device 102 mainly include (3) the video decoding process. Among them, the video encoding device 101 and the video decoding device 102 involve the transmission of video bitstreams. This transmission process can be based on various transmission protocols, including but not limited to: Dynamic Adaptive Streaming over HTTP (DASH) protocol, Dynamic Bitrate Adaptive Transmission (HTTP Live Streaming, HLS) protocol, Smart Media Transport Protocol (SMTP), Transmission Control Protocol (TCP), etc.

[0050] Please refer to the following: Figure 2 , Figure 2 This is a schematic diagram of a video encoding process provided in an embodiment of this application, combined with... Figure 2 The various processes involved in the video encoding device 101 and the video decoding device 102 are described in detail below:

[0051] (1) The process of acquiring and producing video content

[0052] The acquisition of video content can include two methods: capturing sound and visual scenes from the real world through a capture device, and generating them through electronic devices. Specifically, the capture device can refer to hardware components configured in the video encoding device 101, such as microphones, cameras, and various sensors, such as active radar sensors. This capture device can also be a device directly or indirectly connected to the video encoding device 101 via wired or wireless means. For example, if the video encoding device 101 is a server, the capture device could be a camera connected to the server, providing video content acquisition functionality for the video encoding device 101. The capture device can include camera equipment and sensing devices, and may also include audio equipment for acquiring audio content synchronized with the video content. For example, camera equipment can include ordinary cameras, depth cameras, light field cameras, etc. Sensing devices can include laser devices, radar devices, etc. Audio devices can include audio sensors, microphones, etc. The aforementioned capture device is deployed at a specific location in the real space to capture video content and synchronized audio content within that space.

[0053] (2) The process of video encoding

[0054] Video encoding device 101 performs video encoding on the available video signal (video content). The video signal (video content) can be understood as video data in color modes (Red, Green, Blue, RGB) / luminance-bandwidth-chrominance (YUV) video data. Taking the international video coding standards (High Efficiency Video Coding, HEVC), (Versatile Video Coding, VVC), and the Chinese national video coding standard (Audio Video Coding Standard, AVS) as examples, a hybrid coding framework is adopted, and the following series of operations and processing are performed on the input raw video signal:

[0055] 1) Image partitioning / block division structure

[0056] When a video frame is input, it is divided into several non-overlapping processing units based on a block size. Each processing unit performs a similar compression operation. This processing unit is called a Coding Tree Unit (CTU) or Maximum Coding Unit (LCU). The CTU is used to segment the image into independently processable blocks, thereby improving coding efficiency and parallelism. In VVC, the CTU size is 128x128, and it can be further subdivided into one or more basic Coding Units (CUs). The CU is the most basic element in the coding process.

[0057] 2) Predictive Coding

[0058] Predictive coding includes intra-frame prediction and inter-frame prediction. After the original video signal is predicted by the selected reconstructed video signal, the residual video signal is obtained. The encoder needs to decide which predictive coding mode to select for the current coding unit (or coding block) and inform the video decoding device 102.

[0059] a. Intra-frame prediction refers to the prediction of signals from regions within the same image that have already been encoded and reconstructed.

[0060] b. Inter-frame prediction refers to the prediction of signals from other images that have already been encoded and are different from the current image (called reference images). This is because adjacent video frames usually have a high degree of similarity. For the image block to be encoded (such as the CU mentioned above), a reference block can be searched in the reference image, and their image residuals and position information can be recorded.

[0061] This application primarily relates to a Local Illumination Compensation (LIC) prediction method. The LIC prediction method is based on the characteristics of illumination changes, assuming a linear relationship between the current block and a reference block, and establishing a linear model for pixel amplitude changes, which can be specifically expressed as:

[0062] P'=αP+β Formula (1)

[0063] Where P' is the prediction result of the current block; P is the reference block pointed to by the motion vector MV; α and β are parameters obtained from a linear model derived from the template region pixels of the current block and the reference block, where α is the scaling factor and β is the offset term. Figure 3 This is a schematic diagram illustrating the derivation and prediction process of local illumination compensation parameters provided in an embodiment of this application. Figure 3 As shown, the linear model can be derived from the template region pixels in the current block and the reference block, and the parameters (α, β) in the linear model can be obtained. Based on the parameters in the reference block and the linear model, the prediction result of the current block can be obtained.

[0064] 3) Transformation and Quantization

[0065] After transformation and quantization, the residual block yields transform domain coefficients. These coefficients undergo inverse quantization and inverse transform to obtain the residual signal, which is then superimposed with the intra-frame predicted pixels to finally obtain the reconstructed pixels. Due to quantization, the reconstructed residual signal may differ from the difference between the original and predicted pixels. Specifically, the reference pixels come from the left and top regions of the current block. For example, the reference pixels could be one or more columns of pixels to the left of the current block and one or more columns of pixels above the current block. For pixels in the left / top region of the current block, because they are closer to the reference pixels, the correlation between them is statistically strong, resulting in a relatively small absolute value of the predicted residual. For pixels in the bottom / right region of the current block, because they are farther from the reference pixels, the absolute value of the predicted residual is relatively large.

[0066] After the residual video signal undergoes transformation operations such as Discrete Fourier Transform (DFT) and Discrete Cosine Transform (DCT), the signal is transformed into the transform domain, and these are called transform coefficients. The transform coefficients are then subjected to lossy quantization, losing some information to make the quantized signal more suitable for compression. In some video coding standards, there may be more than one transform method to choose from. Therefore, the encoder needs to select one of the transform methods for the current coding unit (or coding block) and inform the decoder. The fineness of quantization is usually determined by the quantization parameter (QP). A larger QP value means that coefficients with a wider range of values ​​will be quantized into the same output, which usually leads to greater distortion and a lower bit rate; conversely, a smaller QP value means that coefficients with a smaller range of values ​​will be quantized into the same output, which usually leads to less distortion and a higher bit rate.

[0067] 4) Entropy coding or statistical coding

[0068] The quantized transform-domain signal is statistically compressed and encoded based on the frequency of each value, ultimately outputting a binary (0 or 1) compressed bitstream. Simultaneously, other information generated during encoding, such as the selected encoding mode and motion vector data, also requires entropy coding to reduce the bit rate. Statistical coding is a lossless encoding method that effectively reduces the bit rate required to represent the same signal. Common statistical coding methods include Variable Length Coding (VLC) and Content Adaptive Binary Arithmetic Coding (CABAC).

[0069] Context-Based Binary Arithmetic Coding (CABAC) primarily involves three steps: binarization, context modeling, and binary arithmetic coding. After binarizing the input syntax elements, the binary data can be encoded using either a regular coding mode or a bypass coding mode. The bypass coding mode eliminates the need to assign a specific probability model to each binary bit; the input binary bit bin value is directly encoded using a simple bypass encoder, thus accelerating the overall encoding and decoding speed. Generally, different syntax elements are not completely independent, and even identical syntax elements possess a certain degree of memory. Therefore, according to conditional entropy theory, using other encoded syntax elements for conditional coding can further improve coding performance compared to independent coding or memoryless coding. This encoded symbol information used as conditions is called the context. In the regular coding mode, the binary bits of the syntax elements sequentially enter the context modeler. The encoder assigns an appropriate probability model to each input binary bit based on the values ​​of previously encoded syntax elements or binary bits; this process is called context modeling. The context model corresponding to a grammatical element can be located using the context index increment (ctxIdxInc) and the context index start (ctxIdxStart). After the bin value and the assigned probability model are fed into the binary arithmetic encoder for encoding, the context model needs to be updated based on the bin value, which is the adaptive process in encoding.

[0070] 5) Loop Filtering

[0071] The transformed and quantized signal is then subjected to inverse quantization, inverse transform, and prediction compensation to obtain a reconstructed image. Due to the effects of quantization, the reconstructed image differs from the original image in some aspects, resulting in distortion. Therefore, filtering operations can be performed on the reconstructed image, such as deblocking filters (DBF), sample adaptive offset (SAO), or adaptive loop filters (ALF), to effectively reduce the distortion caused by quantization. Since these filtered reconstructed images will serve as a reference for subsequent coded images to predict future image signals, the aforementioned filtering operations are also called loop filtering, i.e., filtering operations within the coding loop.

[0072] Currently, mainstream video coding standards such as HEVC, VVC, AV1 (the first-generation video coding standard developed by the Open Media Consortium), AV2 (the second-generation video coding standard developed by the Open Media Consortium), and AVS3 (the third-generation video coding standard developed by the Open Media Consortium) all adopt a block-based hybrid coding framework. Specifically, the original video content is divided into a series of coding blocks, and video coding methods such as prediction, transform, and entropy coding are combined to achieve video data compression.

[0073] In one embodiment of this application, Figure 2 A basic flowchart of a video encoder is shown, illustrating the process using intra-frame prediction as an example. The original image signal s... k [x,y] and the predicted image signal Perform the difference operation to obtain the residual signal u. k [x,y], residual signal u k After transformation and quantization, [x,y] is obtained as quantization coefficients. These coefficients are then used to obtain the encoded bitstream through entropy encoding, and to obtain the reconstructed residual signal u' through inverse quantization and inverse transform. k [x,y], predict image signal With the reconstructed residual signal u' k [x,y] superimposed to generate image signals Image signal On one hand, the signal is input to the intra-frame mode decision module and the intra-frame prediction module for intra-frame prediction processing; on the other hand, the reconstructed image signal s' is output through loop filtering. k [x,y], reconstruct the image signal s' k [x,y] can be used as a reference image for the next frame for motion estimation and motion compensation prediction. Then, based on the result s' of the motion compensation prediction... r [x+m x ,y+m y ] and intra-frame prediction results Obtain the predicted image signal for the next frame. And continue repeating the above process until the coding is complete.

[0074] (3) The process of video decoding

[0075] The decoding process of video decoding device 102 is the reverse of the encoding process of video encoding device 101. As can be seen from the video encoding process described above, when video decoding device 102 performs video decoding, for each CU, after the content playback device obtains the compressed bitstream, it first performs entropy decoding to obtain various mode information (MV, motion estimation information, etc.) and quantized transform coefficients. Each coefficient undergoes inverse quantization and inverse transform to obtain the residual signal (residual data). On the other hand, based on the known encoding mode information, the prediction signal corresponding to that CU can be obtained. Adding the two together yields the reconstructed signal. Finally, the reconstructed value of the decoded image needs to undergo loop filtering to generate the final output signal, which is the restored video content.

[0076] In one implementation, the video stream and video content (each video frame) can both be stored in the blockchain, preventing tampering. Blockchain is a novel application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Essentially, it is a decentralized database, a chain of data blocks linked using cryptographic methods. Each data block contains information about a batch of network transactions, used to verify the validity of the information (anti-counterfeiting) and generate the next block.

[0077] It is understood that the luminance component prediction system described in the embodiments of this application is for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and does not constitute a limitation on the technical solutions provided in the embodiments of this application. As those skilled in the art will know, with the evolution of system architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0078] Based on the aforementioned luminance component prediction system, this application provides a luminance component prediction method. The luminance component prediction method described in this application can be executed by a computer device, which can be... Figure 1 The luminance component prediction system shown includes video processing devices (such as video encoding device 101 and video decoding device 102). Video encoding device 101 and video decoding device 102 can be the same computer device (i.e., the computer device has both encoding and decoding functions). The luminance component prediction method proposed in this application will be described in detail below with reference to the accompanying drawings.

[0079] Please see Figure 4 , Figure 4 This application provides a flowchart of a luminance component prediction method, which can be executed by a decoding device. The luminance component prediction method may include the following steps S401-S404:

[0080] S401. Obtain the image bitstream data.

[0081] The bitstream data is obtained by encoding the image using an encoding device. The image can be a single image, a frame from a video, or a scene from a live stream, etc. This application does not impose any restrictions on this.

[0082] S402. Decode the bitstream data to obtain the residual of the target block and the reference block of the target block.

[0083] The target block is any unreconstructed block in the image, and it may contain one or more basic coding units. The reference block may contain one or more reference pixels, which are reconstructed pixels in the image. The decoding process of the bitstream data is the reverse of the encoding process. Specific implementation methods for the decoding device to decode the bitstream data to obtain the residual of the target block and the reference block of the target block can be found in the above encoding and decoding process, and will not be repeated here.

[0084] S403. Based on the first luminance coefficients of the reference block and the target block, predict the luminance component of the target block to obtain the predicted luminance value of the target block.

[0085] The first luminance coefficient is derived based on the pixels in the target block; the value of the first luminance coefficient can be used as the value of α and β in the above formula (1).

[0086] In one implementation, the decoding device determines the first luminance coefficient of the target block using a second luminance coefficient and a luminance difference value. The luminance difference value indicates the difference between the first luminance coefficient and the second luminance coefficient of the target block, and the value of the second luminance coefficient can be used as the value of α and β in the above formula (1). The difference between the second luminance coefficient and the first luminance coefficient of the target block is that the second luminance coefficient is derived based on pixels in the template region of the target block (not included in the target block), and the template region of the target block can include, but is not limited to, the left and upper regions of the target block. The determination of the first luminance coefficient of the target block using the second luminance coefficient and the luminance difference value can be specifically expressed as any one of the following formulas:

[0087] c i_final =c i_derived +c i_delta Formula (2)

[0088]

[0089] Among them, c i_final c is the first luminance coefficient of the target block. i_derived The second brightness coefficient of the target block. This represents the brightness difference value.

[0090] The decoding device obtains the brightness difference value. Specifically, the decoding device can decode the numerical and symbolic codes corresponding to the brightness difference value in the bitstream data to obtain the brightness difference value; or, the decoding device can determine the brightness difference value from a value index table based on the index information carried in the bitstream data.

[0091] In one embodiment, the encoding device can use quantization to quantize the brightness difference value and use numerical encoding (such as a fixed 4 bits) and symbol encoding (such as a 1-bit identifier) ​​to indicate the quantization result of the brightness difference value. The decoding device can then perform inverse quantization on the numerical encoding and symbol encoding corresponding to the brightness difference value in the bitstream data to obtain the brightness difference value.

[0092] In another embodiment, the encoding and decoding devices can store a value index table for brightness difference values. The encoding device indicates the index information (such as index values) of the brightness difference values ​​through the bitstream data. After decoding the bitstream data, the decoding device determines the brightness difference values ​​according to the index information and the value index table for brightness difference values. The value index table for brightness difference values ​​can be found in Table 1 or Table 2.

[0093] Table 1

[0094]

[0095] Table 2

[0096] Index information 0 1 2 3 4 Value of luminance difference value 0 1 / 16 1 / 8 1 / 4 1 / 2 Bit string 0 10 110 1110 1111

[0097] The difference between Table 1 and Table 2 is that Table 1 includes symbols for the brightness difference values, while Table 2 is only used to determine the values ​​of the brightness difference values. When using Table 2 as an index table for the values ​​of brightness difference values, it is necessary to additionally indicate the symbols for the brightness difference values.

[0098] In another embodiment, the encoding device performs entropy encoding on the brightness difference value. For the obtained target bit string, a CABAC context model can be designed for the value of the i-th bit in the target bit string; where the target bit string indicates the numerical value of the brightness difference value, and i is a positive integer. For example, the target bit string (value of the i-th bit) of the brightness difference value of the previous block can be used as the context of the target bit string (value of the i-th bit) of the brightness difference value of the target block. Accordingly, the decoding device decodes the entropy encoding of the brightness difference value in the bitstream data to obtain the brightness difference value.

[0099] After obtaining the brightness difference value, the decoding device calculates the first brightness coefficient of the target block based on the second brightness coefficient and the brightness difference value. Specific implementation methods can be found in formula (2) or formula (3), which will not be elaborated here. Next, the decoding device determines the predicted brightness value of the target block based on the first brightness coefficients of the reference block and the target block. Referring to formula (1), the value of the first brightness coefficient of the target block is used as the value of α and β. The predicted brightness value of the target block can be directly calculated based on the reference block and α and β.

[0100] In another embodiment, the target block can be divided into at least two regions; for example, the target block is divided into a first region and a second region. After obtaining the brightness difference value and the second brightness coefficient of the target block, on the one hand, the decoding device calculates the brightness prediction value of the first region based on the first brightness coefficient of the reference block and the target block; on the other hand, the decoding device calculates the brightness prediction value of the second region based on the second brightness coefficient of the reference block and the target block. After obtaining the brightness prediction values ​​of the first region and the second region, the decoding device determines the brightness prediction value of the target block based on the brightness prediction values ​​of the first region and the second region.

[0101] For example, for the pixels located in the first M rows and first N columns of the target block (i.e., the pixels in the second region), the decoding device uses the second luminance coefficient of the target block (which can be represented as c). i_derived The values ​​of α and β are used as the values ​​of α and β in the local illumination compensation prediction mode, and the luminance component prediction of the pixels in the second region is performed through the LIC prediction mode to obtain the luminance prediction value of the second region; for the remaining pixels in the target block (i.e., the pixels in the first region), the decoding device uses the first luminance coefficient of the target block (which can be expressed as c) i_final The values ​​of α and β are used as the values ​​of α and β in the local illumination compensation prediction mode, and the luminance component of the pixels in the first region is predicted using the LIC prediction mode to obtain the luminance prediction value of the first region. Where W and H are integers greater than 1, M is a positive integer less than W, and N is a positive integer less than H.

[0102] For example, for pixels located in the first H / T rows and first W / S columns of a target block (columns W * rows H) (i.e., pixels in the second region), the decoding device uses the second luminance coefficient of the target block (which can be represented as c). i_derived The values ​​of α and β are used as the values ​​of α and β in the local illumination compensation prediction mode, and the luminance component prediction of the pixels in the second region is performed through the LIC prediction mode to obtain the luminance prediction value of the second region; for the remaining pixels in the target block (i.e., the pixels in the first region), the decoding device uses the first luminance coefficient of the target block (which can be expressed as c) i_finalThe values ​​of α and β are used as the values ​​of α and β in the local illumination compensation prediction mode, and the luminance component of the pixels in the first region is predicted using the LIC prediction mode to obtain the luminance prediction value of the first region. Here, W and H are integers greater than 1, and T and S are powers of 2.

[0103] For example, the target block (W columns * H rows) is divided from the upper right to the lower left, creating an upper left and lower right region. The upper left region (the second region) uses the target block's second brightness coefficient (which can be represented as c). i_derived The values ​​of α and β are used as the values ​​of α and β in the local illumination compensation prediction mode, and the luminance component prediction of the pixels in the second region is performed through the LIC prediction mode to obtain the luminance prediction value of the second region; for the lower right corner region of the target block (i.e., the pixels in the first region), the decoding device uses the first luminance coefficient of the target block (which can be expressed as c) i_final The values ​​of α and β are used as the values ​​of α and β in the local illumination compensation prediction mode, and the luminance component of the pixels in the first region is predicted using the LIC prediction mode to obtain the luminance prediction value of the first region. The areas of the upper left and lower right regions can be different.

[0104] In another implementation, the target block can be divided into at least two regions; for example, the target block can be divided into a first region and a second region. The decoding device reconstructs some pixels in the target block (such as pixels in the first region of the target block) using the second luminance coefficient of the target block, and then derives the first luminance coefficient of the target block based on the reconstructed pixels (i.e., pixels in the target block); wherein, the second luminance coefficient of the target block is derived based on pixels in the template region of the target block (not included in the target block). Specifically, the decoding device can use the value of the second luminance coefficient of the target block as the value of α, β in the local illumination compensation prediction mode, and reconstruct some pixels in the target block based on the LIC prediction mode. After obtaining the reconstructed pixels in the target block, the decoding device can derive the first luminance coefficient of the target block based on the reconstructed pixels in the target block. The specific derivation method can be found in [reference needed]. Figure 3 The derivation methods (the difference being that the first luminance coefficient is derived from the reconstructed pixels in the target block, while the second luminance coefficient is derived from the pixels in the template region of the target block) will not be elaborated here. After obtaining the first luminance coefficient of the target block, the decoding device uses the first luminance coefficient of the target block to determine the predicted luminance values ​​of the remaining pixels in the target block (such as the pixels in the second region).

[0105] In one implementation, the decoding device uses the value of the first luminance coefficient of the target block as the value of α and β in the local illumination compensation prediction mode, and determines the luminance prediction value of the remaining pixels in the target block (such as pixels in the second region) based on the LIC prediction mode.

[0106] For example, for pixels located in the first H / T rows and first W / S columns of a target block (W columns * H rows) (i.e., pixels in the first region), the decoding device can use the value of the second luminance coefficient of the target block as the coefficient value of the filter, and reconstruct the pixels in the first region using this filter; for the remaining pixels in the target block (i.e., pixels in the second region), the decoding device derives the first luminance coefficient of the target block (which can be expressed as c) based on the reconstructed pixels in the first region. i_final The first luminance coefficient of the target block is used as the values ​​of α and β in the local illumination compensation prediction mode. Then, based on the LIC prediction mode, luminance component prediction is performed on the pixels in the second region to obtain the luminance prediction value of the second region. Where W and H are integers greater than 1, and T and S are powers of 2.

[0107] For example, the target block (W columns * H rows) is divided from the upper right to the lower left, creating an upper left and lower right region. The upper left region (the first region) uses the target block's second brightness coefficient (which can be represented as c). i_derived The values ​​of α and β in the local illumination compensation prediction mode are used to reconstruct the pixels in the first region based on the LIC prediction mode; for the lower right corner region (i.e., the second region) in the target block, the decoding device derives the first luminance coefficient of the target block (which can be expressed as c) based on the pixels reconstructed in the first region. i_final The first luminance coefficient of the target block is used as the α and β values ​​in the local illumination compensation prediction mode. Then, based on the LIC prediction mode, luminance component prediction is performed on the pixels in the second region to obtain the luminance prediction value of the second region. The areas of the upper left and lower right regions can be different.

[0108] In another implementation, the decoding device can use the value of the first luminance coefficient of the target block as the coefficient value of a filter, and use this filter to predict the luminance components of the pixels in the second region, thus obtaining the luminance prediction value of the second region. For example, for the pixels located in the first M rows and first N columns of the target block (W columns * H rows) (i.e., the pixels in the first region), the decoding device uses the second luminance coefficient of the target block (which can be represented as c) i_derived The values ​​of α and β in the local illumination compensation prediction mode are used as the basis for reconstructing the pixels in the first region based on the LIC prediction mode. For the remaining pixels in the target block (i.e., the pixels in the second region), the decoding device derives the first luminance coefficient of the target block (which can be expressed as c) based on the pixels reconstructed in the first region. i_final Next, the value of the first luminance coefficient of the target block is used as the coefficient value of the filter, and the luminance component of the pixels in the second region is predicted by the filter to obtain the luminance prediction value of the second region. Where W and H are integers greater than 1, M is a positive integer less than W, and N is a positive integer less than H.

[0109] Understandably, the first luminance coefficient of the template block is derived from the reconstructed pixels in the target block, and there is no need to indicate the luminance difference value during the encoding and decoding process.

[0110] S404. Display the image based on the predicted brightness value of the target block and the residual of the target block.

[0111] The decoding device obtains the intra-frame prediction result of the target block based on the luminance prediction value of the target block, reconstructs the target block using the intra-frame prediction result and the residual, and displays the image based on the reconstructed target block. The specific implementation of step S604 can be found in the video decoding process described above, and will not be repeated here.

[0112] In this embodiment, the bitstream data of the image is acquired, decoded to obtain the residual of the target block and a reference block of the target block, and the luminance component of the target block is predicted based on the reference block and a first luminance coefficient to obtain the luminance prediction value of the target block. The first luminance coefficient is derived based on the pixels in the target block. Based on the luminance prediction value of the target block and the residual of the target block, the image is displayed. It can be seen that by using the first luminance coefficient in the luminance component prediction process, the loss caused by quantization can be reduced, thereby improving the prediction accuracy of the luminance component.

[0113] Please see Figure 5 , Figure 5 A flowchart of another lumen component prediction method provided in an embodiment of this application is shown. This lumen component prediction method can be executed by an encoding device. The lumen component prediction method may include the following steps S501-S503:

[0114] S501, Obtain the reference block of the target block.

[0115] The target block is any uncoded block in the image, and it may contain one or more basic coding units. The image may be a single image, a frame from a video, or a scene from a live broadcast, etc., and this application does not impose any restrictions on this. The reference block may contain one or more reference pixels, which are already encoded pixels in the image.

[0116] S502. Based on the first luminance coefficients of the reference block and the target block, predict the luminance component of the target block to obtain the predicted luminance value of the target block.

[0117] The encoding device predicts the luminance component of the target block based on the reference pixel and the first luminance coefficient. The first luminance coefficient is derived based on the pixels in the target block; the value of the first luminance coefficient can be used as the value of α and β in the above formula (1).

[0118] The difference between encoding and decoding devices is that encoding devices can directly derive the first luminance coefficient of a target block based on the target block; while decoding devices need to determine the first luminance coefficient of a target block based on the second luminance coefficient and luminance difference value of the target block, or the decoding device reconstructs some pixels in the target block based on the second luminance coefficient of the target block, and then derives the first luminance coefficient of the target block through the reconstructed pixels in the target block.

[0119] The encoding device predicts the luminance component of the target block based on the first luminance coefficients of the reference block and the target block to obtain the predicted luminance value of the target block. For a detailed implementation method, please refer to [reference needed]. Figure 4 The implementation method of the decoding device predicting the luminance component of the target block based on the first luminance coefficients of the reference block and the target block to obtain the luminance prediction value of the target block will not be described in detail here.

[0120] S503. Based on the predicted brightness value of the target block, determine the residual of the target block.

[0121] The residual of the target block is used to generate the bitstream data of the image. The encoding device determines the residual of the target block based on the brightness prediction value of the target block, and the specific implementation of generating the bitstream data of the image based on the residual of the target block can be referred to the video encoding process described above, and will not be repeated here.

[0122] In one implementation, the encoding device indicates the first luminance coefficient of the target block using a luminance difference value. The encoding device can calculate the difference between the first luminance coefficient and the second luminance coefficient of the target block according to formula (2) or formula (3) to obtain the luminance difference value. The second luminance coefficient of the target block is derived based on the pixels in the template region of the target block. The encoding device then adds the luminance difference value indication information to the image's bitstream data; wherein the luminance difference value indication information includes the numerical and symbolic encoding corresponding to the luminance difference value, or includes index information of the luminance difference value. For specific encoding and decoding methods, please refer to... Figure 4 The encoding and decoding implementation methods for the brightness difference value will not be described in detail here.

[0123] In this embodiment, a reference block for the target block is obtained. Based on the reference block and a first luminance coefficient of the target block, the luminance components of the target block are predicted to obtain a predicted luminance value for the target block. The first luminance coefficient is derived based on the pixels in the target block. Based on the predicted luminance value of the target block, a residual for the target block is determined. This residual is used to generate the image's bitstream data. Therefore, by using the first luminance coefficient during luminance component prediction, the loss caused by quantization can be reduced, thereby improving the prediction accuracy of the luminance components.

[0124] The above content introduces the specific execution process of the luminance component prediction method and a luminance component prediction system suitable for implementing this method. The following section will introduce the applicable scenarios for the luminance component prediction method:

[0125] (1) Video live streaming scenario

[0126] Please refer to the following: Figure 6 , Figure 6 This is a schematic diagram of a video live streaming scenario provided in an embodiment of this application. Figure 6 As shown, in a live video streaming scenario, the video encoding device can be an electronic device used by the broadcaster. This device provides video production and encoding functions and can run a live video streaming application. The video decoding device is an electronic device used by the viewers of the broadcast. It provides the function of decoding the video stream, rendering the resulting video content, and playing it. Both the video encoding and decoding devices can be devices including CPUs and GPUs, used for video encoding and decoding processing.

[0127] The video encoding device acquires video content and performs video encoding processing to obtain a video stream. This video encapsulation file containing the video stream can then be streamed to a cloud server via the RTMP protocol using a streaming tool. Upon receiving the encapsulation file, the cloud server can decode it based on user requirements, performing processes such as transcoding, noise reduction, enhancement, and analysis. Alternatively, the video encoding device can also stream the encapsulation file to a Content Delivery Network (CDN) via the RTMP protocol. The cloud server can then retrieve the encapsulation file from the CDN and process it according to user needs. Finally, the viewer's video decoding device can retrieve the encapsulation file through a live video application, decapsulate and decode it, and render the decoded video content based on display parameters to display (play) the video stream.

[0128] In live video streaming scenarios, video encoding and decoding devices can improve the accuracy of luminance component prediction using the aforementioned luminance component prediction method, thereby enhancing encoding and decoding efficiency.

[0129] (2) Video on demand scenario

[0130] Video on Demand (VOD) refers to the ability to play video content according to the request of viewers. It can also be understood as transmitting content that a user wants to watch (click or select) to the requesting user. Please refer to the following: Figure 7 , Figure 7 This is a schematic diagram of a video-on-demand scenario provided in an embodiment of this application. Figure 7 As shown, in a video-on-demand scenario, the video encoding device can be the platform providing the video content or the user's electronic device. This device provides functions for video production, encoding, and encapsulation. The video decoding device is the electronic device used by the viewer using the video-on-demand function. It provides the function of decoding the video stream, rendering, and playing the video. Both the video encoding and decoding devices can be devices including CPUs and GPUs, used for video encoding and decoding processing.

[0131] Specifically, a video encoding device can encode video content to obtain a video stream. Then, the video encoding device can store the video encapsulation file containing this video stream in cloud storage, such as a cloud server. Next, a video decoding device can retrieve the corresponding video encapsulation file from the cloud storage based on the user's selection of desired video content, decapsulate and decode the video encapsulation file, and render the decoded video content based on display parameters to display (play) the video stream.

[0132] In video-on-demand scenarios, video encoding and decoding devices can also improve the accuracy of luminance component prediction by using the aforementioned luminance component prediction method, thereby improving encoding and decoding efficiency.

[0133] The methods of the embodiments of this application have been described in detail above. In order to facilitate better implementation of the above solutions of the embodiments of this application, the apparatus of the embodiments of this application is provided below.

[0134] Please see Figure 8 , Figure 8 This is a schematic diagram of the structure of a luminance component prediction device provided in an embodiment of this application. Figure 8 The luminance component prediction device shown can be mounted in a computer device, specifically a decoding device. Figure 8 The luminance component prediction device shown can be used to perform the above. Figure 4 Some or all of the functionality described in the method embodiments. Please refer to... Figure 8 The luminance component prediction device includes:

[0135] Acquisition unit 801 is used to acquire the image bitstream data;

[0136] The processing unit 802 is used to decode the bitstream data to obtain the residual of the target block and the reference block of the target block, wherein the target block is any unreconstructed block in the image;

[0137] And a first luminance coefficient for predicting the luminance component of the target block based on the reference block and the target block to obtain the luminance prediction value of the target block; the first luminance coefficient is derived based on the pixels in the target block;

[0138] And to display the image based on the brightness prediction value of the target block and the residual of the target block.

[0139] In one embodiment, the processing unit 802 is configured to predict the luminance component of the target block based on a first luminance coefficient of the reference block and the target block, to obtain a predicted luminance value of the target block, specifically configured to:

[0140] Obtain the brightness difference value, which indicates the difference between the first brightness coefficient and the second brightness coefficient of the target block; the second brightness coefficient of the target block is derived based on the pixels in the template region of the target block.

[0141] Calculate the first luminance coefficient of the target block based on the second luminance coefficient and the luminance difference value of the target block;

[0142] The predicted brightness value of the target block is determined based on the brightness components of the reference block and the first brightness coefficient of the target block.

[0143] In one embodiment, the target block includes a first region and a second region; the processing unit 802 is configured to determine a predicted brightness value of the target block based on the brightness components of the reference block and a first brightness coefficient of the target block, specifically for:

[0144] The luminance prediction value of the first region is calculated based on the luminance components of the reference block and the first luminance coefficient of the target block.

[0145] The luminance prediction value of the second region is calculated based on the luminance components of the reference block and the second luminance coefficient of the target block.

[0146] The brightness prediction value of the target block is determined based on the brightness prediction values ​​of the first region and the second region.

[0147] In one embodiment, the processing unit 802 is configured to acquire brightness difference values, specifically for:

[0148] Decode the numerical and symbolic encodings corresponding to the brightness difference values ​​in the bitstream data to obtain the brightness difference values; or,

[0149] Based on the index information carried in the bitstream data, the brightness difference value is determined from the value index table.

[0150] In one embodiment, the target block includes a first region and a second region; the process by which the processing unit 802 predicts the luminance components of the target block based on a reference block and a first luminance coefficient of the target block to obtain a predicted luminance value for the target block includes:

[0151] The pixels in the first region are reconstructed using the second luminance coefficient of the target block; the second luminance coefficient of the target block is derived based on the pixels in the template region of the target block.

[0152] Based on the reconstructed pixels in the first region, the first luminance coefficient of the target block is derived;

[0153] The predicted brightness value of the second region is determined based on the brightness components of the reconstructed pixels and the first brightness coefficient of the target block.

[0154] In one embodiment, the processing unit 802 is configured to determine the predicted brightness value of the second region based on the brightness component of the reconstructed pixel and the first brightness coefficient of the target block, specifically configured to:

[0155] Based on the luminance components of the reconstructed pixels and the first luminance coefficient of the target block, local illumination compensation prediction is performed on the second region to obtain the predicted luminance value of the second region; or,

[0156] The first luminance coefficient of the target block is used as the coefficient of the filter, and the luminance component of the reconstructed pixel is filtered by the filter to obtain the luminance prediction value of the second region.

[0157] According to one embodiment of this application, Figure 4 The steps involved in the luminance component prediction method shown can be derived from... Figure 8 The individual units in the luminance component prediction device shown perform this operation. For example, Figure 4 Step S401 shown can be performed by Figure 8 The acquisition unit 801 shown is executed, and steps S402-S404 can be performed by... Figure 8 The processing unit 802 shown executes. Figure 8 The units in the luminance component prediction device shown can be individually or entirely combined into one or more other units, or some of the units can be further divided into multiple functionally smaller units. This achieves the same operation without affecting the technical effects of the embodiments of this application. The above units are based on logical function division. In practical applications, the function of one unit can be implemented by multiple units, or the function of multiple units can be implemented by one unit. In other embodiments of this application, the luminance component prediction device may also include other units. In practical applications, these functions can also be implemented with the assistance of other units, and can be implemented collaboratively by multiple units.

[0158] According to another embodiment of this application, a general-purpose computing device, such as a computer device including processing elements and storage elements such as a central processing unit (CPU), random access memory (RAM), and read-only memory (ROM), can perform operations such as... Figure 4 The computer program (including program code) for each step involved in the corresponding method shown, to construct such... Figure 8 The luminance component prediction apparatus shown herein, and the luminance component prediction method for implementing the embodiments of this application, are described. The computer program may be recorded on, for example, a computer-readable recording medium, loaded onto the aforementioned computing device via the computer-readable recording medium, and run therein.

[0159] Based on the same inventive concept, the principle and beneficial effects of the luminance component prediction device provided in the embodiments of this application are similar to the principle and beneficial effects of the luminance component prediction method in the embodiments of this application. For details, please refer to the principle and beneficial effects of the method implementation. For the sake of brevity, these will not be repeated here.

[0160] Please see Figure 9 , Figure 9 This is a schematic diagram of another luminance component prediction device provided in an embodiment of this application. Figure 9 The luminance component prediction device shown can be mounted in a computer device, which may specifically be an encoding device. Figure 9 The luminance component prediction device shown can be used to perform the above. Figure 5 Some or all of the functionality described in the method embodiments. Please refer to... Figure 9 The luminance component prediction device includes:

[0161] The acquisition unit 901 is used to acquire a reference block for the target block, where the target block is any uncoded block in the image;

[0162] The processing unit 902 is used to predict the luminance component of the target block based on the first luminance coefficients of the reference block and the target block, and obtain the luminance prediction value of the target block; the first luminance coefficients are derived based on the pixels in the target block.

[0163] And the residual of the target block is used to determine the target block's brightness prediction value, and the residual of the target block is used to generate the image's bitstream data.

[0164] In one embodiment, the processing unit 902 is configured to predict the luminance component of the target block based on a first luminance coefficient of the reference block and the target block, to obtain a predicted luminance value of the target block, specifically configured to:

[0165] Based on the pixels in the target block, derive the first luminance coefficient of the target block;

[0166] The predicted brightness value of the target block is determined based on the brightness components of the reference block and the first brightness coefficient of the target block.

[0167] In one embodiment, the target block includes a first region and a second region; the processing unit 902 is configured to determine a predicted brightness value of the target block based on the brightness components of the reference block and a first brightness coefficient of the target block, specifically configured to:

[0168] The luminance prediction value of the first region is calculated based on the luminance components of the reference block and the first luminance coefficient of the target block.

[0169] The luminance prediction value of the second region is calculated based on the luminance components of the reference block and the second luminance coefficient of the target block; the second luminance coefficient of the target block is derived based on the pixels in the template region of the target block.

[0170] The brightness prediction value of the target block is determined based on the brightness prediction values ​​of the first region and the second region.

[0171] In one embodiment, the processing unit 902 is further configured to:

[0172] The difference between the first luminance coefficient and the second luminance coefficient of the target block is calculated to obtain the luminance difference value; the second luminance coefficient of the target block is derived based on the pixels in the template region of the target block.

[0173] The indication information of the brightness difference value is added to the image bitstream data; the indication information includes the numerical code and symbol code corresponding to the brightness difference value, or includes the index information of the brightness difference value.

[0174] According to one embodiment of this application, Figure 5 The steps involved in the luminance component prediction method shown can be derived from... Figure 9 The individual units in the luminance component prediction device shown perform this operation. For example, Figure 5 Step S501 shown can be performed by Figure 9 The acquisition unit 901 shown is executed, and steps S502 and S503 can be performed by... Figure 9 The processing unit 902 shown executes. Figure 9The units in the luminance component prediction device shown can be individually or entirely combined into one or more other units, or some of the units can be further divided into multiple functionally smaller units. This achieves the same operation without affecting the technical effects of the embodiments of this application. The above units are based on logical function division. In practical applications, the function of one unit can be implemented by multiple units, or the function of multiple units can be implemented by one unit. In other embodiments of this application, the luminance component prediction device may also include other units. In practical applications, these functions can also be implemented with the assistance of other units, and can be implemented collaboratively by multiple units.

[0175] According to another embodiment of this application, a general-purpose computing device, such as a computer device including processing elements and storage elements such as a central processing unit (CPU), random access memory (RAM), and read-only memory (ROM), can perform operations such as... Figure 5 The computer program (including program code) for each step involved in the corresponding method shown, to construct such... Figure 9 The luminance component prediction apparatus shown herein, and the luminance component prediction method for implementing the embodiments of this application, are described. The computer program may be recorded on, for example, a computer-readable recording medium, loaded onto the aforementioned computing device via the computer-readable recording medium, and run therein.

[0176] Based on the same inventive concept, the principle and beneficial effects of the luminance component prediction device provided in the embodiments of this application are similar to the principle and beneficial effects of the luminance component prediction method in the embodiments of this application. For details, please refer to the principle and beneficial effects of the method implementation. For the sake of brevity, these will not be repeated here.

[0177] Please see Figure 10 , Figure 10 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. The computer device can be an encoding device or a decoding device. Figure 10As shown, the computer device includes at least a processor 1001, a communication interface 1002, and a memory 1003. The processor 1001, communication interface 1002, and memory 1003 can be connected via a bus or other means. The processor 1001 (or Central Processing Unit, CPU) is the computing and control core of the computer device. It can parse various instructions within the computer device and process various data. For example, the CPU can parse power-on / off commands issued by objects to the computer device and control the computer device to perform power-on / off operations; it can also transmit various interactive data between internal structures of the computer device, and so on. The communication interface 1002 may optionally include a standard wired interface or a wireless interface (such as Wi-Fi, mobile communication interface, etc.), and can be used to send and receive data under the control of the processor 1001; the communication interface 1002 can also be used for data transmission and interaction within the computer device. The memory 1003 is the storage device in the computer device, used to store programs and data. It is understood that the memory 1003 here may include the built-in memory of the computer device, or it may include the extended memory supported by the computer device. The memory 1003 provides storage space for storing the operating system of the computer device, which may include, but is not limited to, Android, iOS, Windows Phone, etc., and this application does not limit this.

[0178] This application embodiment also provides a computer-readable storage medium (Memory), which is a memory device in a computer device used to store programs and data. It is understood that the computer-readable storage medium here can include both the built-in storage medium in the computer device and extended storage media supported by the computer device. The computer-readable storage medium provides storage space that stores the processing system of the computer device. Furthermore, the storage space also stores computer programs suitable for loading and execution by the processor 1001. It should be noted that the computer-readable storage medium here can be high-speed RAM or non-volatile memory, such as at least one disk storage device; optionally, it can also be at least one computer-readable storage medium located remotely from the aforementioned processor.

[0179] In one embodiment, the processor 1001 performs the following operations by running a computer program stored in the memory 1003:

[0180] Obtain the image bitstream data;

[0181] Decode the bitstream data to obtain the residual of the target block and the reference block of the target block. The target block is any unreconstructed block in the image.

[0182] Based on the first luminance coefficients of the reference block and the target block, the luminance component of the target block is predicted to obtain the luminance prediction value of the target block; the first luminance coefficient is derived based on the pixels in the target block.

[0183] The image is displayed based on the predicted brightness value of the target block and the residual of the target block.

[0184] As an optional embodiment, the processor 1001 predicts the luminance component of the target block based on the first luminance coefficients of the reference block and the target block, and obtains the luminance prediction value of the target block in the following specific embodiment:

[0185] Obtain the brightness difference value, which indicates the difference between the first brightness coefficient and the second brightness coefficient of the target block; the second brightness coefficient of the target block is derived based on the pixels in the template region of the target block.

[0186] Calculate the first luminance coefficient of the target block based on the second luminance coefficient and the luminance difference value of the target block;

[0187] The predicted brightness value of the target block is determined based on the brightness components of the reference block and the first brightness coefficient of the target block.

[0188] As an optional embodiment, the target block includes a first region and a second region; the processor 1001 determines the predicted brightness value of the target block based on the brightness components of the reference block and the first brightness coefficient of the target block in the following specific embodiment:

[0189] The luminance prediction value of the first region is calculated based on the luminance components of the reference block and the first luminance coefficient of the target block.

[0190] The luminance prediction value of the second region is calculated based on the luminance components of the reference block and the second luminance coefficient of the target block.

[0191] The brightness prediction value of the target block is determined based on the brightness prediction values ​​of the first region and the second region.

[0192] As an optional embodiment, the processor 1001 acquires the brightness difference value in the following specific embodiment:

[0193] Decode the numerical and symbolic encodings corresponding to the brightness difference values ​​in the bitstream data to obtain the brightness difference values; or,

[0194] Based on the index information carried in the bitstream data, the brightness difference value is determined from the value index table.

[0195] As an optional embodiment, the target block includes a first region and a second region; the process by which the processor 1001 predicts the luminance components of the target block based on the first luminance coefficients of the reference block and the target block to obtain the predicted luminance value of the target block includes:

[0196] The pixels in the first region are reconstructed using the second luminance coefficient of the target block; the second luminance coefficient of the target block is derived based on the pixels in the template region of the target block.

[0197] Based on the reconstructed pixels in the first region, the first luminance coefficient of the target block is derived;

[0198] The predicted brightness value of the second region is determined based on the brightness components of the reconstructed pixels and the first brightness coefficient of the target block.

[0199] As an optional embodiment, the process by which the processor 1001 determines the predicted brightness value of the second region based on the brightness components of the reconstructed pixels and the first brightness coefficient of the target block includes:

[0200] Based on the luminance components of the reconstructed pixels and the first luminance coefficient of the target block, local illumination compensation prediction is performed on the second region to obtain the predicted luminance value of the second region; or,

[0201] The first luminance coefficient of the target block is used as the coefficient of the filter, and the luminance component of the reconstructed pixel is filtered by the filter to obtain the luminance prediction value of the second region.

[0202] In another embodiment, the processor 1001 performs the following operations by running a computer program stored in the memory 1003:

[0203] Obtain a reference block for the target block, where the target block is any uncoded block in the image;

[0204] Based on the first luminance coefficients of the reference block and the target block, the luminance component of the target block is predicted to obtain the luminance prediction value of the target block; the first luminance coefficient is derived based on the pixels in the target block.

[0205] Based on the predicted brightness value of the target block, the residual of the target block is determined, and the residual of the target block is used to generate the bitstream data of the image.

[0206] As an optional embodiment, the processor 1001 predicts the luminance component of the target block based on the first luminance coefficients of the reference block and the target block, and obtains the luminance prediction value of the target block in the following specific embodiment:

[0207] Based on the pixels in the target block, derive the first luminance coefficient of the target block;

[0208] The predicted brightness value of the target block is determined based on the brightness components of the reference block and the first brightness coefficient of the target block.

[0209] As an optional embodiment, the target block includes a first region and a second region; the processor 1001 determines the predicted brightness value of the target block based on the brightness component of the reference block and the first brightness coefficient of the target block in the following specific embodiment:

[0210] The luminance prediction value of the first region is calculated based on the luminance components of the reference block and the first luminance coefficient of the target block.

[0211] The luminance prediction value of the second region is calculated based on the luminance components of the reference block and the second luminance coefficient of the target block; the second luminance coefficient of the target block is derived based on the pixels in the template region of the target block.

[0212] The brightness prediction value of the target block is determined based on the brightness prediction values ​​of the first region and the second region.

[0213] In another embodiment, the processor 1001 also performs the following operations by running a computer program in the memory 1003:

[0214] The difference between the first luminance coefficient and the second luminance coefficient of the target block is calculated to obtain the luminance difference value; the second luminance coefficient of the target block is derived based on the pixels in the template region of the target block.

[0215] The indication information of the brightness difference value is added to the image bitstream data; the indication information includes the numerical code and symbol code corresponding to the brightness difference value, or includes the index information of the brightness difference value.

[0216] Based on the same inventive concept, the principle and beneficial effects of the computer device provided in the embodiments of this application in solving the problem are similar to the principle and beneficial effects of the audio processing method in the embodiments of this application in solving the problem. Please refer to the principle and beneficial effects of the implementation of the method. For the sake of brevity, they will not be repeated here.

[0217] It should be understood that, in the embodiments of this application, the processor 1201 may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0218] This application also provides a computer-readable storage medium storing a computer program adapted to be loaded by a processor and to execute the audio processing method described in the above method embodiments.

[0219] This application also provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the aforementioned audio processing method.

[0220] The steps in the method of this application embodiment can be adjusted, combined, or deleted according to actual needs.

[0221] The modules in the device of this application embodiment can be merged, divided, and deleted according to actual needs.

[0222] In the embodiments of this application, the term "module" or "unit" refers to a computer program or part of a computer program with a predetermined function, which works together with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0223] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk, etc.

[0224] The above-disclosed embodiments are merely preferred embodiments of this application and should not be construed as limiting the scope of this application. Those skilled in the art will understand that all or part of the processes for implementing the above embodiments and equivalent variations made in accordance with the claims of this application are still within the scope of this application.

[0225] It should also be noted that when the above embodiments of this application are applied to specific products or technologies, if it is necessary to obtain user data, the user's permission or consent must be obtained, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

Claims

1. A method of predicting a luminance component, characterized by, The method comprises: acquiring code stream data of an image; decoding the code stream data to obtain a residual of a target block and a reference block of the target block, the target block being any unreconstructed block in the image; predicting a luminance component of the target block according to a first luminance coefficient of the reference block and the target block to obtain a luminance prediction value of the target block; the first luminance coefficient being derived based on pixels in the target block; displaying the image based on the luminance prediction value of the target block and the residual of the target block.

2. The method of claim 1, wherein, The predicting the luminance component of the target block according to the first luminance coefficient of the reference block and the target block to obtain the luminance prediction value of the target block comprises: acquiring a luminance difference value, the luminance difference value being used to indicate a difference between the first luminance coefficient of the target block and a second luminance coefficient of the target block; the second luminance coefficient of the target block being derived based on pixels in a template region of the target block; calculating the first luminance coefficient of the target block according to the second luminance coefficient of the target block and the luminance difference value; determining the luminance prediction value of the target block according to the luminance component of the reference block and the first luminance coefficient of the target block.

3. The method of claim 2, wherein, The target block comprises a first region and a second region; the determining the luminance prediction value of the target block according to the luminance component of the reference block and the first luminance coefficient of the target block comprises: calculating a luminance prediction value of the first region according to the luminance component of the reference block and the first luminance coefficient of the target block; calculating a luminance prediction value of the second region according to the luminance component of the reference block and the second luminance coefficient of the target block; determining the luminance prediction value of the target block based on the luminance prediction value of the first region and the luminance prediction value of the second region.

4. The method of claim 2, wherein, The acquiring the luminance difference value comprises: decoding a numerical value coding and a sign coding corresponding to the luminance difference value in the code stream data to obtain the luminance difference value; or determining the luminance difference value from a value index table according to index information carried in the code stream data.

5. The method of claim 1, wherein, The target block comprises a first region and a second region; the process of predicting the luminance component of the target block according to the first luminance coefficient of the reference block and the target block to obtain the luminance prediction value of the target block comprises: reconstructing pixels in the first region through a second luminance coefficient of the target block; the second luminance coefficient of the target block being derived based on pixels in a template region of the target block; deriving the first luminance coefficient of the target block based on the reconstructed pixels in the first region; determining a luminance prediction value of the second region according to a luminance component of the reconstructed pixels and the first luminance coefficient of the target block.

6. The method of claim 5, wherein, The determining the luminance prediction value of the second region according to the luminance component of the reconstructed pixels and the first luminance coefficient of the target block comprises: performing local illumination compensation prediction on the second region according to the luminance component of the reconstructed pixels and the first luminance coefficient of the target block to obtain the luminance prediction value of the second region; or The first luma coefficient of the target block is taken as a coefficient of a filter, and a luma component of the reconstructed pixel is filtered by the filter to obtain a luma prediction value of the second region.

7. A method of predicting a luminance component, characterized by, The method comprises: obtaining a reference block of a target block, the target block being any one uncoded block in an image; predicting a luma component of the target block according to the reference block and a first luma coefficient of the target block to obtain a luma prediction value of the target block, the first luma coefficient being derived based on pixels in the target block; determining a residual of the target block based on the luma prediction value of the target block, the residual of the target block being used to generate code stream data of the image.

8. The method of claim 7, wherein, The method comprises: deriving the first luma coefficient of the target block based on pixels in the target block; determining the luma prediction value of the target block according to the luma component of the reference block and the first luma coefficient of the target block.

9. The method of claim 8, wherein, The target block comprises a first region and a second region; the method comprises: calculating the luma prediction value of the first region according to the luma component of the reference block and the first luma coefficient of the target block; calculating the luma prediction value of the second region according to the luma component of the reference block and a second luma coefficient of the target block, the second luma coefficient of the target block being derived based on pixels in a template region of the target block; determining the luma prediction value of the target block based on the luma prediction value of the first region and the luma prediction value of the second region.

10. The method of claim 7, wherein, The method further comprises: calculating a difference value between the first luma coefficient of the target block and the second luma coefficient of the target block to obtain a luma difference value, the second luma coefficient of the target block being derived based on pixels in a template region of the target block; adding indication information of the luma difference value into code stream data of the image, the indication information comprising value coding and sign coding corresponding to the luma difference value, or comprising index information of the luma difference value.

11. A luminance component prediction apparatus characterized by comprising: The luma component prediction device comprises: an obtaining unit configured to obtain code stream data of an image; a processing unit configured to decode the code stream data to obtain a residual of a target block and a reference block of the target block, the target block being any one unreconstructed block in the image; and configured to predict a luma component of the target block according to the reference block and a first luma coefficient of the target block to obtain a luma prediction value of the target block, the first luma coefficient being derived based on pixels in the target block; and configured to display the image based on the luma prediction value of the target block and the residual of the target block.

12. A luminance component prediction apparatus characterized by comprising: The luma component prediction device comprises: an obtaining unit configured to obtain a reference block of a target block, the target block being any one uncoded block in an image; The processing unit is configured to predict a luminance component of the target block according to first luminance coefficients of the reference block and the target block, to obtain a luminance prediction value of the target block; the first luminance coefficients are derived based on pixels in the target block; and configured to determine a residual of the target block based on the luminance prediction value of the target block, the residual of the target block being used to generate code stream data of the image.

13. A computer device, comprising: The method comprises: a memory, wherein a computer program is stored in the memory; a processor configured to load the computer program to implement the luminance component prediction method in any one of claims 1-6; or to load the computer program to implement the luminance component prediction method in any one of claims 7-10.

14. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is adapted to be loaded by the processor and execute the luminance component prediction method in any one of claims 1-6; or to be loaded by the processor and execute the luminance component prediction method in any one of claims 7-10.

15. A computer program product, characterised in that, The computer program product comprises a computer program, and the computer program is adapted to be loaded by the processor and execute the luminance component prediction method in any one of claims 1-6; or to be loaded by the processor and execute the luminance component prediction method in any one of claims 7-10.

Images