Reference block search method and device in live video coding and computer device

Abstract

The application relates to the network live broadcast technical field and the video coding technical field, and proposes a reference block search method, device and computer equipment in live video coding, the method comprising the following steps: when searching for a plurality of reference blocks corresponding to each coding block in a live image in an inter prediction mode, judging whether the currently searched live image is a first live image; if yes, determining target reference blocks corresponding to each coding block in the first live image according to first error information corresponding to the plurality of searched reference blocks in the inter prediction mode, the first error information corresponding to the reference blocks being obtained by obtaining candidate prediction pixel values of each pixel point in a coding block according to pixel values of each reference pixel point in the reference blocks, and calculating absolute errors between the candidate prediction pixel values of each pixel point in the coding block and original pixel values of each pixel point in the coding block. Compared with the prior art, the application can improve the search speed of the target reference blocks and improve the video coding efficiency.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of network live streaming and the technical field of video coding, in particular to a reference block search method in live video coding, a device and a computer device. BACKGROUND

[0002] With the rapid development of the live streaming industry, more and more Internet platforms begin to provide live streaming services to attract users to interact in the live streaming room, so that ordinary people have the opportunity to show their talents and help to alleviate the social employment pressure.

[0003] The live streaming service includes video live streaming service and voice live streaming service, wherein in the video live streaming service process, the video content watched by the user on the client is called live video, and the definition and smoothness of the live video when playing will directly affect the live streaming experience of the user.

[0004] In the network live streaming scenario, the client can be divided into a host client and a viewer client. After the host starts the network live streaming, the host client will collect the live video. After the host client encodes the live video, the encoded live video is sent to the server. The viewer client pulls the encoded live video from the server and decodes and plays it. In the above process, encoding the live video is an important link to ensure the definition and smoothness of the live video and improve the live streaming experience of the user.

[0005] At present, video coding is mostly based on the HECV standard, for example, the X265 encoder based on the HECV standard for video coding. However, the complexity of the video coding method based on the HECV standard is relatively high and the efficiency is relatively low, which is difficult to adapt to the current network live streaming scenario.

[0006] Since in the video coding method based on the HECV standard, the search process of the reference block will not only affect the quality of the video coding, but also affect the efficiency of the video coding, therefore, if the search process of the reference block can be optimized, the higher demand of the user on the definition and smoothness of the video playing in the network live streaming scenario can be better met, and the live streaming experience of the user can be further improved. SUMMARY

[0007] Embodiments of the present application provide a reference block search method in live video coding, a device and a computer device, which can solve the technical problem of how to improve the smoothness of live video playing while ensuring the definition of live video playing, and the technical solution is as follows:

[0008] In a first aspect, the embodiments of the present application provide a reference block search method in live video coding, comprising the steps of:

[0009] Acquire live video; wherein the live video includes several frames of live images;

[0010] When searching for several reference blocks corresponding to each coding block in the live image in the inter-frame prediction mode, it is determined whether the currently searched live image is the first live image; wherein, the coding block is obtained by dividing the live image, and the frame type corresponding to the first live image is a bidirectional difference frame.

[0011] If so, in the inter-frame prediction mode, the target reference block corresponding to each of the coding blocks in the first live image is determined according to the first error information corresponding to the searched reference blocks; wherein, the first error information corresponding to the reference block is obtained by obtaining the candidate predicted pixel value of each pixel in the coding block according to the pixel value of each reference pixel in the reference block, and calculating the sum of the absolute errors between the candidate predicted pixel value of each pixel in the coding block and the original pixel value of each pixel in the coding block.

[0012] Secondly, embodiments of this application provide a reference block search device for live video encoding, comprising:

[0013] The first acquisition unit is used to acquire live video; wherein the live video includes several frames of live images;

[0014] The first judgment unit is used to determine whether the currently searched live image is the first live image when searching for several reference blocks corresponding to each coding block in the live image in the inter-frame prediction mode; wherein the coding block is obtained by dividing the live image, and the frame type corresponding to the first live image is a bidirectional difference frame.

[0015] The first search unit is configured to, if so, in the inter-frame prediction mode, determine the target reference block corresponding to each of the coding blocks in the first live image based on the first error information corresponding to the searched reference blocks; wherein, the first error information corresponding to the reference block is obtained by acquiring the candidate predicted pixel value of each pixel in the coding block based on the pixel value of each reference pixel in the reference block, and calculating the sum of the absolute errors between the candidate predicted pixel value of each pixel in the coding block and the original pixel value of each pixel in the coding block.

[0016] Thirdly, embodiments of this application provide a computer device, a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method described in the first aspect.

[0017] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the method described in the first aspect.

[0018] In this embodiment, when searching for reference blocks corresponding to each coded block in a live image in inter-frame prediction mode, for the first live image with a frame type of bidirectional difference frame, candidate predicted pixel values ​​of each pixel in the coded block are obtained based on the pixel values ​​of each reference pixel in the searched reference block. The sum of absolute errors between the candidate predicted pixel values ​​and the original pixel values ​​of each pixel in the coded block is calculated to obtain the first error information corresponding to the reference block. Then, the first error information corresponding to the reference block is used to determine which reference block is the target reference block. Since the calculation process of the sum of absolute errors is simpler and more efficient, and the first live image with a frame type of bidirectional difference frame has lower importance and a larger number in the live video, in inter-frame prediction mode, by improving the efficiency of searching for target reference blocks corresponding to each coded block in the first live image, the overall coding efficiency of the live video can be improved. Furthermore, the impact on the coding quality of the live video is small, and the clarity and smoothness of the live video during playback can be guaranteed at the same time, improving the user's live streaming experience.

[0019] To better understand and implement this application, the technical solution is described in detail below with reference to the accompanying drawings. Attached Figure Description

[0020] Figure 1 A schematic diagram illustrating an application scenario of the reference block search method in live video encoding provided in this application embodiment;

[0021] Figure 2 A schematic diagram illustrating another application scenario of the reference block search method in live video encoding provided in the embodiments of this application;

[0022] Figure 3 A flowchart illustrating the reference block search method in live video encoding provided in the first embodiment of this application;

[0023] Figure 4 Another flowchart illustrating the reference block search method in live video encoding provided in the first embodiment of this application;

[0024] Figure 5 This is a flowchart illustrating step S106 of the reference block search method in the live video encoding provided in the first embodiment of this application.

[0025] Figure 6 This is a flowchart illustrating step S107 of the reference block search method in the live video encoding provided in the first embodiment of this application.

[0026] Figure 7 This is a flowchart illustrating step S108 of the reference block search method in the live video encoding provided in the first embodiment of this application.

[0027] Figure 8 A schematic diagram of the reference block search device in live video encoding provided in the second embodiment of this application;

[0028] Figure 9 This is a schematic diagram of the structure of a computer device provided in the third embodiment of this application. Detailed Implementation

[0029] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0030] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0031] It should be understood that although the terms first, second, third, etc., may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0032] Those skilled in the art will understand that the terms "client," "terminal," and "terminal device" as used in this application include both devices that are wireless signal receivers, which are devices that only have wireless signal receiver capabilities without transmission capabilities, and devices that have receiving and transmitting hardware, which have receiving and transmitting hardware capable of bidirectional communication on a bidirectional communication link. Such devices may include: cellular or other communication devices such as personal computers or tablets, which have single-line displays or multi-line displays or cellular or other communication devices without multi-line displays; PCS (Personal Communications Service), which can combine voice, data processing, fax, and / or data communication capabilities; PDA (Personal Digital Assistant), which may include a radio frequency receiver, pager, Internet / intranet access, web browser, notepad, calendar, and / or GPS (Global Positioning System) receiver; and conventional laptop and / or handheld computers or other devices that have and / or include radio frequency receivers. As used herein, "client," "terminal," and "terminal device" can be portable, transportable, installed in a means of transportation (air, sea, and / or land), or suitable and / or configured to operate locally and / or in a distributed manner, operating in any other location on Earth and / or in space. "Client," "terminal," and "terminal device" as used herein can also be a communication terminal, an internet access terminal, or a music / video playback terminal, such as a PDA, a MID (Mobile Internet Device), and / or a mobile phone with music / video playback capabilities, or a smart TV, set-top box, etc.

[0033] The hardware referred to by the names "server," "client," and "service node" in this application is essentially a computer device with the equivalent capabilities of a personal computer. It is a hardware device with the necessary components revealed by the von Neumann architecture, such as a central processing unit (including an arithmetic logic unit and a control unit), memory, input devices, and output devices. The computer program is stored in its memory, and the central processing unit loads the program stored in the secondary storage into the main memory to run it, execute the instructions in the program, and interact with the input and output devices to complete specific functions.

[0034] It should be noted that the concept of "server" used in this application can also be extended to the case of server clusters. Based on the network deployment principles understood by those skilled in the art, the servers should be logically divided. Physically, these servers can be independent of each other but accessible through interfaces, or they can be integrated into a single physical computer or a computer cluster. Those skilled in the art should understand this flexibility and should not use it to constrain the implementation of the network deployment method in this application.

[0035] Please see Figure 1 , Figure 1 This is a schematic diagram of an application scenario for the reference block search method in live video encoding provided in this application embodiment. The application scenario includes the broadcaster client 101, server 102, and viewer client 103 provided in this application embodiment. The broadcaster client 101 and the viewer client 103 interact through the server 102.

[0036] The client proposed in this application embodiment includes the broadcaster client 101 and the viewer client 103.

[0037] It should be noted that there are multiple interpretations of the concept of "client" in the existing technology. For example, it can be understood as an application installed on a computer device, or it can be understood as a hardware device corresponding to a server.

[0038] In the embodiments of this application, the term "client" refers to a hardware device corresponding to a server, and more specifically, to a computer device, such as a smartphone, a smart interactive whiteboard, and a personal computer.

[0039] When the client is a mobile device such as a smartphone or smart interactive whiteboard, users can install a matching mobile application on the client or access a web application on the client.

[0040] When the client is a non-mobile device such as a personal computer (PC), the user can install the matching PC application on the client, and can also access the web application on the client.

[0041] Among them, mobile applications refer to applications that can be installed on mobile devices, PC applications refer to applications that can be installed on non-mobile devices, and web applications refer to applications that need to be accessed through a browser.

[0042] Specifically, web applications can be divided into mobile and PC versions depending on the client type, and the page layout and server support provided may differ between the two.

[0043] In this embodiment, the live streaming applications provided to users are categorized into mobile live streaming applications, PC live streaming applications, and web live streaming applications. Users can choose their preferred method of participating in live streaming based on the type of client they use.

[0044] Based on the different user identities used by the client, this application can divide the client into broadcast client 101 and viewer client 103.

[0045] Among them, the broadcaster client 101 refers to the end that sends live video, which is usually the client used by the broadcaster (i.e., the live broadcaster user) in the live broadcast.

[0046] Viewer client 103 refers to the end that receives and watches live online videos. It is usually the client used by viewers (i.e., live stream viewers) watching videos in a live stream.

[0047] The hardware referred to by the broadcaster client 101 and the viewer client 103 essentially refers to computer equipment, specifically, such as... Figure 1 As shown, it can be a computer device such as a smartphone, a smart interactive whiteboard, or a personal computer. Both the broadcaster client 101 and the viewer client 103 can access the Internet through known network access methods and establish a data communication link with the server 102.

[0048] Server 102, acting as a business server, can further connect to related audio data servers, video streaming servers, and other servers providing related support, thus forming a logically interconnected service cluster to serve related terminal devices, such as… Figure 1 The broadcaster client 101 and the viewer client 103 shown provide services.

[0049] In this embodiment, the broadcaster client 101 and the viewer client 103 can join the same live broadcast room (i.e., live broadcast channel). The aforementioned live broadcast room refers to a chat room implemented using Internet technology, which typically has audio and video playback control functions. The broadcaster user conducts live broadcasts in the live broadcast room through the broadcaster client 101, and the viewers of the viewer client 103 can log in to the server 102 to enter the live broadcast room to watch the live broadcast.

[0050] Specifically, the broadcaster logs into the server 102 through the broadcaster client 101, which triggers the broadcaster client 101 to load the broadcasting interface. The broadcasting interface displays broadcasting controls, and the broadcaster can start the live broadcast by clicking the broadcasting controls. If the current mode is video live streaming, the broadcaster client 101 will be triggered to capture the live video.

[0051] The live video is video data collected by a camera that has established a data connection with the broadcaster client 101. This camera can be the built-in camera of the broadcaster client 101 or an external camera connected to the broadcaster client 101.

[0052] The broadcast client 101 encodes the captured live video and pushes the encoded live video to the server 102.

[0053] If a viewer enters the live stream room created by the streamer through the viewer client 103, the viewer client 103 will be triggered to retrieve the encoded live stream video from the server 102, decode it, and output it to the live stream room interface so that the viewer can watch the live stream video in the live stream room.

[0054] There are no restrictions on how to enter the live stream room created by this streamer. Viewers can enter the live stream room created by this streamer through the live stream room recommendation page, manually search for the live stream room, or swipe up and down on the live stream room interface.

[0055] Please see Figure 2 , Figure 2 This is a schematic diagram illustrating another application scenario of the reference block search method in live video encoding provided in the embodiments of this application. Figure 2 In this context, server 102 is a server cluster, which includes at least a business server 1021 and a streaming media server 1022. The business server 1021 is responsible for providing services related to the live streaming business logic, and the streaming media server 1022 is responsible for providing services related to streaming media data. The live video is a type of streaming media data.

[0056] The camera that establishes a data connection with the broadcaster client 101 captures live video. After the broadcaster client 101 encodes the live video, it pushes the encoded live video to the streaming media server 1022. After the viewer client 103 joins the live room created by the broadcaster, it pulls the encoded live video from the streaming media server 1022.

[0057] This application provides a reference block search method in live video encoding. Please refer to... Figure 3 , Figure 3 The flowchart illustrates the reference block search method in live video encoding provided in the first embodiment of this application. The method includes the following steps:

[0058] S101: Acquire live video; wherein, the live video includes several frames of live images.

[0059] S102: When searching for several reference blocks corresponding to each coding block in the live image in the inter-frame prediction mode, determine whether the currently searched live image is the first live image; wherein, the coding block is obtained by dividing the live image, and the frame type corresponding to the first live image is a bidirectional difference frame.

[0060] S103: If so, in the inter-frame prediction mode, the target reference block corresponding to each coding block in the first live image is determined according to the first error information corresponding to several searched reference blocks; wherein, the first error information corresponding to the reference block is obtained by obtaining the candidate predicted pixel value of each pixel in the coding block according to the pixel value of each reference pixel in the reference block, and calculating the absolute error between the candidate predicted pixel value of each pixel in the coding block and the original pixel value of each pixel in the coding block.

[0061] In this embodiment, the method for searching reference blocks in live video encoding is described using the broadcaster client as the execution subject.

[0062] Regarding step S101, the broadcaster's client obtains the live video.

[0063] The live video is video data collected by a camera that has established a data connection with the broadcaster's client. This camera can be the broadcaster's built-in camera or an external camera connected to the broadcaster's client.

[0064] The live video contains several frames of live images.

[0065] Regarding steps S102 to S103, when searching for several reference blocks corresponding to each coded block in the live image in the inter-frame prediction mode, the broadcast client determines whether the currently searched live image is the first live image. If so, in the inter-frame prediction mode, the target reference block corresponding to each coded block in the first live image is determined according to the first error information corresponding to the several searched reference blocks.

[0066] Before explaining steps S102 to S103, let's first explain what a coding block is, what a prediction mode is, and why we need to obtain the target reference block corresponding to each coding block in the live image under different prediction modes.

[0067] (1) The coding block is obtained by dividing the live image. In an optional embodiment, the size of the division can be 64x64, resulting in a 64x64 coding block. That is, the coding block consists of 64 rows and 64 columns of pixels.

[0068] It is understood that the broadcaster's client may further divide the 64x64 encoded block into smaller encoded blocks and perform quantization and other processing on each one, but this will not be specified in detail here.

[0069] If the live image is in YUV format, the pixel value of each pixel is represented by the luminance component Y and the chrominance components U and V. Therefore, the encoding block includes a luminance block and a chrominance block. The luminance block and the chrominance block are quantized and processed separately. In this embodiment, the encoding block will not be divided into luminance block and chrominance block in detail, so as to facilitate the explanation and understanding of the subsequent quantization encoding process.

[0070] (2) Prediction modes include inter-frame prediction mode and intra-frame prediction mode.

[0071] Inter-frame prediction mode searches for the target reference block corresponding to the coded block in the preceding and following live images, while intra-frame prediction mode searches for the target reference block corresponding to the coded block in the current live image.

[0072] In an optional embodiment, the inter-frame prediction modes include Merge mode and Inter mode, and the intra-frame prediction modes include 35 types, namely DC mode, Planar mode and 33 angle modes.

[0073] For a certain coded block in a frame of a live image, there is a corresponding target reference block in different prediction modes.

[0074] (3) Why is it necessary to obtain the target reference block corresponding to each coding block in the live image under different prediction modes?

[0075] The target reference block can be understood as the reference block closest to the coding block. Based on the pixel values ​​of each reference pixel in the target reference block, the predicted pixel values ​​of each pixel in the coding block are obtained. This results in a smaller residual information between the predicted pixel values ​​and the original pixel values ​​of each pixel in the coding block. Consequently, after subsequent transformation and quantization operations on the residual information, fewer bits are required to obtain the quantization information, thus effectively improving the video compression rate.

[0076] The following explains how to obtain the target reference block corresponding to each coded block in a live image under different prediction modes.

[0077] For intra-frame prediction mode, the target reference block corresponding to each coded block is determined by the broadcast client in the current live image.

[0078] Generally, the target reference block corresponding to an coded block includes the pixels in the row above and column to the left of that coded block in the live image. More precisely, for an NxN coded block, the target reference block consists of the N pixels directly above it, the N pixels to its upper right, the N pixels to its left, the N pixels to its lower left, and the 1 pixel in its upper left corner.

[0079] It is understandable that in different intra-frame prediction modes, different calculation methods will be used to obtain the predicted pixel values ​​of each pixel in the coding block based on the pixel values ​​of each reference pixel in the target reference block.

[0080] In this embodiment, the broadcast client does not determine whether the searched live image is the first live image in intra-frame prediction mode. In the same intra-frame prediction mode, the broadcast client uses the same method to determine the target reference block corresponding to each coding block for all live images.

[0081] For inter-frame prediction mode, the target reference block corresponding to each coded block is determined by the broadcast client in the live image of the preceding and following frames.

[0082] In this embodiment, when searching for several reference blocks corresponding to each coded block in the live image in the inter-frame prediction mode, the broadcast client will determine whether the currently searched live image is the first live image. If so, the broadcast client will determine the target reference block corresponding to each coded block in the first live image based on the first error information corresponding to the several searched reference blocks in the inter-frame prediction mode.

[0083] The first live image corresponds to a two-way difference frame.

[0084] In the field of video coding, I-frames are keyframes, which do not require reference to other images during encoding. P-frames are difference frames, which require the use of previous frames during encoding. B-frames are bidirectional difference frames, which require the use of preceding and following frames during encoding.

[0085] In this embodiment, keyframes, difference frames, and bidirectional difference frames are all different frame types. The first live broadcast image is a bidirectional difference frame, also known as a B-frame.

[0086] The first error information corresponding to the target reference block is the smallest.

[0087] The first error information corresponding to the reference block is obtained by summing the absolute errors between the candidate predicted pixel values ​​of each pixel in the coding block and the original pixel values ​​of each pixel in the coding block, based on the pixel values ​​of each reference pixel in the reference block.

[0088] The calculation process of the first error information (SAD, Sum of Absolute Difference) is a relatively simple error calculation method. Therefore, it can shorten the time to determine the target reference block corresponding to the coding block, thereby improving the coding efficiency of live video.

[0089] In this embodiment, there is no limitation on how to search for the target reference block corresponding to the coding block in each inter-frame prediction mode for the second live image other than the first live image.

[0090] In this embodiment, when searching for reference blocks corresponding to each coded block in a live image in inter-frame prediction mode, for the first live image with a frame type of bidirectional difference frame, candidate predicted pixel values ​​of each pixel in the coded block are obtained based on the pixel values ​​of each reference pixel in the searched reference block. The sum of absolute errors between the candidate predicted pixel values ​​and the original pixel values ​​of each pixel in the coded block is calculated to obtain the first error information corresponding to the reference block. Then, the first error information corresponding to the reference block is used to determine which reference block is the target reference block. Since the calculation process of the sum of absolute errors is simpler and more efficient, and the first live image with a frame type of bidirectional difference frame has lower importance and a larger number in the live video, in inter-frame prediction mode, by improving the efficiency of searching for target reference blocks corresponding to each coded block in the first live image, the overall coding efficiency of the live video can be improved. Furthermore, the impact on the coding quality of the live video is small, and the clarity and smoothness of the live video during playback can be guaranteed at the same time, improving the user's live streaming experience.

[0091] In an optional embodiment, step S103, in the inter-frame prediction mode, determines the target reference block corresponding to each coding block in the first live image based on the first error information corresponding to several searched reference blocks, including the following steps:

[0092] The broadcast client obtains the optimization search enable parameters, and determines whether to optimize the target reference block search process based on the optimization search enable parameters. If so, in the inter-frame prediction mode, the target reference block corresponding to each coding block in the first live image is determined based on the first error information corresponding to several searched reference blocks.

[0093] In an optional embodiment, the broadcaster client responds to the parameter configuration instruction sent by the server, parses the parameter configuration instruction, and obtains the optimized search enabling parameters.

[0094] In other words, backend personnel can customize the configuration to enable reference block search optimization for different broadcaster clients during the video encoding process.

[0095] In another alternative embodiment, network latency information and / or client load information are obtained, and optimized search enable parameters are derived based on the network latency information and / or client load information.

[0096] Specifically, the network latency information and / or client load information of the broadcast client are used to dynamically adjust and optimize the search activation parameters. When the broadcast client load is low and the network latency is low, the optimization search activation parameter is set to the first parameter value to indicate that the search process of the target reference block is not optimized. When the broadcast client load is high and the network latency is high, the optimization search activation parameter is set to the second parameter value to indicate the search process of the target reference block is optimized.

[0097] In this embodiment, by dynamically adjusting and optimizing the search activation parameters, it is possible to more reasonably determine whether to optimize the search process for the target reference block based on the real-time network status and client load. This allows for a relatively longer time to determine the most accurate nearest reference block when the broadcaster's client load is low and network latency is low. Conversely, when the broadcaster's client load is high and network latency is high, it allows for a relatively shorter time to search for the nearest reference block without affecting video encoding quality, thereby improving video encoding efficiency and ensuring smooth video playback.

[0098] In an optional embodiment, if the currently searched live image is the second live image, the method further includes step S104, as follows:

[0099] S104: In inter-frame prediction mode, based on the second error information corresponding to several searched reference blocks, determine the target reference block corresponding to each coding block in the second live image; wherein, the second live image is a live image other than the first live image; the second error information corresponding to the reference block is obtained by obtaining the candidate predicted pixel value of each pixel in the coding block based on the pixel value of each reference pixel in the reference block, calculating the first difference between the candidate predicted pixel value of each pixel in the coding block and the original pixel value of each pixel in the coding block, performing a Hadema transform on the first difference to obtain the first Hadema transform value, and then summing the absolute values ​​of the first Hadema transform values.

[0100] In this embodiment, if the currently searched live image is the second live image, then the broadcaster client, in the inter-frame prediction mode, determines the target reference block corresponding to each coded block in the second live image based on the second error information corresponding to several searched reference blocks.

[0101] The second live image is a live image other than the first live image, and the frame type of the second live image is not a bidirectional difference frame.

[0102] The second error information (SATD, Sum of Absolute Transformed Difference) is obtained by acquiring the candidate predicted pixel values ​​of each pixel in the coding block based on the pixel values ​​of each reference pixel in the reference block, calculating the first difference between the candidate predicted pixel values ​​and the original pixel values ​​of each pixel in the coding block, performing a Hademar transform on the first difference to obtain the first Hademar transform value, and then summing the absolute values ​​of the first Hademar transform values.

[0103] The calculation process for the second error information is relatively complex, and the Hadema transformation operation in the process is time-consuming.

[0104] It should also be noted that in each inter-frame prediction mode, in order to facilitate subsequent encoding, the target reference block corresponding to each coded block in the live image is not actually directly output. Instead, the target motion vector corresponding to each coded block in the live image and the frame in which the target reference block is located (also known as the reference frame) are output. As long as the target motion vector can be used to search for the target reference block corresponding to the coded block in the reference frame, it is sufficient.

[0105] Therefore, obtaining the target reference block corresponding to each coded block in the live image under each inter-frame prediction mode means obtaining the target motion vector corresponding to each coded block in the live image under each inter-frame prediction mode.

[0106] Obtaining the target reference block corresponding to each coded block in the live image under each inter-frame prediction mode includes obtaining the target reference block corresponding to each coded block in the first live image under each inter-frame prediction mode and obtaining the target reference block corresponding to each coded block in the second live image under each inter-frame prediction mode.

[0107] The following section will explain how to determine the optimal prediction mode after finding the target reference block corresponding to each coding block, and how to perform video coding under the optimal prediction mode.

[0108] In an optional embodiment, please refer to Figure 4 The method also includes the following steps:

[0109] S105: Obtain the quantization parameters corresponding to each frame of the live image.

[0110] S106: In each prediction mode, the residual information corresponding to each coding block is quantized according to the corresponding quantization parameters to obtain the quantization information corresponding to each coding block in each prediction mode; wherein, the prediction mode includes intra-frame prediction mode and inter-frame prediction mode, and the residual information corresponding to the coding block is obtained by obtaining the predicted pixel value of each pixel in the coding block according to the pixel value of each reference pixel in the target reference block corresponding to the coding block, and calculating the residual value between the predicted pixel value of each pixel in the coding block and the original pixel value of each pixel in the coding block.

[0111] S107: Based on the quantization information corresponding to each coding block under each prediction mode, obtain the distortion information corresponding to each coding block under each prediction mode.

[0112] S108: Obtain the prediction bit information corresponding to each coding block under each prediction mode, and obtain the rate-distortion optimization information corresponding to each coding block under each prediction mode based on the distortion information and prediction bit information corresponding to each coding block under each prediction mode; wherein, the prediction bit information is the bit information required for the predicted coding block.

[0113] S109: Obtain the target prediction mode corresponding to each coding block, and encode each coding block according to the target prediction mode and the quantization information of the coding block under the target prediction mode to obtain the encoded live video; among them, the rate-distortion optimization information corresponding to the coding block under the target prediction mode is the smallest.

[0114] Regarding step S105, the broadcaster client obtains the quantization parameters corresponding to each frame of the live broadcast image.

[0115] Quantization refers to the process of mapping continuous values ​​(or a large number of discrete values) of a signal to a finite number of discrete amplitude values, achieving a many-to-one mapping of signal values. Therefore, quantization can reduce the value space of a signal, thereby achieving better compression results.

[0116] In this embodiment, live video is essentially a signal. Quantizing several frames of live images in the live video and then encoding them can reduce redundant information to be encoded and shorten the length of the image encoding without reducing the visual effect, thereby improving the encoding effect of the live video.

[0117] Since quantization maps signal values ​​to a many-to-one ratio, it causes a loss of pixel values ​​in the live image, resulting in distortion. Therefore, it is necessary to set the quantization parameters appropriately.

[0118] A smaller quantization parameter preserves more details of the live image, reduces distortion, and increases the bit rate required to encode the live image; a larger quantization parameter results in the loss of details, increases distortion, and decreases the bit rate required to encode the live image.

[0119] In this embodiment of the application, the broadcaster client obtains the quantization parameters corresponding to each frame of the live image. The quantization parameter is not a fixed value, but is adjusted according to the live image of different frames.

[0120] The following explains how to obtain the quantization parameters corresponding to each frame of the live broadcast image. In an optional embodiment, the broadcaster client obtains the first bit rate information, the complexity information of each frame of the live broadcast image, and the importance information of each frame of the live broadcast image. Based on the first bit rate information, the complexity information of each frame of the live broadcast image, and the importance information of each frame of the live broadcast image, the quantization parameters corresponding to each frame of the live broadcast image are obtained.

[0121] The first bit rate information is the number of bits transmitted per unit time, measured in bps. In this technical field, bit rate and code rate are the same concept.

[0122] The first bitrate information is used to macroscopically regulate the bitrate information that can be allocated to each frame of the live broadcast image.

[0123] Each frame of a live stream image has varying degrees of importance and complexity due to its position within the live video and the amount of information it carries. To more effectively adjust the quantization parameters, the broadcaster's client needs to obtain information on the complexity and importance of each frame.

[0124] If the complexity and importance of the live image are smaller, then the quantization parameter corresponding to the live image is larger. The larger the quantization parameter corresponding to the live image, the smaller the bit rate information allocated to the live image. In the average bit rate mode, the average bit rate information allocated to each frame of the live image will not exceed the first bit rate information.

[0125] In this embodiment, the broadcast client adjusts the quantization parameters corresponding to each live image frame based on the first bitrate information, the complexity information of each live image frame, and the importance information of each live image frame. This allows more important and complex live images to have smaller quantization parameters, thereby reducing the loss of details in the live image during quantization and increasing the bitrate information allocated to the live image.

[0126] Regarding step S106, the residual information corresponding to each coding block is quantized according to the corresponding quantization parameters under each prediction mode to obtain the quantization information corresponding to each coding block under each prediction mode.

[0127] As mentioned above, prediction modes include intra-frame prediction mode and inter-frame prediction mode.

[0128] The residual information corresponding to the coding block is obtained by obtaining the predicted pixel value of each pixel in the coding block based on the pixel value of each reference pixel in the target reference block corresponding to the coding block, and calculating the residual value between the predicted pixel value of each pixel in the coding block and the original pixel value of each pixel in the coding block.

[0129] The pixel value of each reference pixel in the target reference block refers to the reconstructed pixel value of each reference pixel.

[0130] The method for obtaining the reconstructed pixel values ​​of each reference pixel in the target reference block will be shown in step S107. This is because the target reference block has been encoded. Before encoding, the target prediction mode (optimal prediction mode) corresponding to the target reference block must have been determined. This means that the distortion information corresponding to the target reference block under the target prediction mode has been calculated. In the process of calculating the distortion information, the reconstructed pixel values ​​of the reference pixels can be obtained.

[0131] In an optional embodiment, please refer to Figure 5 S106 includes steps S1061 to S1064, as follows:

[0132] S1061: Obtain the target reference block corresponding to each coding block in each prediction mode.

[0133] S1062: Based on the pixel values ​​of each reference pixel in the target reference block corresponding to the coding block, obtain the predicted pixel values ​​of each pixel in the coding block under each prediction mode.

[0134] S1063: Based on the residual values ​​between the predicted pixel values ​​of each pixel in the coding block and the original pixel values ​​of each pixel in the coding block, obtain the residual information corresponding to the coding block under each prediction mode; wherein, the residual information corresponding to the coding block includes the residual values ​​corresponding to each pixel in the coding block.

[0135] S1064: Perform transformation and quantization operations on the residual information corresponding to the coding block under each prediction mode in sequence to obtain the quantization information corresponding to each coding block under each prediction mode.

[0136] Regarding step 1062, the broadcast client predicts the pixel values ​​of each pixel in the coding block based on the pixel values ​​of each reference pixel in the target reference block, thus obtaining the predicted pixel values ​​of each pixel in the coding block. The specific prediction process may vary depending on the prediction mode, and is not limited here.

[0137] Regarding step S1063, the broadcast client obtains the residual information corresponding to the coding block under each prediction mode based on the residual value between the predicted pixel value of each pixel in the coding block and the original pixel value of each pixel in the coding block.

[0138] The residual information corresponding to the coding block includes the residual value corresponding to each pixel in the coding block.

[0139] Regarding step S1064, the broadcast client sequentially performs transformation and quantization operations on the residual information corresponding to the coding blocks under each prediction mode to obtain the quantization information corresponding to each coding block under each prediction mode.

[0140] The transform operations under the HEVC standard are divided into Discrete Cosine Transform (DCT) and Discrete Sine Transform (DST). Among them, Discrete Sine Transform (DST) is only used to process 4x4 luma blocks in intra-frame prediction mode. For the meaning of luma blocks, please refer to the above.

[0141] Specifically, the broadcast client first performs a transformation operation on the residual information corresponding to the encoded block, and then performs a quantization operation to obtain the quantized information corresponding to the encoded block.

[0142] The transformation and quantization processes are existing technologies in the field of video coding and will not be discussed in detail here.

[0143] Regarding step S107, the broadcaster client obtains the distortion information corresponding to each coding block under each prediction mode based on the quantization information corresponding to each coding block under each prediction mode.

[0144] In an optional embodiment, the broadcaster client can use a unified method to obtain the distortion information corresponding to each coding block under each prediction mode for each frame of live image.

[0145] In another optional embodiment, the broadcast client can optimize the process of obtaining the distortion information corresponding to each coding block under each prediction mode for the third live image with frame type non-reference bidirectional difference frame.

[0146] The following section will first explain how to obtain the distortion information corresponding to the coded block under normal circumstances. Please refer to [link / reference]. Figure 6 S107 includes steps S1071 to S1074, as follows:

[0147] S1071: Perform dequantization on the quantization information corresponding to each coding block under each prediction mode to obtain the dequantization information corresponding to each coding block under each prediction mode.

[0148] S1072: Perform an inverse transform operation on the inverse quantization information corresponding to each coding block under each prediction mode to obtain the inverse transform information corresponding to each coding block under each prediction mode; wherein, the inverse transform information corresponding to the coding block includes the inverse transform value corresponding to each pixel in the coding block.

[0149] S1073: Based on the inverse transform value corresponding to each pixel in the coding block under each prediction mode and the predicted pixel value of each pixel in the coding block under each prediction mode, obtain the reconstructed pixel value of each pixel in the coding block under each prediction mode.

[0150] S1074: Based on the second difference between the reconstructed pixel value of each pixel in the coding block under each prediction mode and the original pixel value of each pixel in the coding block under each prediction mode, perform a Hadema transform on the second difference to obtain the second Hadema transform value. Then, sum the absolute values ​​of the second Hadema transform values ​​to obtain the distortion information corresponding to each coding block under each prediction mode.

[0151] Regarding step S1071, the broadcast client performs dequantization on the quantization information corresponding to each coding block under each prediction mode to obtain the dequantization information corresponding to each coding block under each prediction mode.

[0152] In this context, inverse quantization is the reverse process of quantization.

[0153] Regarding step S1072, the broadcast client performs an inverse transformation operation on the inverse quantization information corresponding to each coding block under each prediction mode to obtain the inverse transformation information corresponding to each coding block under each prediction mode.

[0154] The inverse transform information corresponding to the coded block includes the inverse transform value corresponding to each pixel in the coded block.

[0155] The inverse transformation operation is the reverse process of the transformation operation.

[0156] Regarding step S1073, the broadcast client obtains the reconstructed pixel values ​​of each pixel in the coding block under each prediction mode based on the inverse transform value corresponding to each pixel in the coding block under each prediction mode and the predicted pixel values ​​of each pixel in the coding block under each prediction mode.

[0157] Specifically, the broadcaster client obtains the reconstructed pixel value of each pixel in the coding block under each prediction mode by summing the inverse transform value corresponding to each pixel in the coding block under each prediction mode with the predicted pixel value of each pixel in the coding block under each prediction mode.

[0158] This corresponds to what was mentioned earlier: in the process of obtaining the distortion information corresponding to each coding block under each prediction mode, the reconstructed pixel values ​​of each pixel in the coding block under each prediction mode will be obtained.

[0159] Regarding step S1074, the broadcast client performs a Hademar transform on the second difference between the reconstructed pixel value of each pixel in the coding block under each prediction mode and the original pixel value of each pixel in the coding block under each prediction mode, and then sums the absolute values ​​of the second Hademar transform values ​​to obtain the distortion information corresponding to each coding block under each prediction mode.

[0160] The following describes how to optimize the process of obtaining the distortion information corresponding to each coding block under each prediction mode for a third live image with a frame type of non-reference bidirectional difference frame.

[0161] Since the frame type of the third live image is a non-reference bidirectional difference frame, it will not be used as a reference image for other live images to search for the target reference block. Therefore, there will be no situation where the target reference block is found from the third live image, and consequently, there will be no situation where the pixel values ​​corresponding to each reference pixel in the target reference block are used to obtain the predicted pixel values ​​corresponding to each pixel in the coding block.

[0162] Based on the above, for a third live image with a frame type of non-reference bidirectional difference frame, the distortion information corresponding to each coding block in each prediction mode can be obtained by the difference between the quantization information corresponding to the coding block in each prediction mode and the inverse quantization information corresponding to the coding block in each prediction mode. This simplifies the process of obtaining the distortion information corresponding to each coding block in the third live image. Of course, at this time, it is impossible to obtain the reconstructed pixel value corresponding to each pixel in each coding block in the third live image. However, this will not affect the coding quality of the live video and can also shorten the calculation time and further improve the video coding efficiency.

[0163] Regarding step S108, the broadcaster client obtains the prediction bit information corresponding to each coding block under each prediction mode, and obtains the rate-distortion optimization information corresponding to each coding block under each prediction mode based on the distortion information and prediction bit information corresponding to each coding block under each prediction mode.

[0164] Rate-distortion optimization (RDO) information includes distortion information and prediction bit information.

[0165] RDO = D + λ * R, where D is the distortion information, R is the prediction bit information, and λ is the adjustment parameter determined experimentally.

[0166] The method for obtaining distortion information has already been explained; the following section explains how to obtain the prediction bit information.

[0167] The predicted bit information is the bit information required to encode the coded block. Because different prediction modes use different methods to search for the target reference block corresponding to the coded block, and the target reference block found may also be different, the data used to encode the coded block is different under different prediction modes, and therefore, the bit information required to encode the coded block is also different.

[0168] In an optional embodiment, please refer to Figure 7 In step S108, the prediction bit information corresponding to each coding block under each prediction mode is obtained, including the following steps:

[0169] S1081: Obtain the first parameter to be encoded corresponding to each coding block under each prediction mode; wherein, the first parameter to be encoded is used to obtain the target reference block corresponding to the coding block.

[0170] S1082: Based on the first coding parameter corresponding to each coding block under each prediction mode and the quantization information corresponding to each coding block under each prediction mode, predict the bit information required to encode the coding block under each prediction mode.

[0171] Regarding step S1081, the broadcaster client obtains the first parameter to be encoded corresponding to each coding block under each prediction mode.

[0172] The first parameter to be encoded is used to identify the target reference block corresponding to the encoding block, and the pixel value of each reference pixel in the target encoding block is used to obtain the predicted pixel value of each pixel in the encoding block.

[0173] The first parameter to be encoded that needs to be transmitted may differ depending on the prediction mode.

[0174] For example, in order to find the target reference block corresponding to the coded block using the inter-frame prediction mode, the first parameters to be transmitted include which frame of the live image (also called the reference frame) the target reference block is in, which inter-frame prediction mode it is, and the motion vector (more precisely, the index of the motion vector to be transmitted), etc.

[0175] It is understandable that different inter-frame prediction modes may have different methods for determining the target reference block corresponding to the coded block, and the contents included in the first parameter to be encoded may also increase or decrease accordingly. No detailed restrictions are made here.

[0176] For intra-prediction mode, the first parameter to be encoded includes which intra-prediction mode it is. This is because intra-prediction mode determines the target reference block within the current live image, and it does not need to determine the target reference block based on motion vectors.

[0177] Regarding step S1082, the broadcast client obtains the bit information required to encode the coding block under each prediction mode based on the first to-be-encoded parameter corresponding to each coding block under each prediction mode and the quantization information corresponding to each coding block under each prediction mode.

[0178] The encoding process is the process of converting the data to be encoded into a binary code stream. Therefore, the bit information required to encode the code block in step S1082 can be understood as the number of bits occupied by transmitting the binary code stream.

[0179] Since the first encoding parameter and the quantization information of each coding block may be different under different prediction modes, the bit information required to encode the coding block may also be different. Therefore, it is necessary to obtain the bit information required to encode the coding block under each prediction mode based on the first encoding parameter and the quantization information of each coding block under each prediction mode.

[0180] In an optional embodiment, the broadcaster client can also quantize the quantization parameters corresponding to the encoding block to obtain the quantization information corresponding to the quantization parameters and the second parameter to be encoded corresponding to the encoding block.

[0181] The second parameter to be encoded corresponding to the coded block is used to determine which reference blocks' quantization parameters can be used to predict the quantization parameters of the coded block.

[0182] The first parameter to be encoded corresponding to the coded block, the quantization information corresponding to the coded block, the quantization information corresponding to the quantization parameter, and the second parameter to be encoded corresponding to the coded block are then encoded together to obtain the bit information required to encode the coded block. It can be understood that performing the above process under different prediction modes will yield the bit information required to encode the coded block under different prediction modes.

[0183] Regarding step S109, the broadcaster client obtains the target prediction mode corresponding to each coding block, and encodes each coding block according to the target prediction mode and the quantization information corresponding to the coding block under the target prediction mode to obtain the encoded live video.

[0184] Among them, the rate-distortion optimization information corresponding to the coding block in the target prediction mode is the smallest.

[0185] After obtaining the target prediction mode corresponding to each coding block, the broadcast client encodes each coding block according to the target prediction mode and the quantization information of the coding block under the target prediction mode, thus obtaining the encoded coding block, thereby obtaining the encoded live image, and then obtaining the encoded live video.

[0186] Specifically, the broadcaster client obtains the target to-be-encoded parameters corresponding to each encoding block in the target prediction mode; wherein, the target to-be-encoded parameters are used to confirm the target reference block corresponding to the encoding block in the target prediction mode; the pixel values ​​of each reference pixel in the target reference block are used to obtain the predicted pixel values ​​of each pixel in the encoding block.

[0187] The broadcast client encodes the target parameters to be encoded for each coding block in the target prediction mode and the quantization information corresponding to each coding block in the target prediction mode to obtain the encoded live video.

[0188] In an optional embodiment, the broadcaster client can also quantize the quantization parameters corresponding to the encoding block to obtain the quantization information corresponding to the quantization parameters and the second parameter to be encoded corresponding to the encoding block.

[0189] The second parameter to be encoded corresponding to the coded block is used to determine the quantization parameters of the coded block based on the quantization parameters of the reference blocks.

[0190] Then, the broadcast client encodes the target parameters to be encoded corresponding to the coding block in the target prediction mode, the quantization information corresponding to the coding block in the target prediction mode, the quantization information corresponding to the quantization parameters, and the second parameters to be encoded corresponding to the coding block together to obtain the encoded coding block, thereby obtaining the encoded live image, and then obtaining the encoded live video.

[0191] Please see Figure 8 This is a schematic diagram of the reference block search device in live video encoding provided in the second embodiment of this application. This device can be implemented as all or part of a computer device through software, hardware, or a combination of both. The device 8 includes:

[0192] The first acquisition unit 81 is used to acquire live video; wherein, the live video includes several frames of live images;

[0193] The first judgment unit 82 is used to determine whether the currently searched live image is the first live image when searching for several reference blocks corresponding to each coding block in the live image in the inter-frame prediction mode; wherein, the coding block is obtained by dividing the live image, and the frame type corresponding to the first live image is a bidirectional difference frame.

[0194] The first search unit 83 is configured to, if so, determine the target reference block corresponding to each of the coding blocks in the first live image based on the first error information corresponding to the searched reference blocks in the inter-frame prediction mode; wherein, the first error information corresponding to the reference block is obtained by obtaining the candidate predicted pixel value of each pixel in the coding block based on the pixel value of each reference pixel in the reference block, and calculating the sum of the absolute errors between the candidate predicted pixel value of each pixel in the coding block and the original pixel value of each pixel in the coding block.

[0195] It should be noted that the reference block search device in live video encoding provided in the above embodiments is only illustrated by the division of the above functional modules when executing the reference block search method in live video encoding. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the reference block search device and the reference block search method in live video encoding provided in the above embodiments belong to the same concept, and the implementation process is detailed in the method embodiments, which will not be repeated here.

[0196] Please see Figure 9 This is a schematic diagram of the structure of the computer device provided in the third embodiment of this application. Figure 9 As shown, the computer device 9 may include: a processor 90, a memory 91, and a computer program 92 stored in the memory 91 and capable of running on the processor 90, such as a reference block search method program in live video encoding; when the processor 90 executes the computer program 92, it implements the steps in the first embodiment described above.

[0197] The processor 90 may include one or more processing cores. The processor 90 connects to various parts within the computer device 9 using various interfaces and lines. It executes various functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 91, and by accessing data in the memory 91. Optionally, the processor 90 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 90 may integrate one or more of the following: a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), and a modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content displayed on the touch screen; and the modem handles wireless communication. It is understood that the modem may also be implemented as a separate chip without being integrated into the processor 90.

[0198] The memory 91 may include random access memory (RAM) or read-only memory. Optionally, the memory 91 may include a non-transitory computer-readable storage medium. The memory 91 may be used to store instructions, programs, code, code sets, or instruction sets. The memory 91 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch instructions), instructions for implementing the various method embodiments described above, etc.; the data storage area may store data involved in the various method embodiments described above, etc. Optionally, the memory 91 may also be at least one storage device located remotely from the aforementioned processor 90.

[0199] This application also provides a computer storage medium that can store multiple instructions. These instructions are applicable to being loaded by a processor and executed by the method steps of the above embodiments. For details of the execution process, please refer to the specific description of the above embodiments, which will not be repeated here.

[0200] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0201] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0202] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0203] In the embodiments provided by this invention, it should be understood that the disclosed apparatus / terminal devices and methods can be implemented in other ways. For example, the apparatus / terminal device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0204] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0205] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0206] If integrated modules / units are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms.

[0207] This invention is not limited to the above-described embodiments. If any modifications or variations to this invention do not depart from the spirit and scope of this invention, and if such modifications and variations fall within the scope of the claims and equivalent technologies of this invention, then this invention also intends to include such modifications and variations.

Claims

1. A reference block search method in live video encoding, characterized in that, The method includes the following steps: Acquire live video; wherein the live video includes several frames of live images; When searching for several reference blocks corresponding to each coding block in the live image in the inter-frame prediction mode, it is determined whether the currently searched live image is the first live image; wherein, the coding block is obtained by dividing the live image, and the frame type corresponding to the first live image is a bidirectional difference frame. If so, in the inter-frame prediction mode, the target reference block corresponding to each of the coding blocks in the first live image is determined based on the first error information corresponding to the searched reference blocks; wherein, the first error information corresponding to the reference block is obtained by obtaining the candidate predicted pixel value of each pixel in the coding block based on the pixel value of each reference pixel in the reference block, and calculating the sum of the absolute errors between the candidate predicted pixel value of each pixel in the coding block and the original pixel value of each pixel in the coding block; If the currently searched live image is a second live image, in the inter-frame prediction mode, the target reference block corresponding to each coding block in the second live image is determined according to the second error information corresponding to the searched reference blocks; wherein, the second live image is a live image other than the first live image, and the frame type of the second live image is a non-bidirectional difference frame; the second error information corresponding to the reference block is obtained by obtaining the candidate predicted pixel value of each pixel in the coding block according to the pixel value of each reference pixel in the reference block, calculating the first difference between the candidate predicted pixel value of each pixel in the coding block and the original pixel value of each pixel in the coding block, performing a Hadma transform on the first difference to obtain a first Hadma transform value, and then summing the absolute values ​​of the first Hadma transform values.

2. The reference block search method in live video encoding according to claim 1, characterized in that, The step of determining the target reference block corresponding to each coding block in the first live image based on the first error information corresponding to several searched reference blocks in the inter-frame prediction mode includes the following steps: Obtain optimization search enable parameters, and determine whether to optimize the search process of the target reference block based on the optimization search enable parameters. If so, in the inter-frame prediction mode, determine the target reference block corresponding to each coding block in the first live image based on the first error information corresponding to the searched reference blocks.

3. The reference block search method in live video encoding according to claim 2, characterized in that, The process of obtaining the optimized search enable parameters includes the following steps: In response to a parameter configuration instruction sent by the server, the parameter configuration instruction is parsed to obtain the optimized search enabling parameters; or, network latency information and / or client load information are obtained, and the optimized search enabling parameters are obtained based on the network latency information and / or the client load information.

4. The reference block search method in live video encoding according to any one of claims 1 to 3, characterized in that, The method further includes the following steps: Obtain the quantization parameters corresponding to each frame of the live image; In each prediction mode, the residual information corresponding to each coding block is quantized according to the corresponding quantization parameters to obtain the quantization information corresponding to each coding block in each prediction mode; wherein, the prediction mode includes intra-frame prediction mode and inter-frame prediction mode, and the residual information corresponding to the coding block is obtained by obtaining the predicted pixel value of each pixel in the coding block according to the pixel value of each reference pixel in the target reference block corresponding to the coding block, and calculating the residual value between the predicted pixel value of each pixel in the coding block and the original pixel value of each pixel in the coding block; Based on the quantization information corresponding to each coding block under each prediction mode, the distortion information corresponding to each coding block under each prediction mode is obtained; Obtain the prediction bit information corresponding to each coding block under each prediction mode, and obtain rate-distortion optimization information corresponding to each coding block under each prediction mode based on the distortion information and the prediction bit information corresponding to each coding block under each prediction mode; wherein, the prediction bit information is the bit information required to predict the coding block. Obtain the target prediction mode corresponding to each of the coding blocks, and encode each of the coding blocks according to the target prediction mode and the quantization information corresponding to the coding block under the target prediction mode to obtain the encoded live video; wherein, the rate-distortion optimization information corresponding to the coding block under the target prediction mode is minimized.

5. The reference block search method in live video encoding according to claim 4, characterized in that, The step of quantizing the residual information corresponding to each coding block according to the corresponding quantization parameters under each prediction mode to obtain the quantization information corresponding to each coding block under each prediction mode includes the following steps: Obtain the target reference block corresponding to each coding block under each prediction mode; Based on the pixel values ​​of each reference pixel in the target reference block corresponding to the coding block, the predicted pixel values ​​of each pixel in the coding block under each prediction mode are obtained; Based on the residual values ​​between the predicted pixel values ​​of each pixel in the coding block and the original pixel values ​​of each pixel in the coding block, residual information corresponding to the coding block under each prediction mode is obtained; wherein, the residual information corresponding to the coding block includes the residual values ​​corresponding to each pixel in the coding block; The residual information corresponding to the coding block under each prediction mode is sequentially transformed and quantized to obtain the quantized information corresponding to each coding block under each prediction mode.

6. The reference block search method in live video encoding according to claim 4, characterized in that, The step of obtaining the distortion information corresponding to each coding block under each prediction mode based on the quantization information corresponding to each coding block under each prediction mode includes the following steps: Perform dequantization on the quantization information corresponding to each coding block under each prediction mode to obtain the dequantization information corresponding to each coding block under each prediction mode; An inverse transform operation is performed on the inverse quantization information corresponding to each coding block under each prediction mode to obtain the inverse transform information corresponding to each coding block under each prediction mode; wherein, the inverse transform information corresponding to the coding block includes the inverse transform value corresponding to each pixel in the coding block; Based on the inverse transform value corresponding to each pixel in the coding block under each prediction mode and the predicted pixel value of each pixel in the coding block under each prediction mode, the reconstructed pixel value of each pixel in the coding block under each prediction mode is obtained; Based on the second difference between the reconstructed pixel value of each pixel in the coding block of each prediction mode and the original pixel value of each pixel in the coding block of each prediction mode, a second Hadma transform is performed on the second difference to obtain a second Hadma transform value. Then, the absolute values ​​of the second Hadma transform values ​​are summed to obtain the distortion information corresponding to each coding block of each prediction mode.

7. A reference block search device for live video encoding, characterized in that, include: The first acquisition unit is used to acquire live video; wherein the live video includes several frames of live images; The first judgment unit is used to determine whether the currently searched live image is the first live image when searching for several reference blocks corresponding to each coding block in the live image in the inter-frame prediction mode; wherein the coding block is obtained by dividing the live image, and the frame type corresponding to the first live image is a bidirectional difference frame. The first search unit is configured to, if yes, in the inter-frame prediction mode, determine the target reference block corresponding to each of the coded blocks in the first live image based on the first error information corresponding to the searched reference blocks; wherein, the first error information corresponding to the reference block is obtained by acquiring the candidate predicted pixel value of each pixel in the coded block based on the pixel value of each reference pixel in the reference block, and calculating the sum of the absolute errors between the candidate predicted pixel value and the original pixel value of each pixel in the coded block; if the currently searched live image is a second live image, in the inter-frame prediction mode, determine the target reference block corresponding to each of the searched reference blocks based on the first error information corresponding to the searched reference blocks. The information determines the target reference block corresponding to each of the coding blocks in the second live image; wherein, the second live image is a live image other than the first live image, and the frame type of the second live image is a non-bidirectional difference frame; the second error information corresponding to the reference block is obtained by obtaining the candidate predicted pixel value of each of the coding blocks according to the pixel value of each of the reference pixels in the reference block, calculating the first difference between the candidate predicted pixel value of each of the coding blocks and the original pixel value of each of the coding blocks, performing a Hadema transform on the first difference to obtain a first Hadema transform value, and then summing the absolute values ​​of the first Hadema transform values.

8. A computer device, comprising: A processor, a memory, and a computer program stored in the memory and executable on the processor, characterized in that the processor, when executing the computer program, implements the steps of the method as claimed in any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 6.

Images