A video encoding method, apparatus, device, and computer storage medium

By using a multi-round iterative optimization and gradient information to adjust motion vectors, the problem of time-consuming division operations in video encoding was solved, enabling fast encoding and transmission and improving the efficiency and smoothness of video encoding.

CN123123581APending Publication Date: 2026-11-17TENCENT TECHNOLOGY (SHENZHEN) CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202510643944.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-05-16
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Existing video encoding methods suffer from time-consuming division operations in hardware implementation, resulting in prolonged video recording and transmission times. This is particularly problematic in live streaming scenarios, where stuttering is common, hindering fast encoding and transmission.

Method used

The initial motion vector is optimized through multiple rounds of iteration, and the motion vector is adjusted using the digital information of gradient information to reduce the difference between the predicted frame and the reference frame. The digital processing of gradient information simplifies the calculation process.

Benefits of technology

It improves the speed and efficiency of video encoding, reduces the consumption of computing resources, avoids hardware timing limitations, and enhances the smoothness of video transmission.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123123581A_ABST
    Figure CN123123581A_ABST
Patent Text Reader

Abstract

This application provides a video encoding method, apparatus, device, and computer storage medium, applied in the field of computer technology, to reduce the computational cost of video encoding and improve the video encoding rate. It includes at least: acquiring initial motion vectors for each target point in a prediction frame between a prediction frame and a reference frame; iteratively optimizing the acquired initial motion vectors, wherein in each optimization round, the following steps are performed: predicting pixel information of each target point in a prediction frame based on the reference frame and the output motion vectors of each previous round, and determining the difference information between the obtained prediction result and a prediction frame; obtaining gradient information describing the degree of influence of each previous round's output motion vector on the difference information based on the difference information and the output motion vectors of each previous round; and adjusting the output motion vectors of each previous round based on the digital information of the gradient information, with the goal of reducing the difference information, to obtain the corresponding output motion vectors of each current round.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a video encoding method, apparatus and device, and computer storage medium. Background Technology

[0002] Video coding technology refers to the process of converting raw video signals into compressed digital formats using coding algorithms, thereby significantly reducing the size of video data while ensuring visual quality.

[0003] In practical applications, the motion changes of an object across multiple consecutive video frames in a video recording can be represented by motion vectors. Therefore, see... Figure 1A As shown, when encoding video recordings, the video recording can be compressed into a reference frame and multiple prediction frames associated with the reference frame. The pixel information in the reference frame is recorded in its entirety, while the pixel information in the other prediction frames can be described by the motion vector of the object between the reference frame and the prediction frames and the corresponding residual information.

[0004] Correspondingly, in the video decoding process, for a prediction frame, motion compensation processing is performed using the pixel information of the reference frame and the motion vector corresponding to the prediction frame to obtain the prediction result for the prediction frame. Then, the residual information is superimposed on the prediction result to decode the pixel information in the prediction frame.

[0005] In the above process, only the pixel information of the reference frame needs to be recorded, while the pixel information of other predicted frames can be encoded into corresponding motion vectors and residual information. Therefore, the amount of data in the encoded video data can be greatly reduced.

[0006] It is evident that motion vector selection is a crucial step in video coding technology. The motion of an object can take many complex forms, such as flipping and rotating, requiring affine motion estimation to determine the corresponding motion vectors.

[0007] In affine motion estimation, the determination of motion vectors is divided into: determining the motion vector of the target point and determining the motion vectors of other points. (See also...) Figure 1B As shown, within the target region divided in the prediction frame to be encoded, the target points can be the upper left and upper right corners. The motion vector at each target point represents its displacement from the reference frame to the current prediction frame, and the corresponding motion vector is... Figure 1B By using the motion vectors corresponding to the marked mv0 and mv1 points, as well as the width and height of the region, the motion vectors of other points within the region can be calculated. Thus, based on the motion vectors of each point, corresponding prediction results can be generated through motion compensation.

[0008] In the above process, determining the motion vector of the target point is the foundation of video coding. Under related technologies, the motion vector is usually determined in the following way: after selecting an initial motion vector for each target point, the residual information corresponding to each initial motion vector is obtained, and each initial motion vector is optimized and iterated to reduce the gap between the prediction result and the corresponding prediction frame. In this way, after multiple rounds of iteration, the motion vectors of each target point that meet the conditions can be obtained.

[0009] However, in each round of optimization iteration, Gaussian elimination is typically used to calculate equations for the corresponding residual and gradient information, thereby obtaining the adjustment amount for the motion vector to reduce the residual information between the prediction result and the prediction frame. This calculation method involves a large number of division operations when performing Gaussian elimination on the gradient information, which consumes significant computing resources in hardware implementation. Furthermore, the large amount of data involved in the division operations and the long time required for each operation significantly impact the timing constraints of the hardware, making it difficult for the hardware to quickly and accurately complete the optimization iteration of the motion vector for the target point. Consequently, when video recording is completed and needs to be transmitted to other devices, the inability to quickly encode the video recording results in a prolonged transmission process. For example, in a live streaming scenario, because the captured video recording cannot be quickly encoded and transmitted, noticeable stuttering occurs during playback.

[0010] Therefore, there is an urgent need for a new video coding method to reduce the computational cost of video coding and increase the video coding rate. Summary of the Invention

[0011] This application provides a video encoding method, apparatus, device, and computer storage medium to reduce the computational cost of video encoding and increase the video encoding rate.

[0012] Firstly, a video encoding method is provided, including:

[0013] Obtain the initial motion vector of each target point in a prediction frame between the prediction frame and the reference frame; the prediction frame is any one of the prediction frames associated with the reference frame.

[0014] The obtained initial motion vectors are iteratively optimized until the output motion vectors satisfy the preset optimization conditions; in each round of optimization, the following operations are performed:

[0015] Based on the reference frame and the output motion vectors of each previous round, the pixel information of each target point in the prediction frame is predicted, and the difference information between the obtained prediction result and the prediction frame is determined; in the first round of optimization, the output motion vectors of each previous round are the initial motion vectors.

[0016] Based on the distinguishing information and the output motion vectors of each previous round, corresponding gradient information is obtained; the gradient information is used to describe the degree of influence of the output motion vectors of each previous round on the distinguishing information.

[0017] Based on the digital information of the gradient information, with the goal of reducing the difference information, the output motion vectors of each previous round are adjusted to obtain the corresponding output motion vectors of each current round.

[0018] Secondly, a video encoding apparatus is provided, comprising:

[0019] The acquisition module is used to acquire the initial motion vector of each target point in a prediction frame between the prediction frame and the reference frame; the prediction frame is any one of the prediction frames associated with the reference frame.

[0020] The processing module is used to iteratively optimize each obtained initial motion vector until the obtained output motion vector satisfies the preset optimization conditions; wherein, during each round of optimization, the processing module is further used to:

[0021] Based on the reference frame and the output motion vectors of each previous round, the pixel information of each target point in the prediction frame is predicted, and the difference information between the obtained prediction result and the prediction frame is determined; in the first round of optimization, the output motion vectors of each previous round are the initial motion vectors.

[0022] Based on the distinguishing information and the output motion vectors of each previous round, corresponding gradient information is obtained; the gradient information is used to describe the degree of influence of the output motion vectors of each previous round on the distinguishing information.

[0023] Based on the digital information of the gradient information, with the goal of reducing the difference information, the output motion vectors of each previous round are adjusted to obtain the corresponding output motion vectors of each current round.

[0024] Optionally, when the processing module adjusts the output motion vectors of each previous round based on the digital information of the gradient information, with the goal of reducing the distinguishing information, to obtain the corresponding output motion vectors of each current round, it is specifically used for:

[0025] Determine multiple gradient elements included in the gradient information; each gradient element is used to describe the degree of influence of a vector parameter in the output motion vector of each previous round on the difference element corresponding to a target point included in the difference information.

[0026] Based on the digital information of each selected gradient element, the digital information of each other gradient element in the gradient information is adjusted, and based on the adjusted gradient information and the difference information, the parameter adjustment information of each previous round output motion vector is obtained.

[0027] Based on the parameter adjustment information, with the goal of reducing the difference information, the following steps are performed on each previous round output motion vector: each vector parameter of a previous round output motion vector is adjusted to obtain the corresponding current round output motion vector.

[0028] Optionally, when the processing module adjusts the digits of each other gradient element in the gradient information based on the digit information of each selected gradient element, it specifically performs the following:

[0029] For each other gradient element in the gradient information, perform the following operation:

[0030] Based on a preset correspondence, a gradient element corresponding to another gradient element is selected from the chosen gradient elements and used as the target gradient element.

[0031] Based on a preset number system, the target gradient element and the other gradient element are subjected to number system conversion to obtain the converted target gradient element and the converted other gradient element.

[0032] Based on the digital information of the transformed target gradient element, and in conjunction with the target gradient element, the digital information of another transformed gradient element is adjusted.

[0033] Optionally, when the processing module adjusts the digits of another gradient element after conversion based on the digit information of the converted target gradient element and in conjunction with the target gradient element, it is specifically used for:

[0034] The digital information of the transformed target gradient element and the digital information of another transformed gradient element are fused together.

[0035] When the fused digital information exceeds a set threshold, based on the first difference between the fused digital information and the set threshold, the larger of the digital information of the transformed target gradient element and the digital information of another transformed gradient element is subjected to bit reduction processing.

[0036] When the target gradient element with reduced bit depth is obtained, based on the second difference between the bit depth information of the target gradient element and the first difference, the product between the other gradient information and the target gradient element with reduced bit depth is subjected to bit depth reduction processing.

[0037] When another gradient information after bit reduction is obtained, based on the second difference between the bit information of the target gradient element and the first difference, bit reduction processing is performed on the product between the target gradient information and the other gradient information after bit reduction.

[0038] Optionally, after adjusting the digits of each other gradient element in the gradient information based on the digit information of each selected gradient element, the processing module is further configured to:

[0039] Based on the digital information of each selected gradient element, the digital information of each difference element in the difference information is adjusted; each difference element is used to describe the difference between the pixel information of a target point and the prediction result and the prediction frame.

[0040] The parameter adjustment information for each previous round's output motion vector, obtained based on the adjusted gradient information and the distinguishing information, includes:

[0041] Based on the adjusted gradient information, and combined with each adjusted distinguishing element, the corresponding parameter adjustment element is obtained for each vector parameter, and the obtained parameter adjustment elements are used as the parameter adjustment information.

[0042] Optionally, when the processing module adjusts the digits of each other gradient element in the gradient information based on the digit information of each selected gradient element, it specifically performs the following:

[0043] For each vector parameter, perform the following operations sequentially:

[0044] From multiple gradient elements adjusted in the previous round corresponding to a vector parameter, the gradient element adjusted in the previous round whose value matches the preset filtering condition is selected as the hit gradient element; when adjusting the first vector parameter, the gradient element adjusted in the previous round is the gradient element that has not been adjusted.

[0045] Based on the digit information of the hit gradient element, the following operations are performed on each of the other previous round adjustment gradient elements among the plurality of previous round adjustment gradient elements: the value of one previous round adjustment gradient element is adjusted to a preset value; the hit target point corresponding to the previous round adjustment gradient element is determined, and based on the digit information of the hit gradient element, the digits of the other previous round gradient elements corresponding to the hit target point are adjusted to obtain the corresponding current round adjustment gradient element.

[0046] Optionally, after adjusting the digits of other gradient elements from the previous round corresponding to the hit target point based on the digit information of the hit gradient element to obtain the corresponding adjusted gradient element for the current round, the processing module is further configured to:

[0047] For each of the other previous rounds, adjust the gradient elements by performing the following operations:

[0048] In the previous round adjustment distinction information, a previous round adjustment distinction element corresponding to another previous round adjustment gradient element is determined; when adjusting the first vector parameter, the previous round adjustment distinction information is the distinction information that has not been adjusted; each distinction element contained in the distinction information corresponds to a target point;

[0049] Based on the digit information of the hit gradient element, the digit of the previous round adjustment difference element is adjusted to obtain the corresponding current round adjustment difference element;

[0050] Based on the obtained distinguishing elements of each adjustment in this round, and the distinguishing elements of the previous round corresponding to the hit gradient elements, the distinguishing information of this round of adjustments is obtained.

[0051] Optionally, when the processing module obtains the corresponding gradient information based on the difference information and the output motion vectors of each previous round, it is specifically used for:

[0052] For each of the previous wheel's output motion vectors, the following operations are performed: determine the vector parameters corresponding to a previous wheel's output motion vector;

[0053] In the distinction information, the distinction elements corresponding to each target point are determined respectively; each distinction element is used to describe the difference between the pixel information of a target point and the prediction result and the prediction frame.

[0054] For each of the vector parameters, the following operations are performed: based on the distinguishing elements corresponding to each target point, gradient extraction is performed on a vector parameter to obtain the corresponding gradient elements; each gradient element is used to describe the degree of influence of a vector parameter on a distinguishing element; and the obtained gradient elements are used as the gradient information.

[0055] Thirdly, a computer program product is provided, including a computer program that, when executed by a processor, implements the method described in the first aspect.

[0056] Fourthly, a computer device is provided, comprising:

[0057] Memory, used to store program instructions;

[0058] A processor is configured to invoke program instructions stored in the memory and execute the method described in the first aspect according to the obtained program instructions.

[0059] Fifthly, a computer-readable storage medium is provided, the computer-readable storage medium storing computer-executable instructions for causing a computer to perform the method as described in the first aspect.

[0060] In the video coding method provided in this application embodiment, when obtaining the motion vector corresponding to the target point in the prediction frame, multiple rounds of iterative optimization are performed on the initial motion vector to obtain a target motion vector that meets preset optimization conditions. Thus, after multiple iterations, the obtained target motion vector can describe the displacement of the target point between the prediction frame and the reference frame as accurately as possible, thereby reducing the amount of data containing distinguishing information between the two and improving the compression efficiency of video coding.

[0061] In each optimization round, the following steps are performed: Based on the reference frame and the output motion vectors from each previous round, the pixel information of each target point is predicted. The difference between the predicted result and the predicted frame is then obtained. This difference is used to measure the accuracy of the motion vector's description of the target point's displacement. Furthermore, gradient information is used to describe the degree of influence of each previous round's output motion vector on the difference. In this way, the gradient information guides the adjustment of the motion vector, ensuring that the adjusted motion vector corresponds to the lowest possible difference, thereby improving the accuracy of the motion vector's description of the displacement.

[0062] Furthermore, in the process of adjusting motion vectors using gradient information, digital information of the gradient information is used for related processing. Among them, the digital information of the gradient is quick to obtain and simple to process, which can quickly adjust the motion vectors and avoid processing methods that consume more resources. While improving the efficiency of video encoding processing, it also reduces the computing resources consumed by video encoding processing, which can effectively improve the efficiency and effect of video encoding processing. Attached Figure Description

[0063] Figure 1A This is a schematic diagram of a scenario for video coding methods in related technologies;

[0064] Figure 1B This is a scenario illustration of a video coding method in related technologies. Figure 2 ;

[0065] Figure 1C This is one application scenario of the video encoding method provided in the embodiments of this application;

[0066] Figure 2 This is a schematic flowchart of a video encoding method provided in an embodiment of this application;

[0067] Figure 3A A schematic diagram illustrating the principle of a video encoding method provided in this application embodiment;

[0068] Figure 3B A schematic diagram of the principle of the video encoding method provided in the embodiments of this application. Figure 2 ;

[0069] Figure 3C A schematic diagram three illustrating the principle of the video encoding method provided in this application embodiment;

[0070] Figure 4A A schematic diagram four illustrating the principle of a video encoding method provided in this application embodiment;

[0071] Figure 4B A schematic diagram five illustrating the principle of a video encoding method provided in this application embodiment;

[0072] Figure 4C A schematic diagram six illustrating the principle of a video encoding method provided in an embodiment of this application;

[0073] Figure 5A A schematic diagram of the principle of the video encoding method provided in the embodiments of this application. Figure 7 ;

[0074] Figure 5B A schematic diagram of the principle of the video encoding method provided in the embodiments of this application. Figure 8 ;

[0075] Figure 5C A schematic diagram of the principle of the video encoding method provided in the embodiments of this application. Figure 9 ;

[0076] Figure 5D A schematic diagram of the principle of the video encoding method provided in this application embodiment is shown in Figure 10.

[0077] Figure 5E 11. A schematic diagram illustrating the principle of a video encoding method provided in this application embodiment;

[0078] Figure 5FSchematic diagram twelve illustrating the principle of a video encoding method provided in this application embodiment;

[0079] Figure 6A A schematic diagram thirteen illustrating the principle of a video encoding method provided in this application embodiment;

[0080] Figure 6B Fourteen is a schematic diagram illustrating the principle of a video encoding method provided in an embodiment of this application;

[0081] Figure 6C A schematic diagram fifteen illustrating the principle of a video encoding method provided in this application embodiment;

[0082] Figure 7 A schematic diagram sixteen illustrating the principle of a video encoding method provided in this application embodiment;

[0083] Figure 8 A schematic diagram of a video encoding device provided in an embodiment of this application;

[0084] Figure 9 A schematic diagram of the structure of a video encoding device provided in an embodiment of this application. Figure 2 . Detailed Implementation

[0085] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application. Unless otherwise specified, the embodiments and features in the embodiments of this application can be arbitrarily combined with each other. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than that shown here.

[0086] The following explanations of some terms used in the embodiments of this application are provided to facilitate understanding by those skilled in the art.

[0087] (1) Video encoding:

[0088] Video coding is the process of reducing video data size through compression techniques, with its core being the elimination of temporal, spatial, and statistical redundancy. Key technologies in video coding include intra-frame prediction (utilizing the spatial correlation between adjacent pixels) and inter-frame prediction (eliminating repetitive information between consecutive frames through motion estimation and compensation). For video coding technology, established coding standards define compression algorithms and bitstream formats to balance compression efficiency and video quality, and are widely used in streaming media, video conferencing, digital television, and other fields.

[0089] (2) Affine motion estimation:

[0090] Affine motion estimation is a predictive technique used in video coding to describe complex motion models. Compared to traditional translational motion estimation (which only considers translation vectors), affine motion estimation more accurately depicts the deformation or perspective changes of objects in consecutive frames through affine transformations (such as rotation, scaling, and shearing).

[0091] Specifically, affine motion estimation constructs a motion vector field using a 6-parameter affine model (two translations + four deformation parameters) or a 4-parameter affine model (translation, rotation, scaling). It then uses the motion parameters of adjacent blocks in the prediction frame to predict the motion of the current block, reducing the amount of residual data. This technique is particularly suitable for handling irregular motions (such as cloud diffusion, water ripples, and object rotation), and can significantly improve compression efficiency.

[0092] (3) Affine model:

[0093] Affine models are mathematical modeling methods based on affine transformations used to describe linear geometric transformations in two-dimensional or three-dimensional space, including translation, rotation, scaling, shearing, and combinations thereof. In video coding, affine models more accurately fit complex motions (such as deformation, perspective changes, or non-rigid motion) of objects in consecutive video frames by parameterizing motion vector fields (e.g., a 6-parameter model: translation in the x and y directions, rotation angles, horizontal and vertical scaling factors, and shearing factors) or a simplified 4-parameter model (translation, rotation, and uniform scaling). Compared to traditional translational motion models (which only describe the overall displacement of the object), affine models can capture local deformations or global perspective changes, reducing prediction residuals and improving compression efficiency through more accurate motion compensation.

[0094] (4) Exercise compensation:

[0095] Motion compensation is a key technique for compressing data by leveraging the temporal correlation between adjacent frames. Its core idea is to find the best-matching position of an image patch (or region) in the current frame within a reference frame (such as the previous or next frame) through motion estimation, and represent this displacement relationship using motion vectors (parameters describing the positional offset). Thus, the encoded video data only needs to record the motion vectors and the differences (residuals) between the current and reference blocks, rather than the complete pixel data, significantly reducing the data volume.

[0096] It should be noted that the embodiments of this application involve operations such as acquiring data video and determining prediction frames and reference frames. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, when relevant data is needed, relevant volunteers can be recruited and relevant agreements authorizing the volunteers to authorize data can be signed, and then the data of these volunteers can be used for implementation; or, implementation can be carried out within the authorized scope of an organization, and the following implementation methods can be used to identify internal members by using the data of internal members; or, the relevant data used in the specific implementation are all simulated data, such as simulated data generated in a virtual scene.

[0097] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0098] The application areas of the video encoding method provided in the embodiments of this application will be briefly introduced below.

[0099] With the continuous development and popularization of technology, more and more people are accustomed to recording their daily lives through video recordings, or sharing their knowledge and experiences. At the same time, video recording technology is becoming increasingly integrated into various aspects of people's work and life. Therefore, after recording specific scenes using hardware devices, video encoding technology has emerged to improve the storage efficiency of video recordings.

[0100] Video coding technology mainly uses coding algorithms to convert the original video signal into a compressed digital format, thereby significantly reducing the size of the encoded video data while ensuring visual quality.

[0101] For example, such as Figure 1A As shown, when encoding video recordings, one video frame can be selected as a reference frame, and other adjacent video frames can be used as multiple prediction frames that are key to the reference frame.

[0102] In this way, by using the pixel information recorded in the reference frame, as well as the motion vectors and residual information between the reference frame and the prediction frame, the pixel information in other prediction frames can be described. Specifically, during video decoding, for the pixel information of a prediction frame, motion compensation can be performed using the pixel information of the reference frame and the corresponding motion vector of the prediction frame to obtain the prediction result for that prediction frame. Then, the residual information is superimposed on the prediction result to decode the pixel information in the prediction frame.

[0103] In the above process, only the pixel information of the reference frame needs to be recorded, while the pixel information of other predicted frames can be encoded into corresponding motion vectors and residual information. Therefore, the amount of data in the encoded video data can be greatly reduced.

[0104] It is evident that motion vector selection is a crucial step in video coding technology. The motion of an object can take many complex forms, such as flipping and rotating, requiring affine motion estimation to determine the corresponding motion vectors.

[0105] In affine motion estimation, the determination of motion vectors is divided into: determining the motion vector of the target point and determining the motion vectors of other points. (See also...) Figure 1B As shown, within the target region divided in the prediction frame to be encoded, the target points can be the upper left and upper right corners. The motion vector at each target point represents its displacement from the reference frame to the current prediction frame, and the corresponding motion vector is... Figure 1B By using the motion vectors corresponding to the marked mv0 and mv1 points, as well as the width and height of the region, the motion vectors of other points within the region can be calculated. Thus, based on the motion vectors of each point, corresponding prediction results can be generated through motion compensation.

[0106] In the above process, determining the motion vector of the target point is the foundation of video coding. Under related technologies, the motion vector is usually determined in the following way: after selecting an initial motion vector for each target point, the residual information corresponding to each initial motion vector is obtained, and each initial motion vector is optimized and iterated to reduce the gap between the prediction result and the corresponding prediction frame. In this way, after multiple rounds of iteration, the motion vectors of each target point that meet the conditions can be obtained.

[0107] However, in each round of optimization iteration, Gaussian elimination is typically used to calculate equations for the corresponding residual and gradient information, thereby obtaining the adjustment amount for the motion vector to reduce the residual information between the prediction result and the prediction frame. This calculation method involves a large number of division operations when performing Gaussian elimination on the gradient information, which consumes significant computing resources in hardware implementation. Furthermore, the large amount of data involved in the division operations and the long time required for each operation significantly impact the timing constraints of the hardware, making it difficult for the hardware to quickly and accurately complete the optimization iteration of the motion vector for the target point. Consequently, when video recording is completed and needs to be transmitted to other devices, the inability to quickly encode the video recording results in a prolonged transmission process. For example, in a live streaming scenario, because the captured video recording cannot be quickly encoded and transmitted, noticeable stuttering occurs during playback.

[0108] In view of this, embodiments of this application provide a video encoding method in which corresponding encoding processing is performed on each predicted frame during the video encoding process to improve video encoding efficiency.

[0109] Specifically, when encoding any one of the prediction frames that is critical to the reference frame, it is first necessary to obtain the initial motion vector of each target point in a video frame between a prediction frame and the reference frame. Thus, after obtaining the initial motion vector corresponding to each target point, iterative optimization can be performed on these initial motion vectors until the resulting output motion vector satisfies the preset optimization conditions.

[0110] Specifically, the following operations are performed during each round of optimization:

[0111] Based on the reference frame and the motion vectors output from each previous round, the pixel information of each target point in a prediction frame is predicted, and the difference between the obtained prediction result and a prediction frame is determined. In the first round of optimization, the motion vectors output from each previous round are the initial motion vectors. Thus, starting with the initial motion vectors, the pixel information corresponding to the target points is predicted using the motion vectors and the reference frame. By using the difference between the prediction result and the prediction frame, the reliability of the current motion vector can be determined, and further adjustments can be made to the corresponding motion vectors based on this difference information:

[0112] Based on the distinguishing information and the output motion vectors of each previous round, corresponding gradient information is obtained. This gradient information describes the degree of influence of each previous round's output motion vector on the distinguishing information. Thus, using the gradient information describing the degree of influence, the motion vectors can be adjusted so that the adjusted motion vectors reduce the corresponding distinguishing information. Specifically, when adjusting using gradient information, it is necessary to adjust the output motion vectors of each previous round based on the numerical information of the gradient information, with the goal of reducing the distinguishing information, thereby obtaining the corresponding output motion vector for the current round.

[0113] In the video coding method provided in this application embodiment, when obtaining the motion vector corresponding to the target point in the prediction frame, multiple rounds of iterative optimization are performed on the initial motion vector to obtain a target motion vector that meets preset optimization conditions. Thus, after multiple iterations, the obtained target motion vector can describe the displacement of the target point between the prediction frame and the reference frame as accurately as possible, thereby reducing the amount of data containing distinguishing information between the two and improving the compression efficiency of video coding.

[0114] In each optimization round, the following steps are performed: Based on the reference frame and the output motion vectors from each previous round, the pixel information of each target point is predicted. The difference between the predicted result and the predicted frame is then obtained. This difference is used to measure the accuracy of the motion vector's description of the target point's displacement. Furthermore, gradient information is used to describe the degree of influence of each previous round's output motion vector on the difference. In this way, the gradient information guides the adjustment of the motion vector, ensuring that the adjusted motion vector corresponds to the lowest possible difference, thereby improving the accuracy of the motion vector's description of the displacement.

[0115] Furthermore, in the process of adjusting motion vectors using gradient information, the digital information of gradient information is used for related processing. The digital information of gradient information is quick to obtain and simple to process, which can quickly adjust motion vectors and avoid processing methods that consume more resources. This improves the efficiency of video encoding and processing while reducing the computational resources consumed by video encoding and processing, and can effectively improve the efficiency and effect of video encoding and processing.

[0116] The application scenarios of the video encoding method provided in this application are described below.

[0117] Please refer to Figure 1CThis is a schematic diagram illustrating an application scenario of the video encoding method provided in this application. The application scenario includes a server 101 and a client 102, which can communicate with each other. The communication method can be wired, such as through a network cable or serial cable; or wireless, such as through Bluetooth or Wi-Fi. No specific limitation is imposed.

[0118] Client 102 generally refers to devices that can send data requests, such as terminal devices, third-party applications accessible by terminal devices, or web pages accessible by terminal devices. Server 101 generally refers to devices that can send resources, such as terminal devices or servers.

[0119] Terminal devices include, but are not limited to, mobile phones, computers, smart medical devices, smart home appliances, vehicle terminals, or aircraft. Servers include, but are not limited to, cloud servers, local servers, or associated third-party servers. These can be independent physical servers, server clusters or distributed systems composed of multiple physical servers, or cloud servers providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. Both client 101 and server 102 can use cloud computing to reduce the consumption of local computing resources; similarly, they can also use cloud storage to reduce the consumption of local storage resources.

[0120] As one embodiment, the server 101 and the client 102 can be the same device, or they can be different devices, or they can be different devices with some modules shared, etc., and there are no specific restrictions.

[0121] The following is based on Figure 1C Based on the client, the video encoding method provided in the embodiments of this application will be described in detail. Please refer to... Figure 2 This is a flowchart illustrating a video encoding method provided in an embodiment of this application.

[0122] S21, obtain the initial motion vector of each target point in a prediction frame between a prediction frame and a reference frame, wherein a prediction frame is any one of the prediction frames associated with the reference frame.

[0123] Before introducing the process of obtaining the initial motion vector, it is necessary to first introduce the target point:

[0124] like Figure 3AAs shown, in the video encoding process, each video frame in the sequence of video frames in the video to be processed is typically encoded sequentially. When processing a single video frame, the video is divided into multiple coding units (CUs), and these CUs are encoded sequentially as well. Generally, the size of a coding unit in a video frame is variable, and the number of pixels it contains can be 64×64, 32×32, 16×16, 8×8, etc.

[0125] For a video frame being encoded, the encoding process involves encoding multiple coding units in the video frame sequentially, and the coding unit currently being encoded can be called the current coding unit (hereinafter referred to as the current block).

[0126] As discussed above, a crucial step in the video encoding process is determining the motion vector between two video frames to describe the motion of an object. Thus, by using the pixel information of one video frame and combining it with this motion vector, the position information of pixels in another video frame can be predicted through motion compensation. In this process, the video frame providing the pixel information is called the reference frame, while the video frame whose pixel position information has been predicted is called the prediction frame.

[0127] It should be noted that, as Figure 3B As shown, since video encoding processes each video frame sequentially, the video frame currently being encoded is the prediction frame mentioned above. The reference frame corresponding to this prediction frame can be any video frame with corresponding pixel information. For example, it could be an adjacent video frame that has already been encoded, or an encoded video frame that is separated from the prediction frame by multiple video frames. Correspondingly, for a video frame used as a reference frame, its associated prediction frame is not necessarily only the currently encoded video frame; it can also include multiple other video frames. The specific number can be determined according to actual needs, and this application does not impose any restrictions on this.

[0128] After clarifying the relationship between the predicted frame and the reference frame in the video to be processed, it is necessary to further explain the meaning of the motion vectors between the predicted frame and the reference frame.

[0129] As the name suggests, a motion vector is a vector used to describe motion. In the process of video encoding, a motion vector mainly refers to the displacement of an object between a reference frame and a prediction frame. Specifically, for each pixel, it means the displacement of the same pixel between the reference frame and the prediction frame.

[0130] However, in real life, the types of motion of objects are quite complex, including translation, rotation, scaling, stretching, and so on. Therefore, in the video coding process, in order to describe these types of motion using motion vectors, an affine motion estimation method is proposed. The following will introduce affine motion estimation using a coding unit in a video frame as an example:

[0131] Please refer to Figure 3C For a current block in a prediction frame undergoing video encoding, affine motion estimation estimates the motion of the entire current block using the motion vectors of pixels at fixed positions within that block. These fixed-position pixels are the target points mentioned in S21 above. In affine motion estimation, the target points are typically located in... Figure 3C The selection is made from the top-left corner A, top-right corner B, or bottom-left corner C of the current block shown. Specifically, in a 4-parameter affine model, the top-left corner A and top-right corner B are selected as the target points (also known as control points). In a 6-parameter affine model, the top-left corner A, top-right corner B, or bottom-left corner C are selected as the target points.

[0132] Taking a 4-parameter affine model as an example, after obtaining the motion vectors at points A and B, we can use these motion vectors, along with the width and height of the current block, to calculate the motion vectors of each pixel within the current block. However, in practical applications, due to the enormous computational cost of pixel-level affine mapping, the current block is usually divided into... Figure 4A The diagram shows 16 sub-blocks in a 4×4 pattern. The motion vector corresponding to the center pixel of each sub-block is then calculated, and the displacement of the entire sub-block is described by the motion vector.

[0133] As can be seen from the above process, the most important process for affine motion estimation of the current block is the acquisition of the motion vector of the target point. Therefore, the video coding method provided in this application embodiment will provide a detailed introduction to the acquisition of motion vectors.

[0134] The first thing to introduce is the affine model corresponding to the motion vector in affine motion estimation. Equations 1 and 2 below show the mathematical expressions for the 6-parameter affine model and the 4-parameter affine model, respectively.

[0135] Specifically, the expression for the 6-parameter affine model is:

[0136] x ′ =ax + by + e

[0137] y ′ = cx + dy + f (Formula 1)

[0138] Where (x, y) are the pixel coordinates in the prediction frame currently being encoded, (x... ′ y ′ ) represents the coordinates of the corresponding pixel in the reference frame. Parameters a, b, c, and d control the rotation, scaling, and cropping of the pixel coordinates, while parameters e and f control the translation of the pixel coordinates.

[0139] The 4-parameter affine model assumes that rotation and scaling are uniform, therefore its expression is:

[0140] x ′ =ax-by+e

[0141] y ′ =bx + ay + f (Formula 2)

[0142] As can be seen from the affine model above, determining the motion vector of the target point is actually determining the corresponding vector parameters. In other words, once the corresponding vector parameters are determined, the motion vector corresponding to each target point can be determined.

[0143] Therefore, having clarified the above introduction regarding target points and motion vectors, the following will describe the motion process corresponding to each target point.

[0144] S21 proposes that for each target point in a prediction frame, its corresponding initial motion vector needs to be obtained. The initial motion vector can be obtained in several different ways:

[0145] In one possible implementation, when obtaining the initial motion vector of the target point of the current block, the selection can be based on the motion information of the encoded units around the current block, or the corresponding encoded units in the adjacent frames of the prediction frame where the current block is located.

[0146] Specifically, such as Figure 4A As shown, since the pixels in the coding units surrounding the current block in a prediction frame are spatially similar to the current block, and the coding units corresponding to the current block in adjacent frames are temporally continuous with the current block, the corresponding motion vectors can be obtained from these other coding units associated with the current block as a candidate motion vector set for the initial motion vectors of each target point of the current block.

[0147] After obtaining the candidate motion vector set, the initial motion vector corresponding to each target point needs to be selected. This selection can be based on the Rate Distortion (RD) cost. The RD cost is a core metric for evaluating the quality of a coding scheme, used to balance the trade-off between the encoded bit rate and distortion. By calculating the RD cost, the candidate motion vector with the lowest RD cost can be selected from the set as the initial motion vector for each target point.

[0148] Optionally, the candidate motion vector set may also include some preset motion vectors to address application scenarios where there are no coded adjacent blocks.

[0149] The above has introduced the relevant content of reference frames, prediction frames, target points, and motion vectors in video coding. After obtaining the initial motion vector through the above methods, the following operations can be performed:

[0150] S22: Iteratively optimize each obtained initial motion vector until the obtained output motion vector satisfies the preset optimization conditions.

[0151] After obtaining the initial motion vectors corresponding to each target point, it is necessary to optimize these initial motion vectors in multiple rounds until the final output motion vectors meet the preset optimization conditions.

[0152] The preset optimization condition can be set for either the number of optimization rounds or the output motion vector. For example, the preset optimization condition can be set to: the number of iterations does not exceed a preset number of rounds. Thus, when the preset number of iterations is reached, the output motion vector obtained in this round is used as the target motion vector corresponding to the target point. Alternatively, the preset optimization condition can be set to: the RD cost corresponding to the output motion vector does not exceed a preset cost threshold. Thus, when the RD cost corresponding to the output motion vector in this round does not exceed the preset cost threshold during iteration, the output motion vector in this round can be used as the target motion vector corresponding to the target point.

[0153] Having clarified the termination conditions of the iterative optimization process, the following will describe each round of the optimization process. Specifically, as follows... Figure 2 As shown, the following operations are performed in each round of optimization:

[0154] S221: Based on the reference frame and the motion vectors output by each previous round, predict the pixel information of each target point in the prediction frame, and determine the difference information between the obtained prediction result and the prediction frame.

[0155] In the first round of optimization, the initial motion vectors obtained need to be optimized. In subsequent rounds of optimization, the output motion vectors of the previous round need to be optimized. In other words, in the first round of optimization, the output motion vectors of the previous round are essentially the initial motion vectors.

[0156] Specifically, after the current round of optimization begins, it is necessary to first use the reference frame and the motion vectors output from each previous round to predict the pixel information of the target point in a prediction frame, and determine the difference information between the obtained prediction result and a prediction frame.

[0157] For example, such as Figure 4B As shown, by using the corresponding points in the reference frame and the motion vectors output from each previous round, the pixel information of the predicted points obtained based on the motion vectors can be deduced. Then, by using the pixel information in the prediction result and the pixel information of the actual target points in the prediction frame, the difference information between the prediction result and the prediction frame can be obtained. This difference information includes multiple difference elements, each describing the pixel information of a target point and the difference between the prediction result and the prediction frame.

[0158] In one possible implementation, when acquiring the prediction results, in addition to predicting the target point, prediction can also be performed directly on all pixels in the current block. Specifically, as mentioned above, in affine motion estimation, the motion vectors corresponding to each sub-block in the current block can be directly calculated using the motion vector of the target point and the width and height of the current block. Therefore, during the acquisition of the prediction results, the motion vector corresponding to each sub-block can be calculated using the previous round's output motion vector corresponding to the target point. Then, using the pixel information in the reference frame, prediction can be performed on all pixels in the current block, thus including the prediction information of each pixel in the current block in the prediction results. In this way, after obtaining the distinction information, the distinction elements included in the distinction information can describe the difference between each pixel in the current block and the prediction frame.

[0159] S222: Based on the differentiation information and the output motion vectors of each previous round, obtain the corresponding gradient information, wherein the gradient information is used to describe the degree of influence of the output motion vectors of each previous round on the differentiation information.

[0160] After obtaining the distinguishing information, in order for this distinguishing information to guide the adjustment direction of the motion parameters, it is also necessary to obtain gradient information that describes the degree of influence of the motion vector on the distinguishing information.

[0161] For gradient information, a common approach is to process the first derivative of each previous round's output motion vector based on the acquired distinguishing information, and then use the result as the corresponding gradient information.

[0162] In one possible implementation, the gradient information can be obtained in the following way:

[0163] First, for each previous round's output motion vector, perform the following operation: determine the vector parameters corresponding to a previous round's output motion vector.

[0164] As mentioned above, in the affine model, the motion vector corresponding to the target point is usually described by 4 or 6 parameters. The vector parameters corresponding to each target point are different. Therefore, in this process, it is necessary to determine the vector parameters corresponding to the output motion vector of the previous round.

[0165] Next, in the distinction information, the distinction elements corresponding to each target point are determined respectively. Each distinction element, as mentioned above, is used to describe the difference between the pixel information of a target point and the prediction result and the prediction frame.

[0166] Once the vector parameters and distinguishing elements are defined, the following operations can be performed on each vector parameter: based on the distinguishing elements corresponding to each target point, the gradient of a vector parameter is extracted to obtain the corresponding gradient elements, and the obtained gradient elements are used as gradient information.

[0167] The gradient extraction process is the specific process of the gradient information acquisition process mentioned above. It is applied to the distinguishing element and vector parameters. For a distinguishing element, the first derivative of each vector parameter is calculated, and the result is used as the corresponding gradient element.

[0168] After the above method has been executed for each previous round's output motion vector and each vector parameter, the result obtained is as follows: Figure 4CAs shown, gradient information can be represented by a matrix, where each element describes the influence of a vector parameter on a distinguishing element. Furthermore, each row of this matrix corresponds to a target point, and each column corresponds to a vector parameter. The only variable is the number of vector parameters and target points. In a 4-parameter model, the matrix size can be 4×2, while in a 6-parameter model, it can be 6×3. Optionally, when the prediction result matches the prediction information for each pixel in the current block, each row of the matrix can correspond to a pixel, and the number of rows in the matrix represents the number of pixels included in the prediction information.

[0169] Thus, in this method, based on the distinguishing elements corresponding to each target point, gradient extraction is performed for each vector parameter. After obtaining the gradient elements that describe the degree of influence of the vector parameters on the distinguishing elements, all gradient elements can be used as the corresponding gradient information. Using the degree of influence of the motion vector on the distinguishing information described by this gradient information, subsequent adjustments to the motion vector can be guided, thereby quickly obtaining the output motion vector that meets the preset optimization conditions.

[0170] S223: Based on gradient information, digital information is used to adjust the output motion vectors of each previous round with the goal of reducing discriminative information, so as to obtain the corresponding output motion vectors of each current round.

[0171] Generally, after obtaining the gradient and differentiation information corresponding to the output motion vector of the previous round, a system of four or six equations can be obtained by solving a system of equations for each vector parameter based on the gradient and differentiation information. For example, Figure 5A As shown, each unknown in this system of equations corresponds to a motion vector. The coefficient matrix of the system is the gradient matrix composed of the gradient elements mentioned above, and the constants in each row are the distinguishing elements for each target point. After solving this system of equations, each vector parameter can be assigned an adjustment amount, which can then serve as the data basis for subsequent adjustments to the motion vector.

[0172] In solving systems of equations, Gaussian elimination is a commonly used method. The core idea of ​​Gaussian elimination is to transform the system of equations into a row echelon form matrix through row operations, and then solve it by back substitution. Specifically, for an augmented matrix corresponding to a system of equations, such as... Figure 5BAs shown, after row transformation, the matrix is ​​converted into a row echelon form matrix, where the elements in the lower left corner of the diagonal of the coefficient matrix are reduced to 0. Then, by substituting the solution for the unknowns in the last row back into the row echelon form matrix, the solutions for the remaining unknowns can be solved sequentially.

[0173] When performing elimination, the formula used to update the element in the i-th row is:

[0174]

[0175] Where a represents a coefficient, b represents a constant, and i > k.

[0176] However, in this process, when eliminating elements from the augmented matrix, a large number of division operations are required, as shown in Equation 3: that is, in Equation 3... This division calculation.

[0177] In the field of computer science, division calculations require a large amount of computing resources. On the other hand, the amount of data involved in division operations is enormous, and each division operation takes a long time. This puts a great strain on the timing constraints of the hardware itself, making it difficult for hardware devices to quickly and accurately complete the optimization iteration of motion vectors for target points under such circumstances.

[0178] Therefore, the video coding method proposed in this application suggests that the digital information of the gradient information is directly used to adjust each of the above-mentioned output motion vectors to obtain the corresponding current-round output motion vector.

[0179] In one possible implementation, to obtain the output motion vector for this round, the following operation can be performed:

[0180] First, we determine the multiple gradient elements included in the gradient information. Each gradient element describes the influence of a vector parameter in the output motion vector of each previous round on the difference element corresponding to a target point in the difference information.

[0181] As mentioned above, gradient information contains multiple gradient elements, each corresponding to a vector parameter and a target point. Other details are as described above and will not be repeated here.

[0182] Next, based on the digit information of each gradient element selected from the gradient information, the digits of each other gradient element in the gradient information are adjusted, and based on the adjusted gradient information and the differentiation information, the parameter adjustment information of each previous round output motion vector is obtained.

[0183] After solving the system of equations using gradient information and distinguishing information, each gradient element in the gradient information is a corresponding coefficient element in the system of equations. At this point, the coefficients need to be processed to reduce the difficulty of solving the system of equations.

[0184] Since the proposed solution is a video coding technology applied in computer equipment, in order to improve video coding efficiency, this application proposes to abandon the division operation in the process of solving equations, and instead use digital processing methods in the field of computer technology to predict the result of each division operation.

[0185] Therefore, this process proposes to use the digit information of the selected gradient elements to adjust the digits of other gradient elements.

[0186] In one possible implementation, the process of adjusting the digits of other gradient elements using the digit information of the selected gradient elements can be performed in the following manner:

[0187] For each other gradient element, perform the following operation:

[0188] First, based on the preset correspondence, a gradient element corresponding to another gradient element is selected from the chosen gradient elements and used as the target gradient element.

[0189] The preset correspondence can be established through methods such as... Figure 5C Describe the content shown. For example... Figure 5C As shown, assuming the description involves a coefficient matrix consisting of all gradient elements from the gradient information (the number of elements in this coefficient matrix is ​​for illustrative purposes only and does not represent the actual number of elements in a real application), then the selected gradient coefficients are typically the pivot elements of this coefficient matrix, i.e., the elements on the diagonal. For other gradient elements, the selection of their corresponding target gradient elements needs to be determined by the vector parameters being processed in the elimination process.

[0190] For example, such as Figure 5CAs shown, Gaussian elimination works by sequentially eliminating coefficients in each column of a matrix. Therefore, each elimination operation requires performing a complete elimination process on the column containing a vector parameter. In terms of the selection of the target gradient element, when eliminating the vector parameter in the first column, the pivot element of that column is selected as the target gradient element to apply to the other gradient elements. Similarly, when eliminating the vector parameter in the second column, the pivot element of that column is selected as the target gradient element to apply to the other gradient elements.

[0191] Second, after selecting the target gradient element, the other gradient elements to be processed, as well as the target gradient element, can be converted into a preset number system type to obtain the converted target gradient element and one of the other converted gradient elements. The preset number system type can be a binary number system.

[0192] Finally, based on the digital information of the transformed target gradient element, and in combination with the target gradient element, the digital information of another transformed gradient element can be adjusted.

[0193] For example, with Figure 5D Taking the matrix shown as an example, and combining it with the content of Formula 3 above, the above content will be explained. For example... Figure 5D As shown, assuming Figure 5D In the matrix shown, element D is the element that needs to be adjusted during the current elimination process, element M is the target gradient element corresponding to element D, and element L is the gradient represented in Equation 3. Element C is represented in Formula 3. In other words, the row containing element L is the same as the row containing element D, the column containing element L is the same as the column containing element M, and the row containing element C is the same as the row containing element M, and the column containing element C is the same as the column containing element D. Therefore, for element D, the relationship before and after adjustment is as shown in Formula 4 below:

[0194] D 新 =D 旧 -(L÷M)·C (Formula 4)

[0195] In other words, the content of Formula 4 can be expressed as Formula 5, that is:

[0196] D 新 = (M·D 旧 -L·C)>>log2M (Formula 5)

[0197] In Formula 5, >> indicates a right shift, and log2M represents the bit information of element M in binary.

[0198] Furthermore, regarding the digital adjustment processing for other gradient elements, the following will combine... Figure 5D The content shown, and Figure 5E The content shown will be illustrated with examples.

[0199] For example, such as Figure 5E As shown in Formula 5, for the processing method shown, assuming that it is necessary to perform the following steps: Figure 5D If element D is subjected to bit adjustment, then elements M, L, and C, which are related to D, also require corresponding bit adjustment. Specifically, after multiplying element M by element D, the product is reduced in number by shifting it to the right. The number of bits shifted is determined by the binary representation of element M. For example, assuming element M has a value of 8, its binary representation is 3. Therefore, the number of bits shifted to the right after multiplying M by D is the same as the binary representation of element M, which is 3.

[0200] Similarly, the same process can be applied to the product of element L and element C. After obtaining the result of the shifting process, the difference between the two is the adjusted value of element D.

[0201] Thus, by using the above processing method, the division operation in Formula 4 is transformed into digital processing in computer technology. While meeting the data precision requirements in the field of video encoding, it avoids the occupation of a large amount of complex computing resources in the division operation. Instead, only shift processing is required. Furthermore, since the main body of this digital processing only needs to perform relatively simple bit processing operations, it can significantly improve the acquisition process of the current output motion vector, thereby increasing the processing speed of video encoding.

[0202] In one possible implementation, as shown in Formula 5, during the above-mentioned digit adjustment process, there may be a situation where two gradient elements are multiplied. In order to avoid the two gradient elements being multiplied directly, which would cause the number of digits in the result to overflow and result in an inaccurate result, this application proposes the following method to ensure the accuracy and stability of the digit adjustment process.

[0203] Specifically, the first step is to fuse the digital information of the transformed target gradient element and the digital information of another transformed gradient element.

[0204] Thus, when the fused digital information does not exceed the set threshold, the digital adjustment of another gradient element after conversion can be directly performed in the manner shown in Formula 5.

[0205] When the fused digital information exceeds the set threshold, it is necessary to perform bit reduction processing on the larger value of the digital information of the transformed target gradient element and the digital information of another transformed gradient element, based on the first difference between the fused digital information and the set threshold.

[0206] In this process, because the larger value in the digit information is selected for bit reduction, the precision issues caused by bit reduction can be minimized in the two gradient elements. For example, the precision issues caused by reducing a 10-bit binary number to a 9-bit binary number are much smaller than the precision issues caused by reducing a 3-bit binary number to a 2-bit binary number.

[0207] Furthermore, when the target gradient element with reduced bit depth is obtained, based on the second difference between the bit depth information of the target gradient element and the first difference, the product between another gradient information and the target gradient element with reduced bit depth is subjected to bit depth reduction processing.

[0208] When another gradient information after bit reduction is obtained, the second difference between the bit information of the target gradient element and the first difference is used to perform bit reduction processing on the product between the target gradient information and the other gradient information after bit reduction.

[0209] Since the number of bits to be reduced in Formula 5 refers to the digit information of the target gradient element, and in the previous process, the first difference digit reduction has already been performed on the digit information of the transformed target gradient element, or the digit information of another transformed gradient element. Therefore, after completing the digit reduction based on the first difference, the resulting product needs to undergo a second digit reduction based on the difference between the digit information of the target gradient element and the first difference.

[0210] For example, such as Figure 5F As shown, combined with Figure 5D The matrix information shown, assuming that element D needs to be adjusted, firstly, it is necessary to judge the multiplied elements M and D. When the fusion value of the digital information of the two exceeds the set threshold, it can be determined that the result of multiplying the two will exceed the value that the computer can currently represent. Therefore, it is necessary to perform bit reduction processing on the two in advance.

[0211] To ensure the accuracy of the results, it is necessary to process the element with larger digit information in elements M and D to reduce the error rate.

[0212] Thus, after the first stage of bit reduction processing, the two elements can be multiplied, and the resulting product can be subjected to a second stage of bit reduction processing. In the second stage of bit reduction processing, the number of bits shifted to the right only needs to be the second difference between the digit information of element M and the first difference.

[0213] In this way, the reduction of the number of bits, which was originally done all at once, is divided into two stages. This ensures the accuracy of the result after the reduction of the number of bits and also prevents the result of the element-wise multiplication from exceeding the data range that the computer can express, thus ensuring the reliability of the implementation of the solution.

[0214] Similarly, the product of element L and element C in Formula 5 can be processed in the same way as described above. Then, using Formula 5, another gradient element after bit adjustment can be obtained.

[0215] In the above approach, the two-stage bit reduction process ensures that bit overflow does not occur when multiplying two gradient elements, thus preventing inaccurate data recording and improving the accuracy and reliability of the solution. Furthermore, in the first stage of bit reduction, the gradient element with a larger bit count is selected for processing, which also ensures that the accuracy error during bit reduction is small, avoiding impact on subsequent motion vector acquisition.

[0216] The above describes the processing of gradient elements in gradient information. In addition to processing gradient elements, the distinguishing elements in distinguishing information also need to be processed in the process of solving equations.

[0217] In one possible implementation, the following processing also needs to be performed on the differentiation information:

[0218] Based on the digital information of each selected gradient element, the digital information of each difference element in the difference information is adjusted. The difference element is used to describe the difference between the pixel information of a target point and a prediction frame.

[0219] For each distinguishing element, it can be combined with the gradient elements to form the augmented matrix corresponding to the equation system, where each row of the augmented matrix corresponds to a target point or pixel point.

[0220] Therefore, in addition to performing the aforementioned digit adjustment process on the coefficient elements, the same digit adjustment process is also required on the distinguishing elements. It should be noted that this digit adjustment process can refer to the digit adjustment process performed on the gradient elements, and its specific execution principle is based on the processing method reflected in Formula 5, which will not be elaborated here.

[0221] After adjusting the distinguishing elements in the above manner, based on the adjusted gradient information and combined with each adjusted distinguishing element, the corresponding parameter adjustment elements are obtained for each vector parameter, and the obtained parameter adjustment elements are used as parameter adjustment information.

[0222] Thus, for the augmented matrix corresponding to the system of equations, after the elimination process for each column of the coefficient matrix is ​​completed, the corresponding row echelon form matrix can be obtained.

[0223] At this point, by substituting the solution obtained through the last vector parameter back into the other equations, we can obtain the solution corresponding to each vector parameter.

[0224] It should be noted that, since only a finite number of division operations are required in this process, the division operation can be performed directly during the back substitution solution process, or the digit information of the divisor can be used for digit adjustment. This application does not impose any restrictions on this.

[0225] Thus, after obtaining the parameter adjustment information based on gradient information and differentiation information, the parameter adjustment information can be used to reduce the differentiation information. Then, for each previous round output motion vector, the parameters of each vector of a previous round output motion vector are adjusted to obtain the corresponding current round output motion vector.

[0226] In this method, the digital information of the gradient elements is used to adjust the digital information of other gradient elements. By simply shifting the data, the resource-intensive division operation is replaced. This significantly improves the computational efficiency of video encoding while ensuring controllable computational accuracy, and at the same time reduces the computational resources required for the video encoding process.

[0227] The above describes the digit adjustment processing of gradient elements and distinguishing elements during parameter adjustment. To further improve the efficiency of video encoding, the following methods can also be used during the acquisition of parameter adjustment information:

[0228] In one possible embodiment, when adjusting the gradient elements, the following operations may also be performed:

[0229] like Figure 6A As shown, in the aforementioned process of adjusting the gradient elements, the original augmented matrix is ​​converted into a corresponding row echelon form matrix. Thus, after obtaining the corresponding system of equations, solving for the unknowns requires sequentially substituting back into the equations to obtain the solutions for the remaining unknowns. This process is overly dependent, and the entire back-substitution process is a serial process for the executing entity, which can significantly impact the hardware timing of the executing entity.

[0230] Therefore, in order to improve the parallel processing efficiency of the back-substitution process, the embodiments of this application propose the following implementation methods:

[0231] For each vector parameter, perform the following operations sequentially:

[0232] First, from multiple gradient elements adjusted in the previous round corresponding to a vector parameter, select the gradient element whose value matches the preset filtering condition and use it as the hit gradient element; when adjusting the first vector parameter, the gradient element adjusted in the previous round is the gradient element that has not been adjusted.

[0233] It's important to clarify that the elimination process is performed sequentially for each column. Therefore, when eliminating a column, you first need to select the previous round's adjusted gradient element values ​​that match the preset filtering criteria. These filtering criteria can be either the largest non-zero previous round adjusted gradient element value, or simply any non-zero previous round adjusted gradient element value.

[0234] Next, based on the digit information of the hit gradient element, the following operations are performed on each of the other previous round adjusted gradient elements among the multiple previous round adjusted gradient elements: the value of one previous round adjusted gradient element is adjusted to a preset value; the hit target point corresponding to one previous round adjusted gradient element is determined, and based on the digit information of the hit gradient element, the digits of the other previous round adjusted gradient elements corresponding to the hit target point are adjusted to obtain the corresponding current round adjusted gradient element.

[0235] like Figure 6B As shown, in this elimination process for a vector parameter, assuming the vector parameter is located in the second column, the hit gradient element is the element in the second row of the second column of the matrix. Then, for each other gradient element in that column from the previous round, the following operations must be performed:

[0236] Will Figure 6B The value of one of the elements adjusted in the previous round is set to a preset value. Optionally, this preset value can be zero.

[0237] Next, the target point corresponding to the adjusted element from the previous round is identified as the hit target point. Thus, we can continue to adjust the gradient elements from the previous round corresponding to this hit target point, obtaining the corresponding gradient elements for the current round, which are reflected in… Figure 6B The matrix shown represents the digital adjustment of the gradient elements of the previous round in the row containing the adjusted element of the previous round.

[0238] Thus, in this method, for each vector parameter, during the elimination phase, all gradient elements except the hit gradient element can be eliminated. After completing the elimination for all vector parameters, a set of equations that are independent of each vector parameter can be obtained. This avoids the serial back-substitution operation required for a set of equations with multiple unknowns, thereby achieving parallel processing of unknown solutions, improving the efficiency of obtaining parameter adjustment information, and thus improving the efficiency of video coding.

[0239] Furthermore, after each digit adjustment of the other gradient elements corresponding to a hit target point, the following operations can be performed:

[0240] Specifically, for each of the other gradient elements from the previous round, perform the following operations:

[0241] In the previous round adjustment distinction information, a previous round adjustment distinction element corresponding to another previous round adjustment gradient element is determined; when adjusting the first vector parameter, the previous round adjustment distinction information is the distinction information that has not been adjusted; each distinction element contained in the distinction information corresponds to a target point.

[0242] like Figure 6C As shown, when adjusting other previous round adjustment gradient elements using the hit gradient element corresponding to a vector parameter, it is also necessary to adjust the previous round adjustment differentiation element corresponding to each other previous round adjustment gradient element. That is, based on the digit information of the hit gradient element, the digit of a previous round adjustment differentiation element is adjusted to obtain the corresponding current round adjustment differentiation element.

[0243] In this way, after adjusting each of the other gradient elements from the previous round, we can obtain the adjustment difference information for the current round by matching the gradient element with the corresponding gradient element from the previous round.

[0244] In this approach, while processing other gradient adjustment elements from the previous round, the corresponding distinguishing elements are also adjusted simultaneously, improving the accuracy of parameter adjustment information acquisition.

[0245] In one possible implementation, after adjusting the output motion vectors of each previous round in the above manner, an RD cost calculation can be performed on the adjusted output motion vectors of each previous round. If the RD cost of the adjusted output motion vectors of each previous round is determined to be less than that of the unadjusted output motion vectors of each previous round, then the adjusted output motion vectors of each previous round are used as the output motion vectors of each current round. Conversely, if the RD cost of the adjusted output motion vectors of each previous round is not less than that of the unadjusted output motion vectors of each previous round, then the output motion vectors of each previous round can be directly used as the output motion vectors of each current round.

[0246] In this way, it can be ensured that after each round of optimization, the RD cost corresponding to the output motion vector can generally show a downward trend, so as to quickly obtain the output motion vector that meets the optimization conditions.

[0247] The above describes the video coding method and its various possible implementations provided in this application. In the video coding method provided in this application, when obtaining the motion vector corresponding to the target point in the prediction frame, multiple rounds of iterative optimization are performed on the initial motion vector to obtain a target motion vector that meets preset optimization conditions. Thus, after multiple iterations, the obtained target motion vector can describe the displacement of the target point between the prediction frame and the reference frame as accurately as possible, thereby reducing the amount of data containing distinguishing information between the two and improving the compression efficiency of video coding.

[0248] In each optimization round, the following steps are performed: Based on the reference frame and the output motion vectors from each previous round, the pixel information of each target point is predicted. The difference between the predicted result and the predicted frame is then obtained. This difference is used to measure the accuracy of the motion vector's description of the target point's displacement. Furthermore, gradient information is used to describe the degree of influence of each previous round's output motion vector on the difference. In this way, the gradient information guides the adjustment of the motion vector, ensuring that the adjusted motion vector corresponds to the lowest possible difference, thereby improving the accuracy of the motion vector's description of the displacement.

[0249] Furthermore, in the process of adjusting motion vectors using gradient information, the digital information of gradient information is used for related processing. The digital information of gradient information is quick to obtain and simple to process, which can quickly adjust motion vectors and avoid processing methods that consume more resources. This improves the efficiency of video encoding and processing while reducing the computational resources consumed by video encoding and processing, and can effectively improve the efficiency and effect of video encoding and processing.

[0250] The video encoding method and its various possible implementations provided in the embodiments of this application have been described above. In practical applications, the above-described implementations can be combined in different ways. The process will be described below through a possible combined implementation.

[0251] like Figure 7As shown, in affine estimation for video coding, for the current block undergoing coding, the first step is to obtain the initial motion vector corresponding to the target point. This initial motion vector is mainly obtained from other coded units adjacent to the current block. Specifically, motion vectors at other points corresponding to the target point can be obtained from spatially and temporally adjacent coded units as a candidate motion vector set. Then, the set of motion vectors with the minimum RD cost is selected from the candidate motion vector set as the corresponding initial motion vector.

[0252] Thus, after obtaining the initial motion vector, a multi-round iterative optimization process can begin based on that initial motion vector. Specifically, the following operations can be performed during each iteration:

[0253] By inputting the motion vectors output from each previous round into the current round for adjustment processing, as well as the pixel information in the reference frame, the pixel information of the target point in the prediction frame and the pixel information of the current block where the target point is located can be predicted. After obtaining the corresponding prediction results, the prediction results are compared with the prediction frame to obtain the difference information composed of the difference elements corresponding to each pixel in the current block.

[0254] Furthermore, by utilizing the distinguishing information and the vector parameters in the motion vector, the corresponding gradient information is obtained, so that each gradient element in the gradient information represents the degree of influence of a vector parameter on the distinguishing element corresponding to a pixel.

[0255] Thus, by solving a system of equations based on gradient information and distinguishing information, the corresponding parameter adjustment information is obtained by solving the system of equations. Based on this parameter adjustment information, the output motion vectors of each previous round are adjusted to obtain the output motion vector of the current round.

[0256] In solving the system of equations, the number of digits in the selected gradient elements is used to reduce the number of digits in other gradient elements, thereby adjusting the augmented matrix corresponding to the system of equations so that the augmented matrix is ​​adjusted as follows. Figure 7 In the matrix form shown, each equation in the adjusted augmented matrix contains only one unknown. Therefore, the back-substitution process can be processed in parallel to quickly obtain the solution corresponding to each unknown, that is, to quickly obtain the corresponding parameter adjustment information.

[0257] Based on the same inventive concept, embodiments of this application provide a video encoding apparatus capable of realizing the functions corresponding to the aforementioned video encoding method. Please refer to... Figure 8 The device includes an acquisition module 801 and a processing module 803, wherein:

[0258] The acquisition module 801 is used to acquire the initial motion vector of each target point in a prediction frame between the prediction frame and the reference frame; a prediction frame is any one of the prediction frames associated with the reference frame.

[0259] Processing module 802 is used to iteratively optimize each obtained initial motion vector until the obtained output motion vector meets the preset optimization conditions; wherein, during each round of optimization, processing module 802 is also used to:

[0260] Based on the reference frame and the output motion vectors of each previous round, the pixel information of each target point in a prediction frame is predicted, and the difference information between the obtained prediction result and a prediction frame is determined; in the first round of optimization, the output motion vectors of each previous round are the initial motion vectors.

[0261] Based on the distinguishing information and the output motion vectors of each previous round, the corresponding gradient information is obtained; the gradient information is used to describe the degree of influence of the output motion vectors of each previous round on the distinguishing information.

[0262] Based on gradient information, digital information is used to adjust the output motion vectors of each previous round with the goal of reducing discriminative information, so as to obtain the corresponding output motion vectors of each current round.

[0263] Optionally, the processing module 802, based on gradient information and with the goal of reducing discriminative information, adjusts the output motion vectors of each previous round to obtain the corresponding output motion vectors of each current round. Specifically, it is used for:

[0264] Determine multiple gradient elements included in the gradient information; each gradient element describes the degree of influence of a vector parameter in the output motion vector of each previous round on the distinguishing element corresponding to a target point included in the distinguishing information.

[0265] Based on the digital information of each selected gradient element, the digital information of each other gradient element in the gradient information is adjusted, and based on the adjusted gradient information and the differentiation information, the parameter adjustment information of the output motion vector of each previous round is obtained.

[0266] Based on parameter adjustment information, with the goal of reducing discriminative information, each previous round output motion vector is adjusted to obtain the corresponding current round output motion vector.

[0267] Optionally, when processing module 802 adjusts the digits of each other gradient element in the gradient information based on the digit information of each selected gradient element, it specifically performs the following:

[0268] For each other gradient element in the gradient information, perform the following operation:

[0269] Based on the preset correspondence, a gradient element corresponding to another gradient element is selected from the selected gradient elements and used as the target gradient element.

[0270] Based on the preset number system type, the target gradient element and another gradient element are converted to obtain the converted target gradient element and another gradient element.

[0271] Based on the digital information of the transformed target gradient element, and in conjunction with the target gradient element, the digital information of another transformed gradient element is adjusted.

[0272] Optionally, the processing module 802 is used to adjust the digits of another gradient element after transformation based on the digit information of the transformed target gradient element, specifically for:

[0273] The digital information of the transformed target gradient element and the digital information of another transformed gradient element are fused together.

[0274] When the fused digital information exceeds a set threshold, based on the first difference between the fused digital information and the set threshold, the larger value among the digital information of the transformed target gradient element and the digital information of another transformed gradient element is subjected to bit reduction processing.

[0275] When the target gradient element with reduced bit depth is obtained, based on the second difference between the bit depth information of the target gradient element and the first difference, the product between another gradient information and the target gradient element with reduced bit depth is processed to reduce bit depth.

[0276] When another gradient information after bit reduction is obtained, the second difference between the bit information of the target gradient element and the first difference is used to perform bit reduction processing on the product between the target gradient information and the other gradient information after bit reduction.

[0277] Optionally, after adjusting the digits of each other gradient element in the gradient information based on the digit information of each selected gradient element, the processing module 802 is further configured to:

[0278] Based on the digital information of each selected gradient element, the digital information of each difference element in the difference information is adjusted; each difference element is used to describe the difference between the pixel information of a target point and a prediction frame.

[0279] Based on the adjusted gradient information and the distinguishing information, the parameter adjustment information of the output motion vector of each previous round is obtained, including:

[0280] Based on the adjusted gradient information, and combined with the adjusted distinguishing elements, the corresponding parameter adjustment elements are obtained for each vector parameter, and the obtained parameter adjustment elements are used as parameter adjustment information.

[0281] Optionally, when processing module 802 adjusts the digits of each other gradient element in the gradient information based on the digit information of each selected gradient element, it specifically performs the following:

[0282] For each vector parameter, perform the following operations sequentially:

[0283] From multiple gradient elements adjusted in the previous round corresponding to a vector parameter, select the gradient element whose value matches the preset filtering condition and use it as the hit gradient element; when adjusting the first vector parameter, the gradient element adjusted in the previous round is the gradient element that has not been adjusted.

[0284] Based on the digit information of the hit gradient element, for each of the other previous round adjusted gradient elements among the multiple previous round adjusted gradient elements, the following operations are performed: the value of one previous round adjusted gradient element is adjusted to a preset value; the hit target point corresponding to the previous round adjusted gradient element is determined, and based on the digit information of the hit gradient element, the digits of the other previous round adjusted gradient elements corresponding to the hit target point are adjusted to obtain the corresponding current round adjusted gradient element.

[0285] Optionally, based on the digit information of the hit gradient element, the digits of other previous round gradient elements corresponding to the hit target point are adjusted to obtain the corresponding adjusted gradient elements for this round. The processing module 802 is then further used for:

[0286] For each of the other previous rounds, adjust the gradient elements individually, performing the following operations:

[0287] In the previous round adjustment distinction information, a previous round adjustment distinction element corresponding to another previous round adjustment gradient element is determined; when adjusting the first vector parameter, the previous round adjustment distinction information is the distinction information that has not been adjusted; each distinction element contained in the distinction information corresponds to a target point;

[0288] Based on the digit information of the hit gradient element, the digit of a distinguishing element in the previous round of adjustment is adjusted to obtain the corresponding distinguishing element in the current round of adjustment;

[0289] Based on the obtained distinguishing elements of each adjustment in this round, and the distinguishing elements of the previous round corresponding to the hit gradient elements, the distinguishing information of this round of adjustments is obtained.

[0290] Optionally, when processing module 802 obtains the corresponding gradient information based on the difference information and the output motion vectors of each previous round, it is specifically used for:

[0291] For each previous round's output motion vector, perform the following operations: determine the vector parameters corresponding to a previous round's output motion vector;

[0292] In the differentiation information, the differentiation elements corresponding to each target point are determined respectively; each differentiation element is used to describe the difference between the pixel information of a target point in the prediction result and a prediction frame.

[0293] For each vector parameter, perform the following operations: extract the gradient of a vector parameter based on the distinguishing element corresponding to each target point, and obtain the corresponding gradient element; each gradient element is used to describe the degree of influence of a vector parameter on a distinguishing element; and use the obtained gradient elements as gradient information.

[0294] Please refer to Figure 9 This is a computer device 900 provided in the embodiments of this application. The computer device 900 can, for example, be... Figure 1C The client 91 or server 92 in the system. The current and historical versions of the data storage program and the corresponding application software can be installed on a computer device 900, which includes a processor 980 and a memory 920. In some embodiments, the computer device 900 may include a display unit 940, which includes a display panel 941 for displaying a user-interactive interface, etc.

[0295] In one possible embodiment, the display panel 941 may be configured in the form of a liquid crystal display (LCD) or an organic light-emitting diode (OLED).

[0296] The processor 980 is used to read a computer program and then execute the methods defined by the computer program. For example, the processor 980 reads a data storage program or file, thereby running the data storage program on the computer device 900 and displaying the corresponding interface on the display unit 940. The processor 980 may include one or more general-purpose processors, and may also include one or more DSPs (Digital Signal Processors) for performing related operations to implement the technical solutions provided in the embodiments of this application.

[0297] The memory 920 generally includes main memory and secondary storage. Main memory can be random access memory (RAM), read-only memory (ROM), and cache, etc. Secondary storage can be a hard disk, optical disk, USB flash drive, floppy disk, or magnetic tape drive, etc. The memory 920 is used to store computer programs and other data. The computer programs include applications corresponding to each client, and other data may include data generated after the operating system or applications are run, including system data (e.g., operating system configuration parameters) and user data. In this embodiment, the computer program is stored in the memory 920, and the processor 980 executes the computer program in the memory 920 to implement any of the methods described in the preceding figures.

[0298] The aforementioned display unit 940 is used to receive input digital information, character information, or contact touch operations / non-contact gestures, and to generate signal inputs related to user settings and function control of the computer device 900. Specifically, in this embodiment, the display unit 940 may include a display panel 941. The display panel 941, for example, is a touch screen, which can collect touch operations performed by the user on or near it (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or on the display panel 941), and drive corresponding connection devices according to a pre-set program.

[0299] In one possible embodiment, the display panel 941 may include two parts: a touch detection device and a touch controller. The touch detection device detects the user's touch position and the signal generated by the touch operation, and transmits the signal to the touch controller. The touch controller receives touch information from the touch detection device, converts it into touch point coordinates, and sends it to the processor 980. It can also receive and execute commands from the processor 980.

[0300] The display panel 941 can be implemented using various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the display unit 940, in some embodiments, the computer device 900 may also include an input unit 930. The input unit 930 may include an image input device 931 and other input devices 932, wherein the other input devices may include, but are not limited to, one or more of the following: a physical keyboard, function keys (such as volume control buttons, power buttons, etc.), a trackball, a mouse, and a joystick.

[0301] In addition to the above, the computer device 900 may also include a power supply 990 for powering other modules, an audio circuit 960, a near-field communication module 970, and an RF circuit 910. The computer device 900 may also include one or more sensors 950, such as an accelerometer, a light sensor, and a pressure sensor. The audio circuit 960 specifically includes a speaker 961 and a microphone 962, for example, the computer device 900 can use the microphone 962 to collect the user's voice and perform corresponding operations.

[0302] As one embodiment, the number of processors 980 can be one or more, and the processors 980 and the memory 920 can be coupled together or relatively independent.

[0303] As one example, Figure 9 The processor 980 in the middle can be used to implement, for example Figure 9 The functions of the acquisition module and the processing module.

[0304] As one example, Figure 9 The processor 980 in the text can be used to implement the functions of the server or terminal devices discussed above.

[0305] In some possible implementations, various aspects of the video coding method provided in this application may also be implemented in the form of a computer program product, which includes a computer program that, when run on an electronic device, causes the electronic device to perform the steps of the video coding method according to the various exemplary embodiments of this application described above.

[0306] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0307] Alternatively, if the integrated units of this invention are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this invention, or the parts that contribute to the prior art, can be embodied in the form of software products, for example, through computer program products. These computer program products are stored in a storage medium and include computer programs used to cause a computer device to execute all or part of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.

[0308] The program product of the embodiments of this application may employ a portable compact disc read-only memory (CD-ROM) and include a computer program, and may run on an electronic device. However, the program product of this application is not limited thereto. In this document, the readable storage medium may be any tangible medium that contains or stores a program that may be used by or in conjunction with a command execution system, apparatus, or device.

[0309] A readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying a readable computer program. This propagated data signal may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of sending, propagating, or transmitting a program for use by or in conjunction with a command execution system, apparatus, or device.

[0310] Computer programs contained on readable media may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0311] Computer programs for performing the operations of this application can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The computer program can execute entirely on the user's electronic device, partially on the user's electronic device, as a standalone software package, partially on the user's electronic device and partially on a remote electronic device, or entirely on a remote electronic device. In cases involving remote electronic devices, the remote electronic device can be connected to the user's electronic device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external electronic device (e.g., via the Internet using an Internet service provider).

[0312] It should be noted that although several units or sub-units of the device have been mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to embodiments of this application, the features and functions of two or more units described above can be embodied in one unit. Conversely, the features and functions of one unit described above can be further divided and embodied by multiple units.

[0313] Furthermore, although the operations of the method of this application are described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in that specific order, or that all the operations shown must be performed to achieve the desired result. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0314] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0315] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in one or more blocks of the flowchart illustrations and / or one or more blocks of the block diagrams.

[0316] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowcharts and / or one or more block diagrams.

[0317] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide steps for implementing the functions specified in one or more flowcharts and / or one or more block diagrams.

[0318] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A video encoding method, characterized in that, The method includes: Obtain the initial motion vector of each target point in a prediction frame between the prediction frame and the reference frame; the prediction frame is any one of the prediction frames associated with the reference frame. The initial motion vectors are iteratively optimized until the output motion vectors satisfy the preset optimization conditions. During each optimization round, the following operations are performed: Based on the reference frame and the output motion vectors of each previous round, the pixel information of each target point in the prediction frame is predicted, and the difference information between the obtained prediction result and the prediction frame is determined; in the first round of optimization, the output motion vectors of each previous round are the initial motion vectors. Based on the distinguishing information and the output motion vectors of each previous round, corresponding gradient information is obtained; the gradient information is used to describe the degree of influence of the output motion vectors of each previous round on the distinguishing information. Based on the digital information of the gradient information, with the goal of reducing the difference information, the output motion vectors of each previous round are adjusted to obtain the corresponding output motion vectors of each current round.

2. The method as described in claim 1, characterized in that, The digital information based on the gradient information, with the goal of reducing the distinguishing information, adjusts the output motion vectors of each previous round to obtain the corresponding output motion vectors of each current round, including: Determine multiple gradient elements included in the gradient information; each gradient element is used to describe the degree of influence of a vector parameter in the output motion vector of each previous round on the difference element corresponding to a target point included in the difference information. Based on the digital information of each selected gradient element, the digital information of each other gradient element in the gradient information is adjusted, and based on the adjusted gradient information and the difference information, the parameter adjustment information of each previous round output motion vector is obtained. Based on the parameter adjustment information, with the goal of reducing the difference information, the following steps are performed on each previous round output motion vector: each vector parameter of a previous round output motion vector is adjusted to obtain the corresponding current round output motion vector.

3. The method as described in claim 2, characterized in that, The step of adjusting the digits of each other gradient element in the selected gradient information based on the digit information includes: For each other gradient element in the gradient information, perform the following operation: Based on a preset correspondence, a gradient element corresponding to another gradient element is selected from the chosen gradient elements and used as the target gradient element. Based on a preset number system, the target gradient element and the other gradient element are subjected to number system conversion to obtain the converted target gradient element and the converted other gradient element. Based on the digital information of the transformed target gradient element, and in conjunction with the target gradient element, the digital information of another transformed gradient element is adjusted.

4. The method as described in claim 3, characterized in that, The step of adjusting the digits of another gradient element after conversion, based on the digit information of the converted target gradient element and in conjunction with the target gradient element, includes: The digital information of the transformed target gradient element and the digital information of another transformed gradient element are fused together. When the fused digital information exceeds a set threshold, based on the first difference between the fused digital information and the set threshold, the larger of the digital information of the transformed target gradient element and the digital information of another transformed gradient element is subjected to bit reduction processing. When the target gradient element with reduced bit depth is obtained, based on the second difference between the bit depth information of the target gradient element and the first difference, the product between the other gradient information and the target gradient element with reduced bit depth is subjected to bit depth reduction processing. When another gradient information after bit reduction is obtained, based on the second difference between the bit information of the target gradient element and the first difference, bit reduction processing is performed on the product between the target gradient information and the other gradient information after bit reduction.

5. The method as described in claim 3 or 4, characterized in that, After adjusting the digits of each other gradient element in the selected gradient information based on the digit information of each gradient element, the method further includes: Based on the digital information of each selected gradient element, the digital information of each difference element in the difference information is adjusted; each difference element is used to describe the difference between the pixel information of a target point and the prediction result and the prediction frame. The parameter adjustment information for each previous round's output motion vector, obtained based on the adjusted gradient information and the distinguishing information, includes: Based on the adjusted gradient information, and combined with each adjusted distinguishing element, the corresponding parameter adjustment element is obtained for each vector parameter, and the obtained parameter adjustment elements are used as the parameter adjustment information.

6. The method as described in claim 2, characterized in that, The step of adjusting the digits of each other gradient element in the selected gradient information based on the digit information includes: For each vector parameter, perform the following operations sequentially: From multiple gradient elements adjusted in the previous round corresponding to a vector parameter, the gradient element adjusted in the previous round whose value matches the preset filtering condition is selected as the hit gradient element; when adjusting the first vector parameter, the gradient element adjusted in the previous round is the gradient element that has not been adjusted. Based on the digit information of the hit gradient element, for each of the other previous round adjustment gradient elements among the plurality of previous round adjustment gradient elements, the following operations are performed: the value of one previous round adjustment gradient element is adjusted to a preset value; the hit target point corresponding to the one previous round adjustment gradient element is determined, and based on the digit information of the hit gradient element, the digits of the other previous round gradient elements corresponding to the hit target point are adjusted to obtain the corresponding current round adjustment gradient element.

7. The method as described in claim 6, characterized in that, After adjusting the digits of other gradient elements from the previous round corresponding to the hit target point based on the digit information of the hit gradient element to obtain the corresponding adjusted gradient element for the current round, the method further includes: For each of the other previous rounds, adjust the gradient elements by performing the following operations: In the previous round adjustment distinction information, a previous round adjustment distinction element corresponding to another previous round adjustment gradient element is determined; when adjusting the first vector parameter, the previous round adjustment distinction information is the distinction information that has not been adjusted; each distinction element contained in the distinction information corresponds to a target point; Based on the digit information of the hit gradient element, the digit of the previous round adjustment difference element is adjusted to obtain the corresponding current round adjustment difference element; Based on the obtained distinguishing elements of each adjustment in this round, and the distinguishing elements of the previous round corresponding to the hit gradient elements, the distinguishing information of this round of adjustments is obtained.

8. The method according to any one of claims 1-4, characterized in that, The step of obtaining corresponding gradient information based on the difference information and the output motion vectors of each previous round includes: For each of the previous wheel output motion vectors, perform the following operations: determine the vector parameters corresponding to a previous wheel output motion vector; In the distinction information, the distinction elements corresponding to each target point are determined respectively; each distinction element is used to describe the difference between the pixel information of a target point and the prediction result and the prediction frame. For each of the vector parameters, the following operations are performed: based on the distinguishing elements corresponding to each target point, gradient extraction is performed on a vector parameter to obtain the corresponding gradient elements; each gradient element is used to describe the degree of influence of a vector parameter on a distinguishing element; and the obtained gradient elements are used as the gradient information.

9. A video encoding device, characterized in that, include: The acquisition module is used to acquire the initial motion vector of each target point in a prediction frame between the prediction frame and the reference frame. The predicted frame is any one of the predicted frames associated with the reference frame; The processing module is used to iteratively optimize each obtained initial motion vector until the obtained output motion vector satisfies the preset optimization conditions; wherein, during each round of optimization, the processing module is further used to: Based on the reference frame and the output motion vectors of each previous round, the pixel information of each target point in the prediction frame is predicted, and the difference information between the obtained prediction result and the prediction frame is determined; in the first round of optimization, the output motion vectors of each previous round are the initial motion vectors. Based on the distinguishing information and the output motion vectors of each previous round, corresponding gradient information is obtained; the gradient information is used to describe the degree of influence of the output motion vectors of each previous round on the distinguishing information. Based on the digital information of the gradient information, with the goal of reducing the difference information, the output motion vectors of each previous round are adjusted to obtain the corresponding output motion vectors of each current round.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the method as described in any one of claims 1 to 8.

11. A computer device, characterized in that, include: Memory, used to store program instructions; A processor is configured to invoke program instructions stored in the memory and execute the method as described in any one of claims 1 to 8 according to the obtained program instructions.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions for causing a computer to perform the method as described in any one of claims 1 to 8.