Video processing method and device, electronic equipment, storage medium and program product

Abstract

Embodiments of the present disclosure provide a video processing method and device, electronic equipment, storage medium and program product. The method comprises: first acquiring a to-be-encoded video frame, determining a current coding unit based on the to-be-encoded video frame; then acquiring feature information related to the current coding unit in the encoding process, and determining a motion vector refinement probability based on the feature information related to the current coding unit; and finally determining whether to perform a motion vector refinement process for the current coding unit based on the motion vector refinement probability. In this embodiment, the motion vector refinement probability is determined by extracting the existing feature information in the encoding process, and the motion vector refinement process for the current coding unit is intelligently determined based on the motion vector refinement probability, so as to intelligently control the motion vector refinement process, reduce the computational complexity, and balance the encoding complexity and the encoding performance without affecting the compression efficiency.

Description

Technical Field

[0001] This disclosure relates to the field of computer vision technology, and more specifically, to a video processing method, apparatus, electronic device, storage medium, and program product. Background Technology

[0002] In video coding standards, such as High Efficiency Video Coding (HEVC), Versatile Video Coding (VVC), and Enhanced Compression Model (ECM), which is an extension of VCC, prediction modes include intra-frame prediction and inter-frame prediction. Motion estimation (ME) is the core process of inter-frame prediction.

[0003] However, while motion estimation can improve compression efficiency, it is extremely computationally intensive, especially in high-resolution, complex motion scenarios, which leads to increased coding complexity.

[0004] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] This disclosure provides a video processing method, apparatus, electronic device, storage medium, and program product that reduces computational load and coding complexity without affecting compression efficiency.

[0006] According to a first aspect of the present disclosure, a video processing method is provided, comprising: acquiring a video frame to be encoded; determining a current coding unit based on the video frame to be encoded; acquiring feature information related to the current coding unit during the encoding process; determining a motion vector (MV) refinement probability based on the feature information related to the current coding unit; and determining whether to perform a motion vector refinement (MV refinement) process for the current coding unit based on the motion vector refinement probability.

[0007] In some exemplary embodiments of this disclosure, determining the motion vector thinning probability based on feature information related to the current coding unit includes: using the feature information related to the current coding unit as input to a probability prediction model to obtain the motion vector thinning probability output by the probability prediction model.

[0008] In some exemplary embodiments of this disclosure, the probabilistic prediction model includes a logistic regression model.

[0009] In some exemplary embodiments of this disclosure, determining whether to perform a motion vector refinement process for the current coding unit based on the motion vector refinement probability includes: skipping the motion vector refinement process for the current coding unit if the motion vector refinement probability is less than or equal to a preset probability threshold; and using the first motion vector obtained by the current coding unit in the initial motion estimation process as the optimal motion vector, which is then used for inter-frame prediction of the video frame to be encoded.

[0010] In some exemplary embodiments of this disclosure, determining whether to trigger the motion vector refinement process for the current coding unit based on the motion vector refinement probability further includes: if the motion vector refinement probability is greater than a preset probability threshold, performing motion vector filtering based on the first motion vector to obtain multiple candidate motion vectors; calculating the rate-distortion cost of each candidate motion vector; and selecting the candidate motion vector with the lowest rate-distortion cost as the optimal motion vector.

[0011] In some exemplary embodiments of this disclosure, the feature information associated with the current coding unit includes at least one of the following: the size information of the current coding unit; the proportion of a set coding unit in the video frame to be encoded; the ratio of the rate-distortion cost of inter-frame prediction to the rate-distortion cost of intra-frame prediction of the current coding unit; the distortion information of a first motion vector in the current coding unit; the distribution of distortion information of a set number of second motion vectors associated with the first motion vector; and the prediction mode of the coding units adjacent to the current coding unit.

[0012] In some exemplary embodiments of this disclosure, the method further includes: recording distortion information of each preset motion vector during the initial motion estimation process; obtaining feature information related to the current encoding unit during the encoding process, including: determining the distortion information of a first motion vector and the distortion information of a set number of second motion vectors based on the recorded distortion information of each preset motion vector.

[0013] In some exemplary embodiments of this disclosure, determining the distortion information of a first motion vector and the distortion information of a predetermined number of second motion vectors based on the recorded distortion information of each preset motion vector includes: querying the distortion information of each preset motion vector based on the first motion vector, and using the queried distortion information as the distortion information of the first motion vector; for each second motion vector, querying the distortion information of each preset motion vector based on the second motion vector, and using the queried distortion information as the distortion information of the second motion vector; if the distortion information of the second motion vector is not found in the search information of each preset motion vector, calculating the distortion information of the second motion vector.

[0014] In some exemplary embodiments of this disclosure, in the initial motion estimation process, recording the distortion information of each preset motion vector includes: in the initial motion estimation process, determining the array index corresponding to the preset motion vector; for preset motion vectors that have been searched, using the calculated distortion information as array elements, and for preset motion vectors that have not been searched, using the set distortion information as array elements; based on the distortion information of each preset motion vector, determining the distortion information of a first motion vector and the distortion information of a set number of second motion vectors, including: determining the first array index of the first motion vector, using the array element corresponding to the first array index as the distortion information of the first motion vector; for each second motion vector, determining the second array index corresponding to the second motion vector, and obtaining the array element corresponding to the second array index; if the array element corresponding to the second array index is non-set distortion information, using the array element corresponding to the second array index as the distortion information of the second motion vector; if the array element corresponding to the second array index is set distortion information, calculating the distortion information of the second motion vector.

[0015] According to a second aspect of the present disclosure, a video processing apparatus is provided, comprising: a current coding unit determination module, configured to acquire a video frame to be encoded and determine a current coding unit based on the video frame to be encoded; a related feature information acquisition module, configured to acquire feature information related to the current coding unit during the encoding process; a motion vector thinning probability determination module, configured to determine a motion vector thinning probability based on the feature information related to the current coding unit; and a motion vector thinning execution judgment module, configured to determine whether to execute a motion vector thinning process for the current coding unit based on the motion vector thinning probability.

[0016] According to a third aspect of the present disclosure, an electronic device is provided, characterized in that it includes: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to execute the executable instructions to implement the video processing method of any of the above.

[0017] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided, wherein when the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the video processing method or the video quality assessment method described above.

[0018] According to a fifth aspect of the present disclosure, a computer program product is provided, including a computer program that is executed by a processor using the video processing method described above.

[0019] The video processing method provided in this disclosure first determines the current coding unit based on the video frame to be encoded; then, it acquires feature information related to the current coding unit during the encoding process and determines the motion vector thinning probability based on the feature information related to the current coding unit; finally, it determines whether to execute the motion vector thinning process for the current coding unit based on the motion vector thinning probability. In other words, in this embodiment, the motion vector thinning probability of the current coding unit is determined by extracting existing feature information during the encoding process, and intelligent decisions are made on whether the current coding unit needs to further execute the motion vector thinning process based on the motion vector thinning probability. This achieves intelligent control of the motion vector thinning process, reducing the amount of computation and the encoding complexity without affecting compression efficiency, thus achieving a balance between encoding complexity and encoding performance.

[0020] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0021] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0022] Figure 1 A schematic diagram of an exemplary system architecture to which the video processing method of embodiments of the present disclosure can be applied is shown.

[0023] Figure 2 This is a flowchart illustrating a video processing method according to an exemplary embodiment.

[0024] Figure 3 This is a flowchart illustrating a video processing method according to an exemplary embodiment.

[0025] Figure 4 This is a flowchart illustrating a method for reusing distorted information according to an exemplary embodiment.

[0026] Figure 5 This is a schematic diagram illustrating the search range of a rhombus motion according to an exemplary embodiment.

[0027] Figure 6 This is a flowchart illustrating another method for multiplexing distorted information according to an exemplary embodiment.

[0028] Figure 7 This is a flowchart illustrating yet another method for reusing distorted information according to an exemplary embodiment.

[0029] Figure 8 This is a flowchart illustrating an exemplary video processing method according to an exemplary embodiment.

[0030] Figure 9 This is a block diagram illustrating a video processing apparatus according to an exemplary embodiment.

[0031] Figure 10 This is a schematic diagram illustrating the structure of an electronic device suitable for implementing exemplary embodiments of the present disclosure, according to an exemplary embodiment. Detailed Implementation

[0032] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the embodiments set forth herein; rather, they are provided so that this disclosure will be thorough and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted.

[0033] The features, structures, or characteristics described in this disclosure can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough understanding of embodiments of this disclosure. However, those skilled in the art will recognize that the technical solutions of this disclosure can be practiced with one or more specific details omitted, or other methods, components, apparatuses, steps, etc., can be employed. In other instances, well-known methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this disclosure.

[0034] The accompanying drawings are merely illustrative of this disclosure, and the same reference numerals in the drawings denote the same or similar parts, thus omitting repeated descriptions of them. Some block diagrams shown in the drawings do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in at least one hardware module or integrated circuit, or in different network and / or processor devices and / or microcontroller devices.

[0035] The flowchart shown in the accompanying drawings is merely illustrative and does not necessarily include all content and steps, nor does it require execution in the described order. For example, some steps may be broken down, while others may be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0036] In this specification, the terms “a,” “an,” “the,” “the,” and “at least one” are used to indicate the presence of at least one element / component / etc.; the terms “comprising,” “including,” and “having” are used to indicate an open-ended inclusion and to mean that there may be other elements / components / etc. in addition to the listed elements / components / etc.; the terms “first,” “second,” and “third,” etc., are used only as markings and are not a limitation on the number of objects.

[0037] HEVC, as one of the video compression standards, significantly reduces the bit rate while maintaining image quality. To achieve this goal, the HEVC encoder introduces a more complex coding structure and prediction mechanism. Among them, motion estimation and motion compensation (MC) are the main processes in inter-frame prediction.

[0038] In the HEVC inter-frame prediction process, the encoder first preliminarily selects the optimal motion vector for each coding unit through an initial motion estimation process. Then, it performs rate-distortion optimization (RDO) on this optimal motion vector to obtain the rate-distortion cost (RD Cost) for inter-frame prediction. Based on the RDO cost of inter-frame prediction, it determines whether to adopt the inter-frame prediction mode. Specifically, if the RDO cost of inter-frame prediction is less than the RDO cost of intra-frame prediction, the inter-frame prediction mode is adopted; if the RDO cost of inter-frame prediction is greater than or equal to the RDO cost of intra-frame prediction, the intra-frame prediction mode is adopted.

[0039] To further improve coding accuracy, the encoder will also perform finer-grained rate-distortion cost optimization on multiple candidate motion vectors around the initially selected best motion vector, i.e., motion vector refinement, so as to find better compensation results and further improve coding quality.

[0040] Furthermore, the motion vector process in the HEVC encoder includes: initially selecting the motion vector of the coding unit as the current motion vector through an initial motion estimation process; performing distortion information pre-screening on several candidate motion vectors around the current motion vector (e.g., 8-neighborhood ±1, ±0.5); performing a complete RDO evaluation on the candidate motion vectors that meet the conditions; obtaining the rate-distortion cost of each candidate motion vector; and selecting the candidate motion vector with the minimum rate-distortion cost as the best motion vector. Here, a candidate motion vector that meets the conditions is one whose distortion information is less than a threshold difference from the distortion information of the current motion vector.

[0041] While motion vector thinning can improve compression efficiency, it incurs extremely high computational costs, especially in high-resolution, complex motion scenarios where coding complexity increases dramatically. Furthermore, in many cases, the coding gain from motion vector thinning is very limited, and errors in RDO cost calculations may even lead to incorrect selection of the optimal motion vector. Therefore, identifying and allocating limited computational resources to coding units that can gain significant rate-distortion benefits from motion vector thinning is crucial for achieving a balance between coding efficiency and computational complexity.

[0042] To address the aforementioned technical problems, this disclosure provides a video processing method that determines the motion vector thinning probability of a coding unit by extracting existing multidimensional feature information related to the coding unit during the coding process. Based on the motion vector thinning probability, the method makes an intelligent decision on whether the coding unit needs to further execute the motion vector thinning process, thereby achieving intelligent control of the motion vector thinning process. This reduces the computational load and coding complexity without affecting compression efficiency, achieving a balance between coding complexity and coding performance.

[0043] Figure 1 A schematic diagram of an exemplary system architecture to which the video processing method of embodiments of the present disclosure can be applied is shown.

[0044] like Figure 1 As shown, the system architecture may include server 101, network 102, terminal device 103, terminal device 104, and terminal device 105. Network 102 serves as the medium for providing a communication link between terminal device 103, terminal device 104, or terminal device 105 and server 101. Network 102 may include various connection types, such as wired or wireless communication links or fiber optic cables, etc.

[0045] Server 101 can be a server that provides various services, such as a back-end management server that supports the devices operated by users using terminal devices 103, 104, or 105. The back-end management server can analyze and process received requests and other data, and feed back the processing results to terminal devices 103, 104, or 105.

[0046] Terminal devices 103, 104, and 105 can be smartphones, tablets, laptops, desktop computers, smart speakers, wearable smart devices, virtual reality devices, augmented reality devices, etc., but are not limited to these.

[0047] In this embodiment of the disclosure, server 101 can: acquire a video to be encoded, which includes multiple video frames to be encoded, and divide each video frame to be encoded into multiple encoding units. The following description takes the unit currently being encoded as the current encoding unit as an example.

[0048] Server 101 can: acquire feature information related to the current coding unit during the encoding process; determine the motion vector thinning probability based on the feature information related to the current coding unit; and determine whether to execute the motion vector thinning process for the current coding unit based on the motion vector thinning probability.

[0049] Furthermore, if the motion vector refinement probability is less than or equal to a preset probability threshold, the motion vector refinement process for the current coding unit is skipped; the first motion vector obtained by the current coding unit in the initial motion estimation process is taken as the best motion vector, and the best motion vector is used for inter-frame prediction and coding of the video frame to be encoded.

[0050] If the motion vector refinement probability is greater than a preset probability threshold, motion vector filtering is performed based on the first motion vector to obtain multiple candidate motion vectors; the rate-distortion cost of each candidate motion vector is calculated; and the candidate motion vector with the lowest rate-distortion cost is selected as the optimal motion vector.

[0051] After the current coding unit completes encoding based on the optimal motion vector, the same method can be used to judge the next coding unit until the current video frame is encoded.

[0052] It should be understood that Figure 1 The number of terminal devices 103, 104, 105, network 102, and server 101 in the diagram is merely illustrative. Server 101 can be a single physical server, a server cluster consisting of multiple servers, or a cloud server. Depending on actual needs, it can have any number of terminal devices, networks, and servers.

[0053] The video processing method in the exemplary embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings and examples. The method provided in the embodiments of this disclosure can be executed by any electronic device, such as the one described above. Figure 1 The server and / or terminal equipment in the process, but this disclosure does not limit this.

[0054] The steps of the method in the exemplary embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings and examples.

[0055] Figure 2 This is a flowchart illustrating a video processing method according to an exemplary embodiment. Figure 2The method provided in the embodiments can be executed by any electronic device, such as the one described above. Figure 1 Terminal devices in or Figure 1 The server in, or Figure 1 The terminal devices and servers in the process execute together, but this disclosure does not limit this. For example... Figure 2 As shown, the video processing method provided in this embodiment mainly includes steps S210-S240.

[0056] In step S210, a video frame to be encoded is acquired, and the current encoding unit is determined based on the video frame to be encoded.

[0057] Here, the video frame to be encoded is a discrete, independent static image that is parsed or extracted from the video to be encoded. The encoding unit can be understood as the basic image region in which compression algorithms (e.g., prediction, transformation, quantization) are performed during the video encoding process, and the current encoding unit can be understood as the basic image region that the encoder is currently processing.

[0058] During the video encoding process, video frames to be encoded can be obtained from the sequence of video frames to be encoded. Since the encoding process is carried out in blocks, the current encoding unit is determined from the video frames to be encoded after the video frames to be encoded are obtained.

[0059] For example, the current coding unit can be a coding unit (CU); or a smaller unit obtained by dividing a coding unit; or a macroblock; or a sub-block obtained by dividing a macroblock; or a coding tree unit; or a sub-block obtained by dividing a coding tree unit, etc. In the actual coding process, the size of the coding unit can be determined according to the coding requirements.

[0060] In step S220, feature information related to the current coding unit is obtained during the coding process.

[0061] Feature information related to the current coding unit refers to parameter information extracted or calculated from the pixel data, spatial / temporal neighborhood, and ongoing coding processing of the current coding unit. This feature information can reflect the content characteristics or coding performance of the current coding unit.

[0062] In some embodiments, the feature information associated with the current coding unit includes at least one of the following: the size information of the current coding unit; the proportion of a set coding unit in the video frame to be encoded; the ratio of the rate-distortion cost of inter-frame prediction to the rate-distortion cost of intra-frame prediction of the current coding unit; the distortion information of a first motion vector in the current coding unit; the distribution of distortion information of a set number of second motion vectors associated with the first motion vector; and the mode type of the coding units adjacent to the current coding unit.

[0063] The size information of the current coding unit refers to its geometric size in space, expressed as pixel width × pixel height, such as 64×64 or 32×16. Size information reflects the granularity of coding and is a fundamental parameter that determines coding complexity and distortion characteristics.

[0064] The size information of the current coding unit can be directly read from the coding unit partitioning data. That is, when the encoder determines the current coding unit from the video frame to be encoded, the pixel width and pixel height of the current coding unit have been determined and stored in the corresponding process state variables.

[0065] The coding unit can be configured to use either a non-integer pixel motion vector or a large-amplitude motion vector. A coding unit using a non-integer pixel motion vector means that the resolution of the horizontal and / or vertical components of the optimal motion vector used in inter-frame prediction is higher than integer pixel precision, including but not limited to interpolation positions with 1 / 2 pixel, 1 / 4 pixel, or higher precision. A coding unit using a large-amplitude motion vector means that the amplitude of the optimal motion vector used in inter-frame prediction exceeds a preset amplitude threshold, where the amplitude can be represented by absolute value and / or Euclidean distance.

[0066] The percentage of set coding units in a video frame to be encoded refers to the proportion of the number of set coding units to the total number of coding units in the entire video frame to be encoded. The percentage of set coding units is used to characterize the overall coding characteristics at the frame level or in an image region.

[0067] During the encoding of a video frame, the encoder maintains a counter that accumulates the number of set coding units. The counter increments by 1 when a coding unit uses a non-integer pixel motion vector, or when a coding unit uses a large-amplitude motion vector. At the end of encoding the current coding unit, the accumulated value of the counter is divided by the total number of coding units processed so far to obtain the percentage of set coding units in the video frame to be encoded.

[0068] Intra-prediction rate-distortion cost refers to the rate-distortion cost calculated by the encoder when attempting to use intra-prediction on the current coding unit to evaluate the quality of the intra-prediction mode. Inter-prediction rate-distortion cost refers to the rate-distortion cost calculated by the encoder when attempting to use inter-prediction on the current coding unit to evaluate the quality of the inter-prediction mode.

[0069] The ratio of the rate-distortion cost of inter-frame prediction to the rate-distortion cost of intra-frame prediction for the current coding unit refers to the ratio between the rate-distortion cost of inter-frame prediction and the rate-distortion cost of intra-frame prediction when making coding mode decisions for the current coding unit. This ratio is used to characterize the relative efficiency of temporal prediction and spatial prediction for the current coding unit.

[0070] The encoder performs various mode searches for intra-frame prediction and inter-frame prediction according to a preset procedure. For intra-frame prediction, after trying all or some intra-frame candidate modes, the minimum rate-distortion cost is recorded. For inter-frame prediction, after completing the initial motion estimation, the rate-distortion cost corresponding to its first motion vector is recorded, and the ratio of the minimum rate-distortion cost to the rate-distortion cost corresponding to the first motion vector is calculated. This ratio represents the rate-distortion cost of the current coding unit's inter-frame prediction to the rate-distortion cost of its intra-frame prediction.

[0071] The first motion vector refers to the final motion vector determined by the current coding unit during the initial motion estimation process. The process for determining the first motion vector can be referred to the description in the example below. The distortion information of the first motion vector refers to the error metric calculated after performing motion compensation on the current coding unit using the first motion vector.

[0072] The aforementioned distortion information includes, but is not limited to: SAD (Sum of Absolute Differences), SATD (Sum of Absolute Transformed Difference), and SSE (Sum of Squared Errors).

[0073] The above-mentioned number can be related to the motion search method used in the initial transportation estimation process. For example, when using a rhombus motion search, the number is set to 4, and when using a hexagonal motion search, the number is set to 6.

[0074] The second motion vector associated with the first motion vector refers to a motion vector with a different sub-pixel offset that is located around the finally selected first motion vector during the initial motion estimation process, wherein the first motion vector and the second motion vector are located in the current coding unit.

[0075] The distortion information of the second motion vector refers to the error metric calculated after motion compensation is performed on the current coding unit using the second motion vector. The process for determining the second motion vector can be referred to the description in the following example.

[0076] In this embodiment, the distortion information of the first motion vector and the distortion information of the second motion vector can reuse the distortion information calculated and recorded in the initial motion estimation process (sub-pixel search process), avoiding the repeated calculation of distortion information and saving computing resources.

[0077] The mode type of a coding unit adjacent to the current coding unit refers to the final prediction mode (e.g., intra-frame prediction mode, inter-frame prediction mode, etc.) adopted by the coding units that have completed their coding decisions and are spatially adjacent to the current coding unit (e.g., above, left, right, below, etc.). The mode type of the coding units adjacent to the current coding unit represents local context information and is used to predict the coding mode tendency of the current unit.

[0078] This is obtained by accessing the encoder's context storage area. Specifically, when processing the current coding unit, the encoder reads out the prediction patterns used by its neighboring, already decided coding units.

[0079] Traditional encoders often employ fixed or simple rule-based strategies to determine whether to perform refined motion search, making it difficult to adapt to the complex spatiotemporal changes of video content. This results in excessive computation in simple regions and insufficient computation in complex regions. This embodiment addresses this by comprehensively analyzing diverse feature information from local features (e.g., the size information of the current coding unit, the ratio of rate-distortion cost of inter-frame prediction to intra-frame prediction, the distortion information of the first motion vector in the current coding unit, and the distribution of distortion information of a set number of second motion vectors associated with the first motion vector) to global features (e.g., the proportion of a set number of coding units in the video frame to be encoded). This fully utilizes contextual information during the encoding process to dynamically evaluate the necessity and potential benefits of motion vector refinement for the current coding unit. This allows the encoder to accurately allocate limited computational resources to the critical regions that best improve coding efficiency, while avoiding excessive computational resource consumption in non-critical regions. This reduces coding complexity and time overhead while maintaining or even improving coding efficiency.

[0080] In step S230, the motion vector refinement probability is determined based on the feature information associated with the current coding unit.

[0081] Motion vector refinement can be understood as a more refined motion search process performed after the initial motion estimation (sub-pixel search) has been completed, in order to pursue higher prediction accuracy and coding efficiency. Motion vector refinement includes, but is not limited to: searching with a smaller search step size, expanding the search range to find potential better matching points, or trying to use more complex motion models, etc.

[0082] Motion vector refinement probabilities are used to represent the degree of recommendation or necessity for performing motion vector refinement on the current coding unit. These probabilities are mapped to a continuous numerical range (e.g., [0, 1]) or discretized into several levels (e.g., "high", "medium", "low"). A higher probability value indicates that, based on the feature information of the current code, the expected coding gain from motion vector refinement is greater, or the uncertainty of the first motion vector is higher, thus making it more worthwhile to invest additional computational resources for refinement.

[0083] In one possible implementation, after normalizing the at least one feature information related to the current coding unit obtained in step S220, a feature vector X = [x1, x2, x3, x4, x5, x6] is formed. For example: x1 is the size information of the current coding unit, x2 is the proportion of the set coding unit in the video frame to be encoded, x3 is the ratio of the rate-distortion cost of inter-frame prediction to the rate-distortion cost of intra-frame prediction of the current coding unit, x4 is the distortion information of the first motion vector in the current coding unit, x5 is the distribution of the distortion information of a set number of second motion vectors associated with the first motion vector, and x6 is the prediction mode of the coding units adjacent to the current coding unit.

[0084] It should be noted that this embodiment uses a feature vector X comprising six feature pieces as an example. In practical applications, any one or more of these six feature pieces can be selected to construct the feature vector, depending on the actual situation. Furthermore, the six feature pieces mentioned above are merely illustrative and are not limited to only six; other feature pieces with practical significance during the encoding process can also be included. Moreover, the order of the feature pieces in the feature vector can also be adjusted according to the actual situation.

[0085] The running vector refinement probability P is calculated based on the feature vector X using one or more of the following methods.

[0086] Method 1: Normalize each feature in the feature vector X so that it falls within a set numerical range (e.g., [0, 1]). Set a predefined weight coefficient wi for each feature xi. The weight coefficient wi represents the degree of influence of the feature xi on whether motion vector refinement decision needs to be performed. The weight coefficient wi can be determined through experience or offline optimization.

[0087] Calculate the weighted sum S of each feature xi and each weight coefficient wi, and map the weighted sum S to the interval [0, 1] through a nonlinear function to obtain the final running vector refinement probability P.

[0088] Method 2: Discretize each feature xi in the feature vector X into several levels (e.g., "high", "medium", "low"). Based on the discrete level combinations of all features, a multidimensional record table is predefined. The multidimensional record table stores the probabilities corresponding to each discrete level combination. The content of the multidimensional lookup table is filled offline based on statistical results from numerous coding experiments or expert experience.

[0089] During encoding, the multidimensional lookup table is indexed based on the discrete levels of each feature information of the current encoding unit, and the corresponding probability is directly read out as the running vector refinement probability P.

[0090] Method 3: In the offline phase, a classification model is trained using training data from a large number of encoding units to extract feature vectors X and indicate whether motion vector refinement incurs rate-distortion costs. In the online phase, the feature vector X of the current encoding unit is input into the trained model, and the model directly outputs a probability value between 0 and 1, i.e., the motion vector refinement probability P.

[0091] It should be noted that this embodiment only explains the method for determining the refinement probability P of the running vector, and is not a limitation.

[0092] In step S240, a motion vector thinning process for the current coding unit is determined based on the motion vector thinning probability.

[0093] In this embodiment, the motion vector refinement probability P obtained in step S230 is compared with a preset probability threshold T, and the motion vector refinement process for the current coding unit is determined based on the comparison result.

[0094] Specifically, if the motion vector refinement probability P is greater than the preset probability threshold T, the motion vector refinement process needs to be executed. If the motion vector refinement probability P is less than or equal to the preset probability threshold T, the motion vector refinement process is skipped.

[0095] In this embodiment of the disclosure, the motion vector refinement probability of the current coding unit is determined by extracting the existing feature information during the coding process, and intelligent decision is made on whether the current coding unit needs to further execute the motion vector refinement process based on the motion vector refinement probability. This achieves intelligent control of the motion vector refinement process, reduces the amount of computation and coding complexity without affecting the compression efficiency, and achieves a balance between coding complexity and coding performance.

[0096] Based on the above embodiments, the present disclosure optimizes the video processing method as follows: Figure 3 As shown, the optimized video processing method mainly includes steps S310-S380.

[0097] In step S310, the video frame to be encoded is acquired, and the current encoding unit is determined based on the video frame to be encoded.

[0098] In step S320, feature information related to the current coding unit is obtained during the coding process.

[0099] The implementation of steps S310-S320 in this embodiment is the same as that of steps S210-S220 in the above embodiment. For details, please refer to the description in the above embodiment. This embodiment will not be specifically limited.

[0100] In step S330, the feature information related to the current coding unit is used as the input to the probabilistic prediction model to obtain the motion vector refinement probability output by the probabilistic prediction model.

[0101] The probabilistic prediction model described above can be a trained machine learning model that establishes a mapping relationship from input feature vectors to output probabilities. This model receives multidimensional feature information related to the current encoding unit, performs internal calculations, and outputs a probability within a continuous interval (e.g., [0, 1]).

[0102] The aforementioned probabilistic prediction models include, but are not limited to: logistic regression models, lightweight neural network models (e.g., multilayer perceptrons), decision trees (e.g., binary classification GBDT (Gradient Boosting Decision Tree) models), support vector machines, random forest models, and so on.

[0103] In this embodiment, a large number of video sequence samples are collected. During the encoding process, the relevant feature information of each sample encoding unit is extracted and labeled with a true label of "whether motion vector thinning should be performed". For example, if the rate-distortion cost is reduced by more than a certain threshold after performing motion vector thinning, it is marked as 1, otherwise it is marked as 0. A classification model is trained using the sample data pairs (feature information related to the sample encoding unit, true label).

[0104] Before the encoder starts working, the pre-built or trained probabilistic prediction model is loaded into memory and initialized.

[0105] The preprocessed feature vector X associated with the coding unit is input into the initialized probabilistic prediction model. The probabilistic prediction model processes the input feature vector X according to its structure and outputs a scalar value P, which is the motion vector refinement probability.

[0106] In this embodiment, the probabilistic prediction model can learn the implicit relationship between the feature information related to the current coding unit and the expected benefits brought by performing motion vector refinement operations. This enables the encoder to make decisions based on the learned knowledge, rather than simple conditional judgments, thereby improving the accuracy and reliability of the decisions and allowing computing resources to be allocated to the units that can generate the most coding gain.

[0107] In some embodiments, the probabilistic prediction model includes a logistic regression model.

[0108] In this embodiment, the feature vector X associated with the current coding unit is input into the trained logistic regression model to obtain the motion vector refinement probability output by the logistic regression model.

[0109] The implementation of the logistic regression model is shown in formula (1).

[0110] (1)

[0111] Where X is the feature vector associated with the current coding unit. This is the weight vector of the logistic regression model. This is the bias term in the logistic regression model. This indicates that the weight vector Convert a column vector to a row vector. This represents the conditional probability of event y=1 occurring given an input feature vector X. In this embodiment, This is the predicted probability value for the motion vector refinement process that should be performed on the current coding unit.

[0112] When making predictions, the logistic regression model only requires a weighted summation of the feature vectors and a sigmoid function calculation. This calculation is highly lightweight. The additional computational latency and power consumption introduced by integrating the logistic regression model into the decision loop of the video encoder are negligible. This allows the solution to be applied to scenarios with extremely high real-time requirements, enabling intelligent decision-making with almost no increase in coding complexity.

[0113] In step S340, it is determined whether the motion vector refinement probability is greater than a preset probability threshold. If not, step S350 is executed; if yes, steps S360 to S380 are executed.

[0114] The preset probability threshold T is a predefined or dynamically configurable scalar value, where 0 ≤ T ≤ 1. This preset probability threshold can be dynamically adjusted based on the encoding status. For example, when the server is overheating or encoding deadlines are tight, T can be automatically increased to reduce the triggering of motion vector refinement processes and lower the computational load. Alternatively, a smaller T can be used for keyframes or scenes with intense motion to achieve higher accuracy; or a higher T can be used for B-frames with low reference value or static scenes to save computational resources.

[0115] The motion vector refinement probability P of the current encoding unit obtained in step S330 is obtained, the currently effective preset probability threshold is read from the encoder configuration parameters, and the encoder control logic unit performs a numerical comparison operation.

[0116] In step S350, the motion vector refinement process for the current coding unit is skipped; the first motion vector obtained by the current coding unit in the initial motion estimation process is taken as the optimal motion vector, and the optimal motion vector is used for inter-frame prediction and coding of the video frame to be encoded.

[0117] If the motion vector refinement probability P is less than or equal to the preset probability threshold T, it indicates that the probabilistic prediction model, after calculation, determines that the expected coding gain from performing motion vector refinement on the current coding unit is low given its attributes and context. This indirectly indicates that the first motion vector obtained from the initial motion estimation process is evaluated by the probabilistic prediction model as a relatively optimal motion estimation result under the current coding unit's attributes and context. The marginal benefit of further searching for a better solution is lower than the marginal cost. Therefore, the first motion vector can be used as the optimal motion vector output in the motion estimation stage. Thus, when the motion vector refinement probability is less than or equal to the preset probability threshold, the motion vector refinement process for the current coding unit is skipped, and the first motion vector obtained by the current coding unit in the initial motion estimation process is used as the optimal motion vector. The optimal motion vector is then used for inter-frame prediction of the video frames to be encoded.

[0118] The optimal motion vector is the motion vector that is ultimately selected and used to generate the inter-frame prediction block of the current coding unit and will be encoded into the bitstream.

[0119] When the motion vector refinement probability P of the current encoding unit is less than or equal to the preset probability threshold T, the encoder's program execution flow or hardware state machine will directly bypass the motion vector refinement module, without allocating computing resources to the motion vector refinement module or scheduling its execution. The encoder accesses the buffer storing the initial motion estimation process results (sub-pixel search results) and reads information such as the first motion vector determined by the current encoding unit from this buffer. The first motion vector is positively marked and set as the optimal motion vector for the current encoding unit.

[0120] In this embodiment, when the motion vector refinement probability P is less than or equal to a preset probability threshold T, the initial motion estimation result (sub-pixel search result) is directly used as the optimal motion vector. This avoids wasting computational resources on low-yield coding units. Furthermore, since the initial motion estimation itself already possesses a certain degree of accuracy, coding efficiency is ensured. By skipping the motion vector refinement process, iterative search, multiple candidate calculations, and high-precision interpolation calculations in the motion vector refinement flow are avoided, reducing coding complexity.

[0121] In step S360, motion vector filtering is performed based on the first motion vector to obtain multiple candidate motion vectors.

[0122] A motion vector refinement probability P greater than the set probability threshold T indicates that the probabilistic prediction model, after calculation, determines that the initial motion estimation result (sub-pixel search result) may not be a local optimum and contains significant uncertainty. For the current coding unit, the expected coding benefit of motion vector refinement has exceeded the set threshold. Therefore, the motion vector refinement process is initiated to obtain a better optimal motion vector.

[0123] In this embodiment, the first motion vector MV1 is used as the center point of the local search. Around this center point, candidate motion vectors are generated according to a preset offset pattern and offset step size. For example, for the horizontal offset set H=[-1, -0.5, 0, +0.5, +1] and the vertical offset set V=[-1, -0.5, 0, +0.5, +1], 25 candidate positions are generated. Each candidate position corresponds to a candidate motion vector MV(i, j)=MV1+(H[i], V[j]). Here, H[i] represents the i-th candidate element in the horizontal offset, and V[j] represents the j-th candidate element in the vertical offset.

[0124] A reference coding unit is obtained from a specified reference frame using a first motion vector MV1. For the first motion vector MV1, the SATD value between the pixel of the current coding unit and the prediction block of the reference coding unit is calculated and recorded as the distortion information. Then, for 25 candidate motion vectors, the SATD value between the pixel of the current coding unit and the prediction block of the reference coding unit is calculated sequentially. SATD refers to performing a Hadamard transform on the residual block first, and then summing the absolute values ​​of the transform coefficients, which more closely approximates the frequency domain energy distribution after transform coding.

[0125] The SATD values ​​of the 25 candidate motion vectors are compared with the SATD value of the first motion vector MV1. The candidate motion vectors with SATD values ​​greater than the SATD value of the first motion vector MV1 are selected as candidate motion vectors.

[0126] It should be noted that the above distortion information is illustrated using the SATD value as an example. Distortion information may also include SAD values, SSE values, etc.

[0127] It should be noted that the 25 candidate motion vectors mentioned above are merely illustrative; in practical applications, any number of candidate motion vectors can be selected. The offset mode and offset step size can also be limited according to actual conditions, and are not specifically limited in this embodiment.

[0128] It should be noted that the distortion information of the first motion vector and the candidate motion vector can reuse the distortion information calculated and recorded in the initial motion estimation process (sub-pixel search process), avoiding repeated calculation of distortion information and saving computational resources. For the specific acquisition method, please refer to the acquisition method of the distortion information of the second motion vector, which will not be repeated in this embodiment.

[0129] Furthermore, the selection of candidate motion vectors from the candidate motion vectors can also be achieved by the following measurement: selecting the top N (e.g., N=4 or N=8) candidate motion vectors with the smallest SATD values ​​as candidate motion vectors.

[0130] In step S370, the rate-distortion cost of each candidate motion vector is calculated.

[0131] Rate-distortion cost is a linear weighted sum of distortion (D) and bit rate (R). The form of rate-distortion cost is shown in Equation (2).

[0132] (2)

[0133] Where λ is a Lagrange multiplier, which is a parameter that controls the trade-off between distortion and bit rate. The larger λ is, the higher the weight of bit rate in the cost function, and the more the encoder tends to choose vectors with simpler motion information. J represents the rate-distortion cost.

[0134] The SATD value of the candidate motion vector calculated in step S360 is used as the distortion D for this stage. The candidate motion vector MV(i,j) is compared with the first motion vector MV1 to obtain the motion vector difference MV_DK = MV(i,j) - MV1. Entropy coding is performed on the motion vector difference MV_DK to obtain the required code rate R. The distortion D and code rate R are substituted into formula (2) to calculate the rate-distortion cost J.

[0135] For each candidate motion vector, the rate-distortion cost is calculated using the above process, and the rate-distortion cost of each candidate motion vector is recorded.

[0136] In step S380, the candidate motion vector with the lowest rate-distortion cost is selected as the optimal motion vector.

[0137] The rate-distortion costs of each candidate motion vector calculated in step S370 are compared, and the candidate motion vector with the lowest rate-distortion cost is selected as the optimal motion vector.

[0138] The optimal motion vector determined in step S350 or S380 is used to perform standard encoding operations. Specifically, based on the optimal motion vector and the corresponding reference frame index, the reference pixel block of the reference coding unit is obtained, and the difference (residual block) between the original pixel block of the current coding unit and the reference pixel block of the reference coding unit is calculated. Subsequently, the residual block is transformed and quantized to obtain the quantized transform coefficients. Entropy encoding is performed on the optimal motion vector, and entropy encoding is also performed on the quantized transform coefficients. The optimal motion vector and residual coefficients obtained above are used together as part of the compressed data of the current coding unit and written into the final video bitstream.

[0139] It should be noted that S360-S380 above is an exemplary implementation of motion vector refinement. In practical applications, other motion vector refinement processes can also be used.

[0140] In this embodiment, when the motion vector refinement probability P is greater than the set probability threshold T, a better optimal motion vector is obtained by executing the motion vector refinement process, thereby improving the compression efficiency.

[0141] Based on the above embodiments, this disclosure optimizes the method of "obtaining feature information related to the current coding unit during the coding process," especially optimizing the implementation of "obtaining the distortion information of the first motion vector in the current coding unit and the distribution of distortion information of a set number of second motion vectors associated with the first motion vector," such as... Figure 4 As shown, it mainly includes steps S410-S420.

[0142] In step S410, during the initial motion estimation process, distortion information of each preset motion vector is recorded.

[0143] The initial motion estimation process can be understood as a search process where the current coding unit finds the optimal motion vector, including integer pixel search and sub-pixel search, and finally outputs the first motion vector. The preset motion vector refers to the motion vector at a specific search position that has been evaluated by the algorithm during the initial motion estimation process.

[0144] The initial motion estimation process below will be illustrated using a diamond search as an example. Figure 5 The diagram shown is a schematic representation of the range of the three-round diamond search.

[0145] Using a spatially or temporally predicted motion vector as a center point, and taking a search step size of Step=0.5 as an example, with the current center point C0 as the center and a search step size of Step=0.5, calculate the neighborhood motion vectors in the four directions surrounding center point C0: C1, C2, C3, and C3. Calculate the SATD value corresponding to center point C0 and the four neighborhood motion vectors respectively. Record and compare the SATD values ​​of these five motion vectors, determine the motion vector corresponding to the minimum SATD value, and use the motion vector corresponding to the minimum SATD value as the center point C0' for the next iteration. Repeat the above steps to determine a new minimum SATD value, and use the motion vector corresponding to the new minimum SATD value as the center point C0'' for the next iteration. When the preset convergence condition is met (e.g., after 3 iterations), the iteration terminates. The motion vector corresponding to the minimum SATD value calculated in the last iteration is the first motion vector.

[0146] In one possible implementation, in each round of the iterative search, all the searched motion vectors are the aforementioned preset motion vectors, that is, all the searched motion vectors and their SATD values ​​are recorded in a temporary data structure.

[0147] In another possible implementation, only the determined first motion vector and the searched motion vectors around the first motion vector are used as preset motion vectors. In other words, the SATD of the first motion vector and the searched motion vectors around the first motion vector are recorded in a temporary data structure.

[0148] In this embodiment, the distortion information including SATD value is used as an example for explanation. In practical applications, the distortion information can also include SAD, SSE, etc.

[0149] In step S420, based on the distortion information of each preset motion vector, the distortion information of the first motion vector and the distortion information of a set number of second motion vectors are determined.

[0150] Step S410 above is to record in the initial motion estimation process, and step S420 is to read the distortion information that has value in the motion vector refinement judgment process.

[0151] When it is necessary to obtain the distortion information of the first motion vector and the distortion information of a set number of second motion vectors, the recorded distortion information is read from the data structure.

[0152] In this embodiment, the distortion information of the motion vectors calculated in the initial motion estimation process is saved and passed to the motion vector refinement judgment stage and the motion vector refinement stage. This allows the distortion information of the motion vectors to be reused by the motion vector refinement judgment stage and the motion vector refinement stage, avoiding the repeated calculation of the distortion information of the motion vectors and further saving computing resources.

[0153] exist Figure 4 Based on the previous embodiment, this embodiment describes the process of reusing the distortion information of the recorded motion vectors in the refinement judgment stage, such as... Figure 6 As shown, the steps for obtaining feature information related to the current coding unit during the encoding process include S610-S630.

[0154] In step S610, the distortion information of each preset motion vector is queried based on the first motion vector, and the queried distortion information is used as the distortion information of the first motion vector.

[0155] In this embodiment, Figure 4 In this embodiment, the distortion information of the preset motion vector is stored in a lookup table in the form of key values ​​for illustration.

[0156] During the process of acquiring feature information related to the current encoding unit, a query key is generated based on the determined first motion vector. This query key can be obtained by normalizing the horizontal and vertical components of the first motion vector. The query key is then used to retrieve information from a lookup table that stores preset motion vector distortion information.

[0157] Since the first motion vector must be one of the preset motion vectors calculated in the initial motion estimation process, the query will hit, and the distortion information obtained from the query will be directly used as the distortion information of the first motion vector. The distortion information of the first motion vector will be used as one of the features of the probabilistic prediction model.

[0158] In step S620, for each second motion vector, a query is performed on the distortion information of each preset motion vector based on the second motion vector, and the obtained distortion information is used as the distortion information of the second motion vector.

[0159] For each second motion vector, the same lookup method as for the first motion vector is used to search the lookup table that stores the distortion information of the preset motion vectors.

[0160] Since the second motion vector is not necessarily evaluated in the initial motion estimation process, that is, it may not be found in the lookup table, if the lookup is successful, the distortion information obtained from the lookup will be directly used as the distortion information of the second motion vector.

[0161] In step S630, if the distortion information of the second motion vector is not found in the search information of each preset motion vector, the distortion information of the second motion vector is calculated.

[0162] If the query fails, the distortion information of the failed second motion vector is calculated according to the distortion information calculation method provided in the above embodiment.

[0163] In this embodiment, the distortion information of the motion vector is stored in a lookup table in the form of key values. The size of the lookup table can be dynamically adjusted, occupying only the storage space proportional to the motion vector with calculated distortion information, which can significantly reduce the occupation of memory or on-chip cache.

[0164] Based on the above embodiments, this embodiment optimizes the process of reusing the distortion information of motion vectors, such as... Figure 7 As shown, it mainly includes steps S710-S760.

[0165] The above Figure 6 In one embodiment, the distortion information is stored in a lookup table as a key-value pair. In another embodiment, the distortion information is stored in an array.

[0166] In step S710, during the initial motion estimation process, the array index corresponding to the preset motion vector is determined.

[0167] An array consists of an array index and array elements. An array index is a position identifier used to access a specific element in the array. In this embodiment, the mathematical representation of the preset motion vector is converted into an integer value, which serves as the index for accessing the array. An array element is a data unit stored at each position in the array. Each array element is used to store the distortion information of the preset motion vector corresponding to its index.

[0168] In this embodiment, since the array index is of integer type, and the preset motion vector consists of two components, which may be integers or fractions, it is necessary to convert and map the preset motion vector to obtain its unique corresponding integer value, and use the integer value as the array index.

[0169] In step S720, for the preset motion vectors that have been searched, the calculated distortion information is used as an array element; for the preset motion vectors that have not been searched, the set distortion information is used as an array element.

[0170] The pre-defined motion vectors that have been searched refer to the motion vectors that have been evaluated by the algorithm (i.e., have undergone motion compensation and distortion calculation) in the initial motion estimation process. The pre-defined motion vectors that have not been searched refer to the motion vectors that are within the search range in the initial motion estimation process but have not been evaluated by the algorithm.

[0171] Setting distortion information refers to a predefined value used to fill the array elements of preset motion vectors that have not been searched. For example, setting distortion information is the theoretical maximum distortion value.

[0172] In one possible implementation, an array Ds[] is allocated based on the calculated total number of indices. Before the initial motion estimation process begins, all elements of the array are initialized with preset distortion information. During the initial motion estimation process, for a preset motion vector that has been searched, the distortion information corresponding to that motion vector is calculated and written to the corresponding position in the array. For preset motion vectors that have not been searched, they are not evaluated by the algorithm during the initial motion estimation process, and their corresponding array index positions are not written with new values, i.e., the preset distortion information is retained.

[0173] In another possible implementation, the pre-defined motion vectors found and their corresponding distortion information are recorded during the search process. After determining the first motion vector, an array is created. Taking a preset quantity of 4 as an example, this array has a length of 5 and is used to store the distortion information of the first motion vector and the distortion information of the four surrounding second motion vectors. If the second motion vector is a pre-defined motion vector that has been found, its corresponding distortion information is written to the corresponding position in the array; if the second motion vector is a pre-defined motion vector that has not been found, the pre-defined distortion information is written to the corresponding position in the array. This reduces the array length and lowers memory usage.

[0174] In step S730, the first array index of the first motion vector is determined, and the array element corresponding to the first array index is used as the distortion information of the first motion vector.

[0175] Calculate the corresponding first array index based on the determined first motion vector, use the first data index as the subscript to directly access the array Ds[] that stores distortion information, and read the distortion information as the first array index from the array element.

[0176] Since the first motion vector is necessarily the output of the initial motion estimation process, its corresponding array elements must have been calculated and stored during the initial motion estimation process. Therefore, the read array element values ​​must be non-defined distortion information, and these values ​​will be used as the distortion information for the first motion vector.

[0177] In step S740, for each second motion vector, the second array index corresponding to the second motion vector is determined, and the array element corresponding to the second array index is obtained.

[0178] For each second motion vector, calculate its corresponding second array index based on the second motion vector, and read the corresponding array element in the array Ds[], comparing whether the read array element is the set distortion information.

[0179] In step S750, if the array element corresponding to the second array index is non-defined distortion information, the array element corresponding to the second array index is used as the distortion information of the second motion vector.

[0180] If the array element read is non-defined distortion information, it indicates that the second motion vector has been evaluated in the initial motion estimation process, and the read array element is directly used as the distortion information of the second motion vector.

[0181] In step S760, if the array element corresponding to the second array index is the set distortion information, the distortion information of the second motion vector is calculated.

[0182] If the array element read is equal to the set distortion information, it indicates that the second motion vector was not evaluated by the initial process and no valid data is stored in the array. The distortion information of the missed second motion vector is calculated according to the distortion information calculation method provided in the above embodiment.

[0183] In this embodiment, data is accessed directly through array indexes, eliminating the need to maintain complex data structures and reducing management overhead.

[0184] This disclosure provides an exemplary implementation method, such as... Figure 8 As shown, the main steps include the following: In step S810, an initial motion search is performed.

[0185] Since the SATD values ​​of some motion vectors have already been calculated during the preliminary motion estimation process, redundant calculations can be removed during the refinement and judgment stage.

[0186] Record the coordinate offset of each motion vector during the search process compared to the initial motion vector. Using the motion vector as an index, record the SATD value of each motion vector. For the finally selected first motion vector, save the SATD values ​​of the surrounding searched second motion vectors to an array, setting the elements of any uncalculated second motion vectors to their maximum distortion values.

[0187] The array is passed to the motion vector refinement stage. The array is queried. If an element is smaller than the set distortion information, the element value is directly obtained as the SATD value of the corresponding motion vector without recalculation.

[0188] In step S820, inter RDO is executed.

[0189] Based on the distortion information obtained in S810 above, inter RDO is performed to obtain the rate-distortion cost of inter-frame prediction for the current coding unit.

[0190] In step S830, feature construction takes place.

[0191] Acquire multidimensional feature information related to the current coding unit during the encoding process, including but not limited to: the size information of the current coding unit; the proportion of the set coding units in the video frame to be encoded; the ratio of the rate-distortion cost of inter-frame prediction to the rate-distortion cost of intra-frame prediction of the current coding unit; the distortion information of the first motion vector in the current coding unit; the distribution of distortion information of a set number of second motion vectors associated with the first motion vector; and the proportion of the set coding units in the video frame to be encoded.

[0192] In step S840, model reasoning and judgment are performed.

[0193] The aforementioned feature vectors are used as inputs and fed into a pre-trained logistic regression model to obtain the motion vector refinement probability output by the logistic regression model.

[0194] In step 850, it is determined whether the motion vector refinement probability is greater than a preset probability threshold T. If yes, then step S860 is executed; otherwise, step S870 is executed.

[0195] In step S860, the motion vector refinement process is executed.

[0196] In step S870, the optimal motion vector is output.

[0197] If the motion vector refinement probability P is less than or equal to the preset probability threshold T, then the first motion vector determined by the initial motion estimation process is output.

[0198] If the motion vector refinement probability P is greater than the preset probability threshold T, then the optimal motion vector determined by the motion vector refinement process is output.

[0199] It should be noted that the above-mentioned refinement judgment module is integrated into the encoder's inter-frame prediction process, located after the initial RDO and before the motion vector refinement process, and serves as a decision control module to dynamically enable or disable the motion vector refinement path.

[0200] In this embodiment, by reasonably reusing the sub-pixel search results calculated during inter-frame prediction, redundant calculations of candidate motion vectors are eliminated. Furthermore, a logistic regression-based MV refinement decision method is proposed, applicable to the inter-frame prediction optimization process in HEVC encoders. By extracting existing multi-dimensional feature information from the encoding process, a lightweight logistic regression model is used to intelligently decide whether each coding unit needs further MV refinement, thereby significantly reducing the computational load while maintaining encoding efficiency.

[0201] The following are embodiments of the apparatus disclosed herein, which can be used to execute embodiments of the method disclosed herein. For details not disclosed in the apparatus embodiments of this disclosure, please refer to the embodiments of the method disclosed herein.

[0202] Figure 9 This is a block diagram illustrating a video processing apparatus according to an exemplary embodiment. (Refer to...) Figure 9 The device may include: a current coding unit determination module 910, used to acquire a video frame to be encoded and determine the current coding unit based on the video frame to be encoded; a related feature information acquisition module 920, used to acquire feature information related to the current coding unit during the encoding process; a motion vector thinning probability determination module 930, used to determine the motion vector thinning probability based on the feature information related to the current coding unit; and a motion vector thinning execution judgment module 940, used to determine whether to execute the motion vector thinning process for the current coding unit based on the motion vector thinning probability.

[0203] In some exemplary embodiments of this disclosure, the motion vector refinement probability determination module 930 is specifically used to take feature information related to the current coding unit as input to the probability prediction model to obtain the motion vector refinement probability output by the probability prediction model.

[0204] In some exemplary embodiments of this disclosure, the probabilistic prediction model includes a logistic regression model.

[0205] In some exemplary embodiments of this disclosure, the motion vector refinement execution judgment module 940 is specifically used to skip the motion vector refinement process for the current coding unit if the motion vector refinement probability is less than or equal to a preset probability threshold; and to use the first motion vector obtained by the current coding unit in the initial motion estimation process as the best motion vector, which is then used for inter-frame prediction and encoding of the video frame to be encoded.

[0206] In some exemplary embodiments of this disclosure, the motion vector refinement execution judgment module 940 is further configured to, if the motion vector refinement probability is greater than a preset probability threshold, perform motion vector filtering based on the first motion vector to obtain multiple candidate motion vectors; calculate the rate-distortion cost of each candidate motion vector; and select the candidate motion vector with the minimum rate-distortion cost as the optimal motion vector.

[0207] In some exemplary embodiments of this disclosure, the feature information associated with the current coding unit includes at least one of the following: the size information of the current coding unit; the proportion of a set coding unit in the video frame to be encoded; the ratio of the rate-distortion cost of inter-frame prediction to the rate-distortion cost of intra-frame prediction of the current coding unit; the distortion information of the first motion vector in the current coding unit; the distribution of distortion information of a set number of second motion vectors associated with the first motion vector; and the mode type of the coding units adjacent to the current coding unit.

[0208] In some exemplary embodiments of this disclosure, it further includes: a distortion information recording module, used to record the distortion information of each preset motion vector in the initial motion estimation process; and a related feature information acquisition module 920, specifically used to determine the distortion information of a first motion vector and the distortion information of a set number of second motion vectors based on the distortion information of each preset motion vector.

[0209] In some exemplary embodiments of this disclosure, the relevant feature information acquisition module 920 is specifically used to query the distortion information of each preset motion vector based on the first motion vector, and use the queried distortion information as the distortion information of the first motion vector; for each second motion vector, query the distortion information of each preset motion vector based on the second motion vector, and use the queried distortion information as the distortion information of the second motion vector; if the distortion information of the second motion vector is not found in the search information of each preset motion vector, calculate the distortion information of the second motion vector.

[0210] In some exemplary embodiments of this disclosure, the distortion information recording module is specifically used to, in the initial motion estimation process, use a preset motion vector as an array index; for preset motion vectors that have been searched, use the calculated distortion information as an array element; for preset motion vectors that have not been searched, use the set distortion information as an array element; the related feature information acquisition module 920 is specifically used to determine the first array index of the first motion vector, use the array element corresponding to the first array index as the distortion information of the first motion vector; for each second motion vector, determine the second array index corresponding to the second motion vector, and acquire the array element corresponding to the second array index; if the array element corresponding to the second array index is non-set distortion information, use the array element corresponding to the second array index as the distortion information of the second motion vector; if the array element corresponding to the second array index is set distortion information, calculate the distortion information of the second motion vector.

[0211] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0212] The following reference Figure 10 To describe an electronic device 1000 according to such an embodiment of the present disclosure. Figure 10 The electronic device 1000 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments disclosed herein.

[0213] like Figure 10As shown, the electronic device 1000 is manifested in the form of a general-purpose computing device. The components of the electronic device 1000 may include, but are not limited to: at least one processing unit 1010, at least one storage unit 1020, a bus 1030 connecting different system components (including storage unit 1020 and processing unit 1010), and a display unit 1040.

[0214] The storage unit stores program code, which can be executed by the processing unit 1010 to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of this disclosure. For example, the processing unit 1010 can perform actions such as... Figure 3 The steps shown. For example, electronic devices can achieve, for instance, the following steps. Figure 3 The steps shown.

[0215] Storage unit 1020 may include readable media in the form of volatile storage units, such as random access memory (RAM) 1021 and / or cache memory 1022, and may further include read-only memory (ROM) 1023.

[0216] Storage unit 1020 may also include a program / utility 1024 having a set (at least one) program module 1025, such program module 1025 including but not limited to: operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0217] Bus 1030 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the multiple bus structures.

[0218] Electronic device 1000 can also communicate with one or more external devices 1070 (e.g., keyboard, pointing device, Bluetooth device, etc.), one or more devices that enable a user to interact with electronic device 1000, and / or any device that enables electronic device 1000 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 1050. Furthermore, electronic device 1000 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 1060. As shown, network adapter 1060 communicates with other modules of electronic device 1000 via bus 1030. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 1000, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0219] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, terminal device, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0220] In an exemplary embodiment, a computer-readable storage medium including instructions is also provided, such as a memory including instructions that can be executed by a processor of the device to perform the described method. Optionally, the computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0221] In an exemplary embodiment, a computer program product is also provided, including a computer program that, when executed by a processor, implements the methods described above.

[0222] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0223] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A video processing method, characterized in that, include: Acquire the video frame to be encoded, and determine the current encoding unit based on the video frame to be encoded; Acquire feature information related to the current encoding unit during the encoding process; The motion vector refinement probability is determined based on the feature information associated with the current coding unit; Based on the motion vector thinning probability, determine whether to execute the motion vector thinning process for the current coding unit.

2. The video processing method according to claim 1, characterized in that, The step of determining the motion vector thinning probability based on feature information related to the current coding unit includes: The feature information associated with the current coding unit is used as the input to the probabilistic prediction model to obtain the motion vector refinement probability output by the probabilistic prediction model.

3. The video processing method according to claim 2, characterized in that, The probability prediction model includes a logistic regression model.

4. The video processing method according to claim 1, characterized in that, The step of determining whether to execute the motion vector thinning process for the current coding unit based on the motion vector thinning probability includes: If the motion vector refinement probability is less than or equal to a preset probability threshold, skip the motion vector refinement process for the current coding unit. The first motion vector obtained by the current coding unit in the initial motion estimation process is used as the optimal motion vector, and the optimal motion vector is used for inter-frame prediction of the video frame to be encoded.

5. The video processing method according to claim 4, characterized in that, The step of determining whether to trigger the motion vector thinning process for the current coding unit based on the motion vector thinning probability further includes: If the motion vector refinement probability is greater than the preset probability threshold, motion vector filtering is performed based on the first motion vector to obtain multiple candidate motion vectors; Calculate the rate-distortion cost for each of the candidate motion vectors; The candidate motion vector with the lowest rate-distortion cost is selected as the optimal motion vector.

6. The video processing method according to any one of claims 1-5, characterized in that, The feature information associated with the current coding unit includes at least one of the following: The size information of the current encoding unit; The percentage of encoding units in the video frame to be encoded is set; The ratio of the rate-distortion cost of inter-frame prediction to the rate-distortion cost of intra-frame prediction for the current coding unit; The distortion information of the first motion vector in the current encoding unit; A set amount of distortion information for the second motion vector associated with the first motion vector; The prediction mode of the coding unit adjacent to the current coding unit.

7. The video processing method according to claim 6, characterized in that, Also includes: In the initial motion estimation process, the distortion information of each preset motion vector is recorded; The acquisition of feature information related to the current coding unit during the encoding process includes: Based on the recorded distortion information of each preset motion vector, the distortion information of the first motion vector and the distortion information of a predetermined number of second motion vectors are determined.

8. The video processing method according to claim 7, characterized in that, The step of determining the distortion information of the first motion vector and the distortion information of a predetermined number of second motion vectors based on the recorded distortion information of each preset motion vector includes: Based on the first motion vector, a query is performed on the distortion information of each preset motion vector, and the obtained distortion information is used as the distortion information of the first motion vector. For each of the second motion vectors, a query is performed on the distortion information of each preset motion vector based on the second motion vector, and the obtained distortion information is used as the distortion information of the second motion vector; If the distortion information of the second motion vector is not found in the search information of each preset motion vector, the distortion information of the second motion vector is calculated.

9. The video processing method according to claim 7, characterized in that, In the initial motion estimation process, the distortion information of each preset motion vector is recorded, including: In the initial motion estimation process, the array index corresponding to the preset motion vector is determined; For preset motion vectors that have been searched, the calculated distortion information is used as array elements; for preset motion vectors that have not been searched, the set distortion information is used as array elements. The step of determining the distortion information of the first motion vector and the distortion information of a predetermined number of second motion vectors based on the distortion information of each preset motion vector includes: Determine the first array index of the first motion vector, and use the array element corresponding to the first array index as the distortion information of the first motion vector; For each second motion vector, determine the second array index corresponding to the second motion vector, and obtain the array element corresponding to the second array index; If the array element corresponding to the second array index is non-defined distortion information, the array element corresponding to the second array index is used as the distortion information of the second motion vector; If the array element corresponding to the second array index is the set distortion information, calculate the distortion information of the second motion vector.

10. A video processing apparatus, characterized in that, include: The current encoding unit determination module is used to acquire the video frame to be encoded and determine the current encoding unit based on the video frame to be encoded. The relevant feature information acquisition module is used to acquire feature information related to the current encoding unit during the encoding process; The motion vector refinement probability determination module is used to determine the motion vector refinement probability based on feature information related to the current coding unit; The motion vector refinement execution judgment module is used to determine whether to execute the motion vector refinement process for the current coding unit based on the motion vector refinement probability.

11. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the executable instructions to implement the video processing method as described in any one of claims 1 to 9.

12. A computer-readable storage medium, wherein instructions in the computer-readable storage medium, when executed by a processor of an electronic device, enable the electronic device to perform the video processing method as described in any one of claims 1 to 9.

13. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the video processing method as described in any one of claims 1 to 9.

Images