Video processing methods, devices and related equipment

Abstract

This disclosure provides a video processing method, apparatus, and related equipment, relating to the field of video processing technology. The method includes: performing noise and motion trend analysis on an acquired video to be processed, obtaining noise analysis results and motion trend analysis results; if the noise analysis result indicates a low noise level, and / or the motion trend analysis result indicates the presence of consistent motion, then determining to enable a target inter-frame prediction mode; when the target inter-frame prediction mode is enabled and the current coding block of the video to be processed adopts an inter-frame fusion prediction mode, determining multiple spatial candidate blocks of the current coding block, and obtaining coding parameter information of the multiple spatial candidate blocks; and determining the number of candidate motion information for the current coding block based on the coding parameter information of the multiple spatial candidate blocks. This method can dynamically adjust the number of candidate motion information, avoiding the use of excessive candidate information and reducing the bitrate consumption generated by the coding candidate index.

Description

Technical Field

[0001] This disclosure relates to the field of video processing technology, and in particular to a video processing method, video processing apparatus, electronic device, computer-readable storage medium, and computer program product. Background Technology

[0002] In video coding standards, such as High Efficiency Video Coding (HEVC), Versatile Video Coding (VVC), and Enhanced Compression Model (ECM), which is an extension of VCC, prediction modes include intra-frame prediction and inter-frame prediction. Inter-frame prediction includes ordinary inter-frame prediction mode and inter-frame fusion prediction mode. Compared to ordinary inter-frame prediction mode, inter-frame fusion prediction mode does not require writing information into the bitstream.

[0003] However, the candidate list for inter-frame fusion prediction mode is usually fixed, which leads to the use of too much candidate information in some scenarios. This results in the merge index (the index value used to indicate which candidate motion information to select in the current block) occupying more bits, increasing unnecessary bit rate consumption.

[0004] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] This disclosure provides a video processing method, video processing apparatus, electronic device, computer-readable storage medium, and computer program product to overcome or at least partially solve the above-mentioned problems.

[0006] This disclosure provides a video processing method, the method comprising: acquiring a video to be processed; performing noise analysis and motion trend analysis on the video to be processed to obtain noise analysis results and motion trend analysis results; if the noise analysis result is a low noise level and / or the motion trend analysis result is consistent motion, then determining to enable a target inter-frame prediction mode; when the target inter-frame prediction mode is enabled and the current coding block of the video to be processed adopts an inter-frame fusion prediction mode, determining multiple spatial candidate blocks of the current coding block, and obtaining coding parameter information of the multiple spatial candidate blocks; and determining the number of candidate motion information of the current coding block based on the coding parameter information of the multiple spatial candidate blocks.

[0007] In some embodiments of this disclosure, the encoding parameter information includes motion information; wherein, determining the number of candidate motion information for the current encoding block based on the encoding parameter information of the plurality of spatial candidate blocks includes: for every two spatial candidate blocks among the plurality of spatial candidate blocks, determining the difference between the two spatial candidate blocks based on the motion information of the two spatial candidate blocks; selecting the maximum difference from all the differences between the two spatial candidate blocks, determining a difference threshold based on the maximum difference; and determining the number of candidate motion information for the current encoding block based on the difference threshold.

[0008] In some embodiments of this disclosure, determining the number of candidate motion information for the current coding block based on the difference threshold includes: if the difference threshold is 0, then determining the number of candidate motion information for the current coding block as 1; if the difference threshold is less than or equal to a first threshold, then determining the number of candidate motion information for the current coding block as a preset value.

[0009] In some embodiments of this disclosure, the encoding parameter information further includes: at least one non-motion information; wherein, the method further includes: for each of the at least one non-motion information, obtaining the number of spatial candidate blocks with the same value in the at least one spatial candidate block, determining the number as a specific number of the non-motion information; selecting the minimum number from all the specific numbers of the non-motion information, and determining a specific number threshold based on the minimum number.

[0010] In some embodiments of this disclosure, determining the number of candidate motion information for the current coding block based on the difference threshold includes: if the specific number threshold is the number of the plurality of spatial candidate blocks and the difference threshold is 0, then the number of candidate motion information for the current coding block is determined to be 1; if the specific number threshold is greater than or equal to a second threshold and the difference threshold is less than or equal to a first threshold, then the number of candidate motion information for the current coding block is determined to be a preset value.

[0011] In some embodiments of this disclosure, the method further includes: encoding an index of the candidate motion information of the current coding block based on the number of candidate motion information of the current coding block.

[0012] In some embodiments of this disclosure, the method further includes: if the inter-frame prediction direction of the current coding block is bidirectional prediction, then based on rate-distortion optimization, determining the candidate weight information of the current coding block from the weight information pointed to by the candidate bidirectional weight prediction index.

[0013] In some embodiments of this disclosure, the method further includes: if the inter-frame prediction direction of the current coding block is non-bidirectional prediction, then determining the state information of the local illumination compensation identifier of the current coding block based on rate-distortion optimization.

[0014] In some embodiments of this disclosure, the method further includes: for each video frame to be processed in the video to be processed, determining the noise value of the video frame to be processed; and calculating the average of the noise values ​​of all the video frames to be processed in the video to be processed to obtain the noise value of the video to be processed.

[0015] In some embodiments of this disclosure, the noise analysis includes one or more of the following: determining whether the noise value of the video frame to be processed is less than a first noise threshold; if so, determining that the noise analysis result of the video frame to be processed is a low noise level; determining whether the noise value of the video to be processed is less than a second noise threshold; if so, determining that the noise analysis result of the video to be processed is a low noise level.

[0016] In some embodiments of this disclosure, the motion trend analysis includes: for each video frame to be processed in the video to be processed, obtaining one or more image blocks in the video frame to be processed; for each image block, determining the quadrant corresponding to the motion information of the image block; determining the proportion value of the image blocks corresponding to different quadrants based on the quadrants corresponding to the motion information of all the image blocks, and selecting the maximum proportion value from the proportion values ​​of the image blocks corresponding to different quadrants; if the maximum proportion value is greater than the proportion threshold, then determining that the motion trend analysis result of the video to be processed is that there is consistent motion.

[0017] This disclosure provides a video processing apparatus, comprising: a video analysis unit configured to acquire a video to be processed; perform noise analysis and motion trend analysis on the video to be processed to obtain noise analysis results and motion trend analysis results; a determination unit configured to determine to enable a target inter-frame prediction mode if the noise analysis result is a low noise level and / or the motion trend analysis result is consistent motion; and a mode execution unit configured to, when the target inter-frame prediction mode is enabled and the current coding block of the video to be processed adopts an inter-frame fusion prediction mode, determine multiple spatial candidate blocks of the current coding block, acquire coding parameter information of the multiple spatial candidate blocks, and determine the number of candidate motion information of the current coding block based on the coding parameter information of the multiple spatial candidate blocks.

[0018] This disclosure provides an electronic device, including: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to execute the executable instructions to implement the video processing method described above.

[0019] This disclosure provides a computer-readable storage medium that, when the instructions in the computer-readable storage medium are executed by the processor of an electronic device, enables the electronic device to perform the video processing method described above.

[0020] This disclosure provides a computer program product, including a computer program that, when executed by a processor, implements the video processing method described above.

[0021] The video processing method provided in this disclosure performs noise analysis and motion trend analysis on the video to be processed. If at least one of the following conditions is met (low noise level or consistent motion), the method determines to enable a target inter-frame prediction mode (i.e., an optimized inter-frame fusion prediction mode) for the video. This method enables the target inter-frame prediction mode only in suitable scenarios through noise and motion trend analysis, avoiding unnecessary computational overhead and improving coding efficiency. After enabling the target inter-frame prediction mode, for the current coding block, if the current coding block is determined to be in inter-frame fusion prediction mode, the method determines the number of candidate motion information for the current coding block based on the coding parameter information of its spatial candidate blocks. This dynamically adjusts the number of candidate motion information for each coding block, avoiding the use of excessive candidate information, reducing the bitrate consumption generated by the coding candidate index, and further improving coding efficiency.

[0022] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0023] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0024] Figure 1 A schematic diagram of an exemplary system architecture to which the video processing method of the present disclosure embodiments can be applied is shown;

[0025] Figure 2 A flowchart of a video processing method according to an embodiment of the present disclosure is shown;

[0026] Figure 3 A schematic diagram of the low-frequency and high-frequency regions of an image block according to an embodiment of the present disclosure is shown;

[0027] Figure 4 A mapping diagram of motion information and quadrants for an image block according to an embodiment of this disclosure is shown;

[0028] Figure 5 This diagram illustrates the positions of multiple spatial candidate blocks of the current coded block according to an embodiment of the present disclosure.

[0029] Figure 6 A block diagram of a video processing apparatus according to an embodiment of the present disclosure is shown;

[0030] Figure 7 A schematic diagram of the structure of an electronic device according to an embodiment of the present disclosure is shown. Detailed Implementation

[0031] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the embodiments set forth herein; rather, they are provided so that this disclosure will be thorough and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted.

[0032] The features, structures, or characteristics described in this disclosure can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough understanding of embodiments of this disclosure. However, those skilled in the art will recognize that the technical solutions of this disclosure can be practiced with one or more specific details omitted, or other methods, components, apparatuses, steps, etc., can be employed. In other instances, well-known methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this disclosure.

[0033] The collection, updating, analysis, processing, use, transmission, and storage of user personal information disclosed herein comply with relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. Necessary measures are taken to prevent unauthorized access to user personal information data and to safeguard user personal information security, network security, and national security.

[0034] The accompanying drawings are merely illustrative of this disclosure, and the same reference numerals in the drawings denote the same or similar parts, thus omitting repeated descriptions of them. Some block diagrams shown in the drawings do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in at least one hardware module or integrated circuit, or in different network and / or processor devices and / or microcontroller devices.

[0035] The flowchart shown in the accompanying drawings is merely illustrative and does not necessarily include all content and steps, nor does it require execution in the described order. For example, some steps may be broken down, while others may be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0036] In this specification, the terms “a,” “one,” “the,” “the,” and “at least one” are used to indicate the presence of at least one element / component / etc.; the term “multiple” refers to two or more; the terms “comprising,” “including,” and “having” are used to indicate an open-ended inclusion meaning and that other elements / components / etc. may exist in addition to the listed elements / components / etc.; the terms “first,” “second,” and “third,” etc., are used only as markings and are not a limitation on the number of objects.

[0037] In video coding, inter-frame fusion prediction modes are a common inter-frame prediction technique, such as the Merge mode, which predicts the motion information of the current block by reusing motion information of surrounding encoded blocks.

[0038] Taking the Merge mode in the VVC standard as an example, the Merge mode constructs a candidate list of motion information using adjacent blocks in the temporal or spatial domains, and the list contains 6 candidate motion information. Among them, there are 5 types of candidate motion information, in the following order: spatial candidates based on adjacent blocks, temporal candidates based on co-located blocks, candidates from a First In First Out (FIFO) table constructed based on historical information, paired average candidates, and zero candidates.

[0039] In the Merge mode of HEVC, VVC, and ECM standards, the multiplexing information includes: inter-direction (ID), motion vector (MV), and reference index (RI).

[0040] In this context, ID identifies whether the current block is predicted forward (ID=1), backward (ID=2), or bidirectional (ID=3), and therefore its value ranges from 1 to 3. MV represents the offset of the reference block relative to the current block. RI identifies which reference image the current block references. The value of RI depends on the number of reference frames. When the number of reference frames is M (M is a natural number greater than 0), the RI value ranges from 0 to M-1. When ID=3, there are two RI values, which represent the reference frame indices for forward and backward prediction, respectively.

[0041] Furthermore, in the Merge mode of the VVC and ECM standards, the reused information also includes: bi-prediction with CU-level weight index (BCW index) and half-pelinterpolation filter (HF). In the Merge mode of the ECM standard, the reused information also includes: BCW index, HF, and Local Illumination Compensation flag (LIC flag).

[0042] The BCW index indicates the index of the weights of the two prediction signals during bidirectional prediction. HF indicates whether a half-pixel interpolation filter is used during the motion compensation phase of generating the prediction block; HF values ​​range from 0 to 1, with HF=1 indicating use and HF=0 indicating no use. The LIC flag indicates whether local illumination compensation is used for the current block; LIC flag values ​​range from 0 to 1, with 0 indicating no use and 1 indicating use.

[0043] Because the candidate list in Merge mode is usually predefined and fixed, the same candidate list is used regardless of whether the motion information of the current block is highly consistent with that of surrounding blocks. This means that even in some scenarios where the motion information of the current block is highly consistent with that of surrounding blocks, multiple candidate information is still provided for selection, resulting in the merge index occupying more bits and increasing unnecessary bitrate consumption. Furthermore, the encoding method in Merge mode is usually fixed and does not take into account the differences in motion information correlation between different video content and image regions, leading to low encoding efficiency in consistent motion scenarios.

[0044] Furthermore, the BCW index, HF, and LIC flag each correspond to different optimization algorithms, which guide the current block to generate a more effective prediction block. However, these values ​​are obtained by reusing other blocks in the current block, and the current block itself does not perform validity checks. Therefore, these values ​​may be inaccurate for the current block.

[0045] To address the aforementioned issues, this disclosure provides a video processing method that enables the target inter-frame prediction mode only in suitable scenarios through noise analysis and motion trend analysis, thereby avoiding unnecessary computational overhead and improving coding efficiency. After enabling the target inter-frame prediction mode, the number of candidate motion information in each coding block can be dynamically adjusted to avoid using too much candidate information, reduce the bitrate consumption generated by the coding candidate index, and further improve coding efficiency.

[0046] Figure 1A schematic diagram of an exemplary system architecture to which the video processing method of the present disclosure can be applied is shown.

[0047] like Figure 1 As shown, the system architecture may include server 101, network 102, and terminal device 103. Network 102 serves as the medium for providing a communication link between terminal device 103 and server 101. Network 102 may include various connection types, such as wired or wireless communication links or fiber optic cables, etc.

[0048] In an exemplary embodiment, the terminal device 103 that transmits data with the server 101 may include, but is not limited to, smartphones, tablets, laptops, smart speakers, digital assistants, AR (Augmented Reality) devices, VR (Virtual Reality) devices, smart wearable devices, etc. Alternatively, the terminal device 103 may also be a personal computer, such as a laptop computer or a desktop computer. Optionally, the operating system running on the electronic device may include, but is not limited to, Android, iOS, Linux, Windows, etc.

[0049] Server 101 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. In some practical applications, server 101 can also be a server for a network platform, such as a trading platform, live streaming platform, social platform, or audio platform, etc., which is not limited in this embodiment. The server can be a single server or a cluster of multiple servers; the specific architecture of the server is not limited in this disclosure.

[0050] In some embodiments of this disclosure, the process of server 101 for video processing may include: acquiring a video to be processed; performing noise analysis on the video to be processed to obtain noise analysis results; performing motion trend analysis on the video to be processed to obtain motion trend analysis results; if the noise analysis result is a low noise level, and / or the motion trend analysis result is consistent motion, then determining to enable the target inter-frame prediction mode; after enabling the target inter-frame prediction mode, for the current coding block of the video to be processed, if the prediction mode of the current coding block is the inter-frame fusion prediction mode, determining multiple spatial candidate blocks of the current coding block, and obtaining the coding parameter information of the multiple spatial candidate blocks; and determining the number of candidate motion information of the current coding block based on the coding parameter information of the multiple spatial candidate blocks.

[0051] In addition, it should be noted that, Figure 1 The example shown is merely one application environment of the video processing method provided in this disclosure. Figure 1 The number of terminal devices 103, networks 102 and servers 101 in the diagram is merely illustrative. Depending on actual needs, there can be any number of terminal devices, networks and servers.

[0052] Figure 2 A flowchart of a video processing method according to an embodiment of this disclosure is shown. Figure 2 The execution subject of the method provided in the embodiments can be any electronic device, such as... Figure 1 Server 101 in the embodiment, for example Figure 1 Terminal device 103, for example Figure 1 In this embodiment, server 101 and terminal device 103 jointly implement the video processing method, but this disclosure is not limited thereto. (See also...) Figure 2 The video processing method provided in this disclosure includes the following steps.

[0053] Step S210: Obtain the video to be processed; perform noise analysis and motion trend analysis on the video to be processed, and obtain the noise analysis results and motion trend analysis results.

[0054] In this embodiment of the disclosure, the video to be processed refers to the video that needs to be encoded, and the video data to be processed or encoded is loaded or received from the source device (such as a camera, streaming media, video file, etc.).

[0055] In this embodiment of the disclosure, noise can be understood as unwanted, random, and interfering signals in the video signal. The purpose of noise analysis on the video to be processed is to quantify the noise level in the video in order to determine whether to enable the target inter-frame prediction mode, i.e. the optimized inter-frame prediction mode, for the video to be processed.

[0056] In this embodiment of the disclosure, noise analysis methods based on frequency domain energy distribution, singular value decomposition, convolutional neural networks, and wavelet transform can be used to analyze the noise in the video to be processed. Of course, other noise analysis methods can also be selected, and there is no limitation on them.

[0057] In this embodiment of the disclosure, motion trend analysis focuses on the motion trend of objects in the video to be processed. Motion trend analysis can identify whether there is consistent motion in the video to be processed (such as objects in the video all moving in one direction) or whether the video content is regular (such as regular object frames) in order to determine whether to enable the target inter-frame prediction mode for the video to be processed.

[0058] In this embodiment of the disclosure, motion trend analysis of the video to be processed can be performed using a bitrate control-based analysis method, an optical flow-based motion analysis method, and a feature point matching-based motion analysis method. Of course, other motion trend analysis methods can also be selected, and there is no limitation on this.

[0059] Step S220: If the noise analysis result is a low noise level, and / or the motion trend analysis result is that there is consistent motion, then determine to enable the target inter-frame prediction mode.

[0060] In this embodiment of the disclosure, the target inter-frame prediction mode is an inter-frame prediction mode optimized from the Merge mode. In the Merge mode, the maximum number of candidate motion information in the motion information candidate list is 6. In the target inter-frame prediction mode, the number of candidate motion information for a coding block is related to the information of its spatial candidate blocks, and the number of candidate motion information for each coding block is dynamically adjusted.

[0061] In this embodiment of the disclosure, the target inter-frame prediction mode is determined to be enabled if one or more of the following conditions are met: the noise analysis result is a low noise level; the motion trend analysis result is that there is consistent motion.

[0062] In this embodiment of the disclosure, noise analysis is performed on the video to be processed to evaluate the cleanliness of the video frames. In scenarios with low noise levels, the video data is more accurate and reliable. Therefore, enabling the target inter-frame prediction mode can make more accurate use of video information and reduce prediction errors caused by noise interference.

[0063] In this embodiment of the disclosure, by analyzing the motion trend of the video to be processed, it is determined whether there is consistent motion. When consistent motion is detected, it means that the motion information in the video to be processed has high coherence and predictability. Therefore, enabling the target inter-frame prediction mode can more effectively utilize the motion information of surrounding blocks to predict the motion of the current block, thereby improving the prediction accuracy.

[0064] Step S230: When the target inter-frame prediction mode is enabled and the current coding block of the video to be processed adopts the inter-frame fusion prediction mode, determine multiple spatial candidate blocks of the current coding block and obtain the coding parameter information of multiple spatial candidate blocks.

[0065] Step S240: Determine the number of candidate motion information for the current coding block based on the coding parameter information of multiple spatial candidate blocks.

[0066] In this embodiment of the disclosure, the number of candidate motion information for the current coding block can be understood as the maximum number of candidates in the candidate information list corresponding to the current coding block. After enabling the target inter-frame prediction mode, for the current coding block of the video to be processed, it is determined whether the current coding block uses the inter-frame fusion prediction mode for prediction. If the current coding block determines to use the inter-frame fusion prediction mode for prediction, it indicates that the current coding block uses information from its surrounding coded blocks for prediction. In this case, multiple spatial candidate blocks of the current coding block are searched, such as blocks above, below, to the left, or to the right of the current coding block, and then the coding parameter information of these spatial candidate blocks is obtained to determine the number of candidate motion information for the current coding block using the obtained coding parameter information.

[0067] In the video processing method provided in this disclosure, noise analysis and motion trend analysis are performed on the video to be processed. If at least one of the two conditions of low noise level or consistent motion is met, it is determined that a target inter-frame prediction mode (i.e., an optimized inter-frame fusion prediction mode) will be enabled for the video. By using noise analysis and motion trend analysis, the target inter-frame prediction mode can be enabled only in suitable scenarios, avoiding unnecessary computational overhead and improving coding efficiency. After enabling the target inter-frame prediction mode, for the current coding block, if it is determined that the current coding block is in inter-frame fusion prediction mode, the number of candidate motion information of the current coding block is determined according to the coding parameter information of its spatial candidate blocks. The number of candidate motion information of each coding block can be dynamically adjusted, avoiding the use of too much candidate information, reducing the bitrate consumption generated by the coding candidate index, and further improving coding efficiency.

[0068] To facilitate understanding, the various steps in the video processing method of the embodiments of this disclosure will be explained and described.

[0069] In step S210, the video to be processed is acquired; noise analysis and motion trend analysis are performed on the video to be processed to obtain the noise analysis results and motion trend analysis results.

[0070] In some embodiments of this disclosure, the video processing method further includes: for each video frame to be processed in the video to be processed, determining the noise value of the video frame to be processed; and calculating the average of the noise values ​​of all video frames to be processed in the video to be processed to obtain the noise value of the video to be processed.

[0071] In one possible implementation, a noise analysis method based on frequency domain energy distribution is used to analyze the noise in the video to be processed. Specifically, the implementation is as follows:

[0072] (1) For each video frame to be processed in the video to be processed, the video frame to be processed is divided into multiple image blocks, for example, into MxM image blocks.

[0073] (2) For each image block, perform Discrete Cosine Transform (DCT) processing to obtain a transformed block, which transforms the image block from the spatial domain to the frequency domain, so that the energy of the image is mainly concentrated in the low-frequency part. For example Figure 3 As shown, for each MxM image block, the absolute values ​​of the coefficients in the low-frequency region are summed and concentrated in the upper left corner of the image block to obtain E0; the absolute values ​​of the coefficients in the high-frequency region of the remaining 3 / 4 region are summed to obtain E1. The noise level of the image block is calculated as N = E0 / E1, where E0 represents the low-frequency energy of the image block and E1 represents the high-frequency energy of the image block. The larger N is, the higher the noise level.

[0074] (3) After obtaining the noise level of all image blocks in the video frame to be processed, the average noise level of all image blocks is calculated to obtain the noise level of the video frame to be processed.

[0075] (4) After obtaining the noise level of all video frames to be processed in the video to be processed, the average noise level of all video frames to be processed is calculated to obtain the noise level of the video to be processed.

[0076] In some embodiments of this disclosure, noise analysis is performed on the video to be processed to obtain noise analysis results, including one or more of the following: determining whether the noise value of the video frame to be processed is less than a first noise threshold; if so, determining that the noise analysis result of the video frame to be processed is a low noise level; determining whether the noise value of the video to be processed is less than a second noise threshold; if so, determining that the noise analysis result of the video to be processed is a low noise level.

[0077] In this embodiment of the present disclosure, each video frame to be processed is analyzed to determine whether the noise value of the video frame to be processed is less than a first noise threshold. If it is, the noise analysis result of the video frame to be processed is determined to be low noise level; otherwise, the noise analysis result of the video frame to be processed is determined to be high noise level. The first noise threshold can be set according to the actual situation, and its value range is [0,1].

[0078] In this embodiment of the disclosure, the video to be processed is analyzed to determine whether the noise value of the video to be processed is less than a second noise threshold. If so, the noise analysis result of the video to be processed is determined to be low noise level; otherwise, the noise analysis result of the video to be processed is determined to be high noise level. The second noise threshold can be set according to the actual situation, and its value range is [0,1].

[0079] The video processing method provided in this disclosure provides a noise level judgment at the video frame level, which can provide a more detailed understanding of the noise level of each frame, thereby better adapting to the characteristics of different frames; it also provides a noise level judgment at the video level (i.e., at the video frame sequence level), which can obtain the noise level of the entire video frame to be processed, thereby making a more reasonable encoding decision; it can also choose to perform noise level judgment at the video frame level or at the video level according to actual needs.

[0080] In some embodiments of this disclosure, motion trend analysis is performed on the video to be processed to obtain motion trend analysis results, including: for each video frame to be processed, obtaining one or more image blocks in the video frame to be processed; for each image block, determining the quadrant corresponding to the motion information of the image block; determining the proportion value of the image blocks corresponding to different quadrants based on the quadrants corresponding to the motion information of all image blocks, and selecting the largest proportion value from the proportion values ​​of the image blocks corresponding to different quadrants; if the largest proportion value is greater than the proportion threshold, it is determined that the motion trend analysis result of the video to be processed is that there is consistent motion.

[0081] In one possible implementation, a bitrate-controlled analysis method is used to perform motion trend analysis on the video to be processed. Specifically, this is implemented as follows:

[0082] (1) For each video frame to be processed in the video to be processed, divide the video frame to be processed into multiple image blocks.

[0083] (2) For each image block, search in the reference frame of the video frame to be processed to obtain the matching block of the image block, calculate the MV of the image block and the matching block of the image block, and determine which quadrant the calculated MV maps to, that is, determine the quadrant corresponding to the motion information of the image block.

[0084] Figure 4 A mapping diagram of motion information and quadrants for image blocks according to an embodiment of this disclosure is shown. For example... Figure 4 As shown, if MV{x,y}={4,3}, the MV will be determined to be mapped to the first quadrant; if MV{x,y}={-4,3}, the MV will be determined to be mapped to the second quadrant; if MV{x,y}={-4,-3}, the MV will be determined to be mapped to the third quadrant; if MV{x,y}={4,-3}, the MV will be determined to be mapped to the fourth quadrant.

[0085] For example, the video frame to be processed is divided into multiple 8x8 image blocks. Each 8x8 image block is searched for a matching block in the reference frame {ref0, ref1, ..., refm}. Each 8x8 image block has a corresponding MV in the reference frame, resulting in {MV0, MV1, ..., MVm}. Since m represents the distance between the reference frame and the video frame to be processed, the larger m is, the farther the reference frame refm is from the video frame to be processed. Therefore, the reference frame ref0 is the closest to the video frame to be processed. Thus, MV0 is selected as the motion information of the image block, and the quadrant mapped to MV0 is determined to be the quadrant corresponding to the motion information of the image block.

[0086] (3) After obtaining the quadrants corresponding to the motion information of all image blocks in the video frame to be processed, the proportion of the number of images mapped to different quadrants is counted to the total number of image blocks, i.e., the proportion of image blocks corresponding to different quadrants, and the largest proportion is selected. The specific calculation formula is as follows:

[0087] Q1 = N1 / BlockNum (1)

[0088] Q2 = N2 / BlockNum (2)

[0089] Q3 = N3 / BlockNum (3)

[0090] Q4 = N4 / BlockNum (4)

[0091] Q1max=MAX(Q1,Q2,Q3,Q4) (5)

[0092] Wherein, BlockNum represents the total number of image blocks, N1, N2, N3, and N4 represent the number of image blocks corresponding to the MV mapped in the first, second, third, and fourth quadrants, respectively, Q1, Q2, Q3, and Q4 represent the percentage values, and Q1max represents the maximum percentage value among the percentage values.

[0093] (4) Determine whether the maximum percentage value is greater than the percentage threshold. If it is, then the motion trend analysis result of the video frame to be processed is determined to have consistent motion. If not, then the motion trend analysis result of the video frame to be processed is determined to have no consistent motion. The percentage threshold can be set according to the actual situation, and its value range is (0,1).

[0094] The video processing method provided in this disclosure subdivides the video frame to be processed into multiple image blocks and analyzes the motion information of each image block individually. By mapping the motion information of the image blocks to different quadrants and statistically analyzing the proportion of image blocks in each quadrant, the trend of motion within the video frame to be processed can be intuitively reflected. By setting a proportion threshold, consistent motion is considered to exist in the video frame to be processed only when the maximum proportion value exceeds the threshold. This helps reduce erroneous analysis results caused by misjudgment of individual image blocks or noise interference, improving the accuracy and reliability of the analysis. Furthermore, parameters such as the size and number of image blocks and the proportion threshold can be adjusted according to different application scenarios and needs to adapt to different tasks.

[0095] In step S220, if the noise analysis result is a low noise level and / or the motion trend analysis result is that there is consistent motion, then the target inter-frame prediction mode is determined to be enabled.

[0096] In this embodiment of the disclosure, it is possible to determine whether to enable the target inter-frame prediction mode at the video frame level or at the video level.

[0097] In one possible implementation, if the decision is made at the video level, it can be first determined whether the noise analysis result of the video to be processed indicates a low noise level. If the noise level of the video to be processed is low, then the target inter-frame prediction mode is enabled for that video. If the noise level of the video to be processed is high, then motion trend analysis is performed on each video frame to determine whether the target inter-frame prediction mode should be enabled for each video frame. For each video frame to be processed, if the motion trend analysis result indicates consistent motion, then the target inter-frame prediction mode is enabled for that video frame; if the motion trend analysis result indicates no consistent motion, then the target inter-frame prediction mode is not enabled for that video frame.

[0098] In one possible implementation, if the decision is made at the video level, then each video frame to be processed is analyzed. For each video frame to be processed, if the video frame satisfies at least one of the following two conditions: low noise level and consistent motion, then the target inter-frame prediction mode is determined to be enabled for that video frame; otherwise, the target inter-frame prediction mode is determined not to be enabled for that video frame.

[0099] The video processing method provided in this disclosure can select whether to enable the target inter-frame prediction mode based on the specific situation of each frame, and can control the encoding process more finely; it can also determine whether to enable the target inter-frame prediction mode at the video level, and can consider whether to enable a certain inter-frame prediction mode from the perspective of the entire video, thereby optimizing the overall encoding efficiency while maintaining video quality.

[0100] In step S230, when the target inter-frame prediction mode is enabled and the current coding block of the video to be processed adopts the inter-frame fusion prediction mode, multiple spatial candidate blocks of the current coding block are determined, and the coding parameter information of the multiple spatial candidate blocks is obtained.

[0101] In this embodiment of the disclosure, the spatial candidate block of the current coding block refers to the coded blocks surrounding the current coding block, such as the adjacent left block, top block, top-left block, etc. Motion information includes MV, where the MV of the spatial candidate block is the motion vector of the coded block that is spatially adjacent to the current coding block.

[0102] Figure 5 A schematic diagram showing the positions of multiple spatial candidate blocks of the current coded block according to an embodiment of this disclosure is illustrated. Figure 5 As shown, A1 is the adjacent coding block at the lower left of the current coding block, B1 is the adjacent coding block at the upper right of the current coding block, B2 is the adjacent coding block at the upper left of the current coding block, and A0 and B0 are the coding blocks closest to A1 and B1, respectively. The spatial candidate blocks for the current coding block are determined according to the order B1->A1->B0->A0->B2. For example, if the number of spatial candidate blocks is determined to be 4, then B1, A1, B0, and A0 are selected as the spatial candidate blocks for the current coding block; similarly, if the number of spatial candidate blocks is determined to be 5, then B1, A1, B0, A0, and B2 are selected as the spatial candidate blocks for the current coding block.

[0103] In step S240, the number of candidate motion information for the current coding block is determined based on the coding parameter information of multiple spatial candidate blocks.

[0104] In some embodiments of this disclosure, the encoding parameter information includes motion information. Specifically, determining the number of candidate motion information for the current encoding block based on the encoding parameter information of multiple spatial candidate blocks includes: for every two spatial candidate blocks, determining the difference between the two spatial candidate blocks based on their motion information; selecting the largest difference from all differences between two spatial candidate blocks, and determining a difference threshold based on the largest difference; and determining the number of candidate motion information for the current encoding block based on the difference threshold.

[0105] For every two spatial candidate blocks among multiple spatial candidate blocks, calculate the difference between the two spatial candidate blocks based on their MV (Modular Values). The calculation formula is as follows:

[0106] C(mv1,mv2)=ABS(mv1.x – mv2.x) + ABS(mv1.y – mv2.y) (6)

[0107] Where ABS() represents taking the absolute value, mv1 and mv2 represent the MV of two selected spatial candidate blocks from multiple spatial candidate blocks, and C(mv1, mv2) represents the difference between these two spatial candidate blocks. After calculating the differences between all pairwise combinations of spatial candidate blocks from multiple spatial candidate blocks, the largest difference is selected, and a difference threshold is set to be greater than or equal to the largest difference. This ensures that the differences between all pairwise combinations of spatial candidate blocks from multiple spatial candidate blocks are less than or equal to this difference threshold, as shown in the following formula:

[0108] ABS(mv1.x – mv2.x) + ABS(mv1.y – mv2.y)<=K (7)

[0109] Where K represents the difference threshold, it can be seen from formula (7) that the difference between all pairs of spatial candidate blocks in the multiple spatial candidates is less than the difference threshold K. Then, the number of candidate motion information in the current coding block is determined according to the difference threshold.

[0110] In some embodiments of this disclosure, determining the number of candidate motion information for the current coding block based on a difference threshold includes: if the difference threshold is 0, then determining the number of candidate motion information for the current coding block as 1; if the difference threshold is less than or equal to a first threshold, then determining the number of candidate motion information for the current coding block as a preset value.

[0111] In this embodiment of the disclosure, for the HEVC standard, if the difference threshold K is 0, it indicates that the motion information of multiple spatial candidate blocks of the current coding block is consistent, and the number of candidate motion information of the current coding block is reduced to 1.

[0112] In this embodiment of the disclosure, for the HEVC standard, if the difference threshold K is not 0 and is less than or equal to the first threshold, the number of candidate motion information in the current coding block is reduced to a preset value. For example, the first threshold is set to 4, and the preset value is set to 3. Of course, the values ​​of the first threshold and the preset value can be set according to actual needs, experience, or coding quality requirements, and this embodiment of the disclosure does not limit this.

[0113] In the video processing method provided in this disclosure, the difference in motion information between pairs of spatial candidate blocks is analyzed, and a difference threshold is determined based on the maximum value of the difference. When the difference threshold is small, it means that the motion information of multiple spatial candidate blocks is highly similar. In this case, reducing the number of candidate motion information can reduce coding complexity and improve coding speed. In particular, when the difference threshold is 0, that is, the motion information of all spatial candidate blocks is exactly the same, only one candidate motion information is needed, which greatly simplifies the coding process. When the difference threshold is small (less than or equal to the first threshold), by presetting the number of candidate motion information, a certain coding quality can be maintained while ensuring coding efficiency.

[0114] It should be noted that the method described above for determining the number of candidate motion information for the current coding block based on the motion information of multiple spatial candidate blocks can be applied to the HEVC standard, as well as other coding standards. This disclosure does not limit this application.

[0115] In some embodiments of this disclosure, the encoding parameter information further includes at least one non-motion information. The video processing method further includes: for each non-motion information, obtaining the number of spatial candidate blocks with the same value in at least one spatial candidate block, determining the number as a specific number of non-motion information; selecting the minimum number from all the specific numbers of non-motion information, and determining a specific number threshold based on the minimum number.

[0116] In this embodiment of the disclosure, for both the VVC and ECM standards, when determining the number of candidate motion information for the current coding block, in addition to considering the motion information of multiple spatial candidate blocks, non-motion information of multiple spatial candidate blocks is also considered, i.e., information other than the motion information. In the VVC standard, non-motion information includes the BCW index and HF; in the ECM standard, non-motion information includes the BCW index, HF, and LIC flag. Of course, other non-motion information parameters may also be introduced to further improve coding performance. These parameters may include various identifiers and parameters related to intra-frame prediction, quantization, entropy coding, etc., which are not limited in this embodiment of the disclosure.

[0117] In this embodiment of the disclosure, for each non-motion information, the number of spatial candidate blocks with the same value among multiple spatial candidate blocks of the current coding block is counted, and this data is named the specific quantity of the non-motion information. Then, the minimum quantity is selected from all the specific quantities of non-motion information, and a quantity threshold is set to be less than or equal to the minimum quantity, so that the specific quantities of all non-motion information are greater than or equal to the quantity threshold.

[0118] For example, for the VVC standard, for the BCW index, count the number of spatial candidate blocks with the same BCW index value among the multiple spatial candidate blocks of the current coding block, p1; for HF, count the number of spatial candidate blocks with the same HF value among the multiple spatial candidate blocks of the current coding block, p2; select the minimum number between p1 and p2, and set a number threshold P less than or equal to the minimum number, that is, both p1 and p2 are greater than or equal to the number threshold.

[0119] For example, for the ECM standard, for the BCW index, count the number of spatial candidate blocks with equal BCW index values ​​among multiple spatial candidate blocks of the current coding block, p3; for HF, count the number of spatial candidate blocks with equal HF values ​​among multiple spatial candidate blocks of the current coding block, p4; for LIC flag, count the number of spatial candidate blocks with equal LIC flag values ​​among multiple spatial candidate blocks of the current coding block, p5; select the minimum number among p3, p4, and p5, and set a quantity threshold P less than or equal to this minimum number, that is, p3, p4, and p5 are all greater than or equal to the quantity threshold.

[0120] In some embodiments of this disclosure, determining the number of candidate motion information for the current coding block based on a difference threshold includes: if a specific number threshold is the number of multiple spatial candidate blocks and the difference threshold is 0, then the number of candidate motion information for the current coding block is determined to be 1; if a specific number threshold is greater than or equal to a second threshold and the difference threshold is less than or equal to a first threshold, then the number of candidate motion information for the current coding block is determined to be a preset value.

[0121] In this embodiment of the disclosure, for the VVC standard and the HEVC standard, if the difference threshold K is 0 and the quantity threshold P is the number of multiple spatial candidate blocks of the current coding block, it indicates that the motion information of the multiple spatial candidate blocks of the current coding block is consistent, and the number of candidate motion information of the current coding block is reduced to 1.

[0122] In this embodiment of the disclosure, for both the VVC and HEVC standards, if the difference threshold K is less than or equal to a first threshold and the quantity threshold P is greater than or equal to a second threshold, then the number of candidate motion information for the current coding block is reduced to a preset value. For example, the first threshold and the second threshold are both set to 4, and the preset value is set to 3; or the first threshold and the second threshold are both set to 3, and the preset value is set to 4. Of course, the values ​​of the first threshold, the second threshold, and the preset value can be set according to actual needs, experience, or coding quality requirements, and this embodiment of the disclosure does not limit this.

[0123] In the video processing method provided in this disclosure, in addition to analyzing the motion information differences between pairs of spatial candidate blocks and determining the difference threshold based on the maximum value of the differences, the non-motion information of multiple spatial candidate blocks is also analyzed to obtain a quantity threshold. When the difference threshold is small and the quantity threshold is large, it means that the motion information of multiple spatial candidate blocks is highly similar. In this case, reducing the number of candidate motion information can reduce coding complexity and improve coding speed. In particular, when the difference threshold is 0 and the quantity threshold is the number of multiple spatial candidate blocks, it is determined that the motion information of all spatial candidate blocks is completely identical. At this time, only one candidate motion information is needed, which greatly simplifies the coding process. When the difference threshold is small (less than or equal to the first threshold) and the quantity threshold is large (greater than or equal to the second threshold), by presetting a quantity of candidate motion information, a certain coding quality can be maintained while ensuring coding efficiency.

[0124] It should be noted that the method described above for determining the number of candidate motion information for the current coding block based on the motion information of multiple spatial candidate blocks can be applied to the VVC standard and the HEVC standard, as well as other coding standards. This disclosure does not limit this application.

[0125] In some embodiments of this disclosure, the video processing method further includes: encoding an index of the candidate motion information of the current coding block based on the number of candidate motion information of the current coding block.

[0126] In Merge mode, the maximum number of candidates in the motion information candidate list is 6. The merge index uses a truncated unary code representation, as shown in Table 1 below. Table 1 shows that the merge index in Merge mode requires 1 to 5 bits to encode. The larger the selected candidate index, the higher the code rate required to encode that index.

[0127] Table 1. Meaning of syntax elements in the merge index in the Merge schema.

[0128] value Meaning 0 Select the 0th candidate. 10 Select the first candidate. 110 Choose the second candidate 1110 Choose the third candidate 11110 Select the 4th candidate 11111 Select the 5th candidate

[0129] In this embodiment, the target inter-frame prediction mode is an inter-frame prediction mode optimized from the Merge mode. It can determine the number of candidate motion information for the current coding block based on the coding parameter information of multiple spatial candidate blocks, i.e., determine the maximum number of candidates in the motion information candidate list corresponding to the current coding block. After starting the target inter-frame prediction mode, if the coding parameter information of multiple spatial candidate blocks of the current coding block is highly consistent, the number of candidate motion information for the current coding block is reduced from 6 to 3, resulting in the syntax element meaning table shown in Table 2. Table 2 shows that 1 to 3 bits are needed for encoding. Comparing the two, when the second candidate is selected in Table 2, the corresponding value is 11, while in Table 1, the corresponding value is 110, saving 1 bit.

[0130] Table 2. Meaning of the syntax elements of merge index when the number of candidate motion information is 3.

[0131] value Meaning 0 Select the 0th candidate. 10 Select the first candidate. 11 Choose the second candidate

[0132] In this embodiment of the disclosure, after enabling the target inter-frame prediction mode, when the inter-frame prediction mode is selected for the current coding block, the number of candidate motion information of the current coding block is derived based on the coding parameter information of multiple spatial candidate blocks. In other words, the maximum number of candidates in the motion information candidate list is adjusted, and the truncation coding method for the index of the coding candidate motion information is also optimized.

[0133] If the current coding block selects the Merge mode, then merge_flag equals 1. In the original design for Merge mode, the merge index is directly encoded, that is, the merge index is written into the bitstream according to the syntax element meaning table shown in Table 1. For the target inter-frame prediction mode, if merge_flag equals 1, the number of candidate motion information of the current coding block is determined based on the encoding parameter information of multiple spatial candidate blocks of the current coding block; if the number is determined to be 1, then no merge index needs to be encoded; if the number is determined to be other values, then the merge index is encoded according to the magnitude of the number. For example, if the number is 3, then the merge index is encoded using the syntax element meaning table shown in Table 2.

[0134] In this embodiment of the disclosure, the number of candidate motion information of the current coding block can be derived from the coding parameter information of multiple spatial candidate blocks, and then coding can be performed based on the derived number. This takes into account the differences in motion information correlation between different video contents and image regions, thereby improving coding efficiency in consistent motion scenarios.

[0135] In some embodiments of this disclosure, the video processing method further includes: if the inter-frame prediction direction of the current coding block is bidirectional prediction, then based on the rate-distortion optimization method, determining the candidate weight information of the current coding block from the weight information pointed to by the candidate bidirectional weight prediction index.

[0136] In this embodiment of the disclosure, when the prediction direction ID of the current coding block is equal to 3, that is, the current coding block is bidirectionally predicted, the candidate weight information of the current coding block, i.e., the optimal BCW candidate, is selected through rate-distortion optimization. Specifically, the implementation is as follows:

[0137] (1) Based on the BCW weights pointed to by the candidate BCW index, determine the BCW candidates that need to be selected for rate-distortion optimization in the current coding block.

[0138] Table 3 BCW Candidate List

[0139]

[0140] As shown in Table 3, if the BCW index is less than or equal to 2, there are 3 BCW candidates; if the BCW index is greater than 2, there are 2 BCW candidates. Based on the BCW candidate table shown in Table 3, the encoding method for BCW candidates is further defined.

[0141] Table 4-1 Encoding methods for BCW candidates (BCW index <= 2)

[0142] Candidate 0th position 1st place Candidate 1 0 Candidate 2 1 0 Candidate 3 1 1

[0143] Table 4-2 Encoding methods for BCW candidates (BCW index>2)

[0144] Candidate 0th position Candidate 1 0 Candidate 2 1

[0145] As shown in Table 4-1, when the BCW index <= 2, there are three BCW candidates. When the first candidate is selected, a 0 value is encoded; when the second candidate is selected, a 1 and a 0 value are encoded; and when the third candidate is selected, two 1 values ​​are encoded. As shown in Table 4-2, for cases where the BCW index > 2, the encoding choice depends on whether the first or second candidate is ultimately selected, resulting in either a 0 or a 1.

[0146] (2) Based on the defined BCW candidates, the optimal BCW candidates are traversed through the rate-distortion optimization method. Then, the method to encode the BCW candidates is determined from Table 4-1 and Table 4-2, so as to encode the optimal BCW candidates.

[0147] In some embodiments of this disclosure, the video processing method further includes: if the inter-frame prediction direction of the current coding block is non-bidirectional prediction, then determining the state information of the local illumination compensation identifier of the current coding block based on the rate-distortion optimization method.

[0148] In this embodiment of the disclosure, when the prediction direction ID of the current coding block is equal to 1 or 2, the LIC flag of the current coding block is determined to be on or off by rate-distortion optimization, and the status information of the LIC flag is written into the bitstream according to the encoding method shown in Table 5.

[0149] Table 5. LIC flag encoding method

[0150] Candidate State value Do not enable LIC-off 0 Enable LIC-on 1

[0151] In this embodiment, for both the VVC and ECM standards, rate-distortion optimization is used to select the state values ​​of the BCW index and LIC flag. The selection result is written into the bitstream, and the corresponding syntax elements are modified. Specifically, after obtaining the basic motion information of the merged current coding block, the prediction direction in the merged basic motion information is used to determine whether to encode the BCW index or the LIC flag. If the prediction direction is bidirectional (i.e., ID equals 3), the BCW index is encoded; otherwise, the LIC flag is encoded.

[0152] In the video processing method provided in this disclosure, the BCW candidates and LIC flag state of the current coding block are derived by rate-distortion optimization, rather than by direct reuse. This method can more accurately match the characteristics of the current coding block, ensure that they are optimally configured during the encoding process, reduce redundant information during the encoding process, and improve encoding efficiency.

[0153] It is understood that the same / similar parts between the various embodiments of the methods described above in this specification can be referred to each other. Each embodiment focuses on the differences from other embodiments, and relevant parts can be referred to the description of other method embodiments.

[0154] Figure 6 A block diagram of a video processing apparatus according to an embodiment of the present disclosure is shown. Figure 6 As shown, the device 600 includes a video analysis unit 610, a determination unit 620, and a mode execution unit 630.

[0155] The video analysis unit 610 is configured to: acquire the video to be processed; perform noise analysis and motion trend analysis on the video to be processed, and obtain the noise analysis results and motion trend analysis results. The determination unit 620 is configured to: if the noise analysis result is a low noise level, and / or the motion trend analysis result is the presence of consistent motion, then determine to enable the target inter-frame prediction mode. The mode execution unit 630 is configured to, when the target inter-frame prediction mode is enabled and the current coding block of the video to be processed adopts the inter-frame fusion prediction mode, determine multiple spatial candidate blocks of the current coding block, acquire the coding parameter information of the multiple spatial candidate blocks, and determine the number of candidate motion information of the current coding block based on the coding parameter information of the multiple spatial candidate blocks.

[0156] In some embodiments of this disclosure, the encoding parameter information includes motion information. The mode execution unit 630 is further configured to: for every two spatial candidate blocks among a plurality of spatial candidate blocks, determine the difference between the two spatial candidate blocks based on the motion information of the two spatial candidate blocks; select the maximum difference from all differences between two spatial candidate blocks, and determine a difference threshold based on the maximum difference; and determine the number of candidate motion information for the current encoding block based on the difference threshold.

[0157] In some embodiments of this disclosure, the mode execution unit 630 is further configured to: if the difference threshold is 0, determine that the number of candidate motion information of the current coding block is 1; if the difference threshold is less than or equal to the first threshold, determine that the number of candidate motion information of the current coding block is a preset value.

[0158] In some embodiments of this disclosure, the encoding parameter information further includes at least one non-motion information. The mode execution unit 630 is further configured to: for each non-motion information, obtain the number of spatial candidate blocks with the same value in at least one spatial candidate block, determine the number as a specific number of non-motion information; select the minimum number from all the specific numbers of non-motion information, and determine a specific number threshold based on the minimum number.

[0159] In some embodiments of this disclosure, the mode execution unit 630 is further configured to: if a specific quantity threshold is the number of multiple spatial candidate blocks and the difference threshold is 0, then determine that the number of candidate motion information of the current coding block is 1; if the specific quantity threshold is greater than or equal to a second threshold and the difference threshold is less than or equal to a first threshold, then determine that the number of candidate motion information of the current coding block is a preset value.

[0160] In some embodiments of this disclosure, the device 600 further includes an encoding unit 640 configured to: encode an index of the candidate motion information of the current coding block according to the number of candidate motion information of the current coding block.

[0161] In some embodiments of this disclosure, the mode execution unit 630 is further configured to: if the inter-frame prediction direction of the current coding block is bidirectional prediction, then determine the candidate weight information of the current coding block from the weight information pointed to by the candidate bidirectional weight prediction index based on the rate-distortion optimization method.

[0162] In some embodiments of this disclosure, the mode execution unit 630 is further configured to: if the inter-frame prediction direction of the current coding block is non-bidirectional prediction, determine the state information of the local illumination compensation identifier of the current coding block based on the rate-distortion optimization method.

[0163] In some embodiments of this disclosure, the video analysis unit 610 is further configured to: determine the noise value of each video frame to be processed in the video to be processed; and calculate the average of the noise values ​​of all video frames to be processed in the video to be processed to obtain the noise value of the video to be processed.

[0164] In some embodiments of this disclosure, the video analysis unit 610 is further configured to: determine whether the noise value of the video frame to be processed is less than a first noise threshold; if so, determine that the noise analysis result of the video frame to be processed is a low noise level; and determine whether the noise value of the video to be processed is less than a second noise threshold; if so, determine that the noise analysis result of the video to be processed is a low noise level.

[0165] In some embodiments of this disclosure, the video analysis unit 610 is further configured to: for each video frame to be processed in the video to be processed, obtain one or more image blocks in the video frame to be processed; for each image block, determine the quadrant corresponding to the motion information of the image block; determine the proportion value of the image blocks corresponding to different quadrants according to the quadrants corresponding to the motion information of all image blocks, and select the maximum proportion value from the proportion values ​​of the image blocks corresponding to different quadrants; if the maximum proportion value is greater than the proportion threshold, then determine that the motion trend analysis result of the video to be processed is that there is consistent motion.

[0166] Regarding the apparatus in the above embodiments, the specific manner in which each unit performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0167] Figure 7 A schematic diagram of the structure of an electronic device according to an embodiment of the present disclosure is shown. It should be noted that... Figure 7 The illustrated electronic device 700 is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments disclosed herein.

[0168] like Figure 7As shown, the electronic device 700 is manifested in the form of a general-purpose computing device. The components of the electronic device 700 may include, but are not limited to: at least one processing unit 710, at least one storage unit 720, and a bus 730 connecting different system components (including storage unit 720 and processing unit 710).

[0169] The storage unit stores program code that can be executed by the processing unit 710, causing the processing unit 710 to perform the steps described in the "Exemplary Methods" section above, based on various exemplary embodiments of the present invention. For example, the processing unit 710 can perform actions such as... Figure 2 The steps are shown in the figure.

[0170] Storage unit 720 may include a readable medium in the form of a volatile storage unit, such as random access memory (RAM) 7201 and / or cache memory 7202, and may further include a read-only memory (ROM) 7203.

[0171] The storage unit 720 may also include a program / utility 7204 having a set (at least one) program module 7205, such program module 7205 including but not limited to: an operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0172] Bus 730 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0173] Electronic device 700 can also communicate with one or more external devices 800 (e.g., keyboard, pointing device, Bluetooth device, etc.), and with one or more devices that enable a user to interact with electronic device 700, and / or with any device that enables electronic device 700 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 750. Furthermore, electronic device 700 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 740. As shown, network adapter 740 communicates with other modules of electronic device 700 via bus 730. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 700, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0174] In exemplary embodiments of this disclosure, a computer-readable storage medium is also provided, on which a program product capable of implementing the methods described above is stored. In some possible embodiments, various aspects of the invention may also be implemented as a program product comprising program code that, when the program product is run on a terminal device, causes the terminal device to perform the steps of the various exemplary embodiments of the invention described in the "Exemplary Methods" section of this specification.

[0175] According to embodiments of the present invention, a program product for implementing the above-described method may employ a portable compact disc read-only memory (CD-ROM) and include program code, and may run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto. In this document, a readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, apparatus, or device.

[0176] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0177] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of sending, propagating, or transmitting programs for use by or in conjunction with an instruction execution system, apparatus, or device.

[0178] The program code contained on the readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0179] Program code for performing the operations of this invention can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0180] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0181] Furthermore, although the steps of the method in this disclosure are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in that specific order, or that all the steps shown must be performed to achieve the desired result. Additional or alternative steps may be omitted, multiple steps may be combined into one step, and / or a step may be broken down into multiple steps.

[0182] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, mobile terminal, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0183] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered illustrative only, and the true scope and spirit of this disclosure are indicated by the appended claims.

[0184] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A video processing method, characterized in that, The method includes: Acquire the video to be processed; perform noise analysis and motion trend analysis on the video to be processed, and obtain the noise analysis results and motion trend analysis results; If the noise analysis result is a low noise level, and / or the motion trend analysis result is that there is consistent motion, then the target inter-frame prediction mode is determined to be enabled. When the target inter-frame prediction mode is enabled and the current coding block of the video to be processed adopts the inter-frame fusion prediction mode, multiple spatial candidate blocks of the current coding block are determined, and the coding parameter information of the multiple spatial candidate blocks is obtained. Based on the coding parameter information of the multiple spatial candidate blocks, the number of candidate motion information for the current coding block is determined; The encoding parameter information includes motion information. Determining the number of candidate motion information for the current encoding block based on the encoding parameter information of the plurality of spatial candidate blocks includes: for every two spatial candidate blocks among the plurality of spatial candidate blocks, determining the difference between the two spatial candidate blocks based on their motion information; selecting the maximum difference from all the differences between the two spatial candidate blocks, and determining a difference threshold based on the maximum difference; and determining the number of candidate motion information for the current encoding block based on the difference threshold.

2. The method according to claim 1, characterized in that, Determining the number of candidate motion information for the current coding block based on the difference threshold includes: If the difference threshold is 0, then the number of candidate motion information for the current coding block is determined to be 1; If the difference threshold is less than or equal to the first threshold, then the number of candidate motion information for the current coding block is determined to be a preset value.

3. The method according to claim 1, characterized in that, The encoded parameter information also includes: at least one non-motion information; The method further includes: For each of the at least one non-motion information, obtain the number of spatial candidate blocks with the same value in at least one spatial candidate block, and determine the number as a specific number of the non-motion information; Select the minimum quantity from a specific number of all the non-motion information, and determine a specific quantity threshold based on the minimum quantity.

4. The method according to claim 3, characterized in that, Determining the number of candidate motion information for the current coding block based on the difference threshold includes: If the specific quantity threshold is the number of the multiple spatial candidate blocks, and the difference threshold is 0, then the number of candidate motion information for the current coded block is determined to be 1; If the specific quantity threshold is greater than or equal to the second threshold, and the difference threshold is less than or equal to the first threshold, then the number of candidate motion information for the current coding block is determined to be a preset value.

5. The method according to any one of claims 1 to 4, characterized in that, The method further includes: The index of the candidate motion information of the current coding block is encoded based on the number of candidate motion information of the current coding block.

6. The method according to claim 1, characterized in that, The method further includes: If the inter-frame prediction direction of the current coding block is bidirectional prediction, then based on the rate-distortion optimization method, the candidate weight information of the current coding block is determined from the weight information pointed to by the candidate bidirectional weight prediction index.

7. The method according to claim 1, characterized in that, The method further includes: If the inter-frame prediction direction of the current coding block is non-bidirectional, then the state information of the local illumination compensation identifier of the current coding block is determined based on the rate-distortion optimization method.

8. The method according to claim 1, characterized in that, The method further includes: For each video frame to be processed in the video to be processed, determine the noise value of the video frame to be processed; The noise value of the video to be processed is obtained by averaging the noise values ​​of all the video frames to be processed.

9. The method according to claim 8, characterized in that, The noise analysis includes one or more of the following: Determine whether the noise value of the video frame to be processed is less than a first noise threshold. If so, determine that the noise analysis result of the video frame to be processed is low noise level. Determine whether the noise value of the video to be processed is less than the second noise threshold. If so, determine that the noise analysis result of the video to be processed is low noise level.

10. The method according to claim 1, characterized in that, The motion trend analysis includes: For each video frame to be processed in the video to be processed, obtain one or more image blocks in the video frame to be processed; For each image block, determine the quadrant corresponding to the motion information of the image block; Based on the quadrants corresponding to the motion information of all the image blocks, determine the proportion of image blocks in different quadrants, and select the largest proportion from the proportions of image blocks in different quadrants. If the maximum percentage value is greater than the percentage threshold, then the motion trend analysis result of the video to be processed is determined to show consistent motion.

11. A video processing apparatus, characterized in that, The device includes: The video analysis unit is configured to acquire a video to be processed; perform noise analysis and motion trend analysis on the video to be processed; and obtain noise analysis results and motion trend analysis results. The determining unit is configured to determine to enable the target inter-frame prediction mode if the noise analysis result is a low noise level and / or the motion trend analysis result is the presence of consistent motion. The mode execution unit is configured to, when the target inter-frame prediction mode is enabled and the current coding block of the video to be processed adopts the inter-frame fusion prediction mode, determine multiple spatial candidate blocks of the current coding block, obtain coding parameter information of the multiple spatial candidate blocks, and determine the number of candidate motion information of the current coding block based on the coding parameter information of the multiple spatial candidate blocks. The encoding parameter information includes motion information. The mode execution unit is further configured to: for every two spatial candidate blocks among the plurality of spatial candidate blocks, determine the difference between the two spatial candidate blocks based on the motion information of the two spatial candidate blocks; select the maximum difference from all the differences between the two spatial candidate blocks, and determine a difference threshold based on the maximum difference; and determine the number of candidate motion information for the current encoding block based on the difference threshold.

12. An electronic device, characterized in that, include: processor; A memory for storing processor-executable instructions; wherein the processor is configured to execute the executable instructions to implement the video processing method as described in any one of claims 1 to 10.

13. A computer-readable storage medium, wherein instructions in the computer-readable storage medium, when executed by a processor of an electronic device, enable the electronic device to perform the video processing method as described in any one of claims 1 to 10.

14. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the video processing method as described in any one of claims 1 to 10.

Images