Video processing method and device and related equipment

Abstract

The invention provides a video processing method and device and related equipment, and relates to the technical field of video processing. The method comprises the following steps: respectively carrying out noise and motion trend analysis on an obtained video to be processed to obtain a noise analysis result and a motion trend analysis result; if the noise analysis result is that the noise level is low and / or the motion trend analysis result is that consistent motion exists, determining to start a target inter-frame prediction mode; under the condition that a target inter-frame prediction mode is started and a current coding block of the video to be processed adopts an inter-frame fusion prediction mode, determining a plurality of spatial domain candidate blocks of the current coding block, and obtaining coding parameter information of the plurality of spatial domain candidate blocks; and determining the number of candidate motion information of the current coding block according to the coding parameter information of the plurality of airspace candidate blocks. According to the method, the quantity of candidate motion information can be dynamically adjusted, excessive candidate information is prevented from being used, and code rate consumption generated by encoding candidate indexes is reduced.

Description

Technical Field

[0001] The present disclosure relates to the field of video processing technology, and in particular to a video processing method, a video processing device, an electronic device, a computer-readable storage medium, and a computer program product. Background Art

[0002] In video coding standards, such as High Efficiency Video Coding (HEVC), Versatile Video Coding (VVC), and Enhanced Compression Model (ECM) based on VCC, prediction modes include intra-frame prediction and inter-frame prediction, and inter-frame prediction includes ordinary inter-frame prediction mode and inter-frame fusion prediction mode. Compared with ordinary inter-frame prediction mode, inter-frame fusion prediction mode does not need to write information into the bitstream.

[0003] However, the candidate list of the inter-frame fusion prediction mode is usually fixed, resulting in the use of too much candidate information in some scenarios, causing the merge index value (merge index, an index value used to indicate which candidate motion information is selected for the current block) to occupy more bits, increasing unnecessary bit rate consumption.

[0004] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0005] The present disclosure provides a video processing method, a video processing device, an electronic device, a computer-readable storage medium, and a computer program product to overcome the above problems or at least partially solve the above problems.

[0006] The present disclosure provides a video processing method, which includes: obtaining a video to be processed; performing noise analysis and motion trend analysis on the video to be processed to obtain a noise analysis result and a motion trend analysis result; if the noise analysis result is that the noise level is low, and / or the motion trend analysis result is that there is consistent motion, determining to enable a target inter-frame prediction mode; when the target inter-frame prediction mode is enabled and the current coding block of the video to be processed adopts an inter-frame fusion prediction mode, determining multiple spatial candidate blocks of the current coding block and obtaining coding parameter information of the multiple spatial candidate blocks; and determining the number of candidate motion information of the current coding block according to the coding parameter information of the multiple spatial candidate blocks.

[0007] In some embodiments of the present disclosure, the coding parameter information includes: motion information; wherein, determining the number of candidate motion information of the current coding block based on the coding parameter information of the multiple spatial candidate blocks includes: for every two spatial candidate blocks among the multiple spatial candidate blocks, determining the difference between the two spatial candidate blocks based on the motion information of the two spatial candidate blocks; selecting the maximum difference from all the differences between the two spatial candidate blocks, and determining a difference threshold based on the maximum difference; determining the number of candidate motion information of the current coding block based on the difference threshold.

[0008] In some embodiments of the present disclosure, determining the number of candidate motion information of the current coding block based on the difference threshold includes: if the difference threshold is 0, determining the number of candidate motion information of the current coding block to be 1; if the difference threshold is less than or equal to a first threshold, determining the number of candidate motion information of the current coding block to be a preset value.

[0009] In some embodiments of the present disclosure, the encoding parameter information also includes: at least one non-motion information; wherein the method further includes: for each of the at least one non-motion information, obtaining the number of spatial candidate blocks with the same value in the at least one spatial candidate block, and determining the number as a specific number of the non-motion information; selecting a minimum number from all the specific numbers of the non-motion information, and determining a specific number threshold based on the minimum number.

[0010] In some embodiments of the present disclosure, determining the number of candidate motion information of the current coding block based on the difference threshold includes: if the specific number threshold is the number of the multiple spatial candidate blocks, and the difference threshold is 0, then determining the number of candidate motion information of the current coding block to be 1; if the specific number threshold is greater than or equal to a second threshold, and the difference threshold is less than or equal to the first threshold, then determining the number of candidate motion information of the current coding block to be a preset value.

[0011] In some embodiments of the present disclosure, the method further includes: encoding an index of the candidate motion information of the current coding block according to the number of candidate motion information of the current coding block.

[0012] In some embodiments of the present disclosure, the method also includes: if the inter-frame prediction direction of the current coding block is bidirectional prediction, then based on a rate-distortion optimization method, determining the candidate weight information of the current coding block from the weight information pointed to by the candidate bidirectional weight prediction index.

[0013] In some embodiments of the present disclosure, the method further includes: if the inter-frame prediction direction of the current coding block is non-bidirectional prediction, determining the state information of the local illumination compensation flag of the current coding block based on a rate-distortion optimization method.

[0014] In some embodiments of the present disclosure, the method further includes: determining a noise value of each video frame to be processed in the video to be processed; and calculating the mean of the noise values ​​of all the video frames to be processed in the video to be processed to obtain the noise value of the video to be processed.

[0015] In some embodiments of the present disclosure, the noise analysis includes one or more of the following: determining whether the noise value of the video frame to be processed is less than a first noise threshold, and if so, determining that the noise analysis result of the video frame to be processed is a low noise level; determining whether the noise value of the video to be processed is less than a second noise threshold, and if so, determining that the noise analysis result of the video to be processed is a low noise level.

[0016] In some embodiments of the present disclosure, the motion trend analysis includes: for each video frame to be processed in the video to be processed, obtaining one or more image blocks in the video frame to be processed; for each image block, determining the quadrant corresponding to the motion information of the image block; according to the quadrants corresponding to the motion information of all the image blocks, determining the proportion of image blocks corresponding to different quadrants, and selecting the maximum proportion from the proportions of image blocks corresponding to different quadrants; if the maximum proportion is greater than a proportion threshold, determining that the motion trend analysis result of the video to be processed is that there is consistent motion.

[0017] An embodiment of the present disclosure provides a video processing device, which includes: a video analysis unit, configured to obtain a video to be processed; perform noise analysis and motion trend analysis on the video to be processed to obtain a noise analysis result and a motion trend analysis result; a determination unit, configured to determine to enable a target inter-frame prediction mode if the noise analysis result is a low noise level and / or the motion trend analysis result is the presence of consistent motion; a mode execution unit, configured to determine multiple spatial candidate blocks of the current coding block and obtain coding parameter information of the multiple spatial candidate blocks when the target inter-frame prediction mode is enabled and the current coding block of the video to be processed adopts an inter-frame fusion prediction mode; and determine the number of candidate motion information of the current coding block based on the coding parameter information of the multiple spatial candidate blocks.

[0018] An embodiment of the present disclosure provides an electronic device, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to execute the executable instructions to implement the above-mentioned video processing method.

[0019] An embodiment of the present disclosure provides a computer-readable storage medium. When instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device can execute the above-mentioned video processing method.

[0020] The embodiment of the present disclosure provides a computer program product, including a computer program, and when the computer program is executed by a processor, the above-mentioned video processing method is implemented.

[0021] The video processing method provided by the embodiment of the present disclosure performs noise analysis and motion trend analysis on the video to be processed, and determines to enable a target inter-frame prediction mode (i.e., an optimized inter-frame fusion prediction mode) for the video when it is determined that at least one of the two conditions of low noise level or consistent motion exists is met. The target inter-frame prediction mode can be enabled only in suitable scenarios through noise analysis and motion trend analysis, thereby avoiding unnecessary computational overhead and improving coding efficiency. After enabling the target inter-frame prediction mode, for the current coding block, when it is determined that the current coding block is in the inter-frame fusion prediction mode, the number of candidate motion information of the current coding block is determined according to the coding parameter information of its spatial candidate blocks, and the number of candidate motion information of each coding block can be dynamically adjusted to avoid using too much candidate information, reduce the bit rate consumption caused by the coding candidate index, and further improve the coding efficiency.

[0022] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without creative work.

[0024] Figure 1 A schematic diagram showing an exemplary system architecture to which the video processing method according to an embodiment of the present disclosure can be applied;

[0025] Figure 2 A flow chart of a video processing method according to an embodiment of the present disclosure is shown;

[0026] Figure 3 A schematic diagram showing a low-frequency region and a high-frequency region of an image block according to an embodiment of the present disclosure;

[0027] Figure 4 A diagram showing a mapping relationship between motion information of an image block and a quadrant according to an embodiment of the present disclosure;

[0028] Figure 5 A schematic diagram showing the positions of multiple spatial candidate blocks of a current coding block according to an embodiment of the present disclosure is shown;

[0029] Figure 6 A block diagram of a video processing device according to an embodiment of the present disclosure is shown;

[0030] Figure 7 A schematic structural diagram of an electronic device according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0031] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will be comprehensive and complete and will fully convey the concepts of the example embodiments to those skilled in the art. The same reference numerals in the figures represent the same or similar parts, and thus their repeated description will be omitted.

[0032] The features, structures or characteristics described in the present disclosure may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced while omitting one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, known methods, devices, implementations or operations are not shown or described in detail to avoid blurring the various aspects of the present disclosure.

[0033] The collection, collection, updating, analysis, processing, use, transmission, storage and other aspects of the user personal information involved in this disclosure are in compliance with the relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. Necessary measures are taken for user personal information to prevent illegal access to user personal information data and maintain the security of user personal information, network security and national security.

[0034] The accompanying drawings are only schematic diagrams of the present disclosure, and the same reference numerals in the drawings represent the same or similar parts, so their repeated description will be omitted. Some block diagrams shown in the accompanying drawings do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in at least one hardware module or integrated circuit, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0035] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and steps, nor must they be executed in the order described. For example, some steps can be decomposed, and some steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0036] In this specification, the terms "a", "an", "the", "said" and "at least one" are used to indicate the presence of at least one element / component / etc.; the term "plurality" refers to two or more; the terms "comprising", "including" and "having" are used to express an open-ended inclusion and mean that additional elements / components / etc. may exist in addition to the listed elements / components / etc.; the terms "first", "second" and "third" etc. are used only as labels and are not intended to limit the quantity of their objects.

[0037] In video coding, inter-frame fusion prediction mode is a common inter-frame prediction technology, such as Merge mode, which predicts the motion information of the current block by reusing the motion information of surrounding coded blocks.

[0038] Taking the Merge mode in the VVC standard as an example, the Merge mode uses temporal or spatial adjacent blocks to construct a motion information candidate list, and there are 6 candidate motion information in the list. Among them, there are 5 types of candidate motion information, in the following order: spatial candidates based on adjacent blocks, temporal candidates based on co-located blocks, candidates in the First In First Out (FIFO) table constructed based on historical information, paired average candidates, and zero candidates.

[0039] In the Merge mode of HEVC, VVC and ECM standards, the multiplexing information includes: inter-frame prediction direction (InterDirection, ID), motion vector (Motion Vector, MV) and reference index (Reference index, RI).

[0040] Among them, ID is used to identify whether the current block is forward predicted (ID=1), backward predicted (ID=2), or bidirectionally predicted (ID=3), so the value range is 1 to 3. MV is used to indicate the offset of the reference block relative to the current block. RI is used to identify which reference image the current block refers to. The value of RI depends on the number of reference frames. When the number of reference frames is M (M is a natural number greater than 0), the RI value is (0 to M-1), and when ID=3, there are two RI values, which are used to indicate the reference frame indexes for forward prediction and backward prediction respectively.

[0041] In addition, in the Merge mode of the VVC and ECM standards, the reuse information also includes: Bi-prediction with CU-level weight index (BCW index) and half-pel interpolation filter (HF). In the Merge mode of the ECM standard, the reuse information also includes: BCW index, HF and Local Illumination Compensation flag (LIC flag).

[0042] Among them, BCW index is used to indicate the index of the weight of two prediction signals in bidirectional prediction. HF is used to indicate whether the current block uses a half-pixel interpolation filter in the stage of generating a prediction block by motion compensation. HF has a value of 0 to 1. HF=1 means it is used, otherwise HF=0 means it is not used. LIC flag is used to indicate whether the current block uses local illumination compensation. LIC flag has a value of 0 to 1. 0 means it is not used, and 1 means it is used.

[0043] Since the candidate list in Merge mode is usually pre-defined and fixed, the same candidate list will be used regardless of whether the motion information of the current block is highly consistent with the surrounding blocks. This means that even in some scenarios, the motion information of the current block is highly consistent with the surrounding blocks, multiple candidate information will still be provided for selection, resulting in more bits occupied by the merge index and unnecessary bitrate consumption. For Merge mode, its encoding method is usually fixed and does not take into account the differences in the correlation of motion information between different video content and image regions, which leads to low encoding efficiency in consistent motion scenarios.

[0044] In addition, BCW index, HF and LIC flag correspond to different optimization algorithms, which guide the current block to generate a more effective prediction block. However, these values ​​are obtained by reusing other blocks for the current block, and the current block itself does not make a validity judgment. Therefore, these values ​​may not be accurate for the current block.

[0045] In order to solve the above problems, the embodiments of the present disclosure provide a video processing method, which can enable the target inter-frame prediction mode only in suitable scenarios through noise analysis and motion trend analysis, thereby avoiding unnecessary computing overhead and improving coding efficiency; after enabling the target inter-frame prediction mode, the amount of candidate motion information of each coding block can be dynamically adjusted to avoid using too much candidate information, reduce the bit rate consumption generated by the coding candidate index, and further improve the coding efficiency.

[0046] Figure 1The diagram shows an exemplary system architecture to which the video processing method according to the embodiment of the present disclosure can be applied.

[0047] like Figure 1 As shown, the system architecture may include a server 101, a network 102 and a terminal device 103. The network 102 is used to provide a medium for a communication link between the terminal device 103 and the server 101. The network 102 may include various connection types, such as wired, wireless communication links or optical fiber cables.

[0048] In an exemplary embodiment, the terminal device 103 for data transmission with the server 101 may include but is not limited to a smart phone, a tablet computer, a laptop computer, a smart speaker, a digital assistant, an AR (Augmented Reality) device, a VR (Virtual Reality) device, a smart wearable device, etc., or the terminal device 103 may also be a personal computer, such as a laptop computer and a desktop computer, etc. Optionally, the operating system running on the electronic device may include but is not limited to Android system, IOS system, Linux, Windows, etc.

[0049] Server 101 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. In some practical applications, server 101 can also be a server of a network platform, which can be, for example, a trading platform, a live broadcast platform, a social platform, or an audio platform, etc., which is not limited in the embodiments of the present disclosure. Among them, the server can be a single server or a cluster formed by multiple servers, and the present disclosure does not limit the specific architecture of the server.

[0050] In some embodiments of the present disclosure, the process of video processing by server 101 may be: obtaining a video to be processed; performing noise analysis on the video to be processed to obtain a noise analysis result; performing motion trend analysis on the video to be processed to obtain a motion trend analysis result; if the noise analysis result is a low noise level, and / or the motion trend analysis result is the presence of consistent motion, determining to enable a target inter-frame prediction mode; after enabling the target inter-frame prediction mode, for a current coding block of the video to be processed, when the prediction mode of the current coding block is an inter-frame fusion prediction mode, determining multiple spatial candidate blocks of the current coding block, and obtaining encoding parameter information of the multiple spatial candidate blocks; determining the number of candidate motion information of the current coding block based on the encoding parameter information of the multiple spatial candidate blocks.

[0051] In addition, it should be noted that Figure 1 What is shown is only one application environment of the video processing method provided by the present disclosure. Figure 1 The number of terminal devices 103, networks 102 and servers 101 is merely illustrative, and any number of terminal devices, networks and servers may be provided according to actual needs.

[0052] Figure 2 A flowchart of a video processing method according to an embodiment of the present disclosure is shown. Figure 2 The execution subject of the method provided in the embodiment may be any electronic device, such as Figure 1 The server 101 in the embodiment is also as follows Figure 1 The terminal device 103 in Figure 1 The server 101 and the terminal device 103 in the embodiment jointly implement the video processing method, but the present disclosure is not limited thereto. Figure 2 The video processing method provided by the embodiment of the present disclosure includes the following steps.

[0053] Step S210, obtaining a video to be processed; performing noise analysis and motion trend analysis on the video to be processed to obtain a noise analysis result and a motion trend analysis result.

[0054] In the embodiments of the present disclosure, the video to be processed refers to the video that needs to be encoded, and the video data to be processed or encoded is loaded or received from a source device (such as a camera, streaming media, video file, etc.).

[0055] In the disclosed embodiments, noise can be understood as an unnecessary, random, and interfering signal in a video signal. The purpose of performing noise analysis on a processed video is to quantify the noise level in the video so as to decide whether to subsequently enable a target inter-frame prediction mode, i.e., an optimized inter-frame prediction mode, for the processed video.

[0056] In the embodiments of the present disclosure, a noise analysis method based on frequency domain energy distribution, a noise analysis method based on singular value decomposition, a noise analysis method based on convolutional neural network, and a noise analysis method based on wavelet transform can be used to perform noise analysis on the processed video. Of course, other noise analysis methods can also be selected, and there is no limitation on this.

[0057] In the disclosed embodiment, motion trend analysis focuses on the motion trend of objects in the video to be processed. Motion trend analysis can identify whether there is consistent motion in the video to be processed (such as objects in the video all move in one direction) or whether the video content is regular (such as regular object scenes) to determine whether to enable the target inter-frame prediction mode for the video to be processed later.

[0058] In the disclosed embodiments, a rate control-based analysis method, an optical flow-based motion analysis method, and a feature point matching-based motion analysis method may be used to perform motion trend analysis on the processed video. Of course, other motion trend analysis methods may also be selected, and there is no limitation on this.

[0059] Step S220: If the noise analysis result indicates that the noise level is low, and / or the motion trend analysis result indicates that there is consistent motion, then it is determined to enable the target inter-frame prediction mode.

[0060] In the disclosed embodiment, the target inter-frame prediction mode is an inter-frame prediction mode optimized for the Merge mode. In the Merge mode, the maximum number of candidate motion information in the motion information candidate list is 6. In the target inter-frame prediction mode, the number of candidate motion information of a coding block is related to the information of the spatial candidate block of the coding block, and the number of candidate motion information of each coding block is dynamically adjusted.

[0061] In the embodiment of the present disclosure, if one or more of the following conditions are met, it is determined that the target inter-frame prediction mode is enabled: the noise analysis result is that the noise level is low; the motion trend analysis result is that there is consistent motion.

[0062] In the disclosed embodiment, by performing noise analysis on the video to be processed and evaluating the cleanliness of the video frame, the video data is more accurate and reliable in scenarios with low noise levels. Therefore, enabling the target inter-frame prediction mode can more accurately utilize the video information and reduce the prediction error caused by noise interference.

[0063] In the disclosed embodiment, the motion trend of the video to be processed is analyzed to determine whether there is consistent motion. When consistent motion is detected, it means that the motion information in the video to be processed has high consistency and predictability. Therefore, enabling the target inter-frame prediction mode can more effectively utilize the motion information of the surrounding blocks to predict the motion of the current block, thereby improving the prediction accuracy.

[0064] Step S230, when the target inter-frame prediction mode is enabled and the current coding block of the to-be-processed video adopts the inter-frame fusion prediction mode, multiple spatial candidate blocks of the current coding block are determined, and coding parameter information of the multiple spatial candidate blocks is obtained.

[0065] Step S240: Determine the amount of candidate motion information of the current coding block according to the coding parameter information of the plurality of spatial candidate blocks.

[0066] In the disclosed embodiment, the number of candidate motion information of the current coding block can be understood as the maximum number of candidates in the candidate information list corresponding to the current coding block. After enabling the target inter-frame prediction mode, for the current coding block of the video to be processed, determine whether the current coding block is predicted using the inter-frame fusion prediction mode. If the current coding block is determined to be predicted using the inter-frame fusion prediction mode, it indicates that the current coding block uses the information of the encoded blocks around it for prediction. In this case, multiple spatial candidate blocks of the current coding block are searched, such as blocks above, below, to the left or to the right of the current coding block, and then the coding parameter information of these spatial candidate blocks is obtained to determine the number of candidate motion information of the current coding block using the obtained coding parameter information.

[0067] In the video processing method provided by the embodiment of the present disclosure, noise analysis and motion trend analysis are performed on the video to be processed. When it is determined that at least one of the two conditions of low noise level or consistent motion exists is met, it is determined that the target inter-frame prediction mode (i.e., optimized inter-frame fusion prediction mode) is enabled for the video. The target inter-frame prediction mode can be enabled only in suitable scenarios through noise analysis and motion trend analysis, thereby avoiding unnecessary computational overhead and improving coding efficiency. After enabling the target inter-frame prediction mode, for the current coding block, when it is determined that the current coding block is in inter-frame fusion prediction mode, the number of candidate motion information of the current coding block is determined according to the coding parameter information of its spatial candidate blocks. The number of candidate motion information of each coding block can be dynamically adjusted to avoid using too much candidate information, reduce the bit rate consumption caused by the coding candidate index, and further improve the coding efficiency.

[0068] For ease of understanding, each step in the video processing method of the embodiment of the present disclosure is explained.

[0069] In step S210, a video to be processed is obtained; noise analysis and motion trend analysis are performed on the video to be processed to obtain a noise analysis result and a motion trend analysis result.

[0070] In some embodiments of the present disclosure, the video processing method further includes: determining the noise value of the video frame to be processed for each video frame to be processed in the video to be processed; and calculating the mean of the noise values ​​of all video frames to be processed in the video to be processed to obtain the noise value of the video to be processed.

[0071] In a possible implementation, a noise analysis method based on frequency domain energy distribution is used to perform noise analysis on the video to be processed. The specific implementation is:

[0072] (1) For each to-be-processed video frame in the to-be-processed video, the to-be-processed video frame is divided into a plurality of image blocks, for example, divided into MxM image blocks.

[0073] (2) For each image block, perform discrete cosine transform (DCT) on the image block to obtain a transform block, that is, convert the image block from the spatial domain to the frequency domain so that the energy of the image is mainly concentrated in the low-frequency part. Figure 3 As shown in the figure, for each MxM image block, add up the absolute values ​​of the coefficients in the low-frequency area and concentrate them in the upper left corner of the image block to get E0; add up the absolute values ​​of the coefficients in the high-frequency area of ​​the remaining 3 / 4 area to get E1. Calculate the noise level of the image block N = E0 / E1, where E0 represents the low-frequency energy of the image block and E1 represents the high-frequency energy of the image block. The larger N is, the higher the noise level is.

[0074] (3) After obtaining the noise levels of all image blocks in the video frame to be processed, the noise levels of all image blocks are averaged to obtain the noise level of the processed video frame.

[0075] (4) After obtaining the noise levels of all the video frames to be processed in the video to be processed, the noise levels of all the video frames to be processed are averaged to obtain the noise level of the processed video.

[0076] In some embodiments of the present disclosure, noise analysis is performed on the video to be processed to obtain a noise analysis result, including one or more of the following: determining whether the noise value of the video frame to be processed is less than a first noise threshold, and if so, determining that the noise analysis result of the video frame to be processed is a low noise level; determining whether the noise value of the video to be processed is less than a second noise threshold, and if so, determining that the noise analysis result of the video to be processed is a low noise level.

[0077] In the embodiment of the present disclosure, each video frame to be processed is analyzed to determine whether the noise value of the video frame to be processed is less than the first noise threshold. If so, the noise analysis result of the video frame to be processed is determined to be a low noise level. If not, the noise analysis result of the video frame to be processed is determined to be a high noise level. The first noise threshold can be set according to actual conditions, and its value range is [0,1].

[0078] In the embodiment of the present disclosure, the video to be processed is analyzed to determine whether the noise value of the video to be processed is less than the second noise threshold. If so, the noise analysis result of the video to be processed is determined to be a low noise level. If not, the noise analysis result of the video to be processed is determined to be a high noise level. The second noise threshold can be set according to actual conditions, and its value range is [0,1].

[0079] The video processing method provided in the embodiment of the present disclosure provides noise level judgment at the video frame level, which can provide a more detailed understanding of the noise level of each frame, thereby better adapting to the characteristics of different frames; it also provides noise level judgment at the video level (i.e., video frame sequence level), which can obtain the noise level of the entire video frame to be processed, thereby making more reasonable encoding decisions; it is also possible to choose to perform noise level judgment at the video frame level or to choose noise level judgment at the video level according to actual needs.

[0080] In some embodiments of the present disclosure, a motion trend analysis is performed on a video to be processed to obtain a motion trend analysis result, including: for each video frame to be processed in the video to be processed, obtaining one or more image blocks in the video frame to be processed; for each image block, determining the quadrant corresponding to the motion information of the image block; according to the quadrants corresponding to the motion information of all image blocks, determining the proportions of image blocks corresponding to different quadrants, and selecting the maximum proportion from the proportions of image blocks corresponding to different quadrants; if the maximum proportion is greater than a proportion threshold, determining that the motion trend analysis result of the video to be processed is that there is consistent motion.

[0081] In a possible implementation, a rate control-based analysis method is used to analyze the motion trend of the video to be processed. The specific implementation is:

[0082] (1) For each to-be-processed video frame in the to-be-processed video, the to-be-processed video frame is divided into a plurality of image blocks.

[0083] (2) For each image block, search in the reference frame of the video frame to be processed to obtain a matching block of the image block, calculate the MV of the image block and the matching block of the image block, and determine to which quadrant the calculated MV is mapped, that is, determine the quadrant corresponding to the motion information of the image block.

[0084] Figure 4 FIG. 2 shows a mapping relationship between motion information of an image block and a quadrant according to an embodiment of the present disclosure. Figure 4 As shown, if MV{x,y}={4,3}, the MV will be determined to be mapped to the first quadrant; if MV{x,y}={-4,3}, the MV will be determined to be mapped to the second quadrant; if MV{x,y}={-4,-3}, the MV will be determined to be mapped to the third quadrant; if MV{x,y}={4,-3}, the MV will be determined to be mapped to the fourth quadrant.

[0085] For example, the video frame to be processed is divided into multiple 8x8 image blocks, and each 8x8 image block will search for a matching block in the reference frame {ref0, ref1, ...refm}. Each 8x8 image block has a corresponding MV in the reference frame, and {MV0, MV1, ...MVm} is obtained. Since m represents the distance between the reference frame and the video frame to be processed, the larger the m, the farther the reference frame refm is from the video frame to be processed. Therefore, the reference frame ref0 is closest to the video frame to be processed. Therefore, MV0 is selected as the motion information of the image block, and the quadrant to which MV0 is mapped is determined to be the quadrant corresponding to the motion information of the image block.

[0086] (3) After obtaining the quadrants corresponding to the motion information of all image blocks in the video frame to be processed, the proportion of the number of images mapped in different quadrants to all image blocks is counted, that is, the proportion of image blocks corresponding to different quadrants, and the maximum proportion is selected. The specific calculation formula is:

[0087] Q1=N1 / BlockNum (1)

[0088] Q2=N2 / BlockNum (2)

[0089] Q3=N3 / BlockNum (3)

[0090] Q4=N4 / BlockNum (4)

[0091] Q1max=MAX(Q1,Q2,Q3,Q4) (5)

[0092] Among them, BlockNum represents the total number of image blocks, N1, N2, N3, and N4 represent the number of image blocks corresponding to the MV mapped in the first, second, third, and fourth quadrants respectively, Q1, Q2, Q3, and Q4 correspond to the proportion values ​​respectively, and Q1max represents the maximum proportion value among the proportion values.

[0093] (4) Determine whether the maximum proportion value is greater than the proportion threshold value. If so, determine that the motion trend analysis result of the video frame to be processed is consistent motion. If not, determine that the motion trend analysis result of the video frame to be processed is inconsistent motion. The proportion threshold value can be set according to actual conditions, and its value range is (0,1).

[0094] In the video processing method provided by the embodiment of the present disclosure, by subdividing the video frame to be processed into multiple image blocks, and analyzing the motion information of each image block separately, by mapping the motion information of the image blocks to different quadrants, and counting the image block ratio values ​​of each quadrant, the motion trend in the video frame to be processed can be intuitively reflected; by setting a ratio threshold, only when the maximum ratio value exceeds the threshold, it is considered that there is consistent motion in the video frame to be processed, which helps to reduce the erroneous analysis results caused by misjudgment of individual image blocks or noise interference, and improves the accuracy and reliability of the analysis. In addition, parameters such as the size, number, and ratio threshold of the image blocks can be adjusted according to different application scenarios and requirements to adapt to different tasks.

[0095] In step S220 , if the noise analysis result indicates that the noise level is low, and / or the motion trend analysis result indicates that there is a consistent motion, it is determined to enable the target inter-frame prediction mode.

[0096] In the disclosed embodiment, whether to enable the target inter-frame prediction mode may be determined at the video frame level, or may be determined at the video level.

[0097] In a possible implementation, if the video level judgment is selected, it can be first determined whether the noise analysis result of the video to be processed is a low noise level. If the noise level of the video to be processed is low, then for the video to be processed, it is determined to enable the target inter-frame prediction mode. If the noise level of the video to be processed is high, a motion trend analysis is performed on each video frame to be processed of the processed video to determine whether the target inter-frame prediction mode is enabled for each video frame to be processed. For each video frame to be processed, if the motion trend analysis result of the video frame to be processed is that there is consistent motion, then for the video frame to be processed, it is determined to enable the target inter-frame prediction mode; if the motion trend analysis result of the video frame to be processed is that there is no consistent motion, then for the video frame to be processed, it is determined not to enable the target inter-frame prediction mode.

[0098] In a possible implementation, if the video level judgment is selected, each to-be-processed video frame is analyzed. For each to-be-processed video frame, if the to-be-processed video frame satisfies at least one of the two conditions of low noise level and consistent motion, for the to-be-processed video frame, it is determined to enable the target inter-frame prediction mode; otherwise, for the to-be-processed video frame, it is determined not to enable the target inter-frame prediction mode.

[0099] In the video processing method provided by the embodiment of the present disclosure, it is possible to select whether to enable the target inter-frame prediction mode according to the specific situation of each frame, so that the encoding process can be controlled more finely; it is also possible to determine whether to enable the target inter-frame prediction mode according to the video level, and it is possible to consider whether to enable a certain inter-frame prediction mode from the perspective of the entire video, thereby optimizing the overall encoding efficiency while maintaining the video quality.

[0100] In step S230, when the target inter-frame prediction mode is enabled and the current coding block of the video to be processed adopts the inter-frame fusion prediction mode, multiple spatial candidate blocks of the current coding block are determined, and encoding parameter information of the multiple spatial candidate blocks is obtained.

[0101] In the disclosed embodiment, the spatial candidate block of the current coding block refers to the coded blocks around the current coding block, such as the adjacent left block, upper block, upper left block, etc. The motion information includes MV, and the MV of the spatial candidate block is the motion vector of the coded block that is spatially adjacent to the current coding block.

[0102] Figure 5 FIG. 2 shows a schematic diagram of the positions of multiple spatial candidate blocks of the current coding block in an embodiment of the present disclosure. Figure 5 As shown, A1 is the adjacent coding block at the lower left of the current coding block, B1 is the adjacent coding block at the upper right of the current coding block, B2 is the upper left adjacent coding block of the current coding block, A0 and B0 are the coding blocks closest to A1 and B1 respectively, and the spatial candidate blocks of the current coding block are determined in the order of B1->A1->B0->A0->B2. For example, if the number of spatial candidate blocks is determined to be 4, then B1, A1, B0 and A0 are selected as the spatial candidate blocks of the current coding block; for another example, if the number of spatial candidate blocks is determined to be 5, then B1, A1, B0, A0 and B2 are selected as the spatial candidate blocks of the current coding block.

[0103] In step S240, the amount of candidate motion information of the current coding block is determined according to the coding parameter information of the plurality of spatial candidate blocks.

[0104] In some embodiments of the present disclosure, the coding parameter information includes: motion information. The method of determining the number of candidate motion information of the current coding block according to the coding parameter information of the multiple spatial candidate blocks includes: for each two spatial candidate blocks in the multiple spatial candidate blocks, determining the difference between the two spatial candidate blocks according to the motion information of the two spatial candidate blocks; selecting the maximum difference from the differences between all the two spatial candidate blocks, and determining the difference threshold according to the maximum difference; and determining the number of candidate motion information of the current coding block according to the difference threshold.

[0105] For each two spatial candidate blocks among the multiple spatial candidate blocks, the difference between the two spatial candidate blocks is calculated according to the MVs of the two spatial candidate blocks. The calculation formula is:

[0106] C(mv1,mv2)=ABS(mv1.x – mv2.x) + ABS(mv1.y – mv2.y) (6)

[0107] Among them, ABS() represents taking the absolute value, mv1 and mv2 represent the MVs of two spatial candidate blocks selected from multiple spatial candidate blocks, and C(mv1, mv2) represents the difference between the two spatial candidate blocks. After calculating the difference between all the spatial candidate blocks in the multiple spatial candidates, the maximum difference is selected, and the difference threshold is set to be greater than or equal to the maximum difference, so that the difference between all the spatial candidate blocks in the multiple spatial candidates is less than or equal to the difference threshold, as shown in the following formula:

[0108] ABS(mv1.x – mv2.x) + ABS(mv1.y – mv2.y)<=K (7)

[0109] Wherein, K represents the difference threshold. According to formula (7), it can be seen that the difference between all the spatial candidate blocks in the multiple spatial candidates that are combined in pairs is less than the difference threshold K. Then, the number of candidate motion information of the current coding block is determined according to the difference threshold.

[0110] In some embodiments of the present disclosure, the number of candidate motion information of the current coding block is determined based on a difference threshold, including: if the difference threshold is 0, the number of candidate motion information of the current coding block is determined to be 1; if the difference threshold is less than or equal to a first threshold, the number of candidate motion information of the current coding block is determined to be a preset value.

[0111] In the disclosed embodiment, for the HEVC standard, if the difference threshold K is 0, it means that the motion information of multiple spatial candidate blocks of the current coding block is consistent, and the number of candidate motion information of the current coding block is reduced to 1.

[0112] In the embodiment of the present disclosure, for the HEVC standard, if the difference threshold K is not 0 and is less than or equal to the first threshold, the number of candidate motion information of the current coding block is reduced to a preset value. For example, the first threshold is set to 4, and the preset value is set to 3. Of course, the values ​​of the first threshold and the preset value can be set according to actual needs or experience or coding quality requirements, and the embodiment of the present disclosure does not limit this.

[0113] In the video processing method provided by the embodiment of the present disclosure, by analyzing the difference in motion information between two spatial candidate blocks, the difference threshold is determined according to the maximum value in the difference. When the difference threshold is small, it means that the motion information of multiple spatial candidate blocks is highly similar. In this case, reducing the number of candidate motion information can reduce the encoding complexity and increase the encoding speed. In particular, when the difference threshold is 0, that is, the motion information of all spatial candidate blocks is exactly the same, only one candidate motion information is needed, which greatly simplifies the encoding process; when the difference threshold is small (less than or equal to the first threshold), by presetting the number of candidate motion information, it is possible to ensure the encoding efficiency while maintaining a certain encoding quality.

[0114] It should be noted that the above method of determining the number of candidate motion information of the current coding block based on the motion information of multiple spatial candidate blocks can be applied to the HEVC standard and other coding standards, which is not limited in the embodiments of the present disclosure.

[0115] In some embodiments of the present disclosure, the encoding parameter information further includes: at least one non-motion information. The video processing method further includes: for each non-motion information in the at least one non-motion information, obtaining the number of spatial candidate blocks with the same value in the at least one spatial candidate block, determining the number as a specific number of non-motion information; selecting the minimum number from the specific numbers of all non-motion information, and determining the specific number threshold according to the minimum number.

[0116] In the embodiments of the present disclosure, for the VVC standard and the ECM standard, when determining the number of candidate motion information of the current coding block, in addition to considering the motion information of multiple spatial candidate blocks, the non-motion information of multiple spatial candidate blocks is also considered, that is, information other than the motion information. In the VVC standard, non-motion information includes BCW index and HF, and in the ECM standard, non-motion information includes BCW index, HF and LIC flag. Of course, other non-motion information parameters may be introduced to further improve the coding performance. These parameters may include various identifiers and parameters related to intra-frame prediction, quantization, entropy coding, etc., which are not limited in the embodiments of the present disclosure.

[0117] In the disclosed embodiment, for each non-motion information, the number of spatial candidate blocks with the same value among multiple spatial candidate blocks of the current coding block is counted, and the data is named as the specific number of the non-motion information. Then, the minimum number is selected from the specific numbers of all non-motion information, and the number threshold is set to be less than or equal to the minimum number, so that the specific numbers of all non-motion information are greater than or equal to the number threshold.

[0118] Exemplarily, for the VVC standard, for BCW index, count the number p1 of spatial candidate blocks whose BCW index values ​​are all equal among multiple spatial candidate blocks of the current coding block; for HF, count the number p2 of spatial candidate blocks whose HF values ​​are all equal among multiple spatial candidate blocks of the current coding block; select the minimum number between p1 and p2, and set the number threshold P to be less than or equal to the minimum number, that is, p1 and p2 are both greater than or equal to the number threshold.

[0119] Exemplarily, for the ECM standard, for BCW index, count the number p3 of spatial candidate blocks whose BCW index values ​​are all equal among multiple spatial candidate blocks of the current coding block; for HF, count the number p4 of spatial candidate blocks whose HF values ​​are all equal among multiple spatial candidate blocks of the current coding block; for LIC flag, count the number p5 of spatial candidate blocks whose LIC flag values ​​are all equal among multiple spatial candidate blocks of the current coding block; select the minimum number among p3, p4 and p5, and set the number threshold P to be less than or equal to the minimum number, that is, p3, p4 and p5 are all greater than or equal to the number threshold.

[0120] In some embodiments of the present disclosure, the number of candidate motion information of the current coding block is determined according to the difference threshold, including: if the specific number threshold is the number of multiple spatial candidate blocks and the difference threshold is 0, then the number of candidate motion information of the current coding block is determined to be 1; if the specific number threshold is greater than or equal to the second threshold, and the difference threshold is less than or equal to the first threshold, then the number of candidate motion information of the current coding block is determined to be a preset value.

[0121] In the disclosed embodiment, for the VVC standard and the HEVC standard, if the difference threshold K is 0, and the quantity threshold P is the number of multiple spatial candidate blocks of the current coding block, it means that the motion information of the multiple spatial candidate blocks of the current coding block is consistent, and the number of candidate motion information of the current coding block is reduced to 1.

[0122] In the embodiment of the present disclosure, for the VVC standard and the HEVC standard, if the difference threshold K is less than or equal to the first threshold, and the quantity threshold P is greater than or equal to the second threshold, the number of candidate motion information of the current coding block is reduced to a preset value. For example, the first threshold and the second threshold are both set to 4, and the preset value is set to 3; for another example, the first threshold and the second threshold are both set to 3, and the preset value is set to 4. Of course, the values ​​of the first threshold, the second threshold, and the preset value can be set according to actual needs or experience or coding quality requirements, and the embodiment of the present disclosure does not limit this.

[0123] In the video processing method provided by the embodiment of the present disclosure, in addition to analyzing the motion information difference between two spatial candidate blocks and determining the difference threshold according to the maximum value in the difference, the non-motion information of multiple spatial candidate blocks is also analyzed to obtain a quantity threshold. When the difference threshold is small and the quantity threshold is large, it means that the motion information similarity of multiple spatial candidate blocks is high. In this case, reducing the number of candidate motion information can reduce the encoding complexity and improve the encoding speed. In particular, when the difference threshold is 0 and the quantity threshold is the number of multiple spatial candidate blocks, it is determined that the motion information of all spatial candidate blocks is exactly the same. At this time, only one candidate motion information is needed, which greatly simplifies the encoding process; when the difference threshold is small (less than or equal to the first threshold) and the quantity threshold is large (greater than or equal to the second threshold), by presetting the number of candidate motion information, it is possible to ensure the encoding efficiency while maintaining a certain encoding quality.

[0124] It should be noted that the above method of determining the number of candidate motion information of the current coding block based on the motion information of multiple spatial candidate blocks can be applied to the VVC standard and the HEVC standard, and can also be applied to other coding standards, which is not limited in the embodiments of the present disclosure.

[0125] In some embodiments of the present disclosure, the video processing method further includes: encoding an index of the candidate motion information of the current coding block according to the number of candidate motion information of the current coding block.

[0126] In Merge mode, the maximum number of candidates in the motion information candidate list is 6, and the merge index is represented by a truncated unary code, as shown in Table 1. As can be seen from Table 1, in Merge mode, the merge index requires 1 to 5 bits to encode. When the selected candidate index is larger, the code rate required to encode the index is higher.

[0127] Table 1 Meaning of the syntactic elements of merge index in Merge mode

[0128] value Meaning 0 Select candidate 0 10 Select the first candidate 110 Select the second candidate 1110 Select the third candidate 11110 Select the 4th candidate 11111 Select the 5th candidate

[0129] In the disclosed embodiment, the target inter-frame prediction mode is an inter-frame prediction mode that optimizes the Merge mode. The number of candidate motion information of the current coding block can be determined based on the coding parameter information of multiple spatial candidate blocks of the current coding block, that is, the maximum number of candidates in the motion information candidate list corresponding to the current coding block is determined. After starting the target inter-frame prediction mode, if the coding parameter information of multiple spatial candidate blocks of the current coding block is highly consistent, the number of candidate motion information of the current coding block is reduced from 6 to 3, and the syntax element meaning table shown in Table 2 is obtained. It can be seen from Table 2 that 1 bit to 3 bits are required for encoding. By comparison, when the second candidate is selected in Table 2, the corresponding value is 11, and when the second candidate is selected in Table 1, the corresponding value is 110, which saves 1 bit.

[0130] Table 2 The meaning of the syntax elements of merge index when the number of candidate motion information is 3

[0131] value Meaning 0 Select candidate 0 10 Select the first candidate 11 Select the second candidate

[0132] In the disclosed embodiment, after the target inter-frame prediction mode is enabled, when the inter-frame prediction mode is selected for the current coding block, the number of candidate motion information of the current coding block is derived based on the coding parameter information of multiple spatial candidate blocks, that is, the maximum number of candidates in the motion information candidate list is adjusted. At this time, the truncated coding method of the index of the coding candidate motion information will also be optimized.

[0133] If the current coding block selects the Merge mode, merge_flag is equal to 1. In the original design for the Merge mode, the merge index is directly encoded, that is, the merge index is written into the bitstream according to the syntax element meaning table shown in Table 1. For the target inter-frame prediction mode, if merge_flag is equal to 1, the number of candidate motion information of the current coding block is judged according to the encoding parameter information of multiple spatial candidate blocks of the current coding block; if the number is judged to be 1, the merge index does not need to be encoded at this time; if the number is judged to be other values, the merge index is encoded according to the size of the number. For example, if the number is 3, the merge index is encoded using the syntax element meaning table shown in Table 2.

[0134] In the disclosed embodiment, the number of candidate motion information of the current coding block can be derived based on the coding parameter information of multiple spatial candidate blocks, and then encoding is performed based on the derived number, taking into account the differences in motion information correlation between different video contents and image areas, thereby improving the coding efficiency in consistent motion scenarios.

[0135] In some embodiments of the present disclosure, the video processing method also includes: if the inter-frame prediction direction of the current coding block is bidirectional prediction, then based on the rate-distortion optimization method, determining the candidate weight information of the current coding block from the weight information pointed to by the candidate bidirectional weight prediction index.

[0136] In the embodiment of the present disclosure, when the prediction direction ID of the current coding block is equal to 3, that is, the current coding block is bidirectionally predicted, the candidate weight information of the current coding block, that is, the optimal BCW candidate, is selected by rate-distortion optimization, which is specifically implemented as follows:

[0137] (1) Based on the BCW weight pointed to by the candidate BCW index, determine the BCW candidate that needs to be selected for rate-distortion optimization for the current coding block.

[0138] Table 3 BCW candidate table

[0139]

[0140] It can be seen from Table 3 that if the BCW index is less than or equal to 2, the number of BCW candidates is 3; if the BCW index is greater than 2, the number of BCW candidates is 2. According to the BCW candidate table shown in Table 3, the encoding method of the BCW candidate is further defined.

[0141] Table 4-1 BCW candidate encoding method (BCW index <= 2)

[0142] Candidate No. 0 No. 1 Candidate 1 0 Candidate 2 1 0 Candidate 3 1 1

[0143] Table 4-2 Encoding method of BCW candidates (BCW index>2)

[0144] Candidate No. 0 Candidate 1 0 Candidate 2 1

[0145] As can be seen from Table 4-1, when BCW index <= 2, there are three BCW candidates. When the first candidate is selected, a 0 value is encoded, when candidate 2 is selected, a 1 and a 0 value are encoded, and when candidate 3 is selected, two 1 values ​​are encoded. As can be seen from Table 4-2, for the case of BCW index>2, it is sufficient to select the corresponding 0 or 1 according to whether candidate 1 or candidate 2 is finally selected.

[0146] (2) Based on the defined BCW candidates, the optimal BCW candidate is traversed through the rate-distortion optimization method, and then the method for encoding the BCW candidate is determined from Table 4-1 and Table 4-2 to encode the optimal BCW candidate.

[0147] In some embodiments of the present disclosure, the video processing method further includes: if the inter-frame prediction direction of the current coding block is non-bidirectional prediction, determining the state information of the local illumination compensation flag of the current coding block based on a rate-distortion optimization method.

[0148] In the embodiment of the present disclosure, when the prediction direction ID of the current coding block is equal to 1 or 2, the LIC flag of the current coding block is determined to be on or off by rate-distortion optimization, and the status information of the LIC flag is written into the bitstream according to the encoding method shown in Table 5.

[0149] Table 5 LIC flag encoding method

[0150] Candidate Status value Do not enable LIC-off 0 Turn on LIC-on 1

[0151] In the disclosed embodiment, for the VVC standard and the ECM standard, the state values ​​of the BCW index and the LIC flag are selected by rate-distortion optimization, the selected result is written into the bitstream, and the corresponding syntax elements are modified. The specific syntax element modification is as follows: after obtaining the merged basic motion information of the current coding block, it is determined whether the BCW index or the LIC flag needs to be encoded according to the prediction direction in the merged basic motion information. If the prediction direction is bidirectional prediction, that is, ID is equal to 3, the BCW index is encoded, otherwise, the LIC flag is encoded.

[0152] In the video processing method provided by the embodiment of the present disclosure, the BCW candidates and the LIC flag status of the current coding block are derived by rate-distortion optimization rather than by direct multiplexing, which can more accurately match the characteristics of the current coding block, ensure that they are optimally configured during the encoding process, reduce redundant information during the encoding process, and improve encoding efficiency.

[0153] It can be understood that the same / similar parts between the various embodiments of the above method in this specification can refer to each other, and each embodiment focuses on the differences from other embodiments. For related points, please refer to the description of other method embodiments.

[0154] Figure 6 FIG. 1 is a block diagram of a video processing device according to an embodiment of the present disclosure. Figure 6 As shown, the device 600 includes a video analyzing unit 610 , a determining unit 620 and a mode executing unit 630 .

[0155] Among them, the video analysis unit 610 is configured to: obtain the video to be processed; perform noise analysis and motion trend analysis on the video to be processed to obtain noise analysis results and motion trend analysis results. The determination unit 620 is configured to: if the noise analysis result is a low noise level, and / or the motion trend analysis result is the presence of consistent motion, determine to enable the target inter-frame prediction mode. The mode execution unit 630 is configured to, when the target inter-frame prediction mode is enabled and the current coding block of the video to be processed adopts the inter-frame fusion prediction mode, determine multiple spatial candidate blocks of the current coding block, obtain the encoding parameter information of the multiple spatial candidate blocks; determine the number of candidate motion information of the current coding block based on the encoding parameter information of the multiple spatial candidate blocks.

[0156] In some embodiments of the present disclosure, the encoding parameter information includes: motion information. The mode execution unit 630 is further configured to: for each two spatial candidate blocks in the plurality of spatial candidate blocks, determine the difference between the two spatial candidate blocks according to the motion information of the two spatial candidate blocks; select the maximum difference from the differences between all two spatial candidate blocks, and determine the difference threshold according to the maximum difference; and determine the number of candidate motion information of the current coding block according to the difference threshold.

[0157] In some embodiments of the present disclosure, the mode execution unit 630 is further configured to: if the difference threshold is 0, determine that the number of candidate motion information of the current coding block is 1; if the difference threshold is less than or equal to the first threshold, determine that the number of candidate motion information of the current coding block is a preset value.

[0158] In some embodiments of the present disclosure, the encoding parameter information further includes: at least one non-motion information. The mode execution unit 630 is further configured to: for each non-motion information in the at least one non-motion information, obtain the number of spatial candidate blocks with the same value in the at least one spatial candidate block, determine the number as a specific number of non-motion information; select the minimum number from the specific numbers of all non-motion information, and determine the specific number threshold according to the minimum number.

[0159] In some embodiments of the present disclosure, the mode execution unit 630 is further configured to: if the specific number threshold is the number of multiple spatial candidate blocks, and the difference threshold is 0, then the number of candidate motion information of the current coding block is determined to be 1; if the specific number threshold is greater than or equal to the second threshold, and the difference threshold is less than or equal to the first threshold, then the number of candidate motion information of the current coding block is determined to be a preset value.

[0160] In some embodiments of the present disclosure, the apparatus 600 further includes an encoding unit 640 configured to: encode an index of candidate motion information of the current coding block according to the number of candidate motion information of the current coding block.

[0161] In some embodiments of the present disclosure, the mode execution unit 630 is further configured to: if the inter-frame prediction direction of the current coding block is bidirectional prediction, then based on the rate-distortion optimization method, determine the candidate weight information of the current coding block from the weight information pointed to by the candidate bidirectional weight prediction index.

[0162] In some embodiments of the present disclosure, the mode execution unit 630 is further configured to: if the inter-frame prediction direction of the current coding block is non-bidirectional prediction, determine the state information of the local illumination compensation flag of the current coding block based on the rate-distortion optimization method.

[0163] In some embodiments of the present disclosure, the video analysis unit 610 is further configured to: determine the noise value of the video frame to be processed for each video frame to be processed in the video to be processed; and calculate the mean of the noise values ​​of all video frames to be processed in the video to be processed to obtain the noise value of the video to be processed.

[0164] In some embodiments of the present disclosure, the video analysis unit 610 is further configured to: determine whether the noise value of the video frame to be processed is less than a first noise threshold, and if so, determine that the noise analysis result of the video frame to be processed is a low noise level; determine whether the noise value of the video to be processed is less than a second noise threshold, and if so, determine that the noise analysis result of the video to be processed is a low noise level.

[0165] In some embodiments of the present disclosure, the video analysis unit 610 is further configured to: for each video frame to be processed in the video to be processed, obtain one or more image blocks in the video frame to be processed; for each image block, determine the quadrant corresponding to the motion information of the image block; determine the proportion values ​​of the image blocks corresponding to different quadrants based on the quadrants corresponding to the motion information of all image blocks, and select the maximum proportion value from the proportion values ​​of the image blocks corresponding to different quadrants; if the maximum proportion value is greater than the proportion threshold, it is determined that the motion trend analysis result of the video to be processed is that there is consistent motion.

[0166] Regarding the device in the above embodiment, the specific manner in which each unit performs the operation has been described in detail in the embodiment of the method, and will not be elaborated here.

[0167] Figure 7 FIG. 1 is a schematic diagram showing the structure of an electronic device according to an embodiment of the present disclosure. It should be noted that: Figure 7 The electronic device 700 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0168] like Figure 7As shown, the electronic device 700 is in the form of a general computing device. The components of the electronic device 700 may include but are not limited to: at least one processing unit 710, at least one storage unit 720, and a bus 730 connecting different system components (including the storage unit 720 and the processing unit 710).

[0169] The storage unit stores program codes, which can be executed by the processing unit 710, so that the processing unit 710 performs the steps according to various exemplary embodiments of the present invention described in the above “Exemplary Method” section of this specification. For example, the processing unit 710 can perform the following steps: Figure 2 Follow the steps shown in .

[0170] The storage unit 720 may include a readable medium in the form of a volatile storage unit, such as a random access storage unit (RAM) 7201 and / or a cache storage unit 7202 , and may further include a read-only storage unit (ROM) 7203 .

[0171] The storage unit 720 may also include a program / utility 7204 having a set (at least one) of program modules 7205, such program modules 7205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0172] Bus 730 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0173] The electronic device 700 may also communicate with one or more external devices 800 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 700, and / or communicate with any device that enables the electronic device 700 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 750. Furthermore, the electronic device 700 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 740. As shown, the network adapter 740 communicates with other modules of the electronic device 700 via a bus 730. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 700, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0174] In an exemplary embodiment of the present disclosure, a computer-readable storage medium is also provided, on which a program product capable of implementing the above method of the present specification is stored. In some possible implementations, various aspects of the present invention can also be implemented in the form of a program product, which includes a program code, and when the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps according to various exemplary embodiments of the present invention described in the above "Exemplary Method" section of the present specification.

[0175] The program product for implementing the above method according to the embodiment of the present invention may adopt a portable compact disk read-only memory (CD-ROM) and include program code, and may be run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto, and in this document, a readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, apparatus, or device.

[0176] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0177] Computer readable signal media may include data signals propagated in baseband or as part of a carrier wave, in which readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Readable signal media may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0178] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.

[0179] Program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0180] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0181] In addition, although the steps of the method in the present disclosure are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps, etc.

[0182] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a mobile terminal, or a network device, etc.) to execute the method according to the implementation of the present disclosure.

[0183] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any modification, use or adaptation of the present disclosure, which follows the general principles of the present disclosure and includes common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are to be considered as exemplary only, and the true scope and spirit of the present disclosure are indicated by the appended claims.

[0184] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A video processing method, characterized in that: The method comprises: Acquire a video to be processed; perform noise analysis and motion trend analysis on the video to be processed to obtain a noise analysis result and a motion trend analysis result; If the noise analysis result indicates that the noise level is low, and / or the motion trend analysis result indicates that there is consistent motion, determining to enable the target inter-frame prediction mode; When the target inter-frame prediction mode is enabled and the current coding block of the to-be-processed video adopts the inter-frame fusion prediction mode, determining multiple spatial candidate blocks of the current coding block, and obtaining coding parameter information of the multiple spatial candidate blocks; The amount of candidate motion information of the current coding block is determined according to the coding parameter information of the multiple spatial candidate blocks.

2. The method according to claim 1, characterized in that The encoding parameter information includes: motion information; The step of determining the amount of candidate motion information of the current coding block according to the coding parameter information of the multiple spatial candidate blocks includes: For every two spatial candidate blocks among the multiple spatial candidate blocks, determine a difference between the two spatial candidate blocks according to motion information of the two spatial candidate blocks; Selecting a maximum difference value from the differences between all the two spatial candidate blocks, and determining a difference threshold value according to the maximum difference value; According to the difference threshold, the amount of candidate motion information of the current coding block is determined.

3. The method according to claim 2, characterized in that The determining, according to the difference threshold, the amount of candidate motion information of the current coding block includes: If the difference threshold is 0, determining that the number of candidate motion information of the current coding block is 1; If the difference threshold is less than or equal to the first threshold, it is determined that the number of candidate motion information of the current coding block is a preset value.

4. The method according to claim 2, characterized in that: The encoding parameter information further includes: at least one non-motion information; Wherein, the method further comprises: For each of the at least one non-motion information, obtaining the number of spatial candidate blocks having the same value in the at least one spatial candidate block, and determining the number as a specific number of the non-motion information; A minimum number is selected from all the specific numbers of the non-motion information, and a specific number threshold is determined according to the minimum number.

5. The method according to claim 4, characterized in that The determining, according to the difference threshold, the amount of candidate motion information of the current coding block includes: If the specific number threshold is the number of the plurality of spatial candidate blocks, and the difference threshold is 0, determining the number of candidate motion information of the current coding block to be 1; If the specific number threshold is greater than or equal to the second threshold, and the difference threshold is less than or equal to the first threshold, it is determined that the number of candidate motion information of the current coding block is a preset value.

6. The method according to any one of claims 1 to 5, characterized in that: The method further comprises: According to the number of candidate motion information of the current coding block, an index of the candidate motion information of the current coding block is encoded.

7. The method according to claim 1, characterized in that The method further comprises: If the inter-frame prediction direction of the current coding block is bidirectional prediction, based on a rate-distortion optimization method, candidate weight information of the current coding block is determined from weight information pointed to by a candidate bidirectional weight prediction index.

8. The method according to claim 1, characterized in that: The method further comprises: If the inter-frame prediction direction of the current coding block is non-bidirectional prediction, the state information of the local illumination compensation flag of the current coding block is determined based on a rate-distortion optimization method.

9. The method according to claim 1, characterized in that: The method further comprises: For each to-be-processed video frame in the to-be-processed video, determining a noise value of the to-be-processed video frame; The noise values ​​of all the video frames to be processed in the video to be processed are averaged to obtain the noise value of the video to be processed.

10. The method according to claim 9, characterized in that The noise analysis includes one or more of the following: Determining whether the noise value of the video frame to be processed is less than a first noise threshold, and if so, determining that the noise analysis result of the video frame to be processed is a low noise level; It is determined whether the noise value of the video to be processed is less than a second noise threshold value, and if so, it is determined that the noise analysis result of the video to be processed is a low noise level.

11. The method according to claim 1, characterized in that: The movement trend analysis includes: For each to-be-processed video frame in the to-be-processed video, obtaining one or more image blocks in the to-be-processed video frame; For each of the image blocks, determining a quadrant corresponding to the motion information of the image block; Determine, according to the quadrants corresponding to the motion information of all the image blocks, the proportions of the image blocks corresponding to different quadrants, and select the maximum proportion from the proportions of the image blocks corresponding to the different quadrants; If the maximum proportion value is greater than the proportion threshold, it is determined that the motion trend analysis result of the video to be processed is that there is consistent motion.

12. A video processing device, characterized in that: The device comprises: The video analysis unit is configured to obtain a video to be processed; perform noise analysis and motion trend analysis on the video to be processed to obtain a noise analysis result and a motion trend analysis result; a determining unit configured to determine to enable a target inter-frame prediction mode if the noise analysis result indicates that the noise level is low and / or the motion trend analysis result indicates that there is a consistent motion; The mode execution unit is configured to determine multiple spatial candidate blocks of the current coding block and obtain encoding parameter information of the multiple spatial candidate blocks when the target inter-frame prediction mode is enabled and the current coding block of the video to be processed adopts the inter-frame fusion prediction mode; and determine the number of candidate motion information of the current coding block according to the encoding parameter information of the multiple spatial candidate blocks.

13. An electronic device, characterized in that: include: processor; A memory for storing the processor executable instructions; wherein the processor is configured to execute the executable instructions to implement the video processing method according to any one of claims 1 to 11. 14 . A computer-readable storage medium, when instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the video processing method according to claim 1 .

15. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the video processing method according to any one of claims 1 to 11 is implemented.

Images