Video encoding methods and apparatus, electronic devices, storage media

Abstract

This disclosure provides a video encoding method, apparatus, electronic device, and storage medium, relating to the field of video processing technology. The method includes: in response to a video receiver's computing performance being less than a preset first threshold, acquiring reconstructed frames of a fixed number of second video frames preceding a first video frame to be encoded in the video; performing image matching on every two reconstructed frames to determine a first matching degree between corresponding image blocks in every two reconstructed frames; determining whether the first video frame is a background frame based on each first matching degree; and in response to determining that the first video frame is a background frame, encoding macroblocks of the first video frame using a first mode. This disclosure can solve the problems of poor video playback smoothness and device stuttering in related technologies.

Description

Technical Field

[0001] This disclosure relates to the field of video processing technology, and more specifically, to a video encoding method and apparatus, electronic device, and storage medium. Background Technology

[0002] Video encoding refers to the process of converting a video file in one format into another using compression technology. In existing video encoding standards, an encoding mode decision algorithm is required to determine the encoding mode. However, due to the high complexity of these algorithms, they often consume significant computing resources. In real-time video communication scenarios, this can easily lead to problems such as poor video playback smoothness or device stuttering at the receiving end, especially when device computing resources are limited.

[0003] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0004] The purpose of this disclosure is to provide a video encoding method, apparatus, system, and electronic device, thereby solving to some extent the problems of poor video playback smoothness and device stuttering at the receiving end in related technologies.

[0005] According to a first aspect of this disclosure, a video encoding method is provided, the method comprising: in response to a computing performance of a video receiver being less than a preset first threshold, acquiring reconstructed frames of a fixed number of second video frames preceding a first video frame to be encoded in the video; performing image matching on every two reconstructed frames to determine a first matching degree between corresponding image blocks in every two reconstructed frames; determining whether the first video frame is a background frame based on each first matching degree; and in response to determining that the first video frame is a background frame, encoding macroblocks of the first video frame using a first mode.

[0006] Optionally, determining whether the first video frame is a background frame includes: for each image block, determining a second matching degree of the image block based on each first matching degree of the image block; determining the image block as a background image block in response to the second matching degree being less than a matching threshold; and determining the first video frame as a background frame in response to the proportion of the background image block in the reconstructed frame being greater than a preset second threshold.

[0007] Optionally, the method further includes: obtaining the quantization parameters of the co-occurring blocks in each second video frame, wherein the co-occurring block is a macroblock at the same position as the current coded macroblock in the first video frame; determining the parameter difference between the quantization parameters of every two second video frames; counting the number of times each co-occurring block is referenced; determining the coding mode of the current coded macroblock based on the parameter difference and the corresponding number of times it is referenced, and encoding the current coded macroblock using the coding mode.

[0008] Optionally, determining the coding mode of the current coding macroblock based on the parameter difference and the number of references includes: determining the total parameter difference of the second video frame based on the parameter differences; in response to the total parameter difference being greater than a preset third threshold and the sum of the number of references of each co-occurring block being greater than a preset fourth threshold, determining the coding mode of the current coding macroblock through a target coding mode decision process, wherein the candidate coding modes corresponding to the target coding mode decision process do not include the first mode; in response to the total parameter difference being greater than the preset third threshold and the sum of the number of references of each co-occurring block being less than a preset fifth threshold, comparing the first mode and the second mode to determine the mode with the lower cost as the coding mode of the current coding macroblock, wherein the computational cost of the second mode is greater than that of the first mode.

[0009] Optionally, the method further includes: in response to the computing performance of the video receiver being less than a preset first threshold, adjusting the upper and lower limits of the quantization parameters of the video encoding of the receiver, and / or adjusting the entropy encoding method of the video encoding to a first entropy encoding method.

[0010] Optionally, the method further includes: in response to the computing performance of the video receiver being less than a preset sixth threshold, omitting the filtering process in the video encoding process of the receiver, wherein the sixth threshold is less than the first threshold.

[0011] Optionally, the method further includes: in response to the power-on operation of the video receiver, obtaining the computing performance of the video receiver through a monitoring interface.

[0012] According to a second aspect of this disclosure, a video encoding apparatus is provided, comprising: an acquisition module, configured to acquire reconstructed frames of a fixed number of second video frames preceding a first video frame to be encoded in a video in response to a computational performance of a video receiver being less than a preset first threshold; a first determination module, configured to determine a first matching degree between corresponding image blocks in every two reconstructed frames; a second determination module, configured to determine whether the first video frame is a background frame based on each first matching degree; and an encoding module, configured to encode macroblocks of the first video frame using a first mode in response to determining that the first video frame is a background frame.

[0013] According to a third aspect of this disclosure, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the methods described in any of the above embodiments.

[0014] According to a fourth aspect of this disclosure, an electronic device is provided, comprising: one or more processors; and a storage device for one or more programs, which, when executed by the one or more processors, cause the one or more processors to perform the method described in any of the above embodiments.

[0015] The exemplary embodiments disclosed herein may have some or all of the following beneficial effects:

[0016] In the video encoding method provided in the exemplary embodiments of this disclosure, on the one hand, it can be determined whether the first video frame to be encoded is a background frame based on the first matching degree between the reconstructed frames of a fixed number of second video frames preceding the first video frame to be encoded. If the first video frame is a background frame, it can be directly encoded using the first mode, which can omit the encoding mode decision process for the background frame portion, thereby saving computing resources, improving encoding performance, and solving the problems of poor video playback smoothness and device stuttering in real-time communication scenarios. On the other hand, for cases where the computing performance of the video receiving end is less than a first threshold, i.e., the computing performance of the receiving end device is poor, the video smoothness and system availability of such devices in real-time communication scenarios are guaranteed while ensuring basic video quality.

[0017] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0018] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0019] Figure 1 One of the flowcharts of a video encoding method according to an embodiment of the present disclosure is illustrated schematically.

[0020] Figure 2 The illustration shows a video encoding process and a reconstructed frame according to an embodiment of the present disclosure.

[0021] Figure 3 The illustration schematically depicts a rapid decision-making process according to one embodiment of the present disclosure.

[0022] Figure 4 A flowchart of a video encoding method according to another embodiment of the present disclosure is illustrated schematically.

[0023] Figure 5 A schematic block diagram of a video encoding apparatus according to one embodiment of the present disclosure is shown.

[0024] Figure 6 A block diagram of an electronic device suitable for implementing embodiments of the present disclosure is shown. Detailed Implementation

[0025] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided to make this disclosure more comprehensive and complete, and to fully convey the concept of the exemplary embodiments to those skilled in the art. The described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a full understanding of embodiments of this disclosure. However, those skilled in the art will recognize that the technical solutions of this disclosure can be practiced with one or more specific details omitted, or other methods, components, apparatus, steps, etc., can be employed. In other instances, well-known technical solutions are not shown or described in detail to avoid obscuring various aspects of this disclosure.

[0026] Furthermore, the accompanying drawings are merely illustrative of this disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0027] The video encoding method of this application can be executed on various electronic devices. Unless otherwise specified, the electronic device can perform any step in the embodiments of this application, and the step can be performed by the processor of the electronic device. The electronic device can be various terminal devices capable of video communication, such as tablet computers or smartphones, etc., and this example does not limit it. It should also be noted that any step in the embodiments of this application can be executed independently by the electronic device, that is, when the electronic device performs any step in the following embodiments, it can do so without depending on the execution of other steps.

[0028] The video coding process can include predictive coding (intra-frame prediction and inter-frame prediction), transform coding, quantization, filtering, entropy coding, and other processes. Objects in adjacent images often have temporal correlations, and inter-frame coding utilizes the correlation between video information in consecutive frames for video compression. Inter-frame coding modes can be categorized into Merge, Skip, Affine Merge, Inter_ME, and Affine modes. The time complexity of the inter-frame coding mode decision-making process is relatively high during video coding. This application optimizes the coding process for receivers with lower computational performance, including optimizing the inter-frame decision-making process.

[0029] The video encoding method disclosed in the embodiments of this specification will be described below with reference to specific examples.

[0030] refer to Figure 1 As shown, the video encoding method of an example implementation provided in this disclosure, applied to a business server, may include the following steps.

[0031] Step S110: In response to the computing performance of the video receiver being less than a preset first threshold, a fixed number of reconstructed frames of the second video frame preceding the first video frame to be encoded in the video are obtained.

[0032] Step S120: Perform image matching on every two reconstructed frames to determine the first matching degree between corresponding image blocks in every two reconstructed frames.

[0033] Step S130: Determine whether the first video frame is a background frame based on each first matching degree.

[0034] In step S140, in response to determining that the first video frame is a background frame, the macroblocks of the first video frame are encoded using the first mode.

[0035] In the video encoding method provided in this exemplary embodiment, on the one hand, it can be determined whether the first video frame to be encoded is a background frame based on the first matching degree between the reconstructed frames of a fixed number of second video frames preceding the first video frame to be encoded. If the first video frame is a background frame, it can be directly encoded using the first mode, which can omit the encoding mode decision process for the background frame portion, thereby saving computing resources, improving encoding performance, and solving the problems of poor video playback smoothness and device stuttering in real-time communication scenarios. On the other hand, for cases where the computing performance of the video receiving end is less than a first threshold, i.e., the computing performance of the receiving end device is poor, the video smoothness and system availability of such devices in real-time communication scenarios are guaranteed while ensuring basic video quality.

[0036] The steps of this disclosure are described in more detail below.

[0037] In step S110, in response to the computing performance of the video receiver being less than a preset first threshold, a fixed number of reconstructed frames of the second video frame preceding the first video frame to be encoded in the video are obtained.

[0038] In this example implementation, the video receiver can be a video receiving device in a real-time communication scenario, such as a remote multi-terminal video conferencing scenario. Computational performance can include one or more of the hardware resource performance of the receiver device, such as CPU, memory, and storage space. For example, memory space can be used to indicate the device's computational performance; this example does not limit this. The first threshold can be set according to actual business conditions, and can also be adjusted in different scenarios or time periods; this example does not limit this. This disclosure is designed for the video encoding process of receiver devices with poor computational performance.

[0039] In this example implementation, the first video frame refers to the video frame to be encoded; for example, the first video frame can be the current video frame to be encoded. The second video frame is a number of adjacent video frames preceding the first video frame in the video sequence. The fixed number can be an integer greater than 1 (such as 2, 3, 4, etc.), for example, the second video frame can be two video frames preceding the current frame to be encoded. The reconstructed frame is the reconstructed video frame of the second video frame.

[0040] For example, such as Figure 2 As shown, the first video frame is the i-th video frame, and the second video frame is the (i-1)-th and (i-2)-th frames. The reconstructed (i-1)-th and (i-2)-th frames are obtained.

[0041] In step S120, image matching is performed on every two reconstructed frames to determine the first matching degree between corresponding image blocks in every two reconstructed frames.

[0042] In this example implementation, each reconstructed frame can be divided into image blocks. The division can be based on the macroblock size during video encoding or a custom division can be performed; this example does not limit this approach. For example, each reconstructed frame can be divided into 16×16 image blocks.

[0043] In this example implementation, various image matching algorithms can be used for image matching, such as the Mean Absolute Difference (MAD), Sum of Absolute Errors (SAD), Sum of Squared Errors (SSD), etc., and this example is not limited to any particular algorithm. Accordingly, the first matching degree can be MAD, SAD, or SSD between corresponding image blocks of two reconstructed frames. In this example, when the number of reconstructed frames is greater than two, the first matching degree (such as SAD) between corresponding image blocks of every two reconstructed frames can be calculated separately.

[0044] In step S130, it is determined whether the first video frame is a background frame based on the first matching degree.

[0045] In this example implementation, it can be determined whether an image block is a background image block based on the first matching degree of the image block, and then it can be determined whether the first video frame is a background frame based on the number of background image blocks.

[0046] For example, for each image block, a second matching degree of the image block is determined based on each first matching degree of the image block.

[0047] In this example implementation, the first matching degree refers to the matching degree of an image block in a single image matching process, and the second matching degree refers to the matching degree after fusing multiple image matching processes for that image block. For each image block, the second matching degree can be determined based on the first matching degree of each image block in each pair of reconstructed frames where image matching is performed.

[0048] For example, when the number of reconstructed frames is equal to 2, the first matching degree of the image block can be used as the second matching degree. When the number of reconstructed frames is greater than 2, the multiple first matching degrees of the image block can be weighted and summed, and the weighted sum is used as the second matching degree of the image block. For example, when the number of reconstructed frames is 3, i.e., the reconstructed frames are i-1, i-2, and i-3, then for each image block, the first matching degree of the image block in each image matching process can be determined, which may include 3 image matchings: i-1 and i-2, i-1 and i-3, and i-2 and i-3, resulting in three corresponding first matching degrees. The three first matching degrees are then weighted and summed to determine the second matching degree of the image block. In the weighted summation process, the weights can be determined based on the distance between the second video frame corresponding to the reconstructed frame and the first video frame. For example, a higher weight (e.g., weight 0.5) can be assigned to the first matching degrees corresponding to i-1 and i-2, and a lower weight (e.g., weights 0.3 and 0.2, respectively) can be assigned to the other two first matching degrees.

[0049] In response to the second matching degree being less than the matching threshold, the image block is determined to be a background image block.

[0050] In this example implementation, the matching threshold can be set based on experience; for example, the matching threshold can be set to 320.

[0051] In response to the fact that the proportion of the background image block in the reconstructed frame is greater than a preset second threshold, the first video frame is determined to be a background frame.

[0052] In this example implementation, the number of background image blocks N1 and the total number of image blocks in the reconstructed frame N2 can be counted, and then the proportion of background image blocks in the reconstructed frame is N1 / N2. The second threshold can be set or adjusted according to the actual situation. For example, the second threshold can be set to be greater than 0.8, such as 0.9. When the number of background image blocks in the reconstructed frame is large, it can be considered that the first video frame is more likely to be a background frame.

[0053] In step S140, in response to determining that the first video frame is a background frame, the macroblocks of the first video frame are encoded using the first mode.

[0054] In this example implementation, the first mode refers to the inter-frame coding mode, which is used to characterize the macroblocks of the first video frame as having no motion vectors and no pixel residuals. In one implementation, the first mode can be a skip mode, which can omit the mode decision process of the background frame in the first video frame and reduce the coding computation complexity.

[0055] In some embodiments, in order to improve encoding performance while keeping video quality within a controllable range, refer to Figure 3 The method also includes the following steps S301-310.

[0056] Step S301: Obtain the quantization parameters of the co-position blocks in each second video frame.

[0057] In this example implementation, a co-occurring block is a macroblock that has the same position as the current coded macroblock of the first video frame, and the quantization parameters of the co-occurring block in each second video frame can be obtained from the system.

[0058] Step S302: Determine the parameter difference of the quantization parameters for every two second video frames.

[0059] In this example implementation, the difference between every two quantization parameters can be calculated, similar to the image matching process.

[0060] Step S303: Count the number of times each co-located block is referenced.

[0061] In this example implementation, the number of times each co-location block is referenced throughout the entire video encoding process can be counted. Co-location blocks with a higher number of references can be considered to be of higher importance.

[0062] Step S304: Determine the total parameter difference ref of the second video frame based on the differences of each parameter.

[0063] In this example implementation, when the number of second video frames is 2, the parameter difference between co-position blocks of the two frames can be directly used as the total parameter difference. When the number of second video frames is greater than 2, the parameter differences between co-position blocks within each pair of frames can be weighted and summed, and the weighted sum can be used as the total parameter difference. The parameter differences of second video frames closer to the first video frame can be assigned a larger weight, while those farther away can be assigned a smaller weight, and the sum of the weights of all parameter differences can be 1.

[0064] Step S305: Determine whether the total parameter difference ref is greater than the preset third threshold and the sum of the number of times each co-located block is referenced is greater than the preset fourth threshold. If yes, proceed to step S306; otherwise, proceed to step S307.

[0065] In this example implementation, the third and fourth thresholds can be obtained based on actual business conditions or experience, and this example does not limit them. For example, when the number of second video frames is 2, the third threshold can be set to 3, and the fourth threshold can be set to 5.

[0066] Step S306: Determine the encoding mode of the current encoding macroblock through the target encoding mode decision process, and proceed to step S310.

[0067] In this example implementation, the target coding mode decision process can be a coding decision process that does not include the first mode. That is, it is a mode decision method in existing mode decision methods after removing the operation of judging the Skip mode. In other words, it is a decision process that does not perform the Skip mode judgment operation and directly performs the other mode judgment operation. For example, for OpenH264, the Skip mode judgment operation can be omitted, and the P16x16 mode judgment operation and subsequent P8x8 mode and I4x4 mode judgment operations can be performed directly. The mode with the lowest cost is selected as the optimal macroblock mode, that is, the coding mode of the current coding macroblock. P refers to the method of using previously decoded pixels as reference to realize intra-frame prediction, 16x16 refers to the size of the P macroblock is 16x16, and I refers to the intra-frame mode.

[0068] In this example implementation, the decision-making process can be simplified and computational complexity reduced by omitting the fast decision-making process of the Skip pattern.

[0069] Step S307: Determine whether the total parameter difference ref is greater than the preset third threshold and the sum of the number of times each co-located block is referenced is less than the preset fifth threshold. If yes, proceed to step S308; otherwise, proceed to step S309.

[0070] In this example implementation, the fifth threshold can be determined based on actual business conditions or experience, and this example does not limit it. For example, when the number of second video frames is 2, the fifth threshold can be set to 1.

[0071] Step S308: Compare the first mode and the second mode to determine the mode with the lower cost as the encoding mode of the current encoding macroblock, and proceed to step S310.

[0072] In this example implementation, the computational cost of the second mode is greater than that of the first mode. The second mode is used to characterize each coded macroblock as having pixel residuals but no motion vectors. In one feasible approach, the second mode can be a P16x16 mode; where P refers to the method of using previously decoded pixels as references to achieve intra-frame prediction. The cost function values ​​of the two modes can be calculated, such as calculating the SATD (Sum of Absolute Transformed Difference) for each mode, and the mode corresponding to the smaller SATD value is taken as the coding mode of the current coded macroblock.

[0073] Step S309: Determine the coding pattern based on the existing pattern decision-making process, then proceed to step S310.

[0074] In this example implementation, mode decisions can be made based on the actual video coding standard.

[0075] Step S310: Encode the current coded macroblock using the determined encoding mode.

[0076] In this example implementation, the current coded macroblock can be predictively coded using a defined encoding pattern.

[0077] In some embodiments, the method further includes: in response to the computing performance of the video receiver being less than a preset first threshold, adjusting the upper and lower limits of the quantization parameters of the video encoding of the receiver, and / or adjusting the entropy encoding method of the video encoding to a first entropy encoding method.

[0078] In this example implementation, when the performance of the receiving device is detected to be less than the first threshold, the range of values ​​for the quantization parameters in the encoder can be narrowed, that is, the upper and lower limits of the quantization parameters can be adjusted. For example, the upper limit of the quantization parameters can be reduced and the lower limit of the quantization parameters can be increased. For example, the range of values ​​for the quantization parameters can be adjusted from 0-50 to 28-42.

[0079] In this example implementation, if the performance of the receiving device is detected to be less than the first threshold, the entropy coding method can be adjusted to the first entropy coding method. The first entropy coding method refers to the method with less computational complexity. For example, the entropy coding method of the encoder can be adjusted from CABAC (Context-based Adaptive Binary Arithmetic Coding) to CAVLC (Context-based Adaptive Variable-Length Coding) to reduce computational complexity.

[0080] In some embodiments, the method further includes: omitting filtering processing in the video encoding process of the video receiver in response to the computing performance of the video receiver being less than a preset sixth threshold.

[0081] In this example implementation, the sixth threshold is less than the first threshold, and the sixth threshold can be set according to the actual business requirements. For example, when computational performance is characterized by memory size, the first threshold can be set to 2GB and the sixth threshold to 1GB. This example further simplifies computational complexity by omitting filtering during the encoding process, ensuring smooth video in real-time communication scenarios.

[0082] In some embodiments, the method further includes: in response to a power-on operation of the video receiver, obtaining the computing performance of the video receiver through a monitoring interface.

[0083] In this example implementation, a computing performance monitoring interface can be configured in the receiving device to obtain the device's computing performance when the device is powered on, and the monitoring results can be sent to the encoder through the monitoring interface to optimize the encoder's encoding process when the computing performance is poor.

[0084] For example, such as Figure 4 As shown, the video encoding method disclosed herein can be implemented through the following steps S401-S422.

[0085] Step S401: The video receiver is powered on and the device's computing performance is obtained through the monitoring interface.

[0086] Step S402: Determine whether the computational performance is less than the first threshold. If yes, proceed to step S403; otherwise, proceed to step S422.

[0087] Step S403: Adjust the upper and lower limits of the quantization parameters of the video encoding at the receiving end, and adjust the entropy encoding method of the video encoding to the first entropy encoding method.

[0088] Step S404: Determine whether the computational performance is less than the sixth threshold. If yes, proceed to step S405; otherwise, proceed to step S406.

[0089] Step S405: Turn off the filtering module of the encoder at the receiving end, that is, omit the filtering process during the encoding process.

[0090] Step S406: Obtain the reconstructed frames of the two second video frames preceding the first video frame to be encoded in the video.

[0091] Step S407: Perform image matching on the two reconstructed frames to determine the first matching degree between corresponding image blocks in the two reconstructed frames.

[0092] Step S408: Determine whether the first matching degree of the current image block is less than the matching threshold. If yes, proceed to step S409; otherwise, proceed to step S410.

[0093] Step S409: Determine that the image block is a background image block.

[0094] Step S410: Proceed to the next image block determination.

[0095] Step S411: Determine whether the matching has ended, i.e. whether all image blocks have been matched. If yes, proceed to S412; otherwise, proceed to step S410.

[0096] Step S412: Determine whether the proportion of the background image block in the reconstructed frame is greater than the preset second threshold. If yes, proceed to step S413; otherwise, proceed to step S414.

[0097] Step S413: Determine the first video frame as the background frame, and encode the macroblocks of the first video frame using the first mode.

[0098] Step S414: Obtain the quantization parameters of the co-position blocks in each second video frame.

[0099] Step S415: Determine the parameter difference between the quantization parameters of the two second video frames, and count the number of times each co-position block is referenced.

[0100] Step S416: Determine whether the first condition is met: the parameter difference is greater than the preset third threshold and the sum of the number of times each co-position block is referenced is greater than the preset fourth threshold. If so, proceed to step S417; otherwise, proceed to step S418.

[0101] Step S417: Determine the coding mode of the current coding macroblock through the target coding mode decision process, and proceed to step S421. The candidate coding modes corresponding to the target coding mode decision process do not include the first mode.

[0102] Step S418: Determine whether the second condition is met: the parameter difference is greater than the preset third threshold and the sum of the number of times each co-position block is referenced is less than the preset fifth threshold. If yes, proceed to step S419; otherwise, proceed to step 420.

[0103] Step S419: Compare the first mode and the second mode to determine the mode with the lower cost as the encoding mode of the current encoding macroblock, and proceed to S421, where the computational cost of the second mode is greater than that of the first mode.

[0104] Step S420: Determine the coding pattern using the existing pattern decision-making process, then proceed to step S421.

[0105] Step S421: The encoder performs macroblock encoding of the first video frame using a defined encoding mode.

[0106] In step S422, the encoder performs macroblock encoding of the first video frame using existing video encoding methods. In this example, no encoding optimization is performed if the computing power of the receiving device is sufficient.

[0107] In the above embodiments, the number of second video frames is two. If the number of second video frames is greater than two, it is only necessary to perform weighted processing on the corresponding parameters, as shown in the previous embodiments. This example will not be repeated here.

[0108] This invention addresses the issue of low computing performance of receiving devices in real-time media streaming scenarios. Such devices may be unable to support real-time video encoding and decoding, resulting in video stuttering or even system crashes.

[0109] For this type of equipment, it is necessary to minimize the computational complexity and workload of the video encoding process while ensuring basic video quality. This application significantly reduces the computational complexity and workload of the video encoding process by performing preset configuration pruning operations on the encoder (adjusting the upper and lower limits of quantization parameters and entropy coding method) and optimizing the mode decision process (including predicting background frames, the target coding mode decision process, and a simplified two-mode decision comparison process). This ensures smooth video playback on low-performance receivers, avoids device stuttering or crashes, and guarantees normal communication.

[0110] Furthermore, in this example embodiment, a video encoding apparatus 500 is also provided. This video encoding apparatus 500 can be applied to a video terminal. (See reference...) Figure 5 As shown, the video encoding device 500 may include: an acquisition module 510, a first determination module 520, a second determination module, and an encoding module 530. The acquisition module 510 is used to acquire a fixed number of reconstructed frames of a second video frame preceding the first video frame to be encoded in the video in response to the computing performance of the video receiver being less than a preset first threshold. The first determination module 520 is used to determine a first matching degree between corresponding image blocks in every two reconstructed frames. The second determination module 530 is used to determine whether the first video frame is a background frame based on each first matching degree. The encoding module 540 is used to encode the macroblocks of the first video frame using a first mode in response to the determination that the first video frame is a background frame.

[0111] In one exemplary embodiment of this disclosure, the second determining module 530 may further be used to: for each image block, determine a second matching degree of the image block based on each first matching degree of the image block; determine the image block as a background image block in response to the second matching degree being less than a matching threshold; and determine the first video frame as a background frame in response to the proportion of the background image block in the reconstructed frame being greater than a preset second threshold.

[0112] In one exemplary embodiment of this disclosure, the apparatus 500 further includes a decision module, which can be used to: obtain the quantization parameters of co-occurring blocks in each second video frame, wherein a co-occurring block is a macroblock at the same position as the current coded macroblock of the first video frame; determine the parameter difference between the quantization parameters of every two second video frames; count the number of times each co-occurring block is referenced; determine the encoding mode of the current coded macroblock based on each parameter difference and the corresponding number of times it is referenced; and encode the current coded macroblock using the encoding mode.

[0113] In an exemplary embodiment of this disclosure, the decision module may further be configured to: determine the total parameter difference of the second video frame based on the parameter differences; in response to the total parameter difference being greater than a preset third threshold and the sum of the reference times of each co-occurrence block being greater than a preset fourth threshold, determine the coding mode of the current coding macroblock through a target coding mode decision process, wherein the candidate coding modes corresponding to the target coding mode decision process do not include the first mode; in response to the total parameter difference being greater than the preset third threshold and the sum of the reference times of each co-occurrence block being less than a preset fifth threshold, compare the first mode and the second mode to determine the mode with the lower cost as the coding mode of the current coding macroblock, wherein the computational cost of the second mode is greater than that of the first mode.

[0114] In one exemplary embodiment of this disclosure, the apparatus 500 further includes a pruning module, which can be used to adjust the upper and lower limits of the quantization parameters of the video encoding of the receiving end in response to the computing performance of the video receiving end being less than a preset first threshold, and / or adjust the entropy encoding method of the video encoding to a first entropy encoding method.

[0115] In one exemplary embodiment of this disclosure, the pruning module can also be used to: omit filtering processing in the video encoding process of the receiving end in response to the computing performance of the video receiving end being less than a preset sixth threshold, wherein the sixth threshold is less than the first threshold.

[0116] In one exemplary embodiment of this disclosure, the apparatus 500 may further include a monitoring module, which can be used to obtain the computing performance of the video receiver through a monitoring interface in response to the power-on operation of the video receiver.

[0117] The specific details of each module or unit in the aforementioned video encoding device have been described in detail in the corresponding video encoding methods, so they will not be repeated here.

[0118] On the other hand, this application also provides a computer-readable medium, which may be included in the electronic device described in the above embodiments; or it may exist independently and not assembled into the electronic device. The computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to perform the methods as described in the following embodiments. For example, the electronic device may perform... Figures 1-4 The various process steps are shown.

[0119] It should be noted that the computer-readable medium disclosed herein may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0120] The following reference Figure 6 To describe an electronic device 600 according to such an embodiment of the present disclosure. Figure 6 The electronic device 600 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments disclosed herein.

[0121] like Figure 6As shown, the electronic device 600 is presented in the form of a general electronic device. The components of the electronic device 600 may include, but are not limited to: at least one processing unit 610, at least one storage unit 620, a bus 630 connecting different system components (including storage unit 620 and processing unit 610), and a display unit 640.

[0122] The storage unit stores program code, which can be executed by the processing unit 610, causing the processing unit 610 to perform the steps described in the "Exemplary Methods" section above according to various exemplary embodiments of this disclosure.

[0123] Storage unit 620 may include a readable medium in the form of a volatile storage unit, such as random access memory (RAM) 6201 and / or cache memory 6202, and may further include a read-only memory (ROM) 6203.

[0124] Storage unit 620 may also include a program / utility 6204 having a set (at least one) program module 6205, such program module 6205 including but not limited to: operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0125] Bus 630 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0126] Electronic device 600 can also communicate with one or more external devices 670 (e.g., keyboard, pointing device, Bluetooth device, etc.), and with one or more devices that enable a user to interact with electronic device 600, and / or with any device that enables electronic device 600 to communicate with one or more other electronic devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 650. Furthermore, electronic device 600 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 660. As shown, network adapter 660 communicates with other modules of electronic device 600 via bus 630. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RA identification systems, tape drives, and data backup storage systems.

[0127] In an exemplary embodiment, a computer program product is also provided, which may include, for example, computer-readable code or a non-volatile computer-readable storage medium carrying computer-readable code; when the computer program product is run on a computer, the computer performs the methods described in the above embodiments.

[0128] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause an electronic device (such as a personal computer, server, terminal device, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0129] Furthermore, the above figures are merely illustrative of the processes included in the method according to exemplary embodiments of this disclosure and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Additionally, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0130] It should be noted that although the steps of the method in this disclosure are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in that specific order, or that all the steps shown must be performed to achieve the desired result. Additional or alternative steps, such as omitting certain steps, combining multiple steps into one step, and / or breaking down one step into multiple steps, should all be considered part of this disclosure.

[0131] It should be understood that this disclosure, as disclosed and defined herein, extends to all alternative combinations of two or more individual features mentioned or apparent in the text and / or figures. All these different combinations constitute multiple alternative aspects of this disclosure. The embodiments described in this specification illustrate the best known mode for implementing this disclosure and will enable those skilled in the art to utilize it.

Claims

1. A video encoding method, characterized in that, include: In response to the computing performance of the video receiver being less than a preset first threshold, a fixed number of reconstructed frames of the second video frame preceding the first video frame to be encoded in the video are obtained. For every two reconstructed frames, perform image matching to determine the first matching degree between corresponding image blocks in each pair of reconstructed frames; Based on each of the first matching degrees, determine whether the first video frame is a background frame; In response to determining that the first video frame is a background frame, the macroblocks of the first video frame are encoded using a first mode; The method further includes: Obtain the quantization parameters of the co-occurring block in each of the second video frames, wherein the co-occurring block is the macroblock at the same position as the current coded macroblock of the first video frame; Determine the parameter difference of the quantization parameters for every two second video frames; Count the number of times each of the aforementioned co-position blocks is referenced; Based on the differences in the parameters and the corresponding number of references, the encoding mode of the current coded macroblock is determined, and the current coded macroblock is encoded using the encoding mode.

2. The video encoding method according to claim 1, characterized in that, Determining whether the first video frame is a background frame includes: For each image block, a second matching degree is determined based on each of the first matching degrees of that image block; In response to the second matching degree being less than the matching threshold, the image block is determined to be a background image block; In response to the fact that the proportion of the background image block in the reconstructed frame is greater than a preset second threshold, the first video frame is determined to be a background frame.

3. The video encoding method according to claim 1, characterized in that, Determining the encoding mode of the current coded macroblock based on the parameter difference and the number of references includes: Based on the parameter differences described above, determine the total parameter difference of the second video frame; In response to the total parameter difference being greater than a preset third threshold and the sum of the reference times of each of the co-position blocks being greater than a preset fourth threshold, the coding mode of the current coding macroblock is determined through a target coding mode decision process, wherein the candidate coding modes corresponding to the target coding mode decision process do not include the first mode. In response to the total parameter difference being greater than a preset third threshold and the sum of the reference times of each of the co-occurring blocks being less than a preset fifth threshold, a decision comparison is made between the first mode and the second mode, and the mode with the lower cost is determined as the encoding mode of the current encoding macroblock, wherein the computational cost of the second mode is greater than that of the first mode.

4. The video encoding method according to claim 1, characterized in that, The method further includes: In response to the computing performance of the video receiver being less than a preset first threshold, the upper and lower limits of the quantization parameters of the video encoding at the receiver are adjusted, and / or the entropy encoding method of the video encoding is adjusted to the first entropy encoding method.

5. The video encoding method according to claim 4, characterized in that, The method further includes: If the computing performance of the video receiver is less than a preset sixth threshold, the filtering process in the video encoding process of the receiver is omitted, wherein the sixth threshold is less than the first threshold.

6. The video encoding method according to claim 1, 4, or 5, characterized in that, The method further includes: In response to the power-on operation of the video receiver, the computing performance of the video receiver is obtained through the monitoring interface.

7. A video encoding device, characterized in that, The device includes: The acquisition module is used to acquire a fixed number of reconstructed frames of the second video frame preceding the first video frame to be encoded in the video in response to the computing performance of the video receiver being less than a preset first threshold. The first determining module is used to determine the first matching degree between corresponding image blocks in every two reconstructed frames; The second determining module is used to determine whether the first video frame is a background frame based on each of the first matching degrees. The encoding module is configured to encode the macroblocks of the first video frame in a first mode in response to determining that the first video frame is a background frame. The device further includes a decision module, used to obtain the quantization parameters of co-occurring blocks in each second video frame, where a co-occurring block is a macroblock at the same position as the current coded macroblock in the first video frame; determine the parameter difference between the quantization parameters of every two second video frames; count the number of times each co-occurring block is referenced; determine the encoding mode of the current coded macroblock based on each parameter difference and the corresponding number of references, and encode the current coded macroblock using the encoding mode.

8. An electronic device, characterized in that, include: One or more processors; as well as A storage device for storing one or more programs, which, when executed by one or more processors, cause the one or more processors to implement the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1 to 6.

Images